Companies that have pioneered the use of AI at scale have done so by using their own in-house ML systems (Uber, LinkedIn, Facebook, Airbnb). Many companies are already offering these features as off-the-shelf devices. There are also a number of open-source technologies that handle MLOps. The race to space has generated a new issue—too much variety. Hundreds of tools and at least 40 platforms are currently available:
This is challenging terrain to tackle. However, enterprises must figure it out since getting value from ML is critical. Let’s first identify the major obstacles and then we’ll present some fresh, free content aimed at addressing these issues.
Challenge #1: Overwhelming Choice
In 2020, Chip Huyen acquired data about the ML tool scene. Chip discovered 284 tools—and the number is rising. A solid evaluation does not have to analyze all of these instruments in detail if they can be narrowed down to the most important tools from the start. That won’t be straightforward because we’d need distinct categories to do so, and MLOps categories aren’t very clear.
Challenge #2: Blurred Categories
By categorizing items, we can usually obtain a sense of what software does. This has been attempted with the LFAI Landscape diagram, the MAD landscape, and GitHub lists. However, ML software frequently accomplishes more than one thing and may fall into numerous categories. Platforms are particularly difficult to describe since they expressly aspire to perform more than one task or provide a single service.
Because it is difficult to label software that accomplishes several tasks, MLOps platforms all tend to end up in a single platform” category This obscures the importance of each platform while also blurring the subtleties of how different platforms achieve comparable things in different ways.
Challenge #3: Shifting Landscapes
ML categories are difficult to keep track of, in part because new ones emerge all the time. Feature shops, for example, have only been established for a short time but have quickly grown into a substantial segment. This has an impact on platforms as well since platforms are adding major new features and shifting their emphasis (in part in response to the appearance of new ones).
Challenge #4: Complex Problems
ML is a difficult subject. It is also a large field. Understanding regression, classification, NLP, image processing, reinforcement learning, explainability, AutoML, and other platforms are essential.
Challenge #5: Range of Roles and Stakeholders
Not only are the challenges diverse and complicated, there are also several job roles involved in the ML lifecycle. Data Scientists, Machine Learning Engineers, Data Engineers, Site Reliability Engineers, Product Managers, Architects, Application Developers, Platform Developers, End Users, and so on. Different jobs interact with the program in different ways and have different requirements from it.
Challenge #6: Build vs Buy and Other Controversies
In the ML area, there is a lot of debate over the build vs buy trade-off. How much control do businesses need over their own stack? How much does that control cost? Build vs buy is sometimes presented as an either-or choice, but it is actually a continuum. This is simply one of many perplexing debates in the ML space (considering how controversial AutoML is).
How Do We Get on Top of All This?
Let’s look at specialized tools vs platforms independently.
The mlops.community has some nice profiles for specialty tools that you may delve into on their website. Tools are organized by category, and the website includes feature lists, introductory videos, and a tool for comparing tools side by side.
Thoughtworks has also published a Guide to Evaluating MLOps Platforms to help users comprehend the platform environment and cope with trade-offs.
This is accessible for free and there’s no need to sign up. It discusses how to differentiate MLOps systems. The book examines MLOps platforms via several lenses and presents categories to help categorize them. Here’s an illustration from the handbook that shows the contrast between MLOps platforms and tools:
There is a range, with some all-in-one platforms attempting to handle the whole ML lifetime, specialized tools addressing certain aspects of the lifecycle, and several targeted platforms addressing various elements of the lifecycle—but not the entire lifecycle. Other topics covered in the guide include methods for low-code platforms like AutoML, the roles and personas involved in MLOps, and how to conduct an evaluation based on your organization’s goals.
Understanding the high-level platform landscape is simply one aspect of the evaluation. It is also vital to put this information to use and learn how to compare platforms. On GitHub, an open-source comparison matrix was published to do this.
The matrix is designed to highlight how providers work in their own unique way—while also indicating more extensive information in the product documentation. The writers have put a number of profiles in the repository that briefly, and without marketing, define the regions of goods of major platforms.
Hopefully, you have found this information valuable.
