Artificial intelligence (AI) and machine learning (ML) are hotter than asphalt in Death Valley. But, developing efficiently with these tools can still be a cold, slow process. That's where GPU cloud specialists, such as CoreWeave, Lambda Labs, RunPod and Arc Compute, come in. It’s also the space where Union.ai, an AI pioneer, plays with its new Union Cloud service.
Founded in 2021, Union.ai was created as a workflow orchestration platform catering to AI and ML. Last year it raised $10 million in seed funding from venture capital firm New Enterprise Associates (NEA) and this May bagged another $19.1 million in fresh funding from NEA and Nava Ventures to expand its offerings and grow the business. The latter announcement coincided with the launch of Union Cloud.
Union Cloud works on top of Amazon Web Services (AWS) and Google Cloud Platform (GCP). The whole point of Union Cloud is to speed the AI development process while helping companies maintain the security of and control over their data. To do this, it uses the popular open-source program Flyte to improve ML model-building workflow. This comes as no surprise given Union.ai was founded by the developers who launched Flyte in 2016. Flyte was open-sourced through the Linux Foundation in 2020.
Taking Flyte
So, what's Flyte? It's a cloud-native workflow orchestration platform built on top of Kubernetes, everyone's favorite container orchestration program. It provides an abstraction layer for guaranteed scalability and reproducibility of data and ML workflows. Its containerized, microservices-based architecture is designed to ensure resilience and eliminate single points of failure.
Flyte's more than 500 users include big tech names such as Spotify, Stripe, LinkedIn and ZipRecruiter, showing the program clearly does the job. That is, it helps improve the versioning of data and AI pipelines to vastly increase ML development productivity.
Flyte enables user teams to build workflows using the Python Software Development Kit (SDK) and still easily deploy their workflows to the Flyte backend. That means data scientists, ML practitioners and analytics pipeline builders can work on their own. They don't need to rely on ML and platform engineers to turn models or training pipelines into production-ready pipelines.
Of course, to use Flyte effectively, you still need to know your way around Python and other AI/ML programming tools and Flyte itself. That’s where Union Cloud comes in. In a nutshell, Union Cloud offers everything Flyte does in terms of orchestration for AI/ML without requiring companies to invest in building and maintaining their own infrastructure. It also makes it easier for companies to break down silos and share work across teams with reusable tasks, versioned workflows and an array of plugins.
Union Cloud was launched with general availability for AWS and GCP customers in Europe and North America.
Game changer
Investor and NEA venture partner Greg Papadopoulos noted in a statement “companies are becoming increasingly wary of using services where they lose control over what precisely happens with their data.” But he added Union.ai lets businesses “exploit the power of large language models while maintaining control and ownership of my data.”
Manish Patel, the founder of Nava Ventures, another Union.ai investor, added that companies with limited internal developer resources are disadvantaged. Union Cloud, powered by Flyte, enables these companies to transition quickly from research to production.
Given Flyte's track record and the desperate demand for easy-to-use, top-quality AI/ML services, Union Cloud may become a real player in the nascent AI development space.