You could say artificial intelligence (AI) is both the cloud industry’s blessing and its curse. AI has been touted for its promise in streamlining and even automating mundane tasks, but DoiT International chief Product Officer John Purcell told Silverlinings that training the brains behind the models comes with significant cloud costs.
Purcell said there are two primary kinds of AI workloads: those used to train and evolve models and those used for interacting with the trained AI. He noted AI cloud workloads are “inherently GPU heavy” because they carry a higher computational load than, for instance, running a JavaScript application. This is especially true when it comes to the aforementioned training workloads, and it has big implications for cost since GPU instances tend to carry higher price tags.
A quick glance at AWS and Google Cloud’s pricing bears out this assertion. Google Cloud, for example, charges $3.465 per node hour for classification and object detection training with its Vertex AI AutoML tool, whereas deployment and online prediction runs between $1.375 and $2.002 per node hour and batch prediction costs $2.222. Using AI accelerators for those workloads like NVIDIA’s P100 or A100 GPUs can cost anywhere from $1.679 to $3.374 per node hour.
According to Purcell, most enterprises aren’t – or shouldn’t be – surprised by this fact. And companies with more mature cloud strategies are usually ok with cloud costs increasing so long as they do so in proportion to the benefits the companies are receiving. But they can be caught off guard when costs rise faster than expected or if there’s a disconnect between the teams implementing the workloads and those paying the bills, he said.
There’s no real way to avoid the cost of AI, Purcell said. That’s because “as a tech company or a company that relies on information, data, knowledge to provide value to the market, if you’re not exploring what’s going on here you run the risk of being left behind.”
But there is a way to effectively manage it…using AI. That’s what DoiT is doing.
“I think where AI can potentially help is if you can inject into your model the ability to convert prompts, questions asked, into a data query – ‘tell me what my cost drivers were last month,’ ‘tell me what costs increased more than 5% last month,’ ‘I saw a spike in my bill on this date, what caused the spike in my bill’ – that’s where we are experimenting with a more conversational interface,” he said.
DoiT is one of many companies in the cloud cost management arena, alongside the likes of CoreStack, Flexera, Morpheus and VMware CloudHealth. As noted above, it is hoping to differentiate itself with a fresh, tech-enabled approach to classic cost management that reaches across siloed departments.
“Quite often, the market has been looking for who’s the person at the company that cares about how much you’re spending in the cloud on a monthly basis. Find that person and I’ve got a product for you,” he explained. “What we’re finding…is that ownership of that problem has shifted. It was centralized for a period of time where there was one person who owned that and tried to influence the rest of the organization to make responsible choices. It then became decentralized or federated across the organization…so culturally or organizationally there is an awareness that is required.”