Since last year, the costs of cloud-based GPU computing have been decreasing, providing companies with opportunities for real cost savings depending on how flexibly they utilize computational resources. Cast AI’s recent report analyzed the changing cost structure of NVIDIA A100 and H100 GPU-based cloud computing, comparing prices and availability among major cloud providers like AWS, Microsoft Azure, and Google Cloud Platform.
Laurent Gille, CEO of Cast AI, explained that while a few large companies like OpenAI and Google dominate model training, more startups are focusing on inference tasks that deliver immediate business value. The report highlighted significant price declines for high-demand GPU spot instances, with some prices dropping as much as 88% in certain regions. Such downward trends suggest that cloud providers may have higher capacity than expected, leading to more competitive pricing across the board.
Gille emphasized the need for operational and geographical agility in effectively managing cloud GPU resources. Companies that can dynamically move workloads between regions using AI-based automation could reduce costs by up to 80%. He advised engineers and CTOs to adopt flexibility and automation, moving away from rigid operational models to ensure sustainable AI infrastructure.
👉 Pročitaj original: CIO Magazine
