There’s a common misunderstanding that profiling is only useful for tiny savings that impact infra costs at scale - the so-called “milliseconds matter” approach. But by dogfooding our own profiling tools, we fixed a problem that saved tens of seconds off each user interaction with our AI agent (and for those of you who like math, that’s four orders of magnitude bigger than those milliseconds that matter).

Enterprise GenAI prototypes are everywhere. GenAI in production are far fewer. Why? Management consultancies have different success numbers – we’ve heard 3% or 5% going to prod across tens of thousands of AI proof of concepts. For obvious reasons, it’s not a number LLM enthusiasts want to talk about. But we probably should.

Google’s comprehensive AI offering includes Vertex AI, a cloud-based platform for building and deploying AI applications, AI Studio, a web platform for quickly prototyping and testing AI applications, and Gemini, their multimodal model. Gemini offers advanced capabilities in image, code, and text generation and can be used to implement chatbot assistants, perform complex data analysis, generate design assets, and more.

Amazon has heard FinOps practitioners’ cries asking for new AI tools, and the answer is Titan and AWS Bedrock. These new tools provide the same generative AI abilities of generating images like expected from DALL-E, operating like a Large Language Model (LLM) like ChatGPT, and even transcribing audio to text. But how do these new tools compare to pre-existing ones like Azure’s OpenAI? Most importantly, which of these tools is the best financial investment for your organization?