Context, Cost, & AI: Strategies to Optimise Agent Performance & ROI

After months of experimenting with large language models (LLMs), enterprises are moving from isolated pilots to broad deployments. However, as adoption speeds up, many tech leaders find that the biggest challenge isn't selecting the right AI model but managing the cost of optimising it. While AI models might be getting more expensive owing to geopolitical tensions and expanding use cases, the added challenge is that the price per token continues to rise. The problem is that enterprise AI is using a lot more tokens as companies scale new applications across different departments. In the recent episode of the Don’t Panic It’s Just Data podcast, host Kevin Petrie, VP of Research at BARC, sat down with Eudald Camprubi, Co-Founder of Nuclia (acquired by Progress) and Software Fellow at Progress and Michael Marolda, Senior Product Marketing Manager, Agentic RAG. They talked about how the next phase of enterprise AI will focus less on model selection and more on context engineering, retrieval strategies, and governance. ## What Does the Future of Enterprise AI Rely on? As enterprises progress beyond experimentation, success will rely less on picking the next game-changing model and more on using existing models wisely. That means providing precise context instead of excessive context. Enterprises should select the right model instead of just the latest model. That means they should treat observability, retrieval strategies, and governance as foundational capabilities rather than second priorities. Camprubí said that enterprise leaders should be wary of industry hype. "Don't trust everything you read on LinkedIn.” In enterprise AI, measurable results will determine which enterprises effectively scale intelligent systems. Listen to the full episode of Don't Panic! It's Just Data to hear Michael Marolda and Eudald Camprubí discuss Agentic RAG, token optimisation, context engineering, and the future of enterprise AI at scale. ### Takeaways - Token costs are climbing, making cost management critical for enterprises. - Context is essential for effective AI implementation and user satisfaction. - Enterprises must focus on providing the right context to LLMs to avoid hallucinations. - Modularity in AI platforms allows for flexibility and adaptability in solutions. - Quality metrics are vital for evaluating AI outputs and ensuring reliability. - The latest AI models are not always the best choice for every use case. - Understanding user intent is crucial for effective data retrieval. - FinOps teams are increasingly involved in managing AI costs and strategies. - Enterprises should consider RAG as a service to reduce maintenance burdens. - Collaboration among stakeholders is essential for successful AI implementation. ### Chapters - 00:00 Introduction to AI and Data Context - 03:26 Understanding Token Costs in AI - 09:03 The Importance of Context in AI - 14:37 Exploring the Context Layer and Retrieval Strategies - 22:31 Model Selection and Cost Management in AI - 29:11 Key Takeaways for AI Leaders For more enterprise AI, Agentic RAG, data governance, and enterprise knowledge layer insights, follow Progress Software across its official channels: - Website: Progress Software - YouTube: @ProgressSW - LinkedIn: Progress Software - X: @ProgressSW For more information on enterprise tech analyst-led insights, please visit em360tech.com - EM360Tech YouTube: @enterprisemanagement360 - EM360Tech LinkedIn: @EM360Tech - EM360Tech X: @EM360Tech #EnterpriseAI #ContextEngineering #AgenticRAG #AIROI #TokenCosts #AIStrategy #DataManagement #DonTPanicItsJustData #ProgressSoftware #Nuclia