Running prompts at CI does not make your GenAI app enterprise ready

Speaker: Jakob Frick, CTO, Radiant AI

In this talk, Jakob Frick, CTO of Radiant AI, discusses the challenges of making Generative AI applications enterprise-ready. He highlights the misconception that simply running prompts in CI is sufficient and delves into the inherent stochasticity of LLMs, the need for grounding in reality, and the shortcomings of applying traditional software development and ML practices directly to GenAI. Frick proposes a new approach focused on capturing interactions, deriving key metrics, and building specific evaluations to continuously monitor and improve LLM applications, enabling them to scale reliably and adapt to evolving models and use cases.