Building Composite LLM Systems

Urmish Thakker, Director of Machine Learning at SambaNova Systems, discusses how open-source LLMs have enabled the development of enterprise applications. The talk explores the rise of specialized LLMs that outperform larger models in specific domains and addresses the challenge of providing a single-endpoint user experience similar to proprietary models. Thakker presents research on building composite LLM systems using open-source checkpoints that can effectively map user requests to specialized models or groups of models, demonstrating this capability with a composite model based on the Mistral 7b model. This composite model is shown to outperform several larger models at a significantly lower effective inference cost.