Nebuly's LLM User Feedback Loop

User feedback is crucial for improving LLM-powered products, but traditional methods like explicit feedback (thumbs up/down) are insufficient, with less than 1% of users providing it. This talk, delivered by Zunair Waseem from Nebuly AI, explores how Nebuly helps companies build their own LLM User Feedback Loop by leveraging implicit feedback. Implicit feedback, derived from user prompts and sentiment, is available over 30% of the time and provides deeper insights into user experience. The presentation demonstrates how Nebuly automates the extraction of user behavior, sentiment, and other valuable data to help fine-tune LLM models and enhance user satisfaction. It covers identifying topics, intents, actions, risky behaviors, and sentiment from user interactions, and showcases features like A/B testing and cohort analysis for further optimization.