Mastering LLM Fine-Tuning

Maxime Labonne, Senior Staff Machine Learning Scientist at Liquid AI, delves into the intricacies of fine-tuning Large Language Models (LLMs). The talk covers the entire LLM training lifecycle, from base models to supervised fine-tuning and preference alignment. Labonne discusses best practices for creating high-quality data generation pipelines, techniques for fine-tuning using popular libraries, the mechanics of model merging, and effective methods for evaluating LLMs. He emphasizes that fine-tuning should be considered after exploring prompt engineering and RAG, and highlights its importance for enterprises seeking control and customization of AI models. The presentation also touches upon various fine-tuning approaches, including full fine-tuning, parameter-efficient techniques like LoRA, and preference alignment methods such as DPO. Recommendations for libraries like Hugging Face's `transformers`, `axolotl`, and `unsloth` are provided, along with insights into crucial training parameters and monitoring metrics.