Large Language Model Training and Serving at LinkedIn

Dre Olgiati, Distinguished Engineer at LinkedIn, discusses the fundamental challenges and solutions faced by the LinkedIn team in building innovative products based on Large Language Models (LLMs) and agents. The talk delves into the technology stack, particularly focusing on "lier kernels" developed internally to improve the efficiency of LLM training by reducing memory consumption and transfers. It covers techniques for optimizing training speed and cost, and touches upon the broader applications of LLMs within LinkedIn's product lines.