Fast and Reproducible: Taming AI/ML Dependencies
December 09, 2024
30 min
Free
mlops
ai
ml
dependency-management
python
conda
pip
docker
reproducibility
metaflow
data-science
ml-stack
Description
This talk addresses the critical but often underrated aspect of AI/ML systems: dependency management. Savin Goyal explores common pitfalls that lead to instability in production deployments and slow down development speed. He shares learnings from working with organizations like Netflix and Amazon, demonstrating how to create fast, stable, and reproducible environments for experimentation. The discussion covers scaling dependencies in cloud environments, handling non-Pythonic dependencies, auditing the software supply chain, and accelerating prototyping with fast container image building.