AI Engineering with Chip Huyen

Before we start: a rare giveaway! To celebrate the podcast’s YouTube channel crossing 100K subscribers, I’m doing a giveaway of 100 books (10 of them signed and shipped) and 100 newsletter subscriptions. To enter, just subscribe to the podcast on the platform you use, leave a star review, and fill out this form, indicating you’d like to participate. The giveaway closes in 72 hours. More details here.

Stream the Latest Episode

Available now on YouTube, Apple and Spotify. See the episode transcript at the top of this page, and a summary at the bottom.

Brought to You By

• Swarmia — The engineering intelligence platform for modern software organizations.

• Graphite — The AI developer productivity platform.

• Vanta — Automate compliance and simplify security with Vanta.

—

In This Episode

On today’s episode of The Pragmatic Engineer, I’m joined by Chip Huyen, a computer scientist, author of the freshly published O’Reilly book AI Engineering, and an expert in applied machine learning. Chip has worked as a researcher at Netflix, was a core developer at NVIDIA (building NeMo, NVIDIA’s GenAI framework), and co-founded Claypot AI. She also taught Machine Learning at Stanford University.

In this conversation, we dive into the evolving field of AI Engineering and explore key insights from Chip’s book, including:

• How AI Engineering differs from Machine Learning Engineering

• Why fine-tuning is usually not a tactic you’ll want (or need) to use

• The spectrum of solutions to customer support problems – some not even involving AI!

• The challenges of LLM evals (evaluations)

• Why project-based learning is valuable—but even better when paired with structured learning

• Exciting potential use cases for AI in education and entertainment

• And more!

Takeaways

My biggest takeaways from this episode:

1. “AI engineering” feels closer to software engineering than to ML engineering. The term itself is very new, and “AI engineering” evolved from ML engineering. A big difference is that thanks to LLMs being easy to use (both via APIs, and locally) “AI engineering” is much more about building a product first – and later on, getting around to tweaking the model itself. ML engineering was the other way around: spend a lot of time building a model, and then build an application on it.

2. To get good at AI engineering, focus on the basics. Understand what an LLM is (and how it works), how to evaluate them, how to use RAG, what finetuning is, and how to optimize inference. All of these techniques are foundational, and will remain important in a few years’ time as well. Chip’s book AI Engineering covers all these topics.

3. “The more we want to not miss out on things, the more things we will miss.” When I asked Chip about what she thinks about the fear of missing out (FOMO) across AI: she said it’s ironic that when you are worried about missing out, and keep up with everything new – you stay shallow! Chip doesn’t read news: she chooses to focus on deep research or learning instead. If there’s something that is important today: it will be important next week, after she’s done finishing learning whatever she was focused on.

4. Solving the problem is more important than using the latest AI tools. Amusingly, a lot of teams miss this part: and they build overcompliacated AI solutions that do practically nothing for the business.

Before the interview, I’ve started to go through Chip’s newly published book: AI Engineering. It feels to me this book will become a desk reference, and I recommend it.