Four Hidden Nuggets of Wisdom in Andrew Ng’s ML Engineering for Production (MLOps) Specialization Course on Coursera
Scratch beneath the surface of technical know-how and become a wiser AI engineer.
Like a Disney movie
Andrew Ng’s MOOCs are like Disney movies. They are designed to delight the youngest but contain also philosophy only the older and wiser can appreciate. As a kid, you marveled at The Lion King. What you saw then are the wonderful funny or scary characters, the unforgettable music, and the fascinating storyline. Later in life, you discover the movie's much more profound philosophical dimension. You recognize yourself in the struggles of Simba in the different phases of his life. You relate to the drama of losing his father and meditate on the fragility of life and the heavy burden of responsibility.
In the same way, I believe Andrew’s courses are designed with many intertwined levels of interpretation addressing different audiences, from the junior engineer to the VP. Don’t be fooled by the smooth, enjoyable learning experience of the MOOC.
This course is not only here to give you technical knowledge. It is here to change your mindset to make you a wiser AI engineer.
Andrew Ng is a soft-spoken teacher, but if you pay close attention to what he says, you realize he is dropping -silent- bombs that will shake the whole community and transform the way we build AI everywhere in the world in the long run.
I have taken the course and distilled the most valuable insights I found.
My goal is not to bring you a comprehensive summary of the class, but rather to reveal the learning points that were the most surprising and eye-opening to me.
Like re-watching The Lion King as an adult, I try to decipher the philosophical nuggets of wisdom and discuss them with you.
The article will deep dive into the first four lessons:
- Your model is only 5% of your system in production.
- You don’t know what you don’t know until you have deployed.
- The whole industry needs a paradigm shift from model-centric AI to data-centric AI.
- Don’t miss the blind spot in your accuracy metrics.
Nugget #1 — Your model is only 5% of your system in production.
While modeling is probably a top-of-mind concern for most data scientists and probably one of the most exciting intellectual challenges, it will represent only about 5% of your systems’ code when it reaches full production scale.
The rest, 95% of the code, is everything around the model ensuring it is fed with clean, relevant, reliable data, and managing its performance, as illustrated below. While central, the ML code, represented in the black box is a tiny portion of the total engineering work.
After a successful POC, when you start deploying a proven algorithm, you are only halfway through the journey, and many uncertainties remain. Consequently, you should consider the holistic view from day one. You should ask these kinds of questions early in the project: “Will the API be part of a critical system that requires very high standards of service?”, “What could go wrong with the input data a year from now?”, “How will I know if the model performance drops?”.
Don’t get me wrong though. There are enough problems to solve today. If you try to solve every problem that will arise in the next five years at once, you will get paralyzed by the sheer volume of things that could go wrong. Rather, I encourage building a comprehensive checklist but only focusing on the most structuring engineering issues early to ensure a smooth project onwards. The rest of the risk will be handled along the way, which brings me to the next point about progressive deployment.
Nugget #2 — You don’t know what you don’t know until you have deployed.
It’s wrong to think of deployment as a one-time endeavor. It’s only when you deploy at scale that you get exposed to all real-life issues. You can anticipate and brainstorm problems, and you should do it. But you cannot imagine all possible scenarios. Neither can you design your system to resist every catastrophic scenario, that would be too costly. When building a nuclear power plant, you could design the roof to resist an aircraft attack. But you can’t afford a design that resists a meteorite crash.
So deploy early, with a gradual ramp-up adapted to the business risk associated. This will surface all issues you need to fix and expose you to valuable business feedback. Expect data drift and concept drift to happen sooner or later, detect them, and correct them. For example, when COVID-19 hit the world, during the first lockdown, everybody moved to online shopping. The fraud detection algorithms of credit cards were exposed overnight to a complete shift in payment behaviors and failed massively. You could not have predicted COVID-19, but if you deploy small and early you get more occasions to learn and build resilience to rare events.
Think of deployment as an iterative process, with a strong test & learn dimension.
Nugget #3 — The whole industry needs a paradigm shift from model-centric AI to data-centric AI.
This is not the first time Andrew advocates this mindset shift to data-centric AI. He built a resource hub, shot videos, and even held a Kaggle-like competition around the concept.
In his words,
Data-centric AI is the discipline of systematically engineering the data used to build an AI system.
As explained in this MIT Sloan article “Why it’s time for ‘data-centric artificial intelligence”:
focusing on the quality of data fueling AI systems will help unlock its full power.
Academia historically has been focusing on developing new model architectures, working on fixed benchmark datasets to keep track of the progress in modeling. This approach was mimicked by all data science practitioners in the industry. However, Andrew argues, in most industrial applications we would get much better use of our time and energy if we picked just a good enough model, and focused all efforts on improving the data fed to this fixed model.
In my experience in the retail industry, it is true that data scientists are too model-centric and that working on data would yield a higher return on investment. But this opinion is unpopular, with most data scientists passionate about billion parameters deep neural networks. The community suffers from the “shiny new toy” syndrome and Andrew understands this very well. I admire him for championing those unsexy yet necessary best practices.
Nugget #4 — Don’t miss the blind spot in your accuracy metrics.
Having only one performance metric for your AI system is more convenient. When training your algorithm, it is easier to have only one target to aim at. Unfortunately, the reality is more complex. Andrew once built a speech recognition system based on recordings of adult people speaking. But when deployed, users were sometimes teenagers or younger children and the algorithm was doing poorly. The lesson is that you need to measure accuracy not only on the global dataset but monitor performance on key slices of the dataset, those that have high business value.
To take another example, if you are forecasting demand, you want to be particularly accurate on the most expensive products because they require a lot of working capital. You also want to be accurate on bulky items because they cost a lot to transport and store. And you want to be accurate on perishable items because you want to avoid waste.
Wrap-up
The MLOps specialization is a comprehensive introduction to ML engineering for production. Pay careful attention and you will find nuggets of wisdom hidden between the frameworks and concepts of the course. Today I revealed the first ones I found:
- Your model is only 5% of your system in production.
- You don’t know what you don’t know until you have deployed.
- The whole industry needs a paradigm shift from model-centric AI to data-centric AI.
- Don’t miss the blind spot in your accuracy metrics.
I hope you will appreciate those lessons as much as I did. This is what I love about Andrew Ng’s classes: You come for the tips and tricks, but you stay for the philosophy.
Please share your own nuggets of wisdom in the comment section. Let me know if you like me to publish four other nuggets of wisdom. Here are a few ideas as a teaser: “The world is made of four types of data scientists. Ask the right ones for advice.” and “The most underrated skill of a data scientist”.
