Course Content
ML Design Basics
This topics covers the fundamentals of Machine Learning System Design in real world.
0/11
Requirements and Business objectives
Data Basics
Training Data
Feature Engineering
Feature Engineering II
Model Development and offline evaluation
Model deployment
Monitoring
Test in production
Infrastructure and tooling
User Experience
Interview Expectations
This topic talks about the expectations and focus areas in the interview and shares some insights into what kind of signals the interviewer captures during the interview.
0/7
Introduction
Problem Navigation
Training Data
Feature Engineering
Model Development
Model Evaluation
Model Deployment
Questions
This topics goes through some example questions and how to approach them.
0/5
Restaurant recommender
YouTube recommendations
ChatGPT
Bad advertisement content
Facebook news feed
Machine Learning System Design
About Lesson

ML systems behave differently from traditional software systems.

  • ML systems are probabilistic instead of deterministic.
  • Due to this probabilistic nature, ML systems’ predictions are mostly correct and the hard part is we usually don’t know for what inputs the system will be correct.
  • ML systems can also be large and might take an unexpectedly long time to produce a prediction.

Three challenges that ML systems pose to good user experience:

  • Ensuring user experience consistency
    • For tasks that want to leverage ML to improve users’ experience, the inconsistency in ML predictions can be a hindrance.
    • Consistency–accuracy trade-off: Recommendations deemed most accurate by the system might not be the recommendations that can provide user consistency.
  • Combatting “mostly correct” predictions
    • Mostly correct predictions won’t be very useful if users don’t know how to or can’t correct the responses.
    • An approach is to show users multiple resulting predictions for the same input to increase the chance of at least one of them being correct.
  • Smooth failing 
    • Normally fast models might still take time with certain queries. This can happen especially with models that deal with sequential data like language models or time-series models—e.g., the model takes longer to process long series than shorter series. What should we do with the queries where models take too long to respond?
    • You may use a backup system that is less optimal than the main system but is guaranteed to generate predictions quickly. These systems can be heuristics or simple models. They can even be cached precomputed predictions.
    • Instead of having this simple rule, you may have another model to predict how long it’ll take the main model to generate predictions for a given query and route that prediction to either the main model or the backup model accordingly. Of course, this added model might also add extra inference latency to your system.
    • Speed–accuracy trade-off: A model might have worse performance than another model but can do inference much faster.

Responsible AI

  • Responsible AI is the practice of designing, developing and deploying AI systems with good intention and sufficient awareness to empower users, to engender trust and to ensure fair and positive impact to the society. It consists of areas like fairness, privacy, transparency and accountability.
  • It may be useful to discuss this in certain use-cases like designing an automated grader, chatbot, self-driving car, etc.
Previous
Next
Scroll to Top