I have seen a number of candidates fail here. A lot of times candidates either skip online evaluation or mix-up offline and online metrics. Some candidates start listing all the offline metrics they can think of, without considering the problem at hand.

Here are the list of important topics to cover in this section.

• Offline and online evaluation to determine model performance. 
• Propose offline and online metrics and justify the choices. 
• Details of metric computation.

Successful candidates often start with a set of offline and online metrics relevant to the problem, define them and do some trade-off analysis, before recommending their metric of choice. Online testing methods like A/B testing should be discussed along with statistical significance.