Part 2 · Machine Learning Basics

2.8 Data splits and distribution shift

The gap between training and the real world

10-minute readPrerequisites: 2.6 Overfitting and regularization, 2.7 Evaluation metrics▶ Video (about 5 min)

What you will learn

Explain what causes data leakage, which inflates evaluation results, and how splitting data by groups prevents it
Explain, with examples, the three types of distribution shift: changes in the inputs, in the share of targets, and in the relationship between inputs and answers
Calculate overall performance from per-situation performance and situation mix, and explain how this relates to the ODD and to monitoring after deployment

Build intuition

In 2.6 we learned to measure performance on test data not used for training. Yet a model that did well in testing sometimes works less well than expected on real roads. There are two main reasons. One is a careless split, so the test is not truly “unseen”. The other is a mismatch between the test data and the real driving environment — the character of the data has shifted. This is called distribution shift.

Suppose a pedestrian detector’s recall (2.7) is 96% on clear days but only 62% on rainy nights. If 70% of the test data are clear-day images, overall recall comes out at 92%. Deploy it in a region with more night-time and rain, however, and overall recall drops to 86% — even though its performance in each situation is exactly the same.

The share of each situation in the training and test data (light) and in the region where the car will drive (dark). The detector’s recall differs by situation (numbers below). Even with the same per-situation performance, overall performance falls when difficult situations become more common. The numbers are an illustrative example.

Check with math

① Overall performance = the sum of “per-situation performance × situation share”

With $w_k$ the share of situation k and $R_k$ the recall in that situation, overall recall is:

$$R_{\text{all}} = \sum_{k} w_k\,R_k$$

With the test-data mix: 0.70 × 0.96 + 0.15 × 0.90 + 0.12 × 0.80 + 0.03 × 0.62 ≈ 0.92. With the mix where the car will drive: 0.40 × 0.96 + 0.25 × 0.90 + 0.20 × 0.80 + 0.15 × 0.62 ≈ 0.86. The model has not changed at all; only the mix of situations has.

On top of that, the difficult situations tend to have the least data. Even with 1,000 test images, rainy nights make up 3% — just 30 images. A recall of 62% measured on 30 images carries a 95% confidence interval of roughly ±18 points (1.6).

② How data are split, and “data leakage”

Splitting images one by one at random puts nearly identical images — from the same trip, the same day, the same place — into both the training and the test data. The test then becomes a “question you have already seen”, and the score comes out higher than the model’s real ability. This is data leakage.

How to split	Aim	Example
Split by groups	Keep similar data out of both sets	Assign whole trips, days, regions or vehicles to either training or test
Split by time	See performance on “future” data	Train on an earlier period and test on a later one
Keep the situation mix	Make sure rare situations are not missing from the test	Ensure every split contains a set number of night, rain and child cases

③ Three types of distribution shift

Type	What changes	Example in automated driving	What happens
The inputs change (covariate shift)	The scenery and the state of the sensors	Snowy regions, areas with more night driving, a replacement camera, a dirty windshield	Difficult situations become more common; unfamiliar inputs arrive
The share of targets changes (prior shift)	The mix of answers	A busy downtown with many pedestrians vs quiet suburbs	Precision and similar metrics change (2.7)
The input–answer relationship changes (concept shift)	How the correct answer is determined	New kinds of vehicles such as e-scooters, changes in traffic rules	The learned rules themselves become outdated

④ Noticing the shift

Watch the input statistics: during operation, collect distributions of brightness, object size, weather and so on, and compare them with the training data
Detect unfamiliar inputs: use changes in the model’s confidence, or methods that flag inputs far from the training data (out-of-distribution detection)
Fall back to safety outside the range: when the vehicle is found to be outside the defined range (the ODD), restrict the function or hand over control

Summary

Split data by groups such as trips, days and places so that the test is truly “unseen” (preventing data leakage), and make sure rare situations appear in every split
Distribution shift comes in three types: changes in the inputs, in the share of targets, and in the input–answer relationship. Overall performance is the sum of “per-situation performance × situation share”
Make the range of the data explicit with the ODD, evaluate performance situation by situation, and keep watching for shifts after operation begins

Quick quiz

Q1. Recall is 95% in situation A and 80% in situation B. The test data contain A and B in a 9 : 1 ratio; real operation has them 1 : 1. What is the overall recall in each case?

Test data: 0.9 × 0.95 + 0.1 × 0.80 = 0.935 (93.5%). Real operation: 0.5 × 0.95 + 0.5 × 0.80 = 0.875 (87.5%).

Q2. Images taken on the same day on the same road were split one by one at random into training and test data. What is likely to happen?

Nearly identical images end up in both sets, and the test score comes out higher than the model’s real ability (data leakage). Split by whole trips or days instead.

Q3. E-scooters suddenly become common, and the model’s decisions on whether they are pedestrians or cyclists become inconsistent. Which type of distribution shift is this, mainly?

Mainly a change in the input–answer relationship (concept shift): a new kind of vehicle, for which the rule for the correct classification did not exist at training time. In the sense that unfamiliar inputs are arriving, it is also a change in the inputs.

Q4. Of 1,000 test images, 30 were rainy-night scenes, and recall on them was 62%. How should this figure be treated?

With 30 images, the 95% confidence interval is roughly ±18 points, so the true recall could plausibly be anywhere from 44% to 80%. Collect more rainy-night data and measure again; until then, base decisions on the lower end of the interval.