Part 3 · Deep Learning & AI

3.6 Foundation models and generative AI

LLMs, VLMs and end-to-end driving models

10-minute readPrerequisite: 3.5 Transformers and attention▶ Video (about 5 min)

What you will learn

Explain foundation models (pre-trained on huge amounts of data and reused for many purposes) and the flow from pre-training to fine-tuning
Explain how an LLM writes text by predicting the probability of the next token, and what temperature and hallucination mean
Explain the uses of VLMs and end-to-end driving models in automated driving, and their safety challenges

Build intuition

Until now, the basic pattern has been one model per task — “detect pedestrians”, “classify signs” — each trained on its own labeled data. A foundation model works differently. First, a huge model is trained without labels on text and images from the internet, large volumes of driving logs and so on (pre-training). Afterwards it is reused for many purposes, through extra training on a small amount of data (fine-tuning) or through instructions written in plain language.

Learning without labels is possible because problems can be made from the data itself: “guess how this text continues”, “guess the hidden part”. By guessing continuations across an enormous amount of data, the model builds up something like an understanding of word meanings and knowledge of the world.

The flow of a foundation model. A single model pre-trained on huge amounts of unlabeled data is put to different uses through fine-tuning on a little data or through written instructions.

Foundation models that handle text are called LLMs (large language models); those that handle both images and text are VLMs (vision-language models). Because they can produce new text and images, they are also called generative AI. Inside, most of them are scaled-up versions of the Transformer of 3.5.

Check with math

① An LLM predicts the probability of the next token

Given the sequence of tokens (3.5) so far, an LLM predicts the probability of each possible next token — exactly the conditional probability of 1.2:

$$P(w_t \mid w_1, w_2, \dots, w_{t-1})$$

To write text, it picks one next token according to these probabilities, appends it, predicts the next one, and repeats. Pre-training raises the probability of the token that actually came next in real text — that is, it lowers the cross-entropy (2.4). What the model learns is “is this a plausible continuation?”, not “is this true?” — a point that will matter shortly.

② Temperature: how spread out the answers are

When the candidates’ scores $z_k$ are turned into probabilities, they are divided by the temperature T before the softmax function (2.4):

$$p_k = \frac{e^{z_k / T}}{\sum_{j} e^{z_j / T}}$$

Suppose the candidates to continue “At a red light, the car must” have scores “stop” 4, “halt” 2 and “go” 0.

The probabilities of words continuing “At a red light, the car must”. Raising the temperature spreads out the answers, so the unlikely “go” gets chosen more often; lowering it gives almost the same answer every time. The numbers are an illustrative example.

At temperature 1, “go” has a probability of $1/(e^4 + e^2 + 1) \approx 1.6\%$; at temperature 2 it rises to about 9%. A higher temperature makes the wording richer but also makes an occasional inappropriate answer more likely. A lower temperature gives almost the same answer every time — which does not make it correct.

③ Scale, and how models are reused

Today’s large LLMs have billions to trillions of parameters and are pre-trained on trillions of tokens of text. Increasing model size, data and computation is known to keep lowering the loss in a broadly predictable way (scaling laws), which has driven the push towards ever larger models.

A pre-trained model is adapted to a purpose in ways such as:

Fine-tuning: extra training on a small amount of labeled data for the target task (3.2)
Instructions (prompts): describing the task in writing, such as “list the hazards in this image”
Adjustment from human feedback: tuning the behavior based on answers that people rated as good or bad

Advanced: evaluating generative AI

Answers from generative AI often have no single correct answer, which makes evaluation harder than for image recognition. A common approach is to measure accuracy on a benchmark, a large set of questions with correct answers, but three points deserve care when reading the results.

Accuracy has a range too: 80% on 500 questions has a standard error of $\sqrt{0.8 \times 0.2 / 500} \approx 1.8\%$, so the 95% range is roughly ±3.6 points. A 1–2 point gap on a leaderboard may be chance (2.7, 3.10).
The questions may be in the training data (data contamination): if a benchmark’s questions and answers are published on the internet, they may end up in the training data and be “memorized”. A measured accuracy of 82% on a benchmark where 10% of the questions were memorized means an accuracy of (0.82 − 0.10) / 0.90 ≈ 80% on questions seen for the first time. Check with unpublished or newly written questions.
The answer is not always the same: unless the temperature (②) is 0, the same question can get different answers. Even with 80% accuracy per answer, the probability that 5 answers in a row are all correct is 0.8⁵ ≈ 33%. For safety-related uses, ask several times and evaluate the variation too (3.7).

The quality of text without a fixed correct answer is scored by people, or by another AI (AI-as-judge scoring). AI scoring is fast and cheap, but it is known to be biased: it tends to rate longer answers and answers shown first more highly, and to favour models of its own family. Compare part of it with human scoring and check the gap before relying on it.

Summary

A foundation model is a huge model pre-trained on masses of unlabeled data and reused for many purposes through fine-tuning or instructions. LLMs, VLMs and end-to-end driving models are examples
An LLM writes by predicting the probability of the next token. Raising the temperature spreads out the answers and makes unlikely, inappropriate answers more frequent
What it learns is plausibility, so hallucinations happen. Safe use requires checks by other means, fixed settings and versions, and evaluation situation by situation

Quick quiz

Q1. With scores “stop” 4, “halt” 2 and “go” 0, what is the probability of “go” at temperature 1?

$e^0 / (e^4 + e^2 + e^0) = 1 / (54.6 + 7.4 + 1) \approx 1/63$, about 1.6%.

Q2. With the same scores at temperature 2, what is the probability of “go”? What does this mean?

Dividing the scores by 2 gives 2, 1 and 0, so $1 / (e^2 + e^1 + 1) \approx 1/11.1$, about 9%. Raising the temperature gives unlikely, inappropriate answers more chances to be chosen.

Q3. What is it called when an LLM states something untrue in a confident tone? Why does it happen?

Hallucination. An LLM learns “is this a plausible continuation?”, not “is this true?”, and does not check the facts.

Q4. Given the same image and question, a VLM’s answers differed slightly each time. How should a safety evaluation handle this?

Record and fix settings such as temperature and the model version, test many times under the same conditions, and evaluate including the spread of answers. When the model is updated, repeat the evaluation.