Question answered: does the model separate low risks from high risks, and is its prediction right for each risk segment?
The deviance page ended with D² values of about 1.3%: individual claims are mostly chance, so any average over individual errors is small. A pricing model is judged on something else: whether the policies it calls expensive really cost more, as a group, than the ones it calls cheap. Grouping averages away the noise. This page covers the standard tools for that:
To draw a lift chart (also called a quantile plot):
The deciles hold equal exposure rather than equal numbers of policies, because pure premium is a rate per policy-year: a decile full of short policies would hold far less information than its policy count suggests.
lift = {m: segment_table(y, {m: p[m]}, w, sort_by=p[m]) for m in models}
fig, axes = plt.subplots(1, 2, figsize=(10, 4.2), sharey=True)
for ax, m in zip(axes, models):
plot_segments(lift[m], [m], ax=ax, xlabel=f"Decile of {m} prediction (1 = lowest)", title=m)
axes[1].set_ylabel("")
fig.tight_layout()
A good lift chart shows three things:
pd.DataFrame({
m: {
"actual, decile 1 (EUR)": t.loc[1, "actual"],
"actual, decile 10 (EUR)": t.loc[10, "actual"],
"lift: decile 10 ÷ decile 1": t.loc[10, "actual"] / t.loc[1, "actual"],
"lift: decile 10 ÷ overall": t.loc[10, "actual"] / mean,
"predicted ÷ actual, decile 10": t.loc[10, m] / t.loc[10, "actual"],
}
for m, t in lift.items()
}).round(2)| GLM | GBM | |
|---|---|---|
| actual, decile 1 (EUR) | 73.68 | 51.89 |
| actual, decile 10 (EUR) | 344.87 | 380.61 |
| lift: decile 10 ÷ decile 1 | 4.68 | 7.34 |
| lift: decile 10 ÷ overall | 2.36 | 2.60 |
| predicted ÷ actual, decile 10 | 1.23 | 0.98 |
The GBM’s top decile costs 7.3 times its bottom decile, against 4.7 times for the GLM. Where the models differ most is the top decile: the GLM predicts 23% more than its riskiest policies actually cost, while the GBM is within 2%. The GLM’s predictions are too extreme at the top, the regression-to-the-mean pattern the calibration page described for probabilities.
Each decile holds 12,871 to 20,188 policies, but only 312 to 1,496 of them have a claim, and claim sizes vary enormously. The 95% intervals are wide, so a dip such as the GBM’s 4th decile is well within noise. Don’t redesign a model to remove a wiggle that the intervals cover.
Two lift charts side by side are hard to compare: both look good, and their deciles contain different policies. The double lift chart compares the two models directly, on the policies where they disagree:
Decile 1 holds the policies the GBM rates much lower than the GLM, decile 10 those it rates much higher. In the middle the models agree.
rebalanced = {m: p[m] * mean / np.average(p[m], weights=w) for m in models}
double = segment_table(y, rebalanced, w, sort_by=rebalanced["GBM"] / rebalanced["GLM"])xlabel = "Decile of GBM ÷ GLM (1 = GBM much lower)"
fig, axes = plt.subplots(1, 2, figsize=(10, 4.2))
plot_segments(double, models, ax=axes[0], xlabel=xlabel, legend_loc="upper center")
plot_segments(double, models, ax=axes[1], ratio=True, xlabel=xlabel)
fig.tight_layout()
The left panel answers the question ‘who is right where they disagree?’ In decile 1, the GLM predicts 349 EUR, the GBM 182 EUR, and the actual average is 255 EUR. In decile 10, the GLM predicts 104, the GBM 226, and the actual average is 159. Neither model wins: in both tails the truth lies between them.
The right panel shows the same comparison as actual ÷ predicted. The better model is the one whose curve stays flat at 1. Insurers read this as a loss ratio: if premiums were set by the GLM, actual ÷ GLM is the loss ratio each segment would produce.
Each model knows something the other doesn’t, and each overreacts where it disagrees with the other. A curve that crosses 1 in the opposite direction would mean the model is worse than useless where the two differ: its deviations from the other model point the wrong way.
When both models are partly right, a combination usually beats either. A simple blend is the geometric mean of the two predictions (an average on the log scale, which suits multiplicative pricing models):
blend = np.sqrt(rebalanced["GLM"] * rebalanced["GBM"])
blend *= mean / np.average(blend, weights=w)double_blend = segment_table(y, {"Blend": blend, "GBM": rebalanced["GBM"]}, w, sort_by=blend / rebalanced["GBM"])
ax = plot_segments(double_blend, ["Blend", "GBM"], ratio=True, xlabel="Decile of Blend ÷ GBM (1 = Blend much lower)")
ax.figure.set_size_inches(6, 4)
scores = {**{m: p[m] for m in models}, "Blend": blend}
pd.DataFrame({
m: {"Gini": gini_index(y, s, w), "D²": deviance_metrics(y, s, p["Null"], w, TWEEDIE_POWER)["D²"]}
for m, s in scores.items()
}).round(4)| GLM | GBM | Blend | |
|---|---|---|---|
| Gini | 0.2928 | 0.2944 | 0.3078 |
| D² | 0.0127 | 0.0134 | 0.0147 |
Against the blend, the GBM’s actual ÷ predicted swings away from 1 at both ends, while the blend’s stays near 1 within the noise: in this pairing, the blend wins. The blend also has the highest Gini and D². Two caveats: the comparison is on the test set the double lift chart was read from, so a blend chosen this way should be confirmed on fresh data; and whether its improvement is larger than sampling noise is a question for the model-comparison page.
The lift chart shows separation decile by decile. The Lorenz curve shows it as one continuous curve:
A model that can’t separate risks (the Null model) gives the diagonal: the first 30% of exposure carries 30% of the claim cost. A model that separates well puts little claim cost into its low-risk policies, so its curve sags below the diagonal. The oracle sorts by the actual outcome, the best ranking possible.
ax = plot_lorenz(y, {m: p[m] for m in models}, w)
ax.figure.set_size_inches(5.5, 5.5)
The GBM’s lowest-risk half of the exposure carries only 32% of the claim cost. The Gini index condenses the curve into one number: twice the area between the diagonal and the curve,
\[ \text{Gini} = 1 - 2 \int_0^1 L(x)\,dx, \]
where \(L(x)\) is the share of claim cost in the lowest-predicted share \(x\) of the exposure. It is 0 for the Null model and grows the better the model ranks the policies. The two models score 0.293 (GLM) and 0.294 (GBM), a relative difference of 0.6%. On D² the GBM was 6% ahead, and its top-to-bottom lift was much larger. The Gini weighs the whole ranking, while the lift ratio looks only at the two extreme deciles, each resting on a few hundred claims. A single decile ratio is a much noisier summary than the Gini.
The oracle’s Gini of 0.984 is not a realistic target: 95% of the exposure had no claim at all, and only hindsight can put those policies first. Some authors report a normalized Gini (Gini ÷ oracle Gini), but the oracle mostly reflects how many policies happened to have no claims, so it adds little here.
The ROC page defined Gini = 2·AUC − 1 for a binary target. For a 0/1 outcome with equal weights, the Lorenz Gini equals \((1 - \pi)(2 \cdot \text{AUC} - 1)\), where \(\pi\) is the share of positives (the tests in tests/ check this). The factor \(1 - \pi\) is the oracle’s Gini: dividing by it gives the normalized Gini, which equals 2·AUC − 1. Both measure how well the model ranks. The Lorenz version extends the idea to outcomes that aren’t 0/1, such as claim cost, and to weighted observations.
The Gini depends only on the ranking. Any transformation that keeps the order of the predictions leaves it unchanged, however wrong the predictions become:
squared = p["GBM"] ** 2 * np.average(p["GBM"], weights=w) / np.average(p["GBM"] ** 2, weights=w)
variants = {"GBM": p["GBM"], "2 × GBM": 2 * p["GBM"], "GBM squared": squared}
pd.DataFrame({
name: {
"Gini": gini_index(y, s, w),
"D²": deviance_metrics(y, s, p["Null"], w, TWEEDIE_POWER)["D²"],
"bias (EUR)": regression_metrics(y, s, w)["bias"],
}
for name, s in variants.items()
}).round(4)| GBM | 2 × GBM | GBM squared | |
|---|---|---|---|
| Gini | 0.2944 | 0.2944 | 0.2944 |
| D² | 0.0134 | -0.0050 | -0.0186 |
| bias (EUR) | -3.6890 | 138.9580 | -3.6890 |
Doubling every prediction, or squaring them, keeps the Gini exactly the same. D² drops below 0 (worse than predicting the average for everyone), and a lift chart would show the gap at once. So:
| Tool | Answers | Watch out for |
|---|---|---|
| Lift chart | Does cost rise with the prediction, and does the prediction match per segment? | noisy deciles: read the intervals |
| Double lift chart | Where two models disagree, which is right? | rebalance both models to the same level first |
| Lorenz curve / Gini | How well does the model rank risks, in one number? | blind to calibration and level; not comparable across portfolios |
| Deviance / D² | How good are the individual predictions, ranking and level together? | small values are normal; hard to interpret alone |
The tools complement each other: the Gini and the lift spread measure ranking, the agreement in the lift chart measures calibration by segment, the double lift chart compares two candidate models, and the deviance checks both ranking and level at once.