Why a router network reduces inter-class confusion in GI imaging

Look at the confusion matrix of any flat GI endoscopy classifier and the mistakes aren’t spread evenly — they pile up in small blocks of visually similar classes. The model almost never confuses an anatomical landmark with a polyp; it confuses kinds of similar findings with each other. That structure is an invitation: factor the prediction,

where \(k\) indexes a coarse group. If \(P(k \mid x)\) is nearly free — and empirically it is — then all the hard work lives in \(P(y \mid x, k)\), and a specialist trained only within group \(k\) faces a dramatically easier decision boundary than one trained over all classes at once.

In code the inference path is almost embarrassingly simple:

group_logits = router(x)              # cheap, high-accuracy decision
k = group_logits.argmax(dim=-1)
fine_logits = specialists[k](x)       # capacity spent where confusion lives

Outline of the full post:

1. The block structure of GI confusion matrices — with a real figure from my dissertation experiments.
2. The mixture decomposition above, and what hard routing throws away (when \(\operatorname{argmax}\) is safe, and when it isn’t).
3. Why specialists beat one big model at matched parameter count — an experiment, not a slogan.
4. Failure mode: routing errors are unrecoverable; measuring the router’s ceiling on end-to-end accuracy.
5. What I’d try next: top-2 soft routing, specialist abstention, calibration.