https://josephausterweil.github.io/probintro/complexity/index.htmlThe Winter Model Fair. Chibany judges the student modeling contest, and every entry poses the same dilemma: the flexible models fit everything — including the noise — while the simple ones miss the signal. This Part is about honest model complexity: the bias–variance tradeoff (and its modern double-descent twist), models that grow with the data (Dirichlet process mixtures), and distributions over entire functions (Gaussian processes) — whose infinite-width story is the on-ramp to Part VIII.Hugoen-usThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/complexity/bias-variance/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/complexity/bias-variance/index.htmlHow Complex Should the Model Be? In Chapter 5 we cracked Chibany’s bento mystery with a Gaussian Mixture Model — but only after we fixed the number of components $K$ by hand. Choosing $K=2$ was not a detail; it was a decision about how complex the model is allowed to be. Pick $K$ too small and the model cannot capture the structure that is really there. Pick it too large and it starts inventing structure out of noise.https://josephausterweil.github.io/probintro/complexity/dpmm/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/complexity/dpmm/index.htmlThe Problem with Fixed K In Chapter 5, we solved Chibany’s bento mystery using a Gaussian Mixture Model (GMM) with K=2 components. But we had to specify K in advance and use BIC to validate our choice. What if: We don’t know how many types exist? The number of types changes over time? We want the model to discover the number of clusters automatically? The honest difficulty is that “how should we cluster these points?” has an astronomical number of answers. The number of ways to partition $n$ points into groups is the Bell number $B_n$, and it explodes far faster than the $2^n$ subsets of a set — by $n = 15$ there are already over a billion possible clusterings.https://josephausterweil.github.io/probintro/complexity/gaussian-processes/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/complexity/gaussian-processes/index.htmlFrom Partitions to Functions The Dirichlet Process Mixture Model of the last chapter answered the question “how many clusters are there?” without ever fixing the number. It did this by putting a prior over an infinite object — a partition of the data into groups — and letting the data decide how many groups light up. That is the whole spirit of Bayesian nonparametrics (BNP): don’t fix the size of the model; put a prior over an infinite-dimensional space and let the posterior keep only as much structure as the data support.