Covariance matrices, the noise floor of PCA, and eigenvalues that fill a disc
The semicircle is what you get when the matrix is symmetric and its entries are independent. Both halves of that sentence are assumptions about the system being modelled, and changing either one changes the answer — into a different curve, or off the real line entirely.
Two changes are worth making. Build the matrix out of data instead of positing it directly, as a covariance matrix, and the semicircle becomes the Marchenko–Pastur law — which turns out to be the single most practically useful thing in this module. Drop the symmetry, and the eigenvalues leave the real line and fill a disc.
Measure p quantities n times and compute how each pair covaries. If the quantities are genuinely independent, the truth you are trying to estimate is the identity matrix: every population eigenvalue exactly 1, no structure at all. What you actually get is a whole spread of eigenvalues, and the spread does not shrink to nothing unless observations outpace variables.
Every matrix here is built from data with no correlation structure whatsoever: 80 independent standard normal variables, observed 160 times, with the sample covariance formed and diagonalised in full. The population covariance is the identity, so all 80 population eigenvalues are exactly 1. The spread you are looking at is entirely an artefact of finite data — and it does not shrink unless the number of observations grows faster than the number of variables. At λ = 1 the lower edge reaches zero and the density diverges there, which is the sample covariance matrix becoming singular.
Try it: Push the aspect ratio λ = p / n up toward 1 and watch the lower edge (1 − √λ)² slide down to zero while the upper edge (1 + √λ)² climbs to 4. At λ = 1 — as many variables as observations — the smallest eigenvalues pile up against zero, and the sample covariance matrix stops being invertible. The mean eigenvalue stays pinned at 1 throughout, because the truth has not changed; only your ability to see it has.
Here is where that becomes a rule you can use. Principal component analysis hands you eigenvalues in decreasing order and invites you to keep the big ones. Marchenko–Pastur tells you exactly how big pure noise gets: (1 + √λ)². An eigenvalue below that line is not evidence of anything.
The sharper and less comfortable fact is the converse. Add a genuine signal — a real direction of extra variance in the population — and it does not gradually emerge from the bulk as it grows. It stays entirely invisible until it crosses a threshold, and only then does an eigenvalue detach.
There is no signal here at all, and the largest eigenvalue still sits near 2.91.
Sample covariance matrices of 80 variables from 160 observations, diagonalised in full, with population covariance equal to the identity plus a rank-one bump of size h. Verified offline at λ = 0.5, where the threshold is √0.5 ≈ 0.707: signal strengths of 0.2, 0.4 and 0.6 all produced mean top eigenvalues of 2.88, 2.86 and 2.87, sitting on the noise edge of 2.914 and indistinguishable from h = 0’s 2.83. At h = 1.5 the measured top was 3.325 against a predicted 3.333, and at h = 3 it was 4.700 against 4.667.
Try it: Leave λ at 0.5 and walk the signal strength up from zero. Nothing happens — not a little, nothing — until h passes √λ ≈ 0.707. A population spike of 1.6, which is a genuine 60% excess of variance in one direction, produced a mean top eigenvalue of 2.87 in the runs made for this page, against 2.83 for no signal at all and a noise edge of 2.914. Then keep going: at h = 1.5 the measured top was 3.325 against a predicted 3.333, and at h = 3 it was 4.700 against 4.667.
Two things are worth taking away from that. The first is that a component below the edge tells you nothing, however suggestive it looks. The second is that when a component finally does appear, the sample overstates it: the detached eigenvalue sits at (1 + h)(1 + λ/h), which is strictly larger than the population value 1 + h. At h = 1.5 and λ = 0.5 the population spike is 2.5 and the sample reports about 3.33 — a third too large, from nothing but the aspect ratio.
Symmetry is what forces eigenvalues to be real. Fill all n² entries independently — so that the influence of i on j has nothing to do with the influence of j on i — and the eigenvalues become complex numbers and spread out to fill the unit disc, uniformly.
These are genuine eigenvalues of genuine matrices. Getting them required a real eigensolver for non-symmetric matrices, which is implemented in this module and checked against the identity Σλk = trace(Ak) and against known spectra — the alternative, scattering points drawn from the limiting law, would have been a drawing of the conclusion rather than evidence for it.
The stripe on the real axis is real, not a rendering artefact. A real matrix has a real characteristic polynomial, so its complex eigenvalues come in conjugate pairs — and a few eigenvalues land exactly on the axis instead of pairing up. The count grows like the square root of n, not like n: measured here, an average of 8.5 real eigenvalues per matrix at n = 120 and 12.4 at n = 240, against √(2n/π) = 8.74 and 12.36. So the stripe holds a shrinking fraction of the spectrum — 7.1% at n = 120, 5.2% at n = 240 — and fades relative to the disc as the matrices grow. Fill the matrix with complex entries instead and it disappears entirely.
120 × 120 matrices with all 14,400 entries drawn independently from N(0, 1/n), reduced to Hessenberg form by Householder reflections and diagonalised by the Francis double-shift QR algorithm. Measured over 960 eigenvalues from eight matrices: 96.9% landed inside the unit disc, the median radius was 0.7032 against the uniform-disc value of 0.7071, the lower quartile was 0.5134 against 0.5000, and the largest radius seen was 1.065. The uniform-disc prediction is that the fraction within radius r is exactly r², which is the straight line the right-hand chart is testing.
Try it: Collect a few matrices, then turn on the uniform-disc test. Uniformity on a disc means the fraction of points within radius r is exactly r², so the measured curve should lie on the dashed parabola — which is a much sharper test than looking at the scatter and agreeing that it seems evenly spread. Over the 960 eigenvalues measured for this page, 96.9% sat inside the unit circle, the median radius was 0.7032 against a predicted 0.7071, and the lower quartile 0.5134 against 0.5000.
Notice that the eigenvalues reach out to radius about 1 rather than about 2. Nothing has been rescaled: the same entries, arranged symmetrically, would give a spectrum on [−2, 2]. Imposing symmetry doubles the reach, because it correlates each entry with its partner across the diagonal, and that correlation is worth a factor of two at the edge.
The choice between these is made by the structure of the problem, not by preference. The question to ask is what kind of object the matrix is supposed to be.
Energy levels. Any system whose evolution is generated by a Hermitian operator you cannot write down — a nucleus, a chaotic billiard, a complicated quantum system.
The choice is made by the symmetry of the problem, not by taste. If the quantity being modelled is an energy, the operator is Hermitian and the eigenvalues are real; if it is a covariance, the matrix is a product X Xᵀ and the eigenvalues are non-negative; if the interaction from i to j has nothing to do with the interaction from j to i, there is no symmetry to impose and the eigenvalues leave the real line altogether.