A trough that proves the forest survived

Absorption is normally read as a measurement of what is there. A line’s depth gives a column density, a curve of growth turns the depth into a number of atoms, and the argument runs from what was removed.

Reionisation is measured the other way round. The absorption available is so enormous that it saturates completely at a trace abundance, so a spectrum showing any transmitted light at all is a statement that almost nothing is there. The evidence is the flux that survived, and the argument runs from a detection.

The optical depth #

The calculation is short and its result is the whole essay.

Neutral hydrogen absorbs at 1215.67 Å with a cross-section whose integral over frequency is fixed by the transition’s oscillator strength. In an expanding universe a photon redshifts continuously, so it passes through resonance with the transition at one place along its path, and the resulting optical depth is

$$\tau_{\rm GP}(z) = \frac{3\lambda_\alpha^3\Lambda_\alpha\, n_{\rm HI}(z)}{8\pi H(z)}.$$

Every quantity in it is known. The wavelength and the decay rate are atomic constants. The Hubble rate at redshift six follows from the measured cosmological parameters. And the hydrogen density follows from the baryon density measured two entirely different ways — from the primordial abundances and from the microwave background’s acoustic peaks — times $(1+z)^3$.

Putting the numbers in gives $\tau \approx 6\times10^5$ at $z = 6.3$ for a fully neutral medium.

Six hundred thousand. The transmitted fraction is $e^{-600000}$, which is not a small number but a number with no meaning: it is zero to any conceivable precision. And because $\tau$ is linear in the neutral fraction, a medium with only one part in $10^4$ of its hydrogen neutral still has $\tau = 60$, and transmits $e^{-60} = 10^{-26}$.

The intergalactic medium is opaque long before it is neutral. That is the Gunn–Peterson effect, published in 1965 as a prediction — and immediately as a puzzle, because the quasar spectra then available showed no such trough.

What the absence meant #

Gunn and Peterson looked at 3C 9, at $z = 2.01$, and found flux blueward of its Lyman α emission. Their conclusion was the correct one and it was a strong statement to make from a non-detection: the intergalactic medium at $z = 2$ is ionised to at least a part in $10^5$.

That was surprising. Recombination happened at $z \approx 1100$, when the universe cooled enough for protons and electrons to combine — the event that produced the microwave background — and the gas has been expanding and cooling ever since. Nothing in the expansion re-ionises it. Something must have done so afterwards, and the something had to be luminous.

The forest #

The spectrum blueward of a quasar’s Lyman α emission line is not empty. It is a dense thicket of narrow absorption lines, each from a discrete concentration of neutral hydrogen at a lower redshift than the quasar and therefore at a shorter rest wavelength in the observed frame.

The modern understanding is that those are not clouds in any classical sense. They are the mildly overdense filaments of the cosmic web, photoionised by the background of ultraviolet light from galaxies and quasars, with a residual neutral fraction of $10^{-5}$ set by the balance between photoionisation and recombination. The “lines” are peaks in a continuous field of fluctuating optical depth.

That last point is worth its own paragraph, because it governs how the measurement degrades.

The observable is the mean transmitted flux, and $\langle e^{-\tau}\rangle \ge e^{-\langle\tau\rangle}$ for any fluctuating field — Jensen’s inequality. Most of what gets through comes through the voids, where the density and therefore the optical depth are lowest. So the mean transmission is set by a small fraction of the pixels, and it becomes progressively less sensitive to the mean neutral fraction as that fraction rises.

That is why a trough is a lower limit and not a measurement. Once the transmission is set by a handful of void pixels, multiplying the neutral fraction by ten barely changes it. A spectrum showing no flux says the neutral fraction exceeds about $10^{-4}$; it does not distinguish $10^{-4}$ from $10^{-1}$ from unity.

The transition #

Troughs began to appear in quasar spectra above $z \approx 6$, first in the Sloan survey’s high-redshift quasars from 2001. The effective optical depth of the forest rises smoothly from $z = 2$ to $z = 5$ and then steepens sharply, and by $z = 6$ complete dark gaps stretching across tens of megaparsecs are common.

The natural reading is that reionisation was completing around then. The careful reading is more restrained, and the restraint is the interesting part. Since the trough saturates, what is measured above $z = 6$ is a lower bound on the neutral fraction of order $10^{-4}$ — which is consistent with a medium that is essentially fully ionised, and equally consistent with one that is substantially neutral. The observation that founded the field is the one it cannot use to finish.

What has been used instead is the scatter. Different sight lines at the same redshift show very different amounts of transmission, and the variance between them is a diagnostic that saturation does not destroy: a fully ionised medium with density fluctuations produces one distribution of transmission, and a patchily ionised one — bubbles of ionised gas in a neutral sea — produces a much broader distribution. The observed scatter at $z > 5.5$ is broader than density fluctuations alone can produce, which is evidence that reionisation was still patchy then.

The other clock #

There is a second measurement of the same transition, and it uses different photons, a different physical effect and a different instrument.

Free electrons scatter microwave background photons. If the intergalactic medium is ionised over some range of redshift, a fraction of the background’s photons are scattered on the way out of it, and that fraction is the Thomson optical depth

$$\tau_e = \int \sigma_T\, n_e(z)\, c\,\frac{\mathrm dt}{\mathrm dz}\,\mathrm dz.$$

Scattering does two things: it damps the temperature anisotropies slightly at small angular scales, and — the measurable effect — it generates a distinctive polarisation signal at very large angular scales, because a photon scattering off an electron sitting in a quadrupolar radiation field acquires a polarisation.

Planck’s measurement gives $\tau_e = 0.054 \pm 0.007$.

The agreement is the real result, and it is worth being explicit about why it is not circular. The microwave background measurement integrates the free-electron column over all redshift and is insensitive to when within a broad range the electrons appeared; the forest measurement is a local statement about one epoch and is insensitive to everything before it. Neither could have predicted the other, and a discrepancy would have been a problem.

The remaining tension is over duration. The CMB constrains the integral, so a short sharp reionisation at $z = 7.7$ and a long gradual one from $z = 12$ to $z = 6$ can give the same $\tau_e$ — and the second is disfavoured because it requires more ionising photons than the observed galaxy population appears to produce.

Counting the pixels that are dark #

The saturation problem has one clean workaround, and its virtue is that it requires almost no modelling.

Instead of measuring how much flux gets through, count how many pixels get none. A pixel with flux consistent with zero could be dark because the medium there is neutral, or because it is an ordinary dense region of an ionised medium — but a pixel with flux above zero cannot be neutral, and every such pixel is a piece of the universe that is definitely ionised.

The fraction of dark pixels is therefore a strict upper limit on the neutral fraction, requiring only that the flux measurement be unbiased. No density field, no simulation, no assumption about how the optical depth fluctuates.

The limits it gives are weaker than a full model-dependent analysis and they are trustworthy in a way a model-dependent one is not. At redshift 5.9 the dark-pixel fraction puts the neutral fraction below about ten per cent; by redshift 5.6 it is below a few per cent. Those are upper limits and they are what the field quotes when it wants a statement that no future revision of a simulation can overturn.

The method has one refinement worth noting because it doubles the constraint for free. Each quasar spectrum contains the Lyman α forest and, at shorter wavelengths, the Lyman β forest — absorption by the same gas at the second transition, whose oscillator strength is about five times smaller. So a region opaque in Lyman α may transmit in Lyman β, and a pixel counted as dark only if it is dark in both is a much stronger statement. Requiring both cuts the upper limits by a further factor.

A measurement that gives a bound rather than a value is worth having when the alternative gives a value nobody can check, and the dark-pixel fraction is the one number in this subject that has not moved as the simulations improved.

What did the ionising #

The photon budget is the field’s central open question and it is arithmetic.

Reionising the universe requires at least one ionising photon per hydrogen atom, and in practice several, because recombinations happen and each one has to be undone. The candidates are massive stars in early galaxies and accreting black holes.

Quasars are ruled out as the main source by counting: their number density falls steeply above $z = 3$, and the ionising background they would produce at $z = 7$ is an order of magnitude short. That also fits the spectral evidence, since quasars ionise helium a second time and the helium reionisation is observed to happen much later, around $z = 3$.

Galaxies are the remaining candidate, and the difficulty is not making the photons but getting them out. A young massive star produces ionising photons in quantity; the neutral gas in its own galaxy absorbs most of them. The escape fraction required is around ten to twenty per cent, and the measured escape fractions for galaxies at redshifts where the measurement is possible are typically a few per cent. The budget balances only if the faintest galaxies, which are the hardest to observe, have escape fractions well above the ones that have been measured.

The wing that does not saturate #

There is a second observable in the same spectra that responds where the trough has stopped, and it is now the leading technique.

A quasar sits inside the medium it is illuminating, so light emitted at wavelengths slightly longer than its own Lyman α passes through neutral hydrogen at redshifts slightly below the quasar’s own. That gas is not in resonance with the light, so the resonant absorption does not apply — but the Lorentzian damping wing of the transition does, and a wing falls off slowly enough to matter.

The consequence is that a neutral intergalactic medium imprints a smooth absorption trough on the red side of a quasar’s Lyman α emission line, extending several thousand kilometres a second redward. Its depth and shape depend on the neutral fraction, and crucially the dependence does not saturate: the wing’s opacity is small, so doubling the neutral fraction doubles the absorption.

That converts the measurement from a bound into a number. Fitting the wing gives a neutral fraction directly, and the technique has been applied to the highest-redshift quasars known, returning neutral fractions of order a half at redshift seven.

Its difficulty is the intrinsic spectrum. The absorption is measured as the difference between the observed profile and the profile the quasar would have had, and nobody has observed that quasar without the absorption. What is done instead is to predict the intrinsic profile from the parts of the spectrum the absorption does not touch — the emission lines further to the red — using a relation calibrated on lower-redshift quasars where both are visible.

So the measurement is a difference between an observation and a prediction, and its uncertainty is dominated by how well one quasar’s line profile can be predicted from another’s. That is a very different failure mode from the trough’s, which is what makes the two worth having together.

There is a third technique in the same family that avoids the prediction problem entirely, at the cost of needing a different kind of source. A gamma-ray burst afterglow has a smooth power-law continuum with no emission lines at all, so the damping wing can be fitted against a featureless baseline rather than against a predicted profile. Bursts are found at redshifts beyond any known quasar, they fade within days, and their afterglows are bright enough for spectroscopy for a few hours — so the observation has to be made immediately or not at all.

A handful have been caught at redshifts above six and the constraints they give are consistent with the quasar ones. What limits them is not the physics but the logistics: the whole measurement depends on a telescope being pointed within hours of an event nobody can predict, which is a different kind of difficulty from every other one in this essay and is the reason so few exist.

The same objects carry a second advantage that is easy to overlook. A quasar ionises its own surroundings over several megaparsecs and the near zone has to be excised; a gamma-ray burst is a transient in a small galaxy and ionises almost nothing, so its line of sight samples the ordinary medium right up to the source. Where the two disagree, the burst is the less biased of the two.

Where the model stops #

Four limits.

The Gunn–Peterson expression assumes a smooth medium, and the real one is not, which is the Jensen inequality above. Every quoted neutral fraction from the forest carries a model of the density field, and different simulations give different answers from the same spectrum.

It assumes the absorbing gas is at the same redshift as the resonance, which peculiar velocities and thermal broadening violate at the few per cent level — negligible where the depth is $10^5$ and not where the transmission is being measured to a per cent.

The CMB’s $\tau_e$ is measured from large-angle polarisation, which is the hardest part of the microwave sky to observe: it is contaminated by Galactic dust and synchrotron emission at a level comparable to the signal, and the published value has moved from 0.17 to 0.089 to 0.054 as the foreground treatment improved. The current value is believed and its history is a caution.

And the whole subject has a selection effect in it. The high-redshift quasars whose spectra are used are the most luminous objects of their epoch, they sit in the densest regions, and they ionise their own surroundings — so the medium immediately in front of one is not typical, and the near zone has to be excised before anything is measured.

The two observables the argument rests on are worth reading at a second setting each, since one is a spectrum and the other is an integral over a history.

Where this ladder goes next #

This rung establishes the optical depth, the direction the argument runs, and the reason the trough saturates.

Above it lies the 21-centimetre line, which is the measurement the field is waiting for. Neutral hydrogen emits or absorbs at 21 cm against the microwave background, and unlike Lyman α that transition does not saturate — its optical depth at $z = 8$ is of order a per cent. So a 21-cm map is a direct measurement of the neutral fraction as a function of position and redshift, through the whole transition, and it is the only observable that can watch reionisation happen rather than bracket it.

Beside it lies helium reionisation at $z \approx 3$, which is the same physics on a species with a different ionisation energy, driven by quasars rather than galaxies, and observed in the far ultraviolet.

And below it lies the thing this rung is really an example of: a saturated measurement is a measurement of a bound, and treating a bound as a value is the standing error the whole subject was built to avoid.