A background weighed by what it stops

The optical background light cannot be measured from where anybody is standing. The zodiacal cloud is two orders of magnitude brighter than the thing being looked for, the airglow is worse, and every result is the residual of a subtraction rather than a photometric measurement.

There is a second method, and it has no aperture pointed at the sky at all. A gamma ray crossing intergalactic space can collide with a background photon and turn the pair into an electron and a positron. The gamma ray is then gone, and the deficit in the spectrum of a distant source is a measurement of the density of the photons it collided with. The background is weighed by what it removes from something else, and the zodiacal light does not enter the calculation anywhere.

Two photons and a threshold #

The reaction is $\gamma\gamma \to e^+e^-$ and it has a threshold, because an electron–positron pair has a rest mass and the photons have to supply it.

For two photons of energies $E$ and $\varepsilon$ meeting at an angle $\theta$, the invariant mass squared of the pair is $2E\varepsilon(1-\cos\theta)$, and the reaction is possible when that reaches $(2m_ec^2)^2$. Head-on, the condition is

$$E\varepsilon \ge (m_ec^2)^2 = 0.261\ \mathrm{MeV}^2 .$$

So a gamma ray of 1 TeV can pair with anything above 0.26 eV, which is a wavelength shorter than 4.7 µm. The cross-section is zero at threshold, rises steeply, peaks at roughly twice the threshold energy, and falls slowly afterwards — so what a gamma ray of a given energy actually interacts with is not everything above its threshold but a band near

$$\varepsilon_{\text{peak}} \approx \frac{2(m_ec^2)^2}{E},$$

a wavelength of about 2.4 µm for a TeV photon. That is the near infrared, which is exactly the part of the background the zodiacal light hides best. The coincidence is not one: it is the reason this method exists as a rival to direct photometry rather than as a curiosity.

The angular factor matters too, and it works in the method’s favour. A gamma ray travelling through an isotropic bath meets background photons from every direction, and the head-on collisions are both the most energetic and — because the relative velocity is largest — the most frequent. Averaging over angles therefore does not smear the threshold into uselessness; it shifts the effective pairing wavelength by a factor of order one and leaves the correspondence between energy and wavelength intact. The sharpness of that correspondence is what makes the measurement spectroscopic rather than bolometric: a gamma-ray telescope observing across a decade of energy is sampling the background across a decade of wavelength, one channel at a time.

Running the measurement backwards #

The figure above computes an absorption from an assumed background. The measurement runs the other way, and the awkwardness in it is the whole subject.

What is observed is the spectrum that arrives. What is wanted is the absorption. The difference between them is the spectrum the source emitted — and nobody has ever seen a blazar from close enough to know what that is.

A blazar is an active galactic nucleus whose relativistic jet points within a few degrees of the line of sight, so that the emission is beamed and boosted towards here by factors of tens. The gamma-ray part of its spectrum is produced by electrons that have been accelerated in the jet, and the spectral shape depends on the acceleration mechanism, on the magnetic field, and on the density of the photons the electrons scatter off — which is a set of quantities inferred from the same spectrum whose intrinsic shape is in question.

The escape from the circularity is a physical bound rather than a model. The electrons in a jet are accelerated by shocks, and shock acceleration produces a power law in energy whose index has a hard floor — a photon spectrum harder than $E^{-1.5}$ requires something other than the standard mechanism. So the intrinsic spectrum cannot be arbitrarily hard, which means the absorption cannot be arbitrarily large, which means the background cannot be arbitrarily dense.

The result is an upper limit rather than a measurement, and the upper limit is the interesting number, because it came out close to the sum over counted galaxies. A completely independent physics — pair production at a trillion volts — landed on the same answer as adding up the light of every galaxy in a deep image.

The same trick, in four other places #

Measuring a medium by what it removes rather than by what it emits is one of the most productive habits in the subject, and it is worth seeing the family this belongs to.

Interstellar dust is measured by the light it takes out of a star, and the wavelength dependence of the removal — steeper in the blue — is what identifies the absorber as small grains rather than as anything else. The measurement is of a medium that is, at optical wavelengths, essentially invisible in emission.

The intergalactic medium is measured by the trough it cuts in a quasar’s spectrum, and the astonishing part of that measurement is its sensitivity: neutral hydrogen at one part in $10^{5}$ of the total is enough to absorb completely, so the absence of a trough is a statement about a gas density no emission measurement could reach.

A stellar atmosphere’s composition is read off the lines missing from its continuum, which is the oldest instance of the habit and the one that founded astrophysics as a quantitative subject.

And the cleanest analogue of all is not about photons. An ultra-high-energy proton crossing intergalactic space collides with the microwave background and loses energy to pion production above a threshold set by exactly the same kinematics — and the resulting cutoff in the cosmic-ray spectrum at the highest energies is the same calculation with a different projectile and a different background. Both were predicted within a year of each other, and both took four decades to confirm.

What the family has in common is that the absorber is diffuse, cold, and therefore a poor emitter, while the thing being absorbed is bright, compact, and comes from far enough away to have crossed a great deal of it. An absorption measurement trades the need for a bright target against the need for a bright source, and there is always some source bright enough.

The dispute it adjudicates #

That agreement is what makes this a reply to the photometric measurement rather than an addition to it.

The direct photometric measurement of the optical background finds more light than the counted galaxies add up to. The gamma-ray bound finds no room for it. They cannot both be right, and they fail in unrelated ways: the first fails through a zodiacal model, the second through an assumed intrinsic spectrum.

The escape hatch on the gamma-ray side is that the absorption might be real and partly undone. If photons can oscillate into a hypothetical very light particle that does not pair-produce, travel as that particle through the absorbing region, and oscillate back, then a distant source looks less absorbed than it should. Several analyses have reported a mild preference for exactly that, at a significance nobody treats as decisive.

The sober reading is that the intrinsic spectra are not as well bounded as the argument assumes. The interesting reading is that a measurement of the darkness of the sky has become a laboratory for particle physics, which is not where Olbers left it.

There is a way to tell the two apart, and it is being pursued. An oscillation into a light particle would depend on the intergalactic magnetic field along the path, since the conversion requires one; an over-hard intrinsic spectrum would not. So the anomaly, if it is one, should correlate with the magnetic environment of each source’s sight line and not merely with its redshift. The sample of sources detected at large optical depth is still small enough that the correlation cannot be tested, which is the honest state of the question rather than a hedge.

What can be said is that the size of the claimed effect is uncomfortable. The excess transparency needed to reconcile the direct photometric background with the observed blazar spectra is not a few per cent; it is a factor. A factor in an exponential is a large amount of new physics, and the alternative — that one zodiacal model is wrong at the ten per cent level — is not an extraordinary claim about anything.

Why more background means a nearer horizon #

The quantity the absorption defines is a horizon, and it is worth being careful about which sense of the word is meant.

The particle horizon is set by the time light has had to travel. The gamma-ray horizon is set by opacity, exactly as the surface of last scattering is: it is the distance at which the optical depth reaches unity, and it depends on energy because the opacity does.

The dependence runs the natural way and it is steep, because optical depth is exponential in the column. A background twice as dense does not halve the flux that arrives from a source at $\tau = 3$; it reduces it by a further factor of twenty. Which is why the method is so much more sensitive to the background’s density than a photometric measurement is, and so much less able to say what the density actually equals rather than what it cannot exceed.

How the background grew, and what that adds #

There is a second dimension to the measurement, and it is the one that has come into reach most recently.

The background was thinner in the past — fewer stars had shone — but the photons in it were also denser, because a comoving volume was smaller. The two effects run against each other. Proper number density scales as $(1+z)^3$ from the expansion alone, and the comoving density falls backwards in time as star formation is undone, so the product is a mildly rising function of redshift out to the era when most stars formed.

That is what turns a bound into a history. Stacking the spectra of several hundred blazars in redshift bins, and fitting the change in the absorption with redshift, gives the buildup of the background over cosmic time — which is a measurement of the star formation history, made with gamma rays, at wavelengths where the light itself cannot be seen from here.

The published version of that measurement agrees with the star formation history derived from galaxy counts, to about the accuracy the counts themselves have. A technique that consists of noticing what is missing from a spectrum reproduces a curve built from imaging hundreds of thousands of galaxies.

What the horizon is worth as a distance measure #

A quantity that depends on energy and redshift together offers something no photometric measurement does, and it has been proposed as a cosmological probe in its own right.

The optical depth accumulates along the path, so it depends on the distance–redshift relation and therefore on the expansion history. If the background were known independently, a measured $\tau(E, z)$ would give the Hubble constant and the matter density, from a physics entirely unrelated to standard candles or to the acoustic scale.

It has been tried, and the answer is consistent with everything else and much less precise. The reason is structural rather than technical: the background and the expansion history enter the integral in nearly the same way, so the measurement constrains a combination of them and cannot separate the two without help. That is the same difficulty a chain of calibrated distances has in a different guise — a quantity measured through an intermediary inherits the intermediary’s uncertainty — and there is no version of the gamma-ray measurement that avoids it.

What the method does have, and a chain of standard candles does not, is complete independence from calibration. There is no zero point anywhere in it, no candle, and nothing to be transferred from one step to the next. The threshold is a rest mass, the cross-section is a first-principles calculation, and the only astronomy in it is the assumption about a jet.

Three quantities enter, and only one of them is uncertain #

Three quantities enter, and their reliability is very unequal.

The threshold and the cross-section are quantum electrodynamics, known exactly, with no free parameters anywhere in them. Nothing in the absorption calculation is uncertain on that side.

The path length is the expansion history, which is known to about a per cent over the redshift range that matters. That contributes nothing to the error either.

Everything uncertain is on the third side: the intrinsic spectrum of a source nobody has ever seen unabsorbed. And the measurement’s whole design is an attempt to convert that into something bounded — by requiring only that the intrinsic spectrum be no harder than shock acceleration allows, by using many sources so that a single peculiar one cannot drive the answer, and by fitting the redshift dependence of the absorption rather than its absolute size, since an intrinsic spectrum does not know the redshift of the source it belongs to.

The secondary emission no optical depth contains #

The absorption is not the only thing that happens. An electron–positron pair created in intergalactic space inverse-Compton scatters the microwave background and produces lower-energy gamma rays, which arrive later and from slightly the wrong direction. That secondary emission is a real prediction, its absence or presence constrains the intergalactic magnetic field, and no curve of optical depth contains it.

The background is taken as smooth. A gamma ray crossing 500 Mpc passes through voids and filaments alike, and the background’s local density varies with them by tens of per cent. Averaged over a path that long the variation is small, and for the nearest sources it is not obviously negligible.

And the figure’s optical depths are computed from a modelled background rather than a measured one, which makes them a prediction being tested rather than a measurement being displayed. Read the other way — as the absorption a given background would produce — they are exactly what the method compares an observation against, and that is the only reading the drawing supports.

Why the near infrared is the hard part twice over #

It is worth dwelling on the wavelength where the two methods meet, because both are at their worst there and for unrelated reasons.

Direct photometry is worst in the near infrared because the zodiacal cloud emits thermally as well as scattering: at a few microns the sunlight scattered off the grains and the grains’ own emission are comparable, and the model has to get both right. The foreground is at its most structured exactly where the extragalactic signal sits in the trough between its two humps.

Gamma-ray absorption is worst in the near infrared for the opposite reason — it is best there. The background’s trough is where the gamma-ray method is most sensitive, because the photons in the trough are the ones that pair with the TeV gamma rays that atmospheric Cherenkov telescopes detect most efficiently. So the method with no foreground is most informative exactly where the method with a foreground is least reliable.

That is a good arrangement and a rare one. Two techniques with nothing in common are usually sensitive to the same thing, so that combining them buys a factor of root two. These are sensitive to different parts of one curve, and combining them buys the shape.

The remaining gap in the coverage is the far infrared. Nothing pairs with a 140 µm photon except a gamma ray above 100 TeV, and the sources bright enough to detect at that energy are all Galactic. The dust hump is therefore known from direct photometry alone, which is the part of the background where COBE’s absolute measurement is the whole of the evidence.

An old prediction and a late confirmation #

The reaction was worked out in the 1930s and the astronomical consequence was pointed out in 1962, when the intergalactic radiation field was almost entirely hypothetical. The prediction was that the universe would prove opaque to gamma rays above some energy, and that measuring where the opacity set in would measure the radiation field.

For thirty years nothing could test it, because there were no gamma-ray telescopes with the sensitivity to detect an extragalactic source above 100 GeV. The technique that eventually provided them does not collect gamma rays at all: it images the flash of Cherenkov light that a gamma-ray-induced air shower makes in the atmosphere, using the whole sky above a few hundred square metres as the detector. The first extragalactic detection came in 1992 and the first serious background limit in 2006.

The instrument is therefore looking at the wrong thing on purpose, twice over. It measures a background it cannot see, using photons it does not collect, by observing a flash in the air made by a particle shower. That chain is worth keeping in mind whenever the extragalactic background light is quoted with two error bars, one from photometry and one from gamma rays: the second number was never measured by looking at the sky.

The delay between prediction and test is itself instructive about what the calculation was worth. In 1962 there was no measured background, no detected extragalactic gamma-ray source, and no instrument capable of either. The prediction was made from the threshold alone — a rest mass, a cross-section, and a guess at how much starlight there might be — and it named in advance which energies would be affected and by how much. Everything since has confirmed the structure and refined the number.

That is the best case a figure-first argument can hope for: the shape was derivable from physics that was already settled, and forty years of instrument development were needed only to supply the scale. The threshold relation drawn across the top of every plot in this essay was known before any of the quantities in it had been measured.

Still open: whether a dark sky is a special case #

Both arguments so far have taken it for granted that the sky’s darkness needs explaining, and the explanation has been the same in each: the horizon cuts the integral short, and what is left is faint.

But the horizon is not a property of the universe on its own. It is a property of the universe and of what is doing the travelling — and light is the messenger with the earliest wall in front of it. Two other things cross intergalactic space, both of them detected, and neither has a wall anywhere near as recent.