A background that stops breaking into sources

The sky is dark because the universe is young, and it is not quite dark, because every star that has ever formed has left some of its light behind in a faint glow that fills every direction. That glow is a sum over sources. It was measured first as a sum — a level, in nanowatts per square metre per steradian — by instruments that could not see a single one of the galaxies it came from.

The obvious next step is to take a deeper image and count them. A glow that is a sum over sources should, looked at closely enough, stop being a glow and become a list. In some wavelengths that is exactly what happened: the X-ray background, the first ever found, is now more than four-fifths resolved into individual active galaxies, and the optical background is a catalogue. In the far infrared, where the glow carries as much energy as all the starlight in the universe, the deepest images ever taken resolve about a sixth of it, and no amount of further exposure with the same telescope will resolve more.

The reason is not faintness. It is crowding, and it has a precise form.

The wall that exposure does not move #

A telescope does not record a point source as a point. It records a blur the size of its beam, set by diffraction — the full width at half maximum is about $1.13\lambda/D$ for a mirror of diameter $D$ at wavelength $\lambda$ — and everything inside one beam is added together before any detector sees it.

While the sources are sparse that does not matter: each blur sits on its own and a catalogue can be read off the map. As the image goes deeper, fainter sources appear, and there are more faint sources than bright ones, so the surface density rises. Eventually the blurs overlap. A lump in the map is then two or three or ten sources whose sum happens to be a local maximum, its position is the centre of light of a group rather than a galaxy, and its flux is a statistic rather than a measurement.

The rule astronomers use for where this sets in is empirical and old: a source is reliably recovered, with a trustworthy flux, while there is no more than one source per twenty to fifty beams brighter than it. Thirty is the conventional value, and it is the one used throughout here. Past that density the map does not get deeper with more exposure. The noise of the instrument keeps falling as the square root of the time; the “noise” of the sky — the fluctuating sum of all the unresolved sources in each beam — does not fall at all, because it is not noise in the counting sense that sets every other error bar. It is the sky, and it is the same every time it is observed.

That picture is the whole of the problem. Deepening the right-hand map by a factor of a hundred in exposure time would make its pixels ten times more precise and change nothing about which lumps are galaxies. Only a smaller beam would.

One number decides it #

Whether a survey is confused near the fluxes that matter reduces to a single dimensionless number, and finding it is worth the arithmetic.

The counts of sources — how many per square degree are brighter than a flux $S$ — are the input. For a population spread uniformly through a static Euclidean space they follow $N(>S) \propto S^{-3/2}$, the same shell construction that made Olbers’ sky diverge: the number of sources in a shell rises as $r^2$ and each one’s flux falls as $r^{-2}$. Real counts depart from that law in both directions, for the same reasons a galaxy census acquires a knee. At bright fluxes they are often steeper, because the sources were more luminous or more numerous in the past and the brighter end samples that past. At faint fluxes they become shallower than Euclidean, because the volume runs out and the expansion dims what lies beyond. The flattening produces a break, and the break matters more than anything else about the counts, because it is where the background is made.

The background is the integral of flux times number: $I = \int S\,(dN/dS)\,dS$. Above the break, where $dN/dS$ falls faster than $S^{-2}$, the bright sources are too rare to contribute much. Below it, where $dN/dS$ falls more slowly than $S^{-2}$, the faint ones are too faint. The integral is dominated by sources within a decade or so of the break flux, and that is true in every band where the counts have been measured. A background is made mostly by sources near one flux, and whether a survey resolves the background is a question about whether it resolves the sources at that flux.

So the number that decides it is $X = N_b\,\Omega_{\rm beam}$ — the number of sources brighter than the break that fall inside one beam. If $X$ is much smaller than one in thirty, the survey reaches the break before confusion and can resolve most of the background. If $X$ approaches one in thirty, confusion sets in near the break and most of the background stays a glow.

The curve is universal in shape and the surveys are scattered along it by a factor of ten thousand in $X$, which is the entire observational situation compressed into one axis. The Chandra X-ray Observatory’s on-axis beam is about 0.7 arcseconds and the X-ray sources near the break in the counts number about a hundred and fifty per square degree; the product is six parts in a million. The Herschel Space Observatory’s beam at 250 microns is eighteen arcseconds, a solid angle nearly seven hundred times larger, and there are ten times as many dusty galaxies near its break as there are X-ray sources near Chandra’s. The product is 0.04, which is past the half-way point on the curve.

Chandra is not a more sensitive telescope than Herschel in any sense that decides this. Both were built to detect faint things; both did. One of them sees its background as a list and the other as a haze because of where its beam lands on a curve whose position depends on the sky’s counts and the telescope’s diffraction, and on nothing else.

What the counts do near the line #

The counts themselves show where each beam runs out of room, and they show something the one-number summary hides: how little is gained by moving a confusion line a long way.

The last sentence of that caption is the most useful thing the counts say, and it cuts in two directions at once.

Above the break the counts are steep, so moving the confusion line down by a factor of four buys only a modest step in flux: $4^{1/2.3}$, or about 1.8. That is the regime both Herschel bands sit in, and it is why the Herschel surveys, deep as they were, resolved so little. Below the break the counts are shallow, so the same factor of four in density buys a factor of sixteen in flux. A telescope that brings its confusion line below the break is rewarded enormously; one that stops short of it is rewarded very little. The transition between the two regimes is abrupt, because the break is.

That asymmetry explains a pattern in the history of every background that has been resolved. The X-ray background went from about 20 per cent resolved with the Einstein Observatory in the 1980s, to around 70 per cent with ROSAT in the 1990s, to more than 80 per cent in the softest band with Chandra’s deepest fields. Each step was a telescope whose beam crossed a larger share of the break, and the final step came from a telescope whose beam was so small that confusion stopped mattering at all — Chandra’s deepest field, seven million seconds on one patch, is limited by photon counting, not crowding, and reaches sources a thousand times fainter than the break.

The first confusion, and the cosmology it nearly decided #

Confusion was discovered in the worst possible way: by producing a wrong answer to a question that mattered.

In 1955 the second Cambridge radio survey, the 2C catalogue, published counts of radio sources and found them rising towards faint fluxes far more steeply than the Euclidean law allowed — a slope near $-3$ in $N(>S)$ where a static uniform universe gives $-1.5$. Martin Ryle read it as evidence that radio sources were much more numerous in the past, which is to say that the universe evolves, which is to say that the steady-state theory, then a serious rival to the big bang, was wrong. The steady-state universe looks the same at all times and cannot have more sources in the past.

At the same time, in Sydney, Bernard Mills and his colleagues surveyed the overlapping sky with a different instrument and found that most of the faint 2C sources were not there. The Cambridge interferometer’s beam was large and its sidelobes were strong, and at the faint end of the catalogue a large fraction of the “sources” were blends of several weaker ones, or peaks in the sum of all of them. Blending pushes counts up at the faint end — two sources below the limit add to one above it — and it pushes them up more the deeper the survey goes, which is exactly the signature of a steep slope.

The steep slope was mostly a property of the beam. Peter Scheuer, a Cambridge student, worked out in 1957 how to extract the counts from the statistics of the map itself rather than from the list of peaks, and found a slope consistent with the Sydney result. The evolutionary conclusion eventually turned out right — radio sources were more numerous in the past, and later surveys with better beams showed it — but the 2C evidence for it was confusion, and the argument about cosmology spent several years as an argument about beams.

The episode established two things that have held since. The first is the one-per-several-tens-of-beams rule itself, which Condon later formalised. The second is Scheuer’s method, which treats the confused map as a measurement rather than a failure, and which is where the next part of this argument goes.

How much of the background is below the line #

A confused survey stops counting somewhere. The question it leaves is how much of the background lies below that point, and the answer depends on a slope that the survey, by construction, did not measure.

There is a quiet circularity here that affects every quoted “resolved fraction”. The fraction is a ratio of two numbers: the flux in the sources counted, which is measured, and the total background, which in the far infrared came from the COBE satellite’s absolute photometers in the late 1990s. The second number was measured independently of any counts, which is what makes the ratio meaningful — but it was measured against the glow of the solar system’s own dust, which is a hundred times brighter at those wavelengths — a measurement in which the sky is the signal rather than the nuisance — and its uncertainty is some tens of per cent. A resolved fraction of “15 per cent” at 250 microns is fifteen per cent of a denominator known to perhaps a third.

The published Herschel number is about fifteen per cent at 250 microns, ten at 350 and six at 500, from the individual sources above the confusion limit in the deepest fields. The schematic counts drawn here put 250 microns at a quarter, which is too high by about the amount their break and slopes were guessed rather than fitted; they are there to show where on the curve each survey lies, not to reproduce the published fraction, and they get the order of the three bands right and their spacing roughly right.

A criterion, and what it is worth #

The threshold of one source per thirty beams is a convention, and a conclusion that depended on it strongly would be suspect. The criterion can be moved and the result redrawn.

The looser criterion buys fluxes that are not worth much. A source identified at one per ten beams has a flux boosted by the unresolved sources in its beam — the effect called flux boosting — by an amount comparable to its own error, and its position is uncertain by a large fraction of the beam, so matching it to a galaxy seen at another wavelength becomes a guess. That is the real content of the thirty-beam rule: not that a peak cannot be found at higher density, but that what is found there cannot be trusted as a single galaxy with a single flux. Surveys that push past the rule do it by bringing in information from elsewhere — positions from a sharper image at a shorter wavelength — which is to say by borrowing a smaller beam.

A background resolved by the size of a mirror #

If exposure cannot move the confusion limit, the only lever is the beam, and a beam is set by $\lambda/D$. At a fixed wavelength that means the mirror.

That is an unusual statement to be able to make about a telescope. For most observations the returns on a larger mirror are the returns on collecting area — sensitivity grows as $D^2$, or as $D^4$ for point sources against a background when the beam also shrinks — and they are steady. For a confused survey of a background the returns are set by where the break in the counts falls relative to the beam, and they are anything but steady: nearly nothing, then a great deal, then nearly nothing again once everything is resolved.

It also explains a design argument that has run through every far-infrared mission concept since Herschel. A cold, large far-infrared telescope in space is expensive, and the case for one has always had two halves — sensitivity to spectral lines, which needs collecting area and cold optics, and depth in the continuum, which past a certain point needs only a smaller beam. The second half is a confusion argument, and it is why the far infrared is the one band where an interferometer in space has been seriously proposed for mapping rather than only for resolving individual objects: synthesising a larger aperture is the one way to get a small beam without building a mirror of that size.

The wavelength dependence has a consequence for what the unresolved part of the far-infrared background actually is. The galaxies that dominate the 500-micron background are, on average, further away than those dominating the 250-micron one, because the peak of a dusty galaxy’s emission is redshifted into the longer band from a higher redshift — the same shift that makes a magnitude refer to a band the source never emitted in. So the least-resolved part of the background is also the most distant part of it, and it is the part that carries the star formation hidden from ultraviolet indicators by dust, and the history of dust-obscured star formation at the highest redshifts is precisely the history that the confusion limit keeps out of reach. What has been learned about it has come largely from telescopes on the ground that trade the space telescopes’ sensitivity for a much larger aperture — interferometers at millimetre wavelengths with beams of a fraction of an arcsecond — and they resolve the brightest of these galaxies one at a time, over areas far too small to add up to a background.

The same arithmetic Olbers used #

It is worth seeing that the confusion limit is the paradox itself, applied to a beam instead of to a sky.

Olbers asked how far a sight line has to run before it ends on a star. The answer was the mean free path $\lambda = 1/n\sigma$, and the paradox was that a universe of uniform stars would make every sight line end on one. The confusion limit asks how deep a beam has to reach before it contains a source above a given flux. The answer has exactly the same form — a density times a cross-section, where the cross-section is now the beam’s solid angle rather than a star’s disc — and the “paradox” is that a survey deep enough will find every beam occupied.

In both cases the divergence is stopped by something that has nothing to do with the sources. For Olbers it is the finite age of the universe, which cuts off the shells before the sky fills. For a survey it is the break in the counts, which cuts off the supply of new sources at faint fluxes before every beam fills with bright ones — and the break is itself a consequence of the same finite age and the same expansion. The counts flatten below the break for the reason the sky is dark: there is not enough volume at large distances, and what there is has been redshifted. A universe in which the counts continued at the Euclidean slope forever would be one in which every survey was eventually confused at every flux, and it would also be Olbers’ universe, and its sky would be bright.

What the curve leaves out #

The counts are schematic. Each band is reduced to two power laws joined at a break, with slopes and normalisations chosen to put the published confusion depths and resolved fractions in the right order and the right decade. The real counts curve smoothly through the break, have a bump at bright fluxes from strongly lensed galaxies in the far infrared, and in the X-ray band differ between soft and hard energies — the hard X-ray background, above about 6 keV, is less than two-thirds resolved even in the deepest fields, because the heavily obscured active galaxies that make it are faint in the soft band where Chandra’s sensitivity is greatest.

The sources are drawn as points scattered at random. Galaxies cluster, and clustering puts more sources in some beams than a Poisson distribution would, which makes confusion set in at a lower mean density than the random case — by a modest factor in the far infrared, where the dusty galaxies are strongly clustered. The simulated patch has no clustering and so errs on the side of making confusion look less severe.

The beam is a Gaussian. Real beams have sidelobes, and the 2C episode was as much about sidelobes as about the main beam. And in the optical, the “beam” of a space telescope is smaller than the galaxies themselves, so optical confusion is set by the sizes of galaxies rather than by diffraction; the optical point on the hero figure treats galaxies as points and is placed there to show the plateau, not to model deep optical fields.

And “resolved” means only that a flux can be assigned to a position. It does not mean the source has been identified, has a redshift, or is a single object. The resolved share of a background is an upper bound on how much of it is understood galaxy by galaxy, and in the far infrared the gap between the two is large.

Still open: what the haze is made of #

The confusion limit says how much of a background can be resolved by counting. It does not say that the rest is unknowable. The haze below the limit is not smooth: it is the sum of a finite number of sources in each beam, and a sum of a random number of sources fluctuates from beam to beam in a way that depends on how many there are and how bright. The histogram of the pixel values in a confused map therefore carries the counts below the confusion limit — which is what Scheuer realised in 1957, looking at a map that had misled everyone who read it as a list.

That turns the confusion limit from a wall into a change of method. Above it, a background is measured by counting; below it, by the statistics of the fluctuations, and the statistics reach roughly a decade fainter than any source that can be picked out. How far they reach, and whether the faint counts they recover account for the whole of the measured backgrounds — the radio background above all, where the counts and the absolute level do not obviously agree — is what the confused maps are still being asked.