Light that adds, corrected as if it multiplied

A flat field is an exposure of something that ought to be uniform — a lit screen inside the dome, the twilight sky, the accumulated background of a night’s science frames — and dividing every image by it is the first thing done to an astronomical image. The flat removes two different things: the pixel-to-pixel response of the detector, and the smooth illumination pattern of the optics in front of it. Both are multiplicative. A pixel that is two per cent less sensitive records two per cent less of whatever falls on it, and a corner of the field that the optics vignette receives some fixed fraction of the light that the centre does. Multiply by the response, divide by the response, and the error cancels exactly.

There is a third thing in every flat, and it does not multiply. Some of the light that reaches the focal plane never followed the path a ray from a star follows. It reflected off the back of the filter and then off the detector window and then off the filter again; it grazed a baffle edge; it scattered from dust on the corrector plate or from the micro-roughness of a mirror polished as smoothly as the budget allowed. That light lands at places in the focal plane that the optical design never intended, usually spread smoothly over a large area and often concentrated towards the centre where the reflections overlap. It is scattered light, and it adds to the image rather than scaling it.

This essay is about one consequence of that difference, and it is a strange one. The flat that makes the sky look perfect is the wrong flat for the stars.

The sky and the star are different kinds of light #

The argument depends on one distinction, so it is worth stating carefully.

A flat field is extended illumination: every part of the entrance pupil is lit from every direction within the field of view, the way the sky lights it. Light from that whole wide cone enters the telescope, most of it goes where it should, and a fraction scatters. Because the illumination comes from everywhere, the scattered fraction arriving at any point of the focal plane is the sum of contributions from the whole field — and that sum is what the flat records as extra sensitivity at that point. Write the flat as the true response $R(x)$ times $1 + s(x)$, where $s(x)$ is the scattered light as a fraction of the direct light at position $x$.

A star is a point. Its light arrives from one direction, most of it goes into the point spread function at one place, and the fraction that scatters is spread thinly across the whole focal plane. Inside the star’s own photometric aperture, a few arcseconds across, the star’s own scattered light is negligible: it has been distributed over a field thousands of times larger. So the star’s image carries $R(x)$ and almost none of $s(x)$.

The night sky behind the star is extended illumination again, exactly like the flat. It carries $R(x)\,(1 + s(x))$, if the angular distribution of the night sky resembles that of the source used for the flat.

Now divide the science image by the flat. The sky becomes $R(1+s)/R(1+s) = 1$ — uniform. The star becomes $R/R(1+s) = 1/(1+s)$ — too faint, by the factor $1 + s(x)$, wherever $s$ is not zero. The figure above draws those two quotients against radius. The flat line is the sky and the curve is a star.

The uncomfortable part is the test that the procedure passes. When an observer checks whether a flat-field correction has worked, what is checked is whether the corrected background is flat — whether the sky across the image has the same level in the corners as in the centre, with no rings, no gradient, no dust doughnuts. And on that test this flat is perfect. The defect is invisible in every image it corrects, because it is precisely the component that the sky shares with the flat and the star does not.

A pattern, not an offset #

If $s$ were the same everywhere, the error would be harmless. Every star would come out fainter by the same amount, the zero point measured from standard stars would absorb it, and nothing would ever notice. A magnitude is defined relative to a reference, and a constant multiplier on every flux is just a different reference.

But scattered light is not uniform. The geometry that produces it — reflections between parallel surfaces near the focal plane, light scattered at small angles from an obstruction near the pupil — tends to concentrate it towards the centre of the field, where the reflected cones overlap, and to fall off towards the edges. Ghosts between a filter and the detector window produce a broad central glow; scattering from a central obstruction produces a halo concentrated on axis. The shape varies by instrument but the qualitative pattern is the same: more scattered light in the middle than at the edges.

So the error is a pattern. A star at the centre of the field is made tens of millimagnitudes too faint; the same star at the edge is made almost nothing too faint. A standard star used to set the zero point removes the error at whatever position the standard happened to occupy and leaves the difference everywhere else.

This is why it survives every ordinary check. Compare the same star on two nights with the same pointing, and it lands in the same place, carries the same error, and agrees with itself perfectly. Compare two stars in the same image, and they differ by a real magnitude difference plus the difference in their errors, and nothing distinguishes the second term from the first. Calibrate against a standard field placed at the centre, and every target at the edge comes out too bright relative to the standard by the full amplitude of the pattern. Only a comparison that puts the same star in different places in the field can see it.

Three per cent sounds like a lot of scattered light and is not unusual for a wide-field imager. Wide-field instruments are especially vulnerable, with fast optics, many surfaces close to the focal plane, and a wide cone of sky admitted.

The sky does not have to scatter like the flat #

The argument so far assumed that the night sky and the flat field scatter the same fraction of their light. That is true only if the two sources light the telescope from the same distribution of directions, and they often do not.

A twilight flat is taken when the sun is a few degrees below the horizon. The sky then is bright, and it is brightest towards the sun — a strong gradient across the sky that is much wider than the field, so the light entering the telescope arrives preferentially from one side. A dome flat is a screen a few metres away, lit by lamps, filling the pupil from a short distance rather than from infinity. The night sky is a broad glow of airglow and zodiacal light from every direction at once. Each of these has a different angular distribution, and the scattered fraction depends on exactly that distribution, because light from outside the field of view can scatter into it and light from inside cannot scatter out of it identically.

When the two differ, the corrected sky is no longer flat. If the flat carries more scattered light than the night sky, the corrected background sags towards the centre; if it carries less, the background bulges. That sag is now visible, and here the procedure goes wrong in a second, more insidious way. The standard response to a corrected image whose background is not flat is to fit the background and subtract it: model the sky across the image as a smooth surface and take it away. After that step the background is flat again, and the image looks right.

But the background fit corrects the sky, and the sky’s error is not the star’s error. The sky’s residual is the ratio of its scattered fraction to the flat’s; the star’s residual is the whole of the flat’s scattered fraction, because the star has none. A fitted background makes the sky flat and leaves the stars too faint by the full amount — slightly less visibly than before, but no less wrong. The visible defect in the sky has been removed by a correction that does nothing to the invisible defect in the stars.

This matters because a flat taken in different light from the night it corrects is not an edge case. It is the normal state of affairs. Twilight is the only time the sky is bright enough to make a flat quickly, and also the time when it is least like the night sky. Combining a twilight flat with a night-sky flat, the usual practice, inherits the problem from both: each carries its own scattered light, and neither carries the stars’.

A scattered fraction with a colour #

Scattering is not grey. The reflectance of an anti-reflection coating varies across the band it is designed for, usually rising towards the edges of its design range; the ghosts between a filter and the detector window depend on the reflectance of both surfaces at each wavelength; and scattering from dust and surface roughness is stronger at short wavelengths, in the same sense as the scattering that makes the sky blue. So the scattered fraction at the centre of the field is a function of wavelength, even inside one filter.

A flat records the scattered fraction at the colour of the light that made it. A twilight flat is lit by the twilight sky, which is blue; a dome flat is lit by incandescent lamps, which are red. Every star is then divided by that one scattered fraction, whatever its own colour.

The figure separates two errors that are easy to run together. The first is the error already described: every star is divided by the flat’s scattered fraction, which it never had. That error depends only on where the star lands. The second is the difference between the scattered fraction at the flat’s colour and the scattered fraction at the star’s colour — the difference between what a perfect model of the scattered light would have removed for this star and what the flat removed. That depends on the star’s colour as well as its position, and it is what the figure draws.

Why should that second error matter, when the first is larger? Because of what happens when scattered light is finally modelled and taken out. An instrument team that measures the scattered-light pattern and removes it from the flat has corrected the stars at one colour — whatever colour the measurement was made in. The residual error that remains is the second one, and it is a function of colour.

A colour-dependent error in a magnitude looks exactly like a colour term. A photometric system is calibrated by observing standard stars of a range of colours and fitting the difference between instrumental and catalogue magnitudes as a function of colour; that function is the colour term, and it describes how the instrument’s passband differs from the catalogue’s. A scattered-light residual of fifteen millimagnitudes across a band’s range of colours, at the centre of the field where the standards were put, will be fitted as part of the colour term. It will then be applied to every star at every position, including the edges where the scattered light was negligible — over-correcting them by the amount the centre needed. The calibration has absorbed a positional error into a colour coefficient, and the positional error has been redistributed across the field in a new, colour-dependent pattern.

That is the general form of the trouble with an additive error treated multiplicatively. It does not disappear; it is moved into whatever multiplicative term is fitted next.

Measuring the scattered light with the stars #

The way out is to measure the instrument’s response with point sources — the very thing the flat field gets wrong. A star is the one source whose light contains no scattered component in its own aperture, so a star’s measured brightness at different positions in the field is a direct measurement of the response a star sees, with no assumption about where scattered light went or what colour it had.

The procedure is to observe a field of stars many times, shifting the telescope between exposures so that each star lands at many positions across the detector. A star that is consistently brighter in the corners than at the centre is reporting the pattern the flat introduced. With enough stars at enough positions, the pattern can be solved for, together with each star’s own magnitude — which is not known in advance and does not need to be, since only the differences between positions carry the pattern. The result is called a star flat, and the approach generalises into the self-calibration of whole surveys.

The figure shows the arithmetic. With two positions per star, sixty stars measure an eighteen-millimagnitude pattern to about four millimagnitudes. Each doubling of the positions reduces the error by roughly the square root of two, and thirty-two positions bring it to about half a millimagnitude. The pattern is a handful of numbers and the observations are hundreds, so the noise averages down as fast as data accumulate.

What makes the star flat useful here, rather than merely another method, is its relation to the sky flat. The sky flat measures $R(x)\,(1+s(x))$. The star flat measures $R(x)$ alone on large scales — the response that light from a point source actually experiences. The ratio of the two is the scattered-light pattern. So the combination is not a correction applied blindly; it is a measurement of the scattered light itself, from which the instrument’s stray-light properties can be diagnosed, compared with an optical model, and tracked from season to season as the optics collect dust.

There is a complementary measurement, and it attacks the problem from the other end. Point a bright star just outside the field of view and record what arrives inside it. Every photon that lands on the detector from that star arrived by a path the optical design did not intend, so the image is a map of the instrument’s response to light from that direction. Step the star around the edge of the field and the scattered-light properties of the instrument are mapped directly.

Neither measurement is free. A star flat needs telescope time spent on dithered observations of a dense field, which is time not spent on science; a stray-light map needs bright stars in particular places and good weather. That is why large surveys solve for the star flat from their own overlapping science fields, as an illumination correction derived from the data rather than from separate calibration frames — and why the observing pattern of a survey is chosen in part so that the same stars land in many parts of the camera.

An additive error at the faint end #

Everything so far concerns stars — point sources, measured against a sky that is subtracted. For them the scattered light enters through the flat, as a multiplicative error on the star, and the cure is to measure the response a star sees.

For diffuse light the problem changes character. Measuring the surface brightness of the faint outskirts of a galaxy, of the intracluster light that fills a galaxy cluster, or of a tidal tail drawn out of a galaxy by a passing neighbour means measuring light spread over a large area, at a level far below the sky’s own brightness, after subtracting that sky. Here the scattered light enters a second time, and this time it really is additive: light from bright stars in and around the field, and from the galaxy itself, is spread by the instrument’s scattering into a broad, faint halo that lies on top of everything.

An additive error behaves completely differently from a multiplicative one as the source gets fainter. A multiplicative error of one per cent costs a source one per cent of its light, whatever its brightness. An additive error of one per cent of the sky costs a source an amount that grows in inverse proportion to the source’s own brightness — negligible for something as bright as the sky, comparable to the source for something a hundred times fainter.

The figure draws that inverse relation on logarithmic axes, where it becomes a straight line until the error grows comparable to the source itself and the logarithm of the magnitude bends over. At the sky’s brightness, a 0.3 per cent sky error costs three millimagnitudes; five magnitudes below the sky, which is a factor of a hundred, it costs nearly three tenths of a magnitude; at a thousandth of the sky it is over a magnitude. The faint outskirts of galaxies, intracluster light and stellar streams lie between five and ten magnitudes below the night sky, and at those levels the residual scattered light is not a correction to the measurement — it is a large part of what is measured.

That is why the faintest diffuse structures in the sky have been surrounded by argument over whether they exist. A broad halo around a galaxy, detected at the level of a thousandth of the sky, is exactly what the instrument’s own scattering of that galaxy’s light would produce, at about the same brightness and with about the same shape. Distinguishing the two needs a model of the instrument’s point spread function out to many arcminutes from its centre — the extended wings, carrying a fraction of a per cent of the light, that a photometric aperture never needs and a low-surface-brightness measurement depends on entirely. The telescopes built specifically for this work achieve it by having as few scattering surfaces as possible: unobstructed optics, no secondary mirror in the beam, aggressive anti-reflection coatings, and baffles designed to keep any light from outside the field from reaching the detector at all.

Surface brightness is the quantity that distance cannot change, which is what makes it such a useful probe of how galaxies are built. It is also the quantity most exposed to an additive error, because the faintest surface brightness is the faintest signal, measured against the brightest background, through an instrument whose stray light scales with the brightest thing near the field.

Why the error has to be named as additive #

The lesson of the whole argument is not about stray-light baffling. It is about the algebra of corrections.

A multiplicative error is cured by division, and the cure is exact wherever the error is measured with a source of the right kind. An additive error is cured by subtraction, and dividing by it does not cure it; it rescales it. The flat-field procedure is built on the assumption that every error between the sky and the detector multiplies, and scattered light is the main thing in the optical path for which that assumption is false. So scattered light is not removed by flat-fielding but transformed — taken out of the sky, put into the stars, and handed on to the next correction, which absorbs it into whatever coefficient happens to be fitted there.

That pattern of an error moving from one correction to the next appears in many places. An instrument that adds polarisation to everything it looks at is the same story told in a different quantity: a small term added by the optics, invisible in any single measurement, which must be measured with a source whose own contribution is known before it can be separated from the sky. A faint star measured against a brighter sky is limited by the sky’s noise; a faint diffuse source is limited by the sky’s systematics, of which scattered light is the largest. The defence in every case is the same: measure the term directly, with a source whose relation to it is understood, rather than inferring it from a correction that assumes it away.

It is worth saying plainly what goes wrong when that is not done, because the failure produces published numbers rather than obvious errors. A survey whose stars carry a centrally concentrated scattered-light error will report stars at the centre of every field as fainter than the same stars measured at the edge of an overlapping field. The overlap comparison reveals it if anyone looks. If nobody does, the error enters the catalogue as a pattern repeated on the scale of the camera, and every statistic computed from that catalogue on that scale — counts of galaxies in cells, the clustering of quasars, the inference of a distance from an apparent brightness — inherits a periodic signal at exactly the spatial frequency of the observing pattern. The signal is the telescope’s footprint printed on the sky.

What the figures simplify #

The scattered-light pattern here is a Gaussian bump centred on the field, a stand-in for the broad central glow that ghosts between a filter and a detector window typically produce. Real patterns are more complicated: they can be asymmetric when the optical axis is tilted, they can carry sharp structure from a baffle edge, and they can include a second, displaced ghost image of the pupil. None of that changes the argument, which depends only on the scattered light being present in extended illumination and absent from a point source’s own aperture.

The figures also treat the scattered fraction as a fixed property of the instrument. It is not quite: it depends on how much light falls just outside the field of view, so a flat taken with a bright moon nearby or with dome lights leaking past the screen will carry a different pattern from one taken in darkness. That dependence is another reason why a scattered-light pattern measured once does not stay measured.

The star-flat calculation assumes a radial pattern fitted with a polynomial in the square of the radius. A real star flat fits a two-dimensional surface with many more terms, and the noise in the result rises with the number of terms fitted; the scaling with the number of positions is the same, but the absolute numbers depend on how complex the pattern is allowed to be.

Still open: how faint the faintest light really is #

The measurement this essay ends on is unfinished. Deep images of nearby galaxies, taken with instruments built to minimise scattering, have reported faint extended stellar haloes, thick discs and streams around most of the galaxies examined, down to surface brightnesses below a ten-thousandth of the night sky. Other deep images of some of the same galaxies, with different instruments and different models of the point spread function’s outer wings, have reported less — and the difference between them is of the same order as the scattered-light correction each applied.

The question at stake is how galaxies are assembled. A halo built up from many small accreted satellites should be fainter and more extended than one formed in the galaxy’s own collapse, and the light in it is a direct count of how much of the galaxy arrived from outside. That count lies precisely in the range of surface brightness where an additive error of a fraction of a per cent of the sky is the whole of the signal. Settling it needs an instrument’s scattered light to be measured with stars, out to large distances from their centres and in the colours of the faint light being sought — not inferred from a flat field, which by construction cannot see it.