The atmosphere is a prism as well as a lens

Every observer knows that the atmosphere lifts a star: the light is bent as it enters denser air, so the object is seen higher than it is. At forty-five degrees from the zenith the lift is about a minute of arc, and it is corrected for in every astrometric reduction ever performed.

What is less often corrected is that the lift is not the same for every colour. The refractive index of air rises towards the blue, so the blue image sits above the red one, and at moderate airmass the separation is a few arcseconds. That is not a small quantity by the standards of a ground-based telescope: it is several times the size of the image and comparable to the width of a spectrograph slit.

The effect has a name — atmospheric dispersion, or differential refraction — and a reputation for being a nuisance handled by a piece of hardware. It is worth more attention than that, because it is one of the few systematics in ground-based astronomy that is exactly calculable, exactly correctable, and nevertheless present in a great deal of published data.

Where the wavelength dependence comes from #

Air is dispersive for the same reason glass is: its molecules have resonances in the ultraviolet, and the refractive index of a medium rises as the frequency approaches a resonance from below. The standard expression for dry air puts $n - 1$ at about $2.78\times10^{-4}$ at 550 nanometres, rising to $2.86\times10^{-4}$ at 350 and falling to $2.74\times10^{-4}$ at 950.

Those are differences in the fifth decimal place of a quantity that is itself a small departure from one, and they would be of no consequence except that the refraction angle multiplies them by the tangent of the zenith angle and by 206,265 arcseconds per radian. A fractional difference of four per cent in $n-1$ becomes four per cent of a refraction that is already a minute of arc.

The refraction itself, to sufficient accuracy away from the horizon, is

$$R \approx (n-1)\tan z,$$

so the whole wavelength dependence is carried by $n - 1$ and the whole geometry by $\tan z$. That factorisation is the reason the family of curves in the figure is one curve: the shape of the dispersion against wavelength is fixed by the air, and only its amplitude depends on where the telescope is pointing.

The pressure and temperature enter too, and they enter as a simple scaling: $n-1$ is proportional to density. A telescope at four kilometres’ altitude sees about six tenths of the sea-level refraction and six tenths of the dispersion, which is a real advantage of a high site and is not usually the reason such sites are chosen.

One more property of the factorisation deserves attention, because it decides what a correction has to know. The direction of the dispersion is always along the vertical on the sky — towards the zenith — and the vertical on the sky is not a fixed direction in the instrument. It rotates through the night as the object rises and sets, at a rate that is fastest near the meridian for an object passing close to the zenith. That angle is the parallactic angle, and it is a purely geometric quantity computable from the pointing. A corrector therefore needs two numbers: how much dispersion to apply, from the zenith angle, and in which direction, from the parallactic angle. Both are known exactly and neither is measured.

What it does to a spectrum #

The instrument that suffers most is a slit spectrograph, and the way it suffers is worth following carefully because the damage does not look like what it is.

A slit is a rectangle a fraction of an arcsecond wide. The observer centres the star on it — at some wavelength, usually the one the guider works at — and the rest of the star’s light is displaced along the direction of the dispersion. If that direction is not along the slit, light at other wavelengths falls partly outside it.

The recorded spectrum is therefore the star’s spectrum multiplied by a throughput curve that nobody chose and nobody recorded. It rises to a maximum at the guiding wavelength and falls away on both sides — which is to say it looks like a broad absorption feature, or like a reddening, or like a bluing, depending on where the maximum sat.

Flux calibration is supposed to remove such things by dividing by the spectrum of a standard star. It does, exactly, provided the standard was observed at the same airmass, with the same slit orientation, and centred at the same wavelength. In practice one of those three is usually different, and the residual is a smooth multiplicative error of a few tens of per cent across the band.

There is a second, subtler damage that the throughput curve conceals. Because the loss depends on wavelength, the effective wavelength of a broad-band measurement shifts — the recorded light is weighted towards the guiding wavelength — and an effective wavelength that moves with airmass is a colour term that moves with airmass. In photometry that is exactly the shape of an extinction coefficient, so a magnitude extrapolated to zero airmass absorbs part of it and the rest survives into the published number. The two effects have different colour dependences and can in principle be separated; in practice they are usually not, and the residual sits in the photometric zero point.

It is worth knowing which observations are safe. An imaging observation through a narrow filter is unaffected, because the dispersion within the filter is small. An observation at the zenith is unaffected, because the tangent is zero. An observation in the near infrared is much less affected, because the air’s dispersion falls steeply towards longer wavelengths — between one and two and a half microns it is about a fifth of what it is across the optical band. And an observation whose only product is a position measured relative to nearby stars of similar colour is nearly unaffected, because the displacement is common to all of them. Everything else carries the term, and the observations that carry it worst are broad-band spectroscopy of faint objects at high airmass, which is a large fraction of what large telescopes are used for.

The fix, which is another prism #

The correction is mechanical and old. Two counter-rotating prism pairs are placed in the converging beam, each pair made of two glasses with different dispersions cemented so that the deviation at a reference wavelength is zero and the dispersion is not. Rotating one pair relative to the other varies the net dispersion continuously from zero to maximum, and rotating both together sets its direction.

An atmospheric dispersion corrector is therefore a prism whose job is to cancel a prism, and it is set from the zenith angle and the parallactic angle, both of which are known exactly from the pointing.

There is an alternative that costs nothing and is available whenever the instrument permits: align the slit along the direction of the dispersion. That direction is the vertical on the sky, called the parallactic angle, and a slit set to it loses no light at all — the spectrum is smeared along the slit rather than out of it. The cost is that the slit is then not aligned with anything else, which matters when there are two objects to observe at once or when the seeing is elongated.

The choice between the two is not always free. Multi-object spectrographs place many slits across a field at fixed orientations, so aligning every one with the parallactic angle is impossible and a corrector is the only option. Integral-field units have no slit at all and so lose no light, but they still smear each wavelength to a different position in the field, which has to be undone in the reconstruction. The problem does not disappear with the slit; it moves.

What was actually measured #

The effect is not inferred; it is measured routinely, in three different ways.

Direct imaging of a star through filters. Photographing the same field in blue and red at high airmass and measuring the offset between the two positions gives the dispersion directly. The measurements agree with the standard air-refractivity formulae to a few per cent, and the residual is attributable to the water vapour content, which the dry-air formula does not include and which varies.

The throughput curve of a spectrograph. Observing a standard star at a sequence of airmasses and dividing each spectrum by the one taken overhead gives the loss curve empirically. It has the predicted shape, it scales with the tangent of the zenith angle, and it disappears when the corrector is deployed.

Astrometry. A catalogue position depends on the colour of the star it was measured from, because a bluer star is refracted more. Ground-based astrometric catalogues carried colour-dependent systematics of tens of milliarcseconds for exactly this reason, and correcting for them requires knowing each star’s colour — which was not always available and had to be estimated from the star’s own image.

Radial velocities. The effect reaches even measurements that are not about position or flux at all. A spectrograph fed by a fibre samples a different part of the stellar image at each wavelength when the dispersion is uncorrected, and because a star’s surface is not uniform in velocity — convective motions blueshift the centre of the disc relative to the limb — the measured velocity acquires a term that depends on airmass. For the metre-per-second work that finds planets, this is a real error source and is one of the reasons those instruments insist on a corrector and on a scrambler between the telescope and the fibre.

Where the picture stops #

There are three, and the second is the one that decides whether the effect can be ignored.

The simple formula fails near the horizon. $R = (n-1)\tan z$ diverges at $z = 90°$ and the true refraction does not; below about ten degrees of altitude the calculation requires an actual atmospheric profile, and the answer becomes sensitive to the temperature gradient in the lowest hundred metres. That is why the horizon refraction of 34 arcminutes is a convention rather than a measurement, and why the timing of sunrise is uncertain by tens of seconds.

Water vapour is not in the standard formula. Its refractivity differs from dry air’s and its dispersion differs more, particularly in the infrared where it has strong bands. For most optical work the correction is small; for precise infrared astrometry it is not, and it varies on the timescale of an hour.

And the correction is only as good as the pointing model. A corrector set from a zenith angle that is wrong by a degree leaves a residual dispersion of tens of milliarcseconds. That is negligible for spectroscopy and is not negligible for the adaptive-optics systems that are trying to deliver diffraction-limited images, where an uncorrected dispersion elongates the core of the point spread function and destroys the very thing the correction bought.

And there is a fourth limit worth stating because it is the one that will matter most in the coming decade. Everything above is first order in the flattening of the atmosphere and treats the air as a stack of plane-parallel layers with a single temperature and pressure. For the thirty-metre-class telescopes now being built, whose diffraction limit at two microns is about fifteen milliarcseconds, the residual dispersion after an ideal corrector — set by the mismatch between the glasses available and the air’s actual dispersion curve — is itself comparable to the diffraction limit at the blue end of the range. The correction is becoming a design constraint on the glass rather than a mechanism to be bolted on, which is a familiar trajectory: a term that begins as a nuisance ends as a specification.

Why it belongs with refraction rather than with instruments #

It is tempting to file all of this under instrumental effects, and the filing would obscure the point. The dispersion is not an instrumental effect. It is the same physics as the Sun setting before it sets, differentiated with respect to wavelength, and it is present in the light before it reaches any instrument.

What is instrumental is the consequence, and the consequences are different for different instruments in a way that makes the effect easy to overlook. An imager with a broad filter sees a slightly elongated image and loses nothing. A photometric aperture contains a different fraction of the light in each band, so a colour index measured at high airmass is wrong in a way that mimics a colour term. A slit spectrograph loses light selectively. An interferometer sees a wavelength-dependent path difference. Four instruments, one cause, four apparently unrelated symptoms — and in each case the natural diagnosis is something about the instrument.

That is the signature of an effect worth understanding at the level of the physics rather than at the level of the symptom. The same star measured through two telescopes will disagree in a way that depends on how each was pointed, and no amount of cross-calibration between the two will find the cause, because the cause is in neither of them. The disagreement will also correlate with the stars’ colours, which is what makes it look like a bandpass mismatch — and a colour that is being used as a thermometer inherits the error directly.

End on the size of the thing, because the numbers are easy to lose. The total refraction at sixty degrees from the zenith, at a good high site, is about a minute of arc. The dispersion across the optical band at the same place is about three arcseconds — one part in twenty of the total. The seeing at that site is perhaps six tenths of an arcsecond, and the diffraction limit of a ten-metre mirror is one twentieth. So the dispersion is five times the seeing and sixty times the diffraction limit, sitting inside a correction everybody applies, produced by a term everybody knows about. It survives because it is the derivative of a quantity rather than the quantity, and derivatives are what get left out of first-order corrections.

It is worth stating how the effect scales, because that is what decides which observations can ignore it. The differential refraction between two wavelengths is the difference of two tangents of the zenith distance, so it grows without bound towards the horizon while the total refraction is still modest: at thirty degrees of zenith distance the spread across the visible is under half an arcsecond, at sixty it is nearly one, and at seventy-five it is over two. Since a good site delivers images half an arcsecond across, the effect is negligible near the zenith, comparable to the image at sixty degrees, and larger than the image beyond seventy. That is why the same observation is unaffected in the middle of the night and corrupted at the beginning of it, and why an airmass limit rather than a correction is the usual response for programmes that can afford one.

There is a version of the effect that survives even a perfect corrector, and it is worth knowing about: the atmosphere’s dispersion depends on the water vapour content as well as on the pressure and temperature, so the correction computed from a standard atmosphere is slightly wrong on a humid night. The residual is a few milliarcseconds, which is negligible for spectroscopy and is not negligible for astrometry at the precision a large telescope now reaches.

Where the ladder goes next #

The immediate next rung is the part of the atmosphere this essay has treated as static: the same refractive index, fluctuating, which is the seeing and which has its own colour dependence. Above it again sits the infrared, where water vapour dominates the refractivity, varies on minutes, and is measured rather than modelled.