A square field imaged as a squashed one

Refraction lifts every star towards the zenith, by about a minute of arc at 45 degrees and by more than half a degree at the horizon, and the lift depends on the temperature and pressure of the air along the line of sight. For measuring where a star is on the celestial sphere, that is a serious correction, and a large part of the history of positional astronomy is the history of getting it right.

Most precise astrometry does not measure where a star is on the sphere. It measures where a star is relative to other stars in the same small field, because a relative measurement is far more precise and because the quantities most often wanted — a parallax, a proper motion, the wobble of a star with an unseen companion — are all small displacements relative to the background. The hope is that refraction, common to every star in the field, cancels in the difference. It cancels only in part. What is left is refraction’s variation across the field, and at the precision relative astrometry now reaches, that variation is enormous.

Two reasons a field is squashed #

The displacement pattern has two ingredients, and they come from two different properties of refraction.

The first is that refraction grows towards the horizon. A star at the bottom of the field is lower in the sky than one at the top, so it is lifted more. The lower stars therefore move up more than the upper ones, and the field is compressed vertically — its vertical extent is reduced by the derivative of the refraction with respect to altitude. Near the zenith that derivative is small; refraction there varies as the tangent of the zenith distance, whose derivative is the square of the secant, and it grows rapidly towards the horizon.

The second is geometric. Every star is lifted along its own vertical circle — the great circle through it and the zenith — and the vertical circles through a field’s left and right edges are not parallel. They converge at the zenith. Lifting two stars at the same altitude but different azimuths moves them both towards that point of convergence and so moves them towards each other. The field is compressed horizontally, by an amount equal to the refraction itself times the tangent of the altitude.

The two behave very differently. The horizontal compression is almost constant at about 280 parts per million from the zenith to seventy degrees, because it is the product of a refraction that grows as the tangent of the zenith distance and a convergence that shrinks as the tangent of the altitude, and the two nearly cancel. The vertical compression is the same at the zenith and grows steeply away from it. At 45 degrees they are about 560 and 280 parts per million; at 70 degrees, about 2,400 and 290.

A part per million of a degree is 3.6 milliarcseconds. A one-degree field at 45 degrees is therefore distorted by about two arcseconds vertically and one horizontally from edge to edge. The plate scale — the number of arcseconds per millimetre on the detector — is not a constant of the telescope but a function of where it is pointing, and it is different in the two directions.

Where the distortion stops being small #

The same calculation near the zenith and far from it shows why ground-based astrometry prefers to work high in the sky.

At 45 degrees the compression is a quarter of its size at seventy, and it is closer to uniform: the vertical and horizontal compressions differ by a factor of two rather than eight. A distortion that is a uniform change of scale is harmless in relative astrometry, because every measurement of a field is fitted with a scale factor anyway; only the anisotropy and the non-linearity cost anything. So the useful measure is not the size of the refraction or even of its gradient, but how far the gradient departs from being the same in every direction and at every point — and that departure grows faster towards the horizon than the refraction itself does.

The distortion also changes during an exposure. As the Earth turns, a field rises or sets and its zenith distance changes, and with it the compression. Over an hour, a field at seventy degrees can change its vertical scale by tens of parts per million, which at the edge of a one-degree field moves stars relative to its centre by about a tenth of an arcsecond during the exposure — many times the precision the exposure was taken to reach. The images at the edge of a wide field are smeared by the changing refraction, in a direction that points towards the zenith, and no guiding on a star at the centre can remove it.

The photometric version of the same problem #

The same gradient that distorts positions across a field changes brightnesses across it. The light from a star low in the sky passes through more air and is dimmed more, and a star at the bottom of a field far from the zenith is seen through more air than one at the top. Every published brightness is an extrapolation to zero airmass, and for differential photometry — measuring a star against others on the same frame — the extrapolation is supposed to cancel.

It cancels only for what the atmosphere does to all the stars at once. Across a one-degree field at seventy degrees from the zenith the airmass changes by about 0.14 from top to bottom — five per cent — and with a typical extinction of a fifth of a magnitude per airmass in blue light, the bottom of the field is dimmed by about three hundredths of a magnitude more than the top. For a search for transiting planets, whose signals are a few thousandths of a magnitude, that is a gradient ten times larger than the signal, and it changes through the night as the field rises and sets, and it has exactly the shape the geometric refraction has: largest vertically, growing towards the horizon. And it too has a colour term, since extinction is stronger in the blue, so a blue comparison star and a red target drift apart in brightness as the airmass changes. The positional and photometric versions of the problem are the same atmosphere seen through two different measurements.

The standard remedy, and what it assumes #

The practical response is to fit the distortion rather than to predict it. Every image of an astrometric field is compared with a reference catalogue of stars in it, and a set of plate constants — a scale, a rotation, and terms quadratic and cubic in the position — is fitted to map the detector coordinates onto the catalogue. The refraction’s scale changes and their variation across the field are absorbed into those constants along with the telescope’s own optical distortion.

That works well, and it has a condition. It requires the distortion to be the same for every star in the field, so that one mapping fits them all. For the geometric part of refraction that is true. For the part that depends on wavelength it is not, and that is the part the plate constants cannot absorb.

A colour term in every position #

Refraction depends on wavelength. The atmosphere is a prism as well as a lens: blue light is refracted more than red, and a star seen far from the zenith is drawn out into a short vertical spectrum. Through a filter, the position recorded for a star is the centroid of that spectrum weighted by the star’s own light, so it depends on the star’s colour. A blue star and a red star at the same true position in the same field are imaged at different positions.

The effective wavelength of a star through a broad filter shifts by tens of nanometres between a hot blue star and a cool red one, and at 45 degrees from the zenith a shift of fifty nanometres moves one star relative to the other by 174 milliarcseconds. That is larger than the parallax of any star more than six parsecs away, and it is in a direction — towards the zenith — that changes as the field moves across the sky through the night and through the year. Plate constants cannot absorb it, because it is different for every star.

It is called differential colour refraction, and it is the dominant systematic error in ground-based parallaxes. A parallax is measured from the annual back-and-forth motion of a nearby star relative to distant background stars, which are usually redder, being more distant and seen through more dust. If the observations are made at different hour angles in different seasons, the colour term changes direction and size with the seasons too, and it can imitate or cancel part of the parallactic ellipse. The standard strategy is to observe every field as close to the meridian as possible, where the zenith distance is smallest and the direction of the colour term is the same from night to night, and to calibrate the term from the stars’ own measured colours.

Higher sites, narrower filters #

The size of the colour term depends on the air, and so on where the telescope is.

A high site reduces the refraction and its colour dependence in proportion to the atmospheric pressure, which at the altitudes of the major observatories is about sixty to seventy per cent of sea level. That helps by the same factor and no more. Narrower filters help more directly, because the spread of effective wavelengths between blue and red stars shrinks with the filter’s width; an astrometric programme using a narrow red filter reduces the colour term several-fold at the cost of throughput. And an atmospheric dispersion corrector — a pair of counter-rotating prisms that introduces an equal and opposite dispersion — removes most of it, but introduces its own small distortions that have to be calibrated.

The correction also has to know each star’s colour in the first place, and that is itself a measurement with an error. A parallax programme that measures colours to a few hundredths of a magnitude converts that uncertainty, through the slope of effective wavelength against colour, into an uncertainty of a few nanometres in each star’s effective wavelength — and at 45 degrees a few nanometres is ten milliarcseconds of position. The astrometric error budget therefore contains a photometric term, and the two measurements have to be planned together: a programme that skimps on colours pays for it in parallaxes.

What none of these do is remove the colour term altogether. At the precision of the best ground-based relative astrometry, a few hundred microarcseconds, even a residual of a few per cent of the uncorrected term matters, and the calibration of each star’s colour becomes as important as the measurement of its position.

What only space avoids #

Everything in this essay is a property of the air, and it vanishes above it. A telescope in space has no refraction, no colour term from the atmosphere and no distortion that changes with pointing, which is one of the two reasons that astrometry moved to space — the other being the absence of the atmosphere’s turbulence, which blurs images and moves them randomly from moment to moment.

Radio astrometry has its own version too. An angle measured against a quasar with interferometers spanning continents is limited not by optical refraction but by the delay the troposphere adds to the radio signal at each antenna — mostly from water vapour, which varies on the timescale of weather and differs between antennas thousands of kilometres apart. The delay’s gradient across the sky is the radio counterpart of the refraction gradient here, and it is modelled by fitting it along with the positions, which is the same strategy as the plate constants and fails in the same way when the atmosphere is not smooth.

A space telescope has its own versions of both problems. Its optics have a colour-dependent distortion, since a lens or mirror system images different wavelengths at slightly different places, and the corrections for a space astrometry mission include a chromatic term calibrated star by star from each star’s measured spectrum — the same kind of calibration a ground-based programme needs for the atmosphere, for a smaller and more stable effect. And its field is distorted by the aberration of its own motion, which changes the apparent scale of a field by up to a part in ten thousand as the spacecraft’s velocity changes direction. Relative astrometry is never free of the observer’s circumstances; it is only a question of which circumstances, and how stably they can be modelled.

How the refraction was measured in the first place #

A correction this large, applied to every position ever measured from the ground, had to be measured itself, and the way it was measured is instructive because it needs no knowledge of the atmosphere at all.

A star close enough to the celestial pole never sets. It crosses the meridian twice a day, once above the pole and once below it, and at the two crossings its true altitudes are the pole’s altitude plus and minus its polar distance — two altitudes whose average is the pole’s altitude exactly, by geometry. The observed altitudes are both lifted by refraction, the lower one more, so their average is not the pole’s altitude. The difference is a direct measurement of the refraction at the two altitudes, taken from nothing but the symmetry of a star’s daily circle. Observing many circumpolar stars at many polar distances builds up the refraction as a function of altitude, and nineteenth-century refraction tables were constructed this way, with corrections for the temperature and pressure read at the telescope.

The same method measures the gradient, which is all a relative measurement needs. And it contains a warning: the refraction measured is the refraction on the nights and at the altitudes observed, at one site. Every refraction formula in use, including the one behind these figures, is a fit to such measurements and to a model atmosphere, and its accuracy near the horizon is limited by how representative those nights were.

What the figures leave out #

The figures use a formula for refraction that assumes a standard, horizontally uniform atmosphere. Real refraction varies with the temperature and pressure at the telescope, with the humidity, and with the structure of the air along the line of sight, and near the horizon it varies unpredictably. For the differential quantities drawn here, which depend on the refraction’s gradient across a degree or less, the standard formula is good to a few per cent above thirty degrees altitude and progressively worse below.

They also treat the atmosphere as static. The seeing — the rapid, random refraction by turbulent cells — adds a displacement to every star that varies over seconds and is correlated between stars close together on the sky but not between stars far apart. Over a wide field, averaging over an exposure reduces it but leaves a residual that behaves like a random, time-varying distortion of the field on scales of arcminutes. That residual, rather than the smooth refraction, sets the floor for relative astrometry over wide fields from the ground.

Still open: how far the ground can be pushed #

Ground-based relative astrometry now reaches a few hundred microarcseconds over narrow fields with large telescopes and adaptive optics, which is good enough to measure the parallaxes of faint objects that space astrometry cannot reach and the orbits of stars around the Galaxy’s central black hole. Each factor of improvement has come from modelling the atmosphere better rather than from escaping it — the differential refraction and the colour term to a fraction of a per cent, the turbulence by averaging many short exposures. Whether the same approach can reach the tens of microarcseconds that would make ground-based parallaxes competitive with space for faint stars depends on how well the refraction’s colour dependence can be calibrated for each star — which is, in the end, a question about how well each star’s spectrum is known.