A bend fixed by the air at the telescope

A star’s light reaches a telescope after crossing the whole of the atmosphere, and the atmosphere bends it. Near the zenith the bend is nothing; at forty-five degrees it is about a minute of arc; at the horizon it is more than half a degree, enough that the Sun is watched setting after it has already gone. Every catalogue position measured from the ground has had this bend removed, and the removal has to be good to a fraction of an arcsecond for the catalogue to be worth anything.

The obvious difficulty is that the bend is made along fifty kilometres of air whose temperature at every height is unknown at the moment of observation. A radiosonde is launched twice a day from a few hundred sites on Earth, and almost none of them is beside an observatory. If the refraction depended on the temperature structure of the air the way most quantities in the atmosphere do, every position measured from the ground would carry an error of the size of that ignorance.

It does not, and the reason is one of the cleaner results in positional astronomy. Down to about fifteen degrees above the horizon, the refraction depends on the state of the air at the telescope and on almost nothing else. Two numbers read on the dome — the temperature and the pressure — fix it to a few thousandths of an arcsecond, however the air above is layered. This essay traces rays through four atmospheres that agree at the ground and disagree everywhere above it, shows how little of the disagreement reaches the refraction, and explains the two separate reasons why. The first is a piece of geometry that holds for any layered medium. The second is hydrostatics, and it is the surprising one.

Four atmospheres that agree at the ground #

The test is to construct atmospheres that are as different as possible above the observer while sharing the same surface reading.

The four are not equally realistic, and that is deliberate. The standard atmosphere is the profile every refraction table assumes: temperature falling by six and a half degrees a kilometre to a tropopause near eleven kilometres and constant above. The isothermal atmosphere is physically absurd — no real air stays at ten degrees Celsius up to the stratosphere — and it is included as an extreme, sixty-six degrees warmer than the standard at the tropopause. The inversion is the air of a clear night over snow or a high desert, where the ground radiates its heat to the sky and the lowest few hundred metres end up colder than the air above them. The hot-ground profile is the air of a sunlit afternoon over rock, cooling steeply in the first hundred metres before settling to the normal lapse.

Each atmosphere is built the way the real one is: starting from the surface pressure and integrating upward under hydrostatic balance, so that the pressure at each height is the weight of the air above it. The density follows from the pressure and temperature by the gas law, and the refractive index follows from the density — for dry air in visible light, $n - 1$ is very nearly proportional to density, about $2.8 \times 10^{-4}$ at the surface. Because the four share their surface temperature and pressure, they share their surface density and therefore their surface refractive index exactly. Above the first metre they have nothing in common.

One refraction until the last few degrees #

A ray through a spherically layered atmosphere obeys a single invariant, the spherical form of Snell’s law: the product $n\,r\sin z$ of the refractive index, the distance from the Earth’s centre and the local zenith angle stays constant along the path. Given the refractive index at every height, that is enough to trace the ray from the telescope out to space, adding up the small deflection it suffers in each layer:

$$R = \int_{r_0}^{\infty} \frac{C}{n\sqrt{n^2 r^2 - C^2}}\left(-\frac{dn}{dr}\right) dr, \qquad C = n_0 r_0 \sin z_0 .$$

The integral is performed here on a grid uniform in the square root of height, which keeps it finite even for a ray launched exactly at the horizon, where the path runs parallel to the layers.

The four curves are drawn in four colours, and from the zenith to beyond eighty degrees they are a single line. A temperature difference of sixty-six degrees at ten kilometres makes no visible difference to how much a star at sixty degrees is lifted. Only in the last ten degrees above the horizon do the curves come apart, and there they come apart violently: the horizon refraction ranges from twenty-nine arcminutes over the hot ground to fifty-one over the inversion, a spread nearly as large as the Sun’s diameter. The standard atmosphere gives thirty-four, which is the figure tabulated in every almanac.

How small the agreement really is #

A curve drawn in arcminutes hides everything below a few arcseconds, which is where astrometry lives. The difference between atmospheres, on a logarithmic scale, shows how the agreement degrades.

At sixty degrees from the zenith, the most extreme pair of atmospheres disagrees by less than a thousandth of an arcsecond. That is the size of the parallax of a star a kiloparsec away, in an atmosphere chosen to be as wrong as possible. At seventy degrees the disagreement is seven thousandths, still smaller than the precision of most ground-based astrometry; at eighty it reaches a sixth of an arcsecond; and from there it grows by roughly a factor of ten for every five degrees, until at the horizon it is measured in arcminutes. The profile of the atmosphere is not a small correction anywhere near the horizon. It is a negligible one everywhere above about seventy-five degrees from it.

That is a strong statement, and it would be suspicious if it had no explanation. It has two, one for each of the first two terms of the refraction expanded in powers of $\tan z$.

The first term: a flat atmosphere forgets its layers #

Imagine first that the Earth were flat, so that the layers of the atmosphere were parallel planes. Snell’s law across each boundary says that $n \sin z$ is the same on both sides, and applying it boundary after boundary from the telescope to the top of the atmosphere gives $n_0 \sin z_0 = 1 \cdot \sin z_\infty$: the direction outside the atmosphere is fixed by the refractive index at the bottom and nothing else. Every layer in between bends the ray, and every bend is undone in the accounting, because what one boundary takes the next gives back in exactly the proportion the invariant requires.

So for a flat atmosphere the refraction is exactly $R = \arcsin(n_0 \sin z_0) - z_0$, which for small $n_0 - 1$ is

$$R \approx (n_0 - 1)\tan z_0 .$$

There is nothing approximate about the independence from the profile; it is exact for any stratified slab. The approximation is in the flatness.

Dividing the traced refraction by $\tan z$ shows how close to flat the real atmosphere is. Near the zenith the ratio sits on the flat-Earth value, $58.25$ arcseconds for these surface conditions. It begins to fall measurably only beyond sixty degrees, and the four atmospheres fall together: the curvature correction itself is the same for all of them to well beyond seventy-five degrees. A slab of air fifty kilometres thick wrapped round a ball six thousand kilometres in radius is very nearly flat for any ray that crosses it steeply, and the departure from flatness is controlled by a single length, which is the next part of the story.

The second term: the weight of the column #

The curvature of the Earth enters because a ray crossing a curved atmosphere meets each layer at a slightly smaller zenith angle than it would in a flat one; by the time it has climbed a height $h$, the local vertical has turned by about $h/r_0$ relative to the vertical at the telescope. How much that matters depends on the heights at which the bending happens, weighted by how much bending happens there. The bending in a layer is proportional to the change of refractive index across it, which is proportional to the change of density, so the weighting is by density. Expanding the integral to the next order gives

$$R \approx (n_0-1)(1-\beta)\tan z - (n_0-1)\left(\beta - \tfrac{n_0-1}{2}\right)\tan^3 z, \qquad \beta = \frac{H}{r_0},$$

where $H$ is the height of the homogeneous atmosphere: the thickness the whole column of air would have if it were compressed everywhere to the density at the ground,

$$H = \frac{1}{\rho_0}\int_0^\infty \rho\, dh .$$

On its face, $H$ depends on the whole profile; it is an integral of the density at every height. This is where hydrostatics does something unexpected. The integral of the density over the column is the mass of the column per unit area, and the weight of that mass is exactly what the barometer at the bottom reads. So $\int \rho\,dh = P_0/g$, and

$$H = \frac{P_0}{\rho_0 g} = \frac{k T_0}{\mu m_{\rm H} g} ,$$

a quantity fixed entirely by the surface temperature. However the air above is layered, the column weighs what the barometer says it weighs, and once its density at the bottom is known its compressed thickness is known too.

The figure shows it happening. The isothermal atmosphere holds its density higher and longer and fills its column most slowly; the inversion and the standard atmosphere fill theirs a little faster; all four arrive at 8.28 kilometres. The only property of the air above the telescope that enters the second term is a property the air at the telescope already knows. The first two terms of the refraction are both fixed by the surface state, and the profile enters only at the third — at $\tan^5 z$, multiplied by a coefficient that involves the first moment of the density, $\int \rho\,h\,dh$, which does depend on the layering, and which is why the curves part company where $\tan^5 z$ becomes large.

The result is usually credited to Barnaba Oriani, who showed in the 1780s that the first terms of the refraction series depend only on conditions at the observer, and to Laplace, who gave the treatment its complete form in the fourth volume of the Mécanique céleste. The practical consequence was understood at once. A refraction table does not need a model of the atmosphere for most of the sky. It needs a thermometer and a barometer, and the tables of the nineteenth century were constructed accordingly — a mean refraction for standard conditions, with multiplying factors for the temperature and pressure read at the instrument.

A degree at the thermometer #

If the profile hardly matters, what does? The surface reading, and it matters far more than the whole of the atmosphere above it. The refractivity $n_0 - 1$ is proportional to the surface density, which is proportional to $P_0/T_0$, so an error of one degree in the temperature is an error of about one part in 283 in the refraction — a third of a per cent. An error of one hectopascal in the pressure is one part in a thousand.

At sixty degrees from the zenith a one-degree error in the thermometer moves the computed refraction by a third of an arcsecond, more than four hundred times the spread between the four atmospheres. The air sixty-six degrees warmer at ten kilometres matters less than the reading on a single instrument being slightly off. The two kinds of error cross only at about eighty-five degrees, five degrees above the horizon, and below that the profile wins.

This changes where care should go. A meridian circle of the nineteenth century, measuring declinations to a tenth of an arcsecond, needed its thermometer to be accurate and, just as importantly, to measure the right air: the air in the dome, warmed by the building and the observer, can differ by a degree or two from the air outside, and the refraction is set by the layer the light actually crosses last. Dome seeing and dome refraction were the observatory’s own atmosphere, and they were treated with more care than the stratosphere because they mattered more.

There is a subtlety the surface-state result does not remove. The air immediately around the telescope is not the air whose pressure and temperature appear in the formula if there is a strong gradient in the first few metres — a layer warmed by the ground in daytime or cooled by it at night. A ray at forty-five degrees crosses that layer in a few metres and suffers a bend proportional to the change of refractive index across it, which a thermometer placed at the wrong height will misreport. The invariant says the profile is irrelevant; it does not say which point of the profile is the surface. That is a matter of where the instrument is put.

Where the cancellation fails #

Towards the horizon the series in $\tan z$ stops converging, the ray runs nearly parallel to the layers for a long distance, and what it meets in that distance is the structure of the lowest air. A ray leaving the telescope horizontally spends tens of kilometres within a few hundred metres of the ground, and the refraction it suffers is dominated by the vertical gradient of density there — which is set by the temperature gradient, and the temperature gradient near the ground is the most variable quantity in meteorology.

Hold the thermometer and barometer at the telescope fixed and change only how the temperature varies in the first three hundred metres. The horizon refraction runs from thirty-one arcminutes when that layer cools with height to fifty-one with a fifteen-degree inversion, and to nearly ninety with a thirty-degree inversion of the kind found over polar ice. None of that variation is visible to the instruments at the observer, and none of it changes the refraction at forty-five degrees by a thousandth of an arcsecond.

So the refraction table splits into two different kinds of object. Above fifteen degrees of altitude it is a physical law with two measured inputs. Below about five degrees it is a statistical statement about the weather, and the tabulated thirty-four arcminutes at the horizon is not a measurement of anything — it is the value for one model atmosphere, and the shape of a setting Sun on any particular evening reports the real one. The same lowest layer, steepened past a critical gradient, bends light round the curve of the Earth altogether; that threshold is where the series fails absolutely rather than slowly.

The same theorem at radio wavelengths #

The cancellation is not special to optical light. A radio signal crossing the troposphere is delayed rather than visibly bent, and spacecraft tracking and geodesy have to correct for a delay of about two and a half metres at the zenith that grows more than tenfold towards the horizon. The dry part of that zenith delay is proportional to the surface pressure and nothing else, to a millimetre — and for the same reason. The delay is the integral of $n-1$ along the vertical path, that integral is proportional to the column mass, and the column mass is the surface pressure divided by gravity.

It is the same hydrostatic fact appearing in a second instrument. The optical astronomer’s curvature term and the radio navigator’s zenith delay are both a measurement of the weight of the air, made by a barometer, and both stop being sufficient at the same place — near the horizon, where the path stops being a measurement of the column and becomes a measurement of the lowest layer. In radio work the water vapour breaks the cancellation earlier, because its contribution to the refractivity is not proportional to the dry density and its column is not set by any surface reading; there is no hygrometer equivalent of the barometer, because water does not obey hydrostatics as a separate gas. In visible light water vapour’s refractivity is close enough to that of the dry air it displaces that the optical result survives.

The general point is that a measurement made through an atmosphere can be insensitive to the atmosphere’s structure for one of two reasons: because a conserved quantity makes the intermediate layers cancel, as Snell’s invariant does for the flat term, or because the quantity being measured is an integral that hydrostatics pins to a boundary value, as the column mass does for the curvature term. Both apply to the twinkling of stars in the opposite sense — there the measurement weights the high turbulent layers and ignores the ground, because what matters is a fluctuation, not a mean — and the contrast is a reminder that “the atmosphere” is not one nuisance but several, each sampled by a different moment of its profile.

Why the result was needed, and what it bought #

Before Oriani and Laplace, refraction tables were empirical: values measured at a few zenith angles at one observatory and interpolated. Tycho’s tables used different refractions for the Sun, the Moon and the stars, because his values were contaminated by errors in the solar parallax; Cassini’s assumed a homogeneous atmosphere of fixed height and fitted that height to observations; Bradley’s, which served for most of the eighteenth century, were a fit to his own measurements at Greenwich. Each worked for its observatory and its climate and transferred badly.

The theorem turned a fitted table into a portable one. If the first two terms depend only on the surface state, a table computed for standard conditions and scaled by the local thermometer and barometer is good anywhere, at any altitude and in any season, down to about fifteen degrees above the horizon. The coefficients could then be measured once, very well, by the method of circumpolar stars observed above and below the pole, and applied everywhere. Bessel’s refraction tables of 1818 and their descendants, Pulkovo’s in the nineteenth century, are of this form, and the positional catalogues of that century — the ones in which the first stellar parallaxes were found — rest on it.

The result also explains why ground-based astrometry learned to stay high. The fixed rule of observing near the meridian and avoiding low altitudes is often justified by the extinction and the seeing, which do worsen towards the horizon. But it is equally justified by the refraction: above fifteen degrees of altitude the correction is a law, and below it the correction is a guess. Differential colour refraction inherits the same structure, since it is the derivative of the same quantity with respect to wavelength, and so does the airmass used in photometry, which is the path length through the same column and departs from the secant law by the same curvature term.

What the traces leave out #

The four atmospheres are dry, horizontally uniform and spherically symmetric. Real air has water vapour, whose refractivity per molecule differs from that of the dry air it displaces and whose distribution is not hydrostatic in the same sense; it adds a small profile-dependent term that the surface reading cannot fully remove. Real air also has horizontal gradients — a front, a coastline, a mountain — which tilt the layers and add a refraction in azimuth as well as in altitude; the theorem assumes layers parallel to the ground, and a tilt of a few hundredths of a degree in the isopycnic surfaces near the telescope produces shifts of a few hundredths of an arcsecond that no surface reading can see. These are the residuals that limit the best modern ground-based astrometry.

The gravitational acceleration is held constant with height and the refractivity is treated as strictly proportional to density, with no dependence on wavelength. Neither assumption touches the argument: a gravity falling with height changes $H$ by a fraction of a per cent and changes all four atmospheres alike, and the dispersion of air multiplies the whole refraction curve by a factor that depends on colour without changing its shape.

And the profiles were chosen to be extreme rather than typical. Real atmospheres sharing a surface state differ from each other far less than these four do, which only strengthens the conclusion.

Still open: whether the low sky can be calibrated #

The theorem leaves a clear division of the sky: a cap within about seventy-five degrees of the zenith where refraction is a solved problem, and a ring near the horizon where it is a property of the weather. Most astrometry has simply stayed out of the ring. Surveys that must observe low — satellites tracked across the whole sky, near-Earth asteroids found in twilight close to the Sun, occultations that happen where they happen — have to work in it, and there the correction is only as good as the model of the lowest air.

What could make the ring usable is measurement rather than modelling: a profile of the lowest few hundred metres taken at the moment of observation, by a tethered sensor, a temperature-sensing laser or the refraction of reference stars low in the same direction, whose catalogue positions are known. Each samples the gradient that the surface reading cannot. Whether any of them can bring refraction at five degrees of altitude down to the tenth of an arcsecond that the cap delivers routinely — so that a quarter of the sky that astrometry abandons could be reclaimed — has not been demonstrated, and the theorem is precise about why it is hard: below fifteen degrees, nearly all the information is in air that the observatory’s instruments do not touch.