Luck that reaches as far as the quiet air

A telescope a couple of metres across, looking through ordinary air, collects a wavefront corrugated on the scale of ten or twenty centimetres, and its long exposures are blurred to the size that corrugation sets rather than its own aperture. Its short exposures are different from one another. The corrugation changes every few milliseconds, and now and then, by chance, it is nearly flat across the whole aperture; an exposure taken then is nearly as sharp as the mirror allows. Lucky imaging takes thousands of short frames, measures the sharpness of a reference star in each, keeps the best one or ten per cent, and adds them. The probability of a lucky frame falls exponentially with the square of the aperture in units of the corrugation scale, which confines the method to telescopes a few times larger than that scale and to red light, where the scale is larger. Within those limits it produces images close to the diffraction limit with no deformable mirror at all.

The sharpness is measured on one star. The question the method has always had to answer is how far from that star the sharpness extends — how large a field a lucky frame corrects. The standard answer comes from the error budget of adaptive optics, where a correction measured on a guide star degrades with angle from it because the light from a target at a different angle crosses the turbulent layers at different places. The degradation is set by the isoplanatic angle, a few arcseconds in the visible and several times larger in the near infrared, and it depends on how high the turbulence is: turbulence at the ground is crossed at the same place by every line of sight and is common to the whole field; turbulence ten kilometres up is crossed at places that differ by the separation angle times ten kilometres.

The lucky-imaging version of the argument has a twist, and it comes from the fact that the method does not correct anything. It selects. The frames it keeps were chosen because the reference star was sharp in them, and what else was true of the air in those frames is determined by what made the star sharp.

Two layers, and a strength that comes and goes #

The atmosphere drawn here is two layers of Kolmogorov turbulence — one at the ground, one high — each a random phase screen drawn fresh for every frame, as they effectively are for exposures separated by more than a few milliseconds. The reference star’s light crosses both at the centre. A target at angle θ crosses the same ground screen and the high screen shifted by θ times the layer’s height. The frame’s sharpness on each star is the Strehl ratio computed from its own phase error across the aperture, after removing the tilt that merely shifts a short exposure.

One further property is added, and it is the one that matters. A layer’s strength — the amount of turbulence in it, its share of the refractive-index fluctuations integrated along the line of sight — is not constant. Profiles of the turbulence measured night after night with scintillation-based instruments show individual layers strengthening and weakening by factors of two on timescales of seconds to minutes, as eddies of wind shear form and decay. In the model, each layer’s strength is multiplied in each frame by an independent log-normal factor with a width of 0.6 in the natural log, which is a stated assumption about the size of that variation rather than a fit to any site. The frames are then selected on the reference star alone.

Two sets of lucky frames are compared. When the layers’ strengths are steady, the selected frames are sharp on the reference star and degrade with separation as the fixed-layer theory expects: by half an aperture of shift in the high layer — about thirty arcseconds for a two-and-a-half-metre telescope and a layer at eight kilometres — they are no better than an average frame. When the strengths vary, the selected frames are sharper on axis to begin with, and they stay sharp much further out; at the same separation they keep two-thirds of their central Strehl ratio. Most of the difference is a property of the selection, not of the atmosphere’s geometry, and the next figure shows what it selected.

Why a varying atmosphere is luckier on axis too #

The first figure has a second feature that is easy to pass over: the lucky frames are sharper on axis when the layers’ strengths vary, 0.52 against 0.32, with the same average turbulence. The variation itself makes luck more common, and the reason is the shape of the function that turns phase error into sharpness.

The Strehl ratio of a frame is approximately the exponential of minus its phase variance, and the chance of a frame being lucky is governed by how often that variance falls well below its mean. With steady layers, the variance fluctuates only because the random pattern of each layer happens to be flatter or rougher across the aperture — a fluctuation whose size is fixed by the Kolmogorov statistics and which becomes relatively smaller as the aperture grows, which is why the lucky fraction falls as the exponential of the square of the aperture. With varying strengths there is a second source of fluctuation, multiplying the first: a frame in which the whole layer is half as strong as usual has half the variance, whatever its pattern. The low tail of the variance’s distribution is then much heavier, and the exponential turns a heavier low tail into a much larger number of sharp frames.

That is a general property of convex functions averaged over a fluctuating argument, and here it has a practical consequence. Fried’s estimate of the probability of a lucky frame assumes a stationary atmosphere of fixed strength, and it underestimates the rate of lucky frames on nights when the turbulence is intermittent — which, by the profiles measured at real sites, is most nights. The same intermittency that widens the field also raises the yield. Both effects come from the same place: the atmosphere’s strength is itself a random variable, and a method that selects on sharpness selects on it.

What the selection actually chose #

The frames with a sharp reference star are, overwhelmingly, frames in which both layers were weaker than usual. That is not surprising: the phase variance on the star is the sum of the two layers’ contributions, and the easiest way for a sum to be small is for both terms to be small. What follows from it is the important part. The high layer’s pattern is different along the target’s line of sight — the shifted screen has different corrugations — but its strength is the same, because strength is a property of the whole layer in that frame. A target that looks through a quiet high layer in a different place is still looking through a quiet high layer. Selecting on the reference star therefore selects, for the target too, a frame in which the high turbulence was weaker than average, and the target inherits part of the luck through the strength even where it inherits none of it through the pattern.

In the fixed-strength case there is nothing to inherit. The selected frames are those in which the particular pattern of the high layer happened to be flat over the reference’s footprint; a few tens of centimetres further along the layer the pattern is unrelated, and the target sees an average frame. The whole reach of the luck then comes from the ground layer, which is common to all directions.

Frame by frame #

The inheritance can be seen directly by plotting, for each frame, the sharpness on the star against the sharpness on a target a quarter of an aperture’s shift away.

The points cluster along the diagonal with a wide scatter. The correlation is strong — frames sharp on the star are usually sharp beside it — and the scatter is the part of the high layer that is not shared. The lucky frames, to the right of the selection line, have target sharpness spread from nearly as good as the star’s to barely better than average. Adding them gives an image whose centre is sharp and whose field is sharp on average, with a point-spread function that varies from frame to frame across the field in a way the final image does not record. That variation matters for photometry and astrometry of the field stars, which assume a single point-spread function at each position, and it is one reason lucky-imaging photometry of crowded fields has proved harder than its sharpness suggests.

A field as wide as the turbulence is low #

The reach depends on where the turbulence is.

With most of the turbulence at the ground the lucky frames are lucky everywhere: the ground layer is crossed at the same place by every line of sight, so a frame in which it was quiet is quiet for the whole field, and the target keeps nearly all of the reference’s sharpness at thirty arcseconds. With most of it high, the field is narrower, but even then the target keeps more than half, because the strength of the high layer is shared. The fixed-layer theory would put the second number much lower. The difference between the two theories is largest exactly where lucky imaging is most often used — at sites and in seasons where a strong jet-stream layer ten kilometres up contributes much of the seeing, and where the high layer’s strength varies most.

This makes a lucky-imaging run an instrument as well as an image. The fraction of the reference’s sharpness retained at each separation, measured on the field stars of a lucky image, is a function of the share and variability of high turbulence during the run, and a sequence of such runs through a night traces the height of the turbulence as it changes. Instruments built for that purpose — scintillation detection and ranging, multi-aperture scintillation sensors — measure the profile more directly, but they are separate instruments on separate telescopes, and the lucky image carries its own record of the air it was taken through.

What the reach is worth in arcseconds #

The separations in the figures are drawn as the shift of the high layer in units of the aperture, because that is what the physics depends on, and the conversion to angle depends on the telescope and the layer. For a two-and-a-half-metre telescope and a layer at eight kilometres, a shift of half an aperture is thirty-two arcseconds, and the fixed-layer theory’s reach — the separation at which the lucky frames lose four-tenths of their central sharpness — is about thirteen. For a one-metre telescope under the same layer, every angle is two and a half times smaller: the fixed-layer reach is five arcseconds and the extended one perhaps twice that. For a layer at four kilometres instead of eight, every angle doubles.

Those numbers decide what lucky imaging is for. A reach of five arcseconds is enough for close binary stars, whose companions sit within an arcsecond or two of the primary, and for resolving a star from a faint neighbour that has contaminated a transit measurement; it is the regime where the method has been most productive, surveying thousands of planet hosts for close companions. A reach of thirty arcseconds is enough to image the core of a globular cluster or a small galaxy, where the scientific interest is in the field rather than in any one star, and there the difference between the fixed-layer estimate and the extended one is the difference between a sharp spot in the middle of a blurred frame and a sharp frame.

It also decides where the method can be used at all. Every lucky image needs a reference bright enough to measure its sharpness in a few milliseconds — a star of about fifteenth or sixteenth magnitude in the red for a two-metre telescope — within the reach of the target. The fraction of the sky with such a star within a given angle grows as the square of the angle, so a reach twice as large makes the method usable over four times as much of the sky, and at high Galactic latitudes, where bright stars are scarce, that is the difference between a technique for special fields and one for most fields. Measuring a faint target against the sky is limited by photons; measuring it at all, with lucky imaging, is limited first by the chance of a reference star close enough.

Tighter selection, narrower luck #

A lucky-imaging reduction has one free choice: how many frames to keep.

Keeping every frame gives the ordinary short-exposure image with its tilt removed, uniform across the field and poor everywhere. Keeping fewer frames sharpens both the star and the field, but the star faster: the tighter the selection, the more it rewards the particular pattern along the reference’s line of sight, which the field does not share, over the strength, which it does. At a half per cent kept the star is six times sharper than the unselected average and the target five times sharper. The ratio falls from one to four-fifths. That is a choice between a sharper star and a more uniform field, and it is made in every lucky-imaging reduction, usually on the basis of the star alone, because the star is what is measured.

The same logic has a direct counterpart in adaptive optics. A correction applied faster than the air changes is a correction measured on one star and applied everywhere, and the field over which it is good is limited by exactly the unshared part of the turbulence. Lucky imaging and adaptive optics differ in how they exploit the shared part — one waits for it to be small, the other removes it — and the most effective systems combine them, selecting frames from a stream already corrected for the ground layer, so that the only turbulence left to be lucky about is the high part. Interferometers face the same division on a larger scale: the fringes on one star are stabilised by tracking another nearby, and how far away the tracking star can be is set by the same unshared turbulence.

What the model leaves out #

The two-layer atmosphere is the simplest one that distinguishes shared from unshared turbulence. A real profile has several layers, including a strong surface layer a few tens of metres thick, a boundary layer up to a kilometre or two, and one or more high layers associated with the jet stream; each intermediate height has its own reach, and the retained sharpness at a given separation is an average over them. The frames are independent draws, whereas real frames a few milliseconds apart are strongly correlated because the turbulence is carried across the aperture by the wind rather than redrawn. That correlation does not change which frames are lucky, but it means the lucky frames come in bursts, and the effective number of independent frames is smaller than the number taken.

The strength variation is the most important assumption and the least tested. The log-normal width of 0.6 is plausible for high layers on timescales of seconds; if the real variation within the few minutes of a lucky-imaging run is smaller, the effect is correspondingly weaker, and if it is larger, stronger. The model also ignores scintillation — the fluctuation in brightness of each star caused by the high layer’s focusing — which is itself correlated with the high layer’s strength and could be used to select frames in which the high layer was quiet, rather than inferring it from the reference’s sharpness.

Why this was noticed late #

The fixed-layer theory of anisoplanatism was developed for adaptive optics, where it is the right theory. A correction applied in real time is applied to every frame, lucky or not, and its error at an angle from the guide star is an average over frames. Averaged over frames, the strength fluctuation contributes nothing special: the mean variance is the mean variance. The selection effect exists only for a method that chooses a subset. Lucky-imaging observers have reported that on some nights their field stars stayed sharp further from the reference than the conventional isoplanatic angle allows, and conditioning on the reference star is one explanation for it; a ground-dominated night is another, and the two are not easily told apart after the fact. It is the same statistical point as choosing the best aperture for photometry: the quantity that decides what is kept is correlated with quantities that were not used to decide, and the kept set is biased in all of them together.

Still open: whether the high layer can be chosen directly #

The effect drawn here comes from the high layer’s strength being correlated with the reference’s sharpness, and the correlation is imperfect: some kept frames are lucky because the ground layer happened to be quiet, and those carry no benefit for the field beyond what the ground layer already gives. A selection that looked at the high layer directly — through the scintillation of the reference star, which is produced almost entirely by high turbulence, or through the correlation of image motion between two stars in the field — could choose frames for the field rather than for the star, and would give up some central sharpness for a wider corrected region. Whether real high layers vary enough within a run for such a selection to pay, and on which sites and nights it would, is a question the profiles now measured routinely at large observatories could answer, and lucky-imaging runs taken beside them are how it would be tested.