A calibration chosen by the measurement it calibrates

Every distance beyond a few kiloparsecs is a brightness turned into a distance, and every brightness is turned into a distance through an absolute magnitude that was calibrated somewhere. The calibration is made on stars of the same kind close enough for the triangle to reach them: measure their parallaxes, convert each to a distance, compare the distance with the apparent magnitude, and average. The answer is the zero point that every later distance, out to the Cepheids in other galaxies and beyond, inherits.

Any such calibration has to choose which stars to use. A parallax measured to a fifth of its own size gives a distance good to about twenty per cent; one measured to its own size gives almost nothing. So the habit is to keep the stars whose parallaxes are “good” — whose measured parallax is at least five, or ten, times its quoted error — and leave the rest out. It seems the most conservative thing to do. It is a selection on the measured parallax, made in order to calibrate a relation that is then tested against the measured parallax, and that circle is the subject here.

The stars a precision cut keeps #

The figures use a simulated sample, so that the right answer is known. A population of stars of one kind — a standard candle of absolute magnitude 0.5, with an intrinsic scatter of 0.15 magnitudes from star to star — is spread uniformly through a disc 15 kiloparsecs in radius and 300 parsecs thick, centred on the Sun, and catalogued down to apparent magnitude 16. Each star’s parallax is then measured with the error a modern astrometric catalogue quotes at its brightness: about 25 microarcseconds at magnitude 15, more for fainter stars, and never less than 15. The measured parallax is the true one plus that error, and it can come out negative.

At bright magnitudes the cut does nothing. Every star there is near, its parallax is large, and the error is a tiny fraction of it; all of them pass. Fainter than about magnitude 14 the stars are further away and their parallaxes smaller, while their errors grow, and the dashed curve rises into the cloud of points. By magnitude 15.5 it runs through the middle of it.

What happens there is the whole problem. A star at the faint end whose parallax error happened to be positive measured a larger parallax than its true one, and is kept; an identical star whose error happened to be negative is dropped. The stars that survive are disproportionately those that measured themselves as nearer than they are. Inverting their parallaxes puts them too close, makes them too faint for their apparent magnitude, and calibrates the candle as fainter than it is.

The cut keeps 41 per cent of the sample here and drops nearly everything beyond 8 kiloparsecs, and the 45 stars whose parallax came out negative are dropped by any cut at all. No inversion can use those: a negative distance is not a distance. They are stars like any other, at the far end of the sample, whose errors happened to exceed their parallaxes. Leaving them out is a selection in the same direction as the cut.

The answer depends on where the line is drawn #

Moving the cut moves the answer, and not monotonically.

With almost no cut — every star whose parallax is at least half its error — the inversion biases the answer the other way, by 0.07 magnitudes too bright. That is the effect described for the inversion of a single noisy parallax: the reciprocal of a noisy quantity is skewed, and for stars at low ϖ/σ the long tail of the reciprocal dominates. As the threshold rises past one, the cut begins to select stars with upward errors and the bias swings positive, peaking at 0.15 magnitudes for a threshold of 3. It falls again as the threshold rises, because a strict cut keeps only stars so near that their errors are tiny and neither effect matters.

A strict cut therefore does work, after a fashion. At a threshold of 10, the bias is gone. So is 79 per cent of the sample, and the part that is gone is precisely the distant part. A calibration intended for use at large distances is then calibrated entirely on near stars. If the candle’s luminosity depends at all on anything that varies with distance in the sample — age, metallicity, position in the Galaxy, extinction — the strict cut has traded a bias of the method for a bias of the sample. And the statistical precision falls with the number of stars kept.

The size of the bias at moderate thresholds is not small by the standard of what calibrations are now asked to deliver. The present disagreement between the two leading measurements of the expansion rate is about 0.17 magnitudes in the distance scale. A calibration biased by 0.06 magnitudes is a third of that, from a choice that is usually made in one line and rarely reported as a choice.

A per-star correction is only as good as its prior #

The standard remedy, since Lutz and Kelker described the bias in 1973, is to correct each star’s distance rather than invert its parallax. The modern form of that remedy is Bayesian: for each star, combine the likelihood of its measured parallax with a prior on where stars like it are likely to be, and use the posterior distance. For a single star that is exactly right, provided the prior is right. The question is which prior.

A common generic choice is an exponentially decreasing space density: the number of stars rises with the volume of each distance shell and falls exponentially beyond a length $L$. One of the first catalogues of Bayesian distances to the stars of an astrometric survey used, among its priors, a single length of 1.35 kiloparsecs for the whole sky. Applied to this sample, that prior calibrates the candle 0.32 magnitudes too faint using every star — more than twice the worst bias of the cut, and five times the bias of the common cut at 5. The sample’s stars lie mostly between 5 and 12 kiloparsecs, and the prior insists they are near. For a star whose parallax is uncertain, the prior wins, and the star is moved inward.

With a longer length the correction gets better, crosses zero at about 3 kiloparsecs, and then goes wrong in the other direction: at 8 kiloparsecs the candle is calibrated half a magnitude too bright. Applying the correction only to stars that pass a cut at ϖ/σ = 5 makes everything less sensitive, because the stars kept have strong parallaxes that override any prior. But it reintroduces the cut, and at 1.35 kiloparsecs the result is still 0.08 magnitudes too faint.

The prior did what it was asked. It was asked a question about a different sample. The later distance catalogues recognised this and let the length vary across the sky, taken from a model of the Galaxy, so that the prior for a star towards the Galactic centre differs from the prior for a star towards the pole. That is a better prior for a typical star in each direction. It is still not the prior for this sample, because the stars in a calibration sample are not typical. They are the stars of one kind that are bright enough to have been catalogued.

The prior depends on the luminosity being measured #

Which distances a sample’s stars lie at depends on how luminous they are.

A magnitude limit means that a star is in the sample only if it is within the distance at which its luminosity makes it brighter than the limit. For a candle of absolute magnitude 0.5 and a limit of 16, that distance is 12.6 kiloparsecs; for a candle 0.3 magnitudes brighter, 14.5. So the brighter the candle really is, the further out its sample reaches, and the more of its stars lie at large distances. The distribution of distances in the sample — which is the prior a correct per-star correction would need — is a function of the absolute magnitude.

That is the circularity in its plainest form. To correct each star’s distance correctly, the calibration must already know the luminosity it is trying to measure. A per-star correction with any fixed prior is, in effect, an assumption about the answer, and the result will be pulled towards whatever that assumption implies. For a sample this deep, the pull can be several tenths of a magnitude.

There is no way out of the circle star by star, because no single star’s measurement contains the information. There is a way out by solving for the luminosity and the distribution of distances together, so that each is consistent with the other — and the simplest version of that does not need a distance for any star at all.

Predicting the parallax instead of inverting it #

A parallax has a symmetric, nearly Gaussian error. A distance does not. So the honest comparison between a calibration and the data is made in the quantity that has the honest error: predict each star’s parallax from its apparent magnitude and a trial absolute magnitude, compare the prediction with the measurement, and adjust the trial value until the predictions fit. For a star of apparent magnitude $m$ and absolute magnitude $M$, the predicted parallax in milliarcseconds is

$$\varpi = 10^{(M - m + 10)/5} .$$

Nothing is inverted, and every star can be used. A star with a negative measured parallax is a star whose prediction is small and positive and whose measurement came out below it, which is exactly the information a fit needs about how large the errors are and where the faint stars really are.

The full fit lands at 0.497, within its own uncertainty of the true value. It uses all 8,000 stars and needs no prior for any star’s distance. Removing the 45 negative parallaxes changes almost nothing here — they are few — but in a sample dominated by stars at low ϖ/σ they would be a large share of the information about its far end.

Two lessons sit in the other curves. The first is that the parallax-space fit does not remove the bias of the cut. Applied only to stars with ϖ/σ above 5, it finds 0.527, too faint by 0.03 magnitudes, because the cut still keeps the stars whose errors went upward, and a fit that takes their measured parallaxes at face value inherits that. Working in parallax space removes the problem of inversion. Only keeping every star removes the problem of selection.

The second is that one bias does remain in the simplest version of the fit, and it has nothing to do with the parallaxes. At a given apparent magnitude, a star of the candle’s kind could be intrinsically slightly brighter and further away, or slightly fainter and nearer, and there are more stars to be had further away, because there is more space there. So the stars of a given apparent magnitude are, on average, slightly brighter than the candle’s mean. That is the Malmquist bias of a magnitude-limited sample, here for a scatter of 0.15 magnitudes: the fit that ignores it finds 0.484, too bright by 0.016. Weighting each possible luminosity by the number of stars at the corresponding distance removes it.

That weighting does need the spread of the population through space — the disc’s size and thickness in this simulation, a model of the Galaxy in practice. But it enters differently from a per-star prior. It is one description of the population, the same whatever luminosity is being tried; it is reweighted against each trial value rather than fixed in advance; and its effect is proportional to the square of the candle’s scatter, which for a good candle is small. A mistake in it costs hundredths of a magnitude, where a mistaken per-star prior cost tenths.

The faint stars pin the zero point #

There is a second reason to keep the faint stars, and it concerns the error that averaging cannot remove: the small offset common to every parallax in a catalogue, of the order of tens of microarcseconds, that an astrometric solution leaves behind because a uniform shift in all parallaxes is nearly degenerate with the instrument’s own calibration. A calibration that ignores the offset inherits it. One that fits it needs stars that distinguish it from a change in the candle’s luminosity.

An offset adds the same amount to every parallax. A change in the absolute magnitude multiplies every predicted parallax by the same factor. Over a narrow range of parallax — a sample of near, bright stars, all with parallaxes of a milliarcsecond or more — the two look nearly the same, and the fit cannot separate them: the ellipse for the bright stars is long and tilted, and freeing the offset triples the uncertainty on the calibration. The faint stars have parallaxes of a tenth of a milliarcsecond, where a fixed offset of tens of microarcseconds is a large fraction of the parallax and a change in luminosity is a small one. They break the degeneracy.

So the stars a precision cut throws away are the ones that measure the offset, and the offset is the error a calibration on bright stars cannot escape. Recent calibrations of the Cepheid period–luminosity relation on astrometric parallaxes have fitted the catalogue’s offset alongside the calibration for exactly this reason, and found that the offset appropriate to bright Cepheids differs by some microarcseconds from the one measured on faint quasars, which have no parallax at all — a difference that matters at the Cepheids’ distances, and that a fit with the offset held fixed would have folded silently into the calibration.

A habit older than the catalogues #

The approach is not new. When parallaxes from the first astrometric satellite were used in 1997 to calibrate the Cepheid relation, almost every Cepheid’s parallax was smaller than a few times its error, and any cut on precision would have left a handful of stars. The calibration was therefore made in the parallaxes themselves, weighting each star by its error and keeping the negative ones, for want of any alternative. The same reasoning was formalised afterwards as a general method: transform the calibration, not the data, so that the quantity carrying the error is the one compared.

The calibrations that reached furthest before the satellites made the same choice for a different reason. They used no individual parallax at all, but the statistical parallax of a group of stars, in which the Sun’s motion through the group supplies a baseline and the stars’ own motions are averaged as noise. That is a fit of a whole sample, with a model of how its members move, and it yields a mean luminosity without assigning a distance to any one star. It was never tempted to keep only the best-measured members, because no member was well measured; the method only worked by using every one.

What has changed since is the size of the samples. A calibration now draws on tens of thousands of stars with excellent parallaxes, and with that many it is tempting to throw away the poor ones on the grounds that they add little precision. The figures say that they add little precision and remove a great deal of bias. The good parallaxes are good because of who they belong to, and the sample of good parallaxes is a sample selected on the quantity it is used to test.

The same structure appears wherever a catalogue’s own measurement decides its membership. A sample of planets selected because their signals were detectable describes the instrument as much as the planets. A sample of galaxies selected because they are bright describes the luminosity function at its upper end. In each case the cure is the same: write down how the sample was selected, and fit the selection along with everything else, rather than correcting each member for a selection it did not know about.

What the simulation leaves out #

The disc in the figures is uniform and smooth, and real calibrators are not: they cluster in spiral arms, lie behind patchy dust, and are concentrated towards the Galactic centre in ways a model of the Galaxy describes only roughly. Extinction is the largest omission. It dims distant stars more than near ones, so it adds its own distance-dependent error, and a calibration in the infrared reduces it without removing it. The parallax errors are taken as exactly Gaussian with exactly the quoted size, while real catalogues underestimate their errors for some stars by tens of per cent, which inflates the scatter at the faint end and — through the cut — the bias.

The candle is also simplified to a single absolute magnitude with Gaussian scatter. Real candles have relations — with period, colour or metallicity — whose slopes are fitted along with the zero point, and a selection that preferentially removes faint, distant stars can distort the slope too if the distant stars differ in period or colour from the near ones. None of these changes the direction of the argument; each adds a reason to model the sample rather than trim it.

Still open: a selection nobody wrote down #

The parallax-space fit removes the bias of a selection it knows about. The calibration samples actually in use were often assembled from several earlier catalogues, each with its own magnitude limit, its own completeness and its own reasons for including a star — spectroscopic follow-up chosen for bright targets, variability surveys that reach different depths in different directions, lists cleaned of stars with suspicious astrometry using criteria that themselves depend on the parallax. No single selection function describes such a sample, and no fit can model a selection that was not recorded.

Whether the existing calibrations of the period–luminosity relation, the red-giant branch tip and the red clump carry residual biases of a few hundredths of a magnitude from undocumented selections is not known. The test would be to rebuild a calibration from a sample defined entirely by a single, documented criterion — every star of the kind brighter than a stated limit in a stated region — and compare. The catalogues are now complete enough to make such samples; whether the calibrations rebuilt from them agree with those in use is an open question, and at the precision the expansion rate now requires, the answer matters.