A forecast that fails on a schedule

A transit is only useful if someone is watching when it happens. A planet found by a space telescope is followed up by instruments on the ground and in orbit that book their time months in advance, and every booking rests on a forecast: a transit time, extrapolated from the ones already seen, with an uncertainty attached. For a planet on an undisturbed orbit the forecast is a straight line and its uncertainty grows slowly and honestly, in proportion to how far ahead it reaches.

For a planet whose transits run late and early because a neighbour is pulling on it, the straight line is the wrong model, and its uncertainty is a statement about a planet that does not exist. The forecast does not merely get worse; it fails, on a schedule the neighbour sets, and the size of the failure has nothing to do with how precisely the transits were timed.

The wander the line is fitted to #

The planets in these figures are an ordinary near-resonant pair: the outer orbit takes 1.524 times as long as the inner, just wide of the 3:2 commensurability. Integrated on its own and set against the best straight line through all four years, the inner planet’s transits look like this.

Two features of that curve decide everything that follows. The first is its period. At 317 days, a single super-period is longer than a typical season of ground-based observations and far longer than the few months over which a newly found planet’s ephemeris is first measured. A fit to a short window therefore sees not a sinusoid but a fragment of one, and a fragment of a sinusoid is very well described by a line with the wrong slope.

The second is its amplitude, eleven minutes each way. That is small beside a 10-day period — a hundredth of a per cent — and large beside the half-minute to which a good transit can be timed. The planet’s clock is overwhelmingly regular and not regular enough, which is exactly the combination that makes a straight-line fit look excellent and forecast badly.

Why the error grows and does not wander back #

A straight line fitted to a fragment of a sinusoid has two faults. Its intercept absorbs where on the wave the fragment sat, which does no lasting harm. Its slope absorbs how fast the wave was rising or falling across the fragment, and that is converted into an error in the period.

A period error does not stay put. A transit’s predicted time is the fitted time of the first one plus the fitted period multiplied by the number of orbits since, so an error of a few seconds in the period becomes an error of a few seconds times the number of orbits, and grows without limit. The wander itself swings back after half a super-period; the line does not follow it. That is why the opening figure’s residuals rise steadily through four years, with the super-period’s swing riding on top of a climb, rather than oscillating about zero.

The same shape turns up wherever a period is estimated from a short arc of a longer motion. An orbit determined from a short stretch of observations carries almost all of its uncertainty along the track, because the thing a short arc measures worst is how long the orbit takes, and that error accumulates into position one orbit at a time. The transit ephemeris is the same problem in one dimension: a clock whose rate was measured over too short an interval to average out its own fluctuations.

The size of the period error is modest by any ordinary standard, which is what makes it dangerous. The 120-day fit’s worst forecast error comes some forty orbits after the middle of its window, so the forty-three minutes it accumulates there correspond to a period that is wrong by roughly a minute out of fourteen thousand — a fractional error of less than one part in ten thousand, from a fit whose residuals inside the window were two minutes. A period quoted to six significant figures is being quoted to about its fifth, and the sixth digit is where the neighbour lives.

Nor is the fitted period a poor estimate of anything in particular. It is an excellent estimate of the rate at which the planet’s transits were advancing during those 120 days, which is a real, physical rate: the planet’s orbit at that phase of the exchange really was that long. An ephemeris is a table that is a fit, and a fit reports the model that best describes the data it was given; what it cannot report is how much of the planet’s future lies outside the family of models it was allowed to choose from.

A band that describes a different planet #

The narrow band in the opening figure is not a strawman. It is the uncertainty the fit reports, computed in the standard way from the scatter a timing precision of half a minute would put on each transit, and at the date of the worst error it is ±5.1 minutes. Nothing about that calculation is wrong for the model it assumes, which is a planet whose transits scatter independently about a straight line.

The planet in the figure does not do that. Its departures from a line are correlated from one transit to the next, because they are samples of a smooth wave. A least-squares fit that treats correlated departures as independent noise is overconfident in a specific way: it believes it has as many independent pieces of information as transits, when it has something closer to one piece of information per super-period.

That figure contains a result that is easy to misread. Degrading the timing precision widens the band and leaves the error untouched, so the ratio of error to band improves — from 8.4 to 2.1 — and a naive check of whether the forecast failed “significantly” would find it had failed less. A forecast’s honesty is not a property of its error bar. The band here scales with the noise, the error scales with the planet’s neighbour, and the two are unrelated quantities that happen to be expressed in the same units.

The prudent forecast in practice is not a straight line with an inflated band but a model that contains the perturber. When the neighbour transits too, its period is known, the super-period is known, and a two-planet integration fitted to the transits predicts the wander rather than being surprised by it. Every well-characterised timing system is forecast that way. The difficulty is that the ephemeris is needed first — to schedule the observations that would reveal the wander — and the first ephemeris of any planet is a line.

A longer window buys less than it seems #

The obvious remedy is to wait: fit more of the wander before forecasting.

The forecast improved and its honesty got worse. A window that spans nearly a full super-period averages most of the wave out of the slope, so the period error falls and the extrapolation drifts more slowly. At the same time the fit has twenty-nine transits instead of twelve, spread over a longer lever arm, and its formal uncertainty collapses. The residuals inside the window, twelve minutes against a claimed precision of half a minute, are the warning, and a fit that reports a reduced chi-squared of several hundred is announcing that its error bar is meaningless. The warning is available only if someone looks at the residuals rather than the band.

The general statement is that a linear ephemeris for a planet with timing variations is only as good as the fraction of a super-period it spans, and a window of several super-periods is needed before the line’s slope converges on the planet’s mean period. For a pair near resonance that can be years, and the pairs nearest resonance — with the largest, most easily detected timing variations — have the longest super-periods and so the slowest convergence.

Chains of planets make it worse in a way no single pair does. In a system where four or five planets sit near successive commensurabilities, as they do in a chain that could not have been assembled in place, every adjacent pair contributes its own super-period, and the planets in the middle of the chain feel two at once. Their timing wanders at several incommensurable periods simultaneously, so there is no window length that spans a whole cycle of all of them, and the fitted slope keeps changing as the window grows. The forecasts for such systems are only ever made from full dynamical fits, and those fits have themselves been redone each time new transits arrived, because the masses they return depend on how many beats of each super-period the data contain.

The reverse is also true. A planet with no neighbour near any commensurability has timing variations too small to matter, and its linear ephemeris is exactly as good as its error bar. The forecasts that fail are therefore not scattered at random through a catalogue; they are concentrated in the most dynamically interesting systems, which are the ones follow-up observers most want to catch.

The calendar decides how soon #

Two observers fitting the same planet over windows of the same length, at different times, do not get the same forecast.

Over a whole year the three forecasts fail about equally, because a year is longer than the super-period and each extrapolation meets every phase of the wave. What the fitted phase decides is how soon. A window that caught the wander while it was bending — near the top or bottom of a swing — extrapolates a slope that is already turning wrong, and is off by a quarter of an hour within two months. A window that caught it running nearly straight through its mean extrapolates a slope that stays roughly right for a while, and holds to five minutes over the same two months.

For a follow-up programme the difference is practical. Observations in the two months after an ephemeris is published fall inside a transit window for one of those forecasts and outside it for another, for the same planet with the same timing precision. A missed transit is then commonly attributed to weather or to an instrument, when the cause is the calendar of the original observations.

When the model with the neighbour fails too #

A two-planet model predicts the wander of a two-planet system, and when the system has a third planet the prediction fails in a way that is more informative than any success.

The figure’s two curves have the same cause and different contents. The two-planet residual is the part of a two-planet system’s own signal that the analytic model leaves out — terms of higher order in the masses and the eccentricities — and it has no single period. The three-planet residual is dominated by one period that none of the modelled terms has, and a period that is not in the model is a statement about a body that is not in the model. From that period and the transiting planet’s own, the list of candidates for where the extra body sits can be drawn up, and a velocity search can be pointed at the right periods.

The detail in the caption that the super-period fitted from the data is 305 days, not the 317 days the pair’s starting periods predict, is not an error in either. Orbital elements are osculating — each is the orbit a body would follow if every perturbation vanished at that instant — and a near-resonant pair’s instantaneous periods differ from their long-run averages by a fraction of a per cent. Over four super-periods that fraction becomes a visible phase drift, and a model built from the osculating values would itself leave a structured residual. The periods that matter for forecasting are the mean ones, and they are only available from the data.

A residual of that kind can only be read once everything known has been taken out of it, and taking it out is itself the craft. The subtraction in the figure is the timing version of the series of known perturbations that is subtracted from a planet’s observed positions before anything new is claimed: each term removed is a modelled effect of a known body, and whatever survives every subtraction is attributed to something unknown. The claim is only as good as the completeness of the terms. A two-planet model that omitted the chopping, as the first version of this subtraction did, left a residual of nearly two minutes from the two known planets alone — larger than the signal of a small third planet — and would have attributed the known system’s own conjunctions to an imaginary neighbour.

The whole construction has a famous ancestor. The positions of Uranus refused to follow their tables in the 1830s and 1840s, and the residual had the shape a further planet would produce. Urbain Le Verrier and John Couch Adams computed where that planet had to be, and Neptune was found in 1846 within a degree of the prediction. A decade later the same method was turned on the unexplained advance of Mercury’s perihelion, and predicted a planet inside Mercury’s orbit that was never found, because that residual was not a planet but a correction to Newton’s law of gravity. A structured residual is evidence that the model is missing something. It is not, by itself, evidence of what.

What was actually measured #

Kepler-9, the first system in which timing variations were measured, is two giant planets near a 2:1 commensurability whose transits wandered by tens of minutes over the telescope’s first months. Its early ephemerides, extended by a few years, mispredicted later transits by amounts larger than their quoted uncertainties, and only a dynamical fit containing both planets forecast them correctly. The same experience recurred across the Kepler catalogue once other telescopes began re-observing its planets: for systems with strong timing variations, the transit windows computed from Kepler-era linear ephemerides had drifted by hours within a decade, and dedicated networks now maintain ephemerides for planets that future missions intend to observe, precisely because a stale forecast is a missed transit.

The third-planet signature has been seen as well. Systems in which a two-planet fit to the transit times left a periodic residual have been followed by velocity surveys, and some of those residuals turned out to be non-transiting planets at one of the periods the timing allowed. Others have turned out to be stellar activity, whose spots distort individual transit shapes and shift their fitted times, and whose rotation period can masquerade as a periodic timing term.

What the forecast leaves out #

The perturber is on a fixed orbit. Over four years its orbit precesses and its eccentricity exchanges slowly with the transiting planet’s, and the figures include that exactly because they are integrations; a forecast based on a two-planet analytic model does not, which is part of the 0.87-minute floor.

The timing noise is uniform and independent. Real transit times from different instruments carry different precisions and different systematic offsets, and a fit that combines them without allowing for an offset between instruments produces a slope error of its own.

The system is coplanar. A mutual inclination between the two planets makes the transiting planet’s impact parameter drift, which changes its transit durations and, at the precision of a few seconds, the fitted mid-times. That is a further way for a forecast to fail, and none of the figures contains it.

Still open: how long a forecast should be trusted before the neighbour is known #

A newly found planet has a linear ephemeris and, usually, no way of knowing whether it has a near-resonant neighbour until its timing has been followed for longer than the super-period it would have. The question a follow-up programme actually faces is therefore statistical: given the census of timing variations among planets of this kind and size, how wide should the transit window be at each date ahead, so that the chance of missing the transit is kept below some level? The census that would answer it is biased towards the systems whose variations were large enough to find, which are the ones with the longest super-periods and the worst forecasts, and correcting for that bias is the same problem as a prediction whose expiry date is itself uncertain.