Great Circle Distance (Haversine)

Two points on a sphere are joined by exactly one shortest path, and it is the arc of the great circle through them — the circle whose centre is the centre of the earth. Every meridian is a great circle. The equator is a great circle. No other parallel of latitude is, which is the first surprise the subject hands you: the 60th parallel looks like a perfectly good road on a school globe, and following it is not the shortest way anywhere.

The formula on this page is R. W. Sinnott's 1984 restatement of a very old identity, published in Sky & Telescope under the title "Virtues of the Haversine". The haversine itself is just $\operatorname{hav}\theta = \sin^{2}(\theta/2)$, and the virtue Sinnott was defending is numerical. The older spherical law of cosines, $\cos d = \sin\varphi_1\sin\varphi_2 + \cos\varphi_1\cos\varphi_2\cos\Delta\lambda$, is algebraically correct and quietly useless on short legs: when the two points are close the cosine is within a hair of 1, and a machine carrying seven digits has nothing left to work with. Sinnott's example was a calculator returning zero distance for two points a kilometre apart. The haversine form never forms that difference and holds its precision all the way down.

The radius used here is 6 371 008.8 m, the IUGG mean radius of the WGS-84 ellipsoid. Be honest with yourself about what that costs. The earth is flattened by about one part in 298, and a spherical distance can differ from a true geodesic on the ellipsoid by up to roughly half a percent depending on how the track lies. On a thirty-mile coastal passage that is a couple of hundred metres and it does not matter. On a five-thousand-mile ocean crossing it is twenty-five miles, which is a real fuel figure and a real arrival time. When it matters, the tool is Vincenty's iterative method or a modern geodesic library — not a bigger radius.

The classic mistake here is not in the formula at all; it is in the longitudes going into it. This shard takes east as positive and west as negative, without exception, and a west longitude entered as a positive number is the single most common error in every coordinate calculation ever written. A Halifax longitude is −63.6°, not 63.6°. The second most common is mixing degrees-minutes-seconds with decimal degrees: 40°38'23" is 40.6398°, not 40.3823°, and the two are close enough to look plausible and far enough apart to lose a ship.