Navigation & Position formulas — 12 solvers

Compass to True Heading (Variation and Deviation)

T = C + D + V

Navigation & PositionThe TVMDC chain in one line: a compass heading corrected by deviation for the ship's own iron and by variation for the earth's field, giving the true heading a chart is drawn in. East corrections are positive and are added going this way.

Cross Track Error

e_{xt} = R\arcsin\left(\sin\frac{d_{13}}{R}\,\sin\left(\theta_{13}-\theta_{12}\right)\right)

Navigation & PositionThe perpendicular distance from the intended track to where you actually are, on a sphere: the number a plotter shows on its deviation bar. Positive is right of track, negative is left.

Dead Reckoning Position

\Delta\varphi = \frac{S\,t\cos C}{R}

Navigation & PositionThe classic DR advance: speed times time gives the distance run, and the cosine of the course splits it into the change of latitude. The oldest position-finding method there is, and still the one that catches an instrument failure.

Destination Latitude from Course and Distance

\varphi_2 = \arcsin\left(\sin\varphi_1\cos\frac{d}{R} + \cos\varphi_1\sin\frac{d}{R}\cos\theta\right)

Navigation & PositionWhere a great circle course of a given length puts you: the latitude reached from a starting latitude, an initial true bearing and a distance run. The answer carries the change of longitude with it.

Distance to the Visible Horizon

D = \sqrt{h\left(2R + h\right)}

Navigation & PositionHow far away the sea horizon is from a given height of eye: the tangent line from the observer to the sphere. The geometric figure, before atmospheric refraction pushes it out by about a tenth.

Estimated Time En Route

t = \frac{d}{V_g}

Navigation & PositionDistance divided by ground speed. Trivial arithmetic, and the single most common place a navigation plan goes wrong, because the speed put into it is so often the airspeed or the log speed rather than the speed actually made good.

Great Circle Distance (Haversine)

d = 2R\arcsin\sqrt{\sin^{2}\frac{\varphi_2-\varphi_1}{2} + \cos\varphi_1\cos\varphi_2\sin^{2}\frac{\lambda_2-\lambda_1}{2}}

Navigation & PositionThe shortest distance between two points on a sphere, from their latitudes and longitudes. Sinnott's haversine form, which keeps its precision on short legs where the older law-of-cosines version loses it.

Initial Great Circle Bearing

\theta = \operatorname{atan2}\!\left(\sin\Delta\lambda\,\cos\varphi_2,\; \cos\varphi_1\sin\varphi_2 - \sin\varphi_1\cos\varphi_2\cos\Delta\lambda\right)

Navigation & PositionThe course at the moment of departure along the great circle to a destination, measured clockwise from true north. It is the starting bearing only — on any track but a meridian or the equator it changes the whole way.

Rhumb Line Distance

d = R\sqrt{\left(\varphi_2-\varphi_1\right)^{2} + q^{2}\left(\lambda_2-\lambda_1\right)^{2}}

Navigation & PositionThe length of the track that crosses every meridian at the same angle: the route you actually steer when you set one course and hold it. Longer than the great circle, and the stretching factor q comes from the Mercator latitude.

Set and Drift of the Current

D_r = \frac{\sqrt{\Delta n^{2} + \Delta e^{2}}}{t}

Navigation & PositionThe current the water was making, worked backwards from the gap between where dead reckoning said you were and where the fix put you. Drift is its speed; set is the true direction it flows toward.

Wind Correction Angle

\mathrm{WCA} = \arcsin\left(\frac{W\sin\theta}{V}\right)

Navigation & PositionHow far off the desired track to point the nose so the wind carries you along it instead of off it. Only the crosswind component matters: the along-track part of the wind changes the speed, never the heading.

Wind Triangle Ground Speed

V_g = \sqrt{V^{2} - W^{2}\sin^{2}\theta} - W\cos\theta

Navigation & PositionWhat the aircraft is actually making over the ground once the wind has had its say: true airspeed, wind speed, and the angle between the wind and the intended track. The E6B's whole reason for existing.