Response to the ATSB Report and DSTG Book on MH370
Richard Godfrey and Victor Iannello
9th December 2015
Introduction
The Australian Transport Safety Bureau (ATSB) published an update to their report entitled “MH370 – Definition of Underwater Search Areas” on 3rd December 2015. At the same time, the Australian Defence Science and Technology Group (DSTG; formerly the DSTO) through the ATSB made available a book entitled “Bayesian Methods in the Search for MH370”. Both may be downloaded as PDFs (1.76 MB and 8.76 MB respectively) via this webpage.
At first sight, the search area centred on a point located at 38.0S 88.4E aligns very well with the end point published by the IG (over 14 months ago, on 26th September 2014; see here also) at 37.71S 88.75E. In fact, these two points are only 24 NM apart.
However, when one digs a little deeper, there are obvious flaws in the Bayesian approach adopted by the DSTG team and the belated confirmation of the IG end point by the DSTG is more due to error than design. There appears to be a fundamental disconnect between the mathematics employed and the constraints imposed on possible paths based on the aircraft dynamics and satellite communication data. In this brief post we summarise how and why this is so.
Bayesian approach
The BFO and BTO data are stochastic in nature, so assigning random variables and calculating probability density functions for the derived end point from paths based on the uncertainty of these variables seems reasonable, especially if other constraints are considered such as human factors and aircraft performance characteristics.
The path travelled, however, was most likely not stochastic: there was a deterministic reason for whatever path the plane took, be it human intention or a particular failure mode. And that particular path could have a very low probability if it is just one of many possible random paths that are generated in the way the DSTG researchers generated their possible paths.
Bias toward straight paths and against curved paths
The DSTG posterior distribution has a high incidence of straight paths, which peaks the end point distribution around 38S. In fact, if all speeds and turn times are equally probable, the distribution should be much flatter. By virtue of the way the DSTG generate paths using random manoeuvres and random changes in heading and speed, curved paths (which require more turns) are not represented as often in the posterior distribution.
This bias for straight paths can be seen in Chapter 7 of the DSTG book. Figures 7.2 and 7.4 show paths for which the BTO and BFO are ignored completely. There is a hot spot for the end point in Oman, corresponding to a straight flight starting at 18:02 and proceeding northwest.
The assumption made, then, is that the plane flew straight between manoeuvres, so a flight with few manoeuvres flies relatively straight. This produces a bias in the analysis in favour of straight paths and against curved paths.
Bias toward faster speeds and against slower speeds
In the DSTG study, the speeds after manoeuvres are chosen to be randomly distributed between Mach 0.73 and Mach 0.84 based on fuel considerations. In this brief section we indicate why the choice of lower speed limit is erroneous if fuel consumption is the only consideration.
At FL350 and DTisa=0, these Mach speeds corresponds to True Air Speeds (TAS) of 421 knots and 484 knots respectively. At DTisa = 10K, the TAS range is 430 to 495 knots.
The Holding Speed of an aircraft is defined as the speed at which fuel flow rate is at its minimum, for any specified altitude. Over the interval from 18:28 to fuel exhaustion the average weight of the plane is about 192.3 tonnes. Using this average as a constant weight throughout that interval, at Holding Speed at FL200, and with DTisa = 0, the appropriate Indicated Air Speed (IAS) is 213 knots, which corresponds to Mach 0.47 and a TAS of 288 knots. At DTisa = 10K, Mach 0.47 corresponds to a TAS of 293 knots.
There is therefore a fundamental error made in setting a lower limit of TAS equal to 421 knots when speeds of around 288 knots would have sufficient endurance if the aircraft flew at a level around FL200.
Because fuel considerations allow lower speeds than M0.74, the DSTG model has an inherent bias toward faster speeds.
Cost Index
The Cost Index (CI) has finally been made public (a request for its publication was made in September 2014) and we are told that the flight was planned on the basis of a CI of ECON 52. The resulting MH370 Flight Path Model V16.0 is published elsewhere and shows an end point of 38.4S 87.7E.
Radar Data
The DSTG Bayesian approach is based on a prior defined by the Malaysian Military Primary Radar at 18:01 (penultimate 10-second radar capture).
The single capture from the Malaysian Military Primary Radar at 18:22 was not used by the DSTG because it is thought likely to be less accurate, near the radar system’s maximum range. However, this single radar capture has been shown to align with the satellite data a few minutes later.
The Malaysian authorities’ previous public releases of their Military Primary Radar data is not consistent with the Malaysian Military Primary Radar available to the team at DSTG, as it shows a detailed radar trace with a large number of data points after 18:01:49 up to 18:22:12. For example, see the photograph below.
We request the following:
1. Publication the 10-second Primary Radar data from 16:42:27 to 18:01:49.
2. An explanation of the source of the Primary Radar trace between 18:01:49 and 18:22:12 (as shown above) and whether the Malaysian authorities have made these radar-determined positions available to the ATSB and the DSTG.
BFO Bias
The BFO bias was found by the DSTG researchers to have a geographic dependency, but it was not possible for them to determine the cause. A geographic dependency can be ruled out, as the laws of physics do not change with location and time. As this uncertainty in the bias causes a large uncertainty in the overall BFO measurement, it needs to be better understood.
Conclusion
In summary, the recent DSTG report presents a generalized, Bayesian method to prioritize the search zone. Although the ATSB is using this report as justification of its current search area, the alignment between the new results and previous results are a result of an inherent bias towards straight, high speed paths. More generalized methods would allow a larger range of end points. This bias should be acknowledged.