OAS constants - History

14.06.2006

Submission:

HISTORY OF THE PANS-OPS OAS CALCULATION ANOMALY
1 INTRODUCTION
During the validation of the new OAS C-ROM program it was found that there were anomalies between the CD-ROM results, the output from PC version of the NLR "Book" program, and PANS-OPS Volume II, Part III, Attachment I. There were differences between all three. This paper describes the validation process, and the resolution of these differences.

The tests were updated in October 2004 to reflect the changes in aircraft dimensions in Amendment 12 to PANS-OPS Volume II.
2 BACKGROUND
2.1 NLR Memorandum IN-81-007 U
The original program to calculate the PANS-OPS obstacle assessment surfaces (OAS) was described in NLR Memorandum IN-81-007 U (1981). This program, implemented on a mainframe computer using punched cards, calculated the OAS constants for specific sets of input variables (LLZ-THR distance, GP angle and aircraft size). Originally two aircraft sizes were programmed, but subsequently the Panel decided to print only that for the Standard size aircraft. This program was then used to generate printout that became the master copy for subsequent reproduction in all subsequent issues of PANS-OPS Volume II.
2.2 ICAO PANS-OPS OAS CD-ROM
The ICAO PANS-OPS OAS program is written in Fourth Dimension (4D) - the programming language used in the new PANS-OPS Software). It uses the same source code for the calculation routines as the original FORTRAN program, except for one significant difference - the results are stored at full machine precision in memory instead of being written to an intermediate file for subsequent printing.
3 VALIDATION PROCESS
3.1 Validation Data
The data available for validation consisted of:
- Output from the 4D implementation of the original NLR program,
- The printed tables of PANS-OPS Volume II Attachment I, which had been reproduced from the printout accompanying the original NLR Memorandum in 1981, and
- Printout from the PC implementation of the NLR program developed for ICAO in the 1990s.
3.2 Method of validation
Because the neither the new 4D implementation nor the PC implementation of the FORTRAN source code could reproduce exactly the results of Attachment I, the differences between the three sources were investigated to determine which source provided correct results.
4 VALIDATION RESULTS
4.1 OAS constants
4.1.1 Differences between CD-ROM and Attachment I
The anomalies between the ICAO OAS CD-ROM and Attachment I involved about 2% of the OAS A and B constants. In all the anomalies, the CD-ROM value was 0.000001 smaller than that in Attachment I - such anomalies being symptomatic of incorrect rounding of numbers.

As noted above, the new OAS CD-ROM program calculates and stores the OAS constants in memory at the full machine precision. They are only rounded to 6 decimal places (A and B constants), and 2 decimal places (C constants) for display and printing. However, in the NLR program source code the OAS constants are written to an external file and subsequently read back in for printing. This was necessary because of the memory limitations of early computing systems. The constants in the intermediate files are stored in scientific floating point notation, rounded to a 7 digit mantissa and a 2 digit signed exponent. However, when these values are printed for Attachment I, they are rounded again to 6 decimal places fixed point format.

Double rounding is of course a mathematical error, since with some combinations of digits it can result in the number being rounded up instead of down.
4.1.1.1 Example of rounding errors
4.1.1.1.1 Rounding used in FORTRAN source code:

Number in computer full precision = 0.123414489......
Rounded to 7 places for intermediate file = 0.1234145
Rounded to 6 places for printed result = 0.123415
4.1.1.1.2 Correct rounding:
The answer must be either 0.123414 or 0.123415

Since 0.123414489 - 0.123414 = 0.00000049
whilst 0.123415 - 0.123414489 = 0.00000051
Clearly 0.123414489 in nearer to 0.123414 than to 0.123415

Hence the correct result to six decimals is 0.123414, not the 0.123415 given by double rounding.
4.1.1.2 Confirmation of rounding errors in Attachment I

Since the intermediate data is stored in scientific notation with 7 significant digits in the mantissa, the B constants with (exponent +00) are effectively rounded to 7 digits then again to six, whilst the A constants (with exponent of -01) are rounded to 8 digits and then to six.

This was confirmed by adding two Incorrect double rounding routines in the OAS CD-ROM implementation as follows:
- double rounding (to 7 then to 6 decimals) for the B constants.
- double rounding (to 8 then 6 decimals) to the A constants.
This change removed all discrepancies between the constants in the CD-ROM OAS and Attachment I. The constants from the OAS CD-ROM were checked manually against PANS-OPS Vol II part III Attachment I, and no discrepancies were found..

At this point it was clear that all the anomalies were attributable to incorrect rounding in the NLR program, and that the results of the new OAS CD_ROM were in fact correct.

From the operational point of view, the effect of the errors in Attachment I are relatively insignificant, some 4cm difference in the related OAS surface height at 300m. Nevertheless, this detailed review was considered justified to ensure user confidence in the new implementation of the original OAS program.
4.1.2 The PC implementation of the NLR Book program
The discrepancies between Attachment I and the output from the PC implementation of the NLR program involved similar 0.000001 differences from Attachment I, but at different values of LLZ-THR distance, GP and missed approach gradient.

The ICAO PC version was produced by compiling the original source code to run on a PC. However, subsequent checks showed that if the same source code was compiled using a different compiler (Leahey), similar differences of 0.000001 appeared, but in different places. It has been concluded that this was due to the incorrect rounding noted above, together with the way different FORTAN compilers treat floating point arithmetic and rounding, and deficiencies in the early FORTRAN Standards.

The results of the PC version of the program have therefore been discounted.
4.2 OAS CD-ROM template coordinates
The ICAO OAS CD-ROM template coordinates (and those in Attachment I) are shown to a resolution of one meter. If the value is not an integer, it is changed to be the next integer value nearer to threshold or centre-line. This accuracy is of course more than adequate for manual plotting. It also ensures that if the template coordinates are entered in the OAS height calculator, it never gives an "Outside OAS" warning. However, if computer graphics programs are used, the template coordinates should be calculated from the OAS constants. These constants give more precise results, and avoid anomalies for obstacles very close to the template boundaries.
4.3 OAS adjustments
To validate the adjustments of the OAS constants, a series of tests were made covering a range of the possible adjustments. No anomalies were found.
4.4 OAS height calculator
The OAS height calculator was checked by entering the x y values shown in the template table and confirming that the OAS height calculated corresponded with the threshold level, 150 or 300m contour height as appropriate. Apart from small height differences due to the use of integer values for the template coordinates, no anomalies were noted.
The extension of the W and X surfaces above 300m and towards the intermediate approach area were also checked and confirmed.
Checks were also made of obstacles located just outside the areas covered by the templates, and the appropriate "Outside OAS" warnings were given.
4.5 Program operation
All program functions have been checked. The following anomalies exist in this version:
a. The options for printing a "selection", the "current page" and a "specified range" of pages do not work.
b. The "print to a file" option will not send output to a generic text print file. However, it will send output to an Adobe Distiller ".prn" file, or to PDF maker.
c. If the printout from "Print Attachment I" is stopped before completion, any constants and template coordinates already in the program screen reset to zero. They may be regained by re-entering any of the setup values.
5 VALIDATION RESULTS - POST PANS-OPS AMENDMENT 12

Amendment 12 to PANS-OPS Volume II, effective 1 November 2004 changed the aircraft dimensions associated with the aircraft categories as follows:

Cat. Semi-span (old) GP-wheel (old) Semi-span (new)GP-wheel (new)
A 30 6 30 6
B 30 6 30 6
C 30 6 32.5 7
D 30 6 32.5 7
DL - - 40 8

The program was updated to incorporate the new values as the default values in the program. Further validation checks were made to confirm correct operation with these new default values.

From
Robert Woodhouse - ICAO OCP member (retired) - Canada

Answer or Commentary:

(I.W.
It is good that this historical information is not lost)