During the process of measuring and recording weather data, errors may be introduced such that the actual weather is not captured correctly in the records. For example, it may be 20 ºC outside, but due to a mistake the temperature is reported as 22 ºC. Such errors may be caused by the following factors:
- Equipment failure: Weather sensors may fail or malfunction due to e.g. aging components, extreme weather conditions, and electrical or communication faults.
- Human error: Instruments that are manually operated by human observers may be misread, recorded incorrectly in logbooks or typed incorrectly into electronic databases. Measured values may be recorded at the wrong date or the values themselves may be recorded incorrectly.
- External influences: Changes in equipment and the environment surrounding a weather station could also introduce errors in the data. For example, a tree may slowly grow and cast a shadow over the weather station, which will cause a shift in temperature readings over time.
Data errors should be identified and removed (or corrected) through a quality control (QC) process. A number of automatic QC checks can be performed as a first step to identify these errors. QC checks may include the following (also see BoM and WMO QC documentation):
- Climatology checks: Test whether an observation is within physical limits (e.g. air temperature within -80 and 60 ºC) and also test the plausibility of the observation at the given time of the year (e.g. can it be -5 ºC midday in the summer in an arid region?).
- Internal checks: Comparing an observation of a given station with other observations within the same time frame to ensure consistency over time. For example, if a thermometer measured 5 ºC at 9AM, is it possible that the same thermometer would measure 34 ºC at 3PM?
- Consistency with neighbours: Testing whether a measurement agrees with historical patterns in comparison with the closest surrounding weather stations. For example, if it is known that the local post office is on average 2 ºC warmer than the airport, can the current measurement at the post office be correct – if it is 10 ºC warmer than the airport?
The BoM approach to dealing with data values that failed one or more of the above tests, is to place these values on a priority list for a skilled QC operator to investigate further. Information from other sources (e.g. type of measuring equipment used, site location, satellite and radar images, weather charts, etc.) is then used to decide what the relative quality of these potentially wrong observations is. If sufficient evidence exists that an observation is indeed wrong, it will either be removed or amended, with an appropriate QC flag added.
To address problems in the data caused by changes in the sensor network (e.g. site moves), the BoM applies homogenisation techniques.
Jennifer Marohasy BSc PhD has worked in industry and government. She is currently researching a novel technique for long-range weather forecasting funded by the B. Macfie Family Foundation.