Guidance Document on Model Quality Objectives and Benchmarking

5.4. Observation uncertainty

1. General expression

In Thunis et al., 2013 a general expression for the observation uncertainty is derived by considering that the combined uncertainty, of a measurement , can be decomposed into a component that is proportional, to the concentration level and a non-proportional contribution, :

(18)

The non-proportional contribution, is by definition independent of the concentration and can therefore be estimated at a concentration level of choice that is taken to be the reference value (). If represents the estimated relative measurement uncertainty around the reference value () for a reference time averaging, e.g. the daily/hourly Limit Values of the AQD then can be defined as a fraction α (0-1) of the uncertainty at the reference value:

(19)

Similarly the proportional component can be estimated from:

(20)

From the combined uncertainty, an expanded uncertainty can be estimated by multiplying with a coverage factor :

(21)

Each value of gives a particular confidence level so that the true value is within the confidence interval bounded by . Coverage factors of = 1.4, = 2.0 and = 2.6 correspond to confidence levels of around respectively 90, 95 and 99%.

Combining (18) – (21) the uncertainty of a single observation value can be expressed as:

(22)

From Equation (23) it is possible to derive an expression for (equation 5) as:

(23)

where and are respectively the mean and the standard deviation of the measured time series.

2. Derivation of parameters for the uncertainty

To be able to apply (24) it is necessary to estimate , the relative uncertainty around a reference value and α, the non-proportional fraction around the reference value. If we define the relative expanded uncertainty as , equation 23 can be rewritten as

(24)

with the absolute expanded uncertainty around the reference value, . This is a linear relationship with slope, and intercept, which can be used to derive values for and α by fitting measured squared uncertainties to squared observed values .

An alternative procedure for calculating and α can be derived by rewriting (25) as

(25)

where L is a low range concentration value (i.e. close to zero) and its associated absolute expanded uncertainty. Comparing the two formulations we obtain:

(26)

(27)

The two above relations (26) and (27) allow switching from one formulation to the other. The first formulation (24) requires defining values for both α and around an arbitrarily fixed reference value () and requires values of over a range of observed concentrations, while the second formulation (27) requires defining uncertainties around only two arbitrarily fixed concentrations ( and ).

For air quality models that provide yearly averaged pollutant concentrations, the MQO is modified into a criterion in which the mean bias between modelled and measured concentrations is normalized by the expanded uncertainty of the mean concentration (equation 7). For this case, Pernigotti et al (2013) derive the following expression for the uncertainty:

(28)

where and are two coefficients that are only used for annual averages and that account for the compensation of errors (and therefore a smaller uncertainty) due to random noise and other factors like periodic re-calibration of the instruments. In equation (28) the standard deviation term is assumed to be linearly related to the observed mean value in the annual average formulation

(i.e. ). The calculation of the coefficient accounts for the correction resulting from this assumption. To determine and a similar procedure is used as for α and above. Once α and are known from the uncertainties on the hourly observations, values and are derived from the uncertainties for the yearly averaged values by using a linear fit between and . You could also again simplify the fitting of the coefficients for the annual expression using the same methodology using two arbitrarily fixed concentrations as presented by equations 26 and 27 above.

The following values are currently proposed for the parameters in (22) and (28) based on Thunis et al. (2012), Pernigotti et al. (2013) and Pernigotti et al. (2014). Note that the value of α for PM_2.5 referred to in the Pernigotti et al. (2014) working note has been arbitrarily modified from 0.018 to 0.050 to avoid larger uncertainties for PM₁₀ than PM_2.5 in the lowest range of concentrations.

Table : List of the parameters used to calculate the uncertainty

				α
NO2	2.00	0.120	200 µg/m^3	0.040	5.2	5.5
O3	1.40	0.090	120 µg/m^3	0.620	NA	NA
PM10	2.00	0.140	50 µg/m^3	0.018	40	1
PM25	2.00	0.180	25 µg/m^3	0.050	40	1