When comparing breathalyzer results with reference concentrations, safety managers and quality or technical staff tend to look first at the mean error in the table. But a mean of zero does not mean every reading matched the reference value. Readings that were high at some test points and low at others may have cancelled each other out in the total.
A breathalyzer is a device that detects the alcohol in breath and displays it as a concentration, and in testing the device's displayed values are compared with reference values. Here, error is usually calculated as "measured value − reference value." Looking at positive and negative errors separately shows in which concentration range and in which direction the device deviated, and helps you find points needing further review that a single mean hides.
The sign of an error is set by the direction of the calculation
The International Vocabulary of Metrology (VIM) defines measurement error as the measured quantity value minus a reference quantity value. This article follows the same direction. For example, if the reference value is 0.20 mg/L and the device displays 0.22 mg/L, the error is +0.02 mg/L. If the displayed value is 0.18 mg/L, it is −0.02 mg/L.
What matters is applying the same direction of calculation to every test result. If one table uses "measured value − reference value" and another uses "reference value − measured value," the meanings of + and − are reversed. Only by keeping the raw readings, the reference values used for comparison, the formula, and the units together can the same results be recalculated later or the table reviewed by someone else.
A mean of zero does not mean there is no difference
The following is a hypothetical test example to explain the calculation principle. All values are assumed to be in mg/L and compared under the same conditions, and they do not represent the performance or acceptance criteria of any particular breathalyzer. Device A reads higher than the reference at the low test point and lower than the reference at the high test point.
| Reference value | Device A display | Error (measured minus reference) |
|---|---|---|
| 0.10 mg/L | 0.12 mg/L | +0.02 mg/L |
| 0.20 mg/L | 0.20 mg/L | 0.00 mg/L |
| 0.30 mg/L | 0.28 mg/L | −0.02 mg/L |
The three errors add up to 0 mg/L, so the mean error including signs is also 0 mg/L. Yet each result's distance from the reference value is 0.02 mg/L at the first and third test points. A mean of zero does not mean these two differences were absent; it means the positive value on one side and the negative value on the other cancelled out arithmetically.
So the mean error can be used as an indicator of whether the overall results lean in one direction, but on its own it cannot show how close each test point was to the reference value. Especially in data where several concentrations were each measured once, a pattern in which the direction of error changes with concentration can be buried in the mean.
Direction and size answer different questions
A signed mean error is suited to seeing whether "the device tends overall to read high or low." Absolute error, on the other hand, removes the sign and looks only at the distance from the reference value. In the hypothetical example above, the absolute errors are 0.02, 0.00, and 0.02 mg/L, and their mean is about 0.0133 mg/L. This value confirms that a distance between measured and reference values remains even though the signed mean is zero.
The largest absolute error also needs to be looked at. Even if the mean absolute error is small, if a large difference occurred only at a particular concentration, it is hard to notice that test point from the overall mean alone. Conversely, looking only at absolute error does not tell you whether the device keeps reading high, or reads high at low concentrations and low at high concentrations. The two indicators are not better or worse than each other; they serve different observational purposes.
Repeated measurement results also need to be interpreted separately. The spread among several measurements at the same reference concentration is material for seeing how scattered the results were under that condition. A single average of values obtained at several reference concentrations is used to find deviations by concentration range. Lumping the two kinds of data under the same name, "mean error," loses what was actually averaged.
Once you spot cancellation, go back to the raw values
At the observation stage, lay out the reference value, the device's displayed value, and the signed error side by side for each concentration. At the interpretation stage, see whether positive errors cluster in one range and negative errors in another, or whether repeated results at the same reference value are widely spread. Going through this sequence lets you identify specifically which results cancelled out, instead of reading a mean of zero as "no problem."
Possible actions include re-checking the raw data for the test points concerned, confirming that the units of the device being compared and the reference values are the same, and organizing whether each test point was included in the mean the same number of times. For example, if there is one result at 0.10 mg/L and several at 0.30 mg/L, the overall mean is influenced more by the point with more repetitions. Entering missing measurement results as zero error to balance the mean changes the meaning of the actual test data.
The completion criterion is not obtaining a single mean value. You should be able to trace each test point's raw values and error direction, and explain separately what the signed mean, the absolute error, and the largest absolute error each mean. Only then is there a basis for judging whether a mean of zero reflects true agreement or the cancellation of positive and negative errors.
Means can be compared only when the items compared are the same
To compare summary values for two breathalyzers, you first need to make sure the results were obtained at the same reference concentrations, in the same units, and with the same number of repetitions. If one device was tested mainly at low concentrations and the other mainly at high concentrations, it is hard to attribute the difference in mean error to the devices. Whether temperature or other test conditions differed should also be recorded as separate items in the raw data.
How decimals are displayed also affects interpretation. It is better to keep the precision of the raw data during calculation and round to the necessary number of digits only when displaying values in a report. If small positive and negative errors both look like 0 after rounding, both the reason the mean became 0 and the differences in each result can be hidden.
In the end, the reason to check positive and negative errors separately is not to dismiss the mean. It is to read the overall direction the mean shows together with the size and direction of differences shown by the raw data at each concentration. With this distinction, you avoid over-interpreting a summary value of zero and can decide on a sound basis which test points to examine further.
