Raw records to compare before deleting a single outlier

5 min read

When you spot one unusually high or low value in a list of breath alcohol test results, you may be tempted to delete that row to prevent aggregation errors. But a number that looks like a candidate for deletion may be in a state where several possibilities have not yet been told apart: a test error, a difference in the displayed unit, another person's result linked by mistake, or a duplicate created during transmission. Raw records here mean not screen statistics or edited lists, but the result values first stored by the device or connected system, along with the time, display unit, and identification information attached to them.

A breathalyzer is a device that analyzes a breath sample and displays the result, and the same number can be interpreted differently depending on the unit it was displayed in and the conditions under which it was recorded. For example, DATECH's AF-40BT is described as using an electrochemical sensor, displaying ranges in %BAC, ‰, and mg/L, and storing up to 10,000 test results. Before deleting a single outlier, do not look at that number in isolation; compare it with the raw records stored before and after it on the same device and with the path by which the result was produced.

Preserve the original row before looking at the number

The starting point of an outlier review is making a copy to analyze. If you edit or delete the raw record itself, then even if you later discover a unit setting or duplicate transmission problem, you cannot compare the original result with the changes. Even if you mark it as excluded only on the aggregation screen, it is better to leave the original result value, storage time, and device identification information as they are.

In particular, if you have been managing only result values copied into a separate table, you also need to keep the text originally displayed on screen. If only numbers remain with the decimal places, unit symbols, and warning messages stripped out, you may end up comparing values from different display systems by the same standard. Also state clearly in the record that "outlier" is a marker that further review is needed, not a cause.

Four points of comparison to tell whether it is the same result

There may seem to be many things to compare, but the four groups below come first for a deletion decision. If the connected system does not store every item, compare only the items actually available, and do not fill in missing information by guessing.

What to compare What to look at in the raw record Question to decide after comparing
Testing device Product model, device identification, settings in use Did the result come from the same device under the same display conditions?
The result itself Result value as first displayed, unit, test sequence number or result identifier Is the number a duplicate, or does it only look different because of the unit?
Time order Time recorded by the device, and the time the system received it if available Is the order of results before and after natural, or did the same result come in again?
Result status Failure, warning, or analysis status messages left on screen or in the transmitted value Was a normal analysis result read, or was a status message confused with a value?

The official AF-40BT information gives the display range as 0.000–0.500 %BAC, 0.00–5.00 ‰, and 0.00–2.50 mg/L. So rather than judging that a value stands out just from the size of the numbers in one column, first align which unit the result was stored and displayed in. The product's accuracy figures are also given together with specific reference conditions such as 0.100 %BAC and 25 ℃, so you cannot conclude there is a sensor problem from a single record outside those conditions.

Records before and after are not evidence that settles the cause

The test history before and after is useful for checking whether a result is isolated, but it does not in itself prove the cause of a fault. Look at whether several results stored in close sequence on the same device continue in the same format and unit, and whether only a particular result has different identification information or status messages. In an environment that uses both device time and server receipt time, sort by each separately so that transmission delays are not mistaken for changes in the test order.

Let's take a hypothetical example. Suppose that in a monthly aggregation one result looked much larger than the other rows, and a sensor problem was suspected at first. On checking the raw records, that row looked similar in its digits to other values but had a different unit, and the history before and after in the same time window showed no duplicates or failure statuses. In this case, the observation is not "a large number" but "a result in a different unit," and the possible action is not to delete the row as a device fault but to fix the analysis screen to see whether aggregation across units was mixed. The completion criterion is confirming, with the original value and unit kept, that the result has been included in the correct unit group according to the aggregation criteria.

Separate the exclusion decision from determining the cause

Even when a particular record must be left out of an analysis for statistical purposes, excluding and deleting are different actions. For example, if a result is confirmed as a duplicate transmission, only one can be counted in the analysis list. In that case, keep both results in the raw records and note separately which result was treated as the duplicate and the basis for the comparison, so the decision can be reproduced later.

Conversely, if the unit has not been confirmed or the link between person and device is unclear, it is more accurate not to label it "sensor failure." If you mark only the facts confirmed so far, such as "unit not yet confirmed," "duplicate status under review," or "cause not determined," the same record can be re-judged when more data comes in. Deleting a result whose cause is unknown may not remove an error but rather remove the trace of one.

Set the criteria for concluding a deletion decision

The completion criterion for deletion or exclusion from analysis is not whether the numbers look neat. The criterion is whether the original result, display unit, link between device and result, time order, and reason for exclusion can be explained without contradicting one another. If the cause has not been determined, it is appropriate to keep the record preserved and to record confirmed facts separately from assumptions.

This procedure does not target only high values. Even values that look within normal range can distort the analysis if they are linked to another person or if unit columns were merged incorrectly. So after reviewing one row that stands out, also check the linking method and unit notation for the same period, so that errors that slipped in quietly can be found by the same standard.

References

#outlier
#raw records
#exclusion decision
Considering our products?
We'll help you decide.