Instrument-Specific (Table)
Checks whether values match the measurement instrument they were collected with. A one-to-five Likert score cannot be six, a Glasgow Coma Scale value cannot be below three, and an integer-scale response cannot carry several decimals. The indicator matches headers exactly to named instruments and checks scale bounds, precision, and, for integer instruments, the granularity a reported mean must obey.
Technical description
Loads a dictionary of instruments (each with min, max, type, aliases), extracts the table grid by OCR, and matches each header by EXACT equality with an instrument key or alias. For matched columns it flags out-of-scale values (below min or above max) as critical, and for integer instruments flags values with more than two decimals as excess precision. When an integer instrument is present it applies the GRIM test to mean/SD/N triplets, restricted to means within the instrument's scale so a continuous-variable mean is not tested under the integer assumption. Score: 4.5 for a GRIM failure, 4.0 for out-of-scale, 2.0 for excess precision, 0 otherwise.
How it works
Layer 1 (deterministic): matches each header exactly to an instrument, then flags values outside the instrument's scale (critical), integer-instrument values with more than two decimals (info), and, when an integer instrument is matched, mean/SD/N triplets whose in-scale mean fails GRIM (warning). The score is 4.5 for a GRIM failure, 4.0 for an out-of-scale value, 2.0 for excess precision, and 0 otherwise.
Why this matters
Measurement instruments impose exact, documented constraints, so a value that violates them is an unambiguous error or fabrication. Comparing values against an instrument's valid range and properties is a core data-cleaning and integrity check. For integer scales the constraint goes further: responses are whole numbers, so a reported mean is an integer total over the sample size and must satisfy the GRIM granularity test, which exposes impossible means on the Likert scales ubiquitous in surveys. Out-of-scale values and impossible means require no modelling assumptions to interpret.
Score thresholds
- 0-1
- Values conform to their instruments' scales, precision, and granularity
- 2-3
- Integer-instrument values carry excess decimal precision
- 4-5
- Values fall outside an instrument's scale, or a mean fails GRIM for an integer instrument, consistent with errors or fabrication
Limitations
The check applies only to instruments in its dictionary and only when a header names one exactly, so an unrecognised instrument, an unusual abbreviation, or a combined header is missed; exact matching trades coverage for precision. The precision check assumes the column holds instrument values rather than summary statistics, so a column of means could be flagged. Decimal counting works on parsed values, so trailing zeros are lost. The GRIM step infers granularity from the reported mean under the integer-instrument assumption, guarded but not fully established by the in-scale restriction. General physiological plausibility is indicator T11, and the granularity tests on means and SDs are indicators T2 and T3.
References
- Brown NJL, Heathers JAJ. (2017). The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology. Social Psychological and Personality Science 8(4):363-369
- Van den Broeck J, Cunningham SA, Eeckels R, Herbst K. (2005). Data cleaning: detecting, diagnosing, and editing data abnormalities. PLoS Medicine 2(10):e267
- Carlisle JB. (2017). Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals. Anaesthesia 72(8):944-952