S17Statistical analysisStatistical ConsistencyLayer 2 (Contextual)

Correlation Impossibilities

Builds the correlation matrix from a study's raw participant-level data and looks for structures unlikely in genuine measurements: two variables almost perfectly correlated, a matrix so redundant its determinant collapses toward zero, five or more variables implausibly independent of one another, and suspiciously uniform eigenvalues. Each points to data that were derived, duplicated, or simulated rather than measured.

Technical description

A contextual screen on the correlation structure of individual-patient data. It selects numeric columns with at least five non-missing values and more than one distinct value (constant columns are excluded, since a zero-variance column has no defined correlation and would propagate NaN through the matrix and disable the checks). With at least three columns it forms the Pearson correlation matrix over complete cases and computes the largest off-diagonal |r|, the determinant, the eigenvalues, the maximum variance inflation factor, and the condition number. Four checks: max off-diagonal |r| > 0.99 (near-perfect relationship, usually a derived or duplicated variable); determinant < 0.001 or maximum variance inflation factor > 10 (severe multicollinearity, the latter a dimension-robust cutoff of Belsley, Kuh and Welsch 1980 and Fox and Monette 1992); with at least five columns, determinant > 0.95 (suspicious independence) and eigenvalue coefficient of variation < 0.1 (suspiciously uniform spectrum). The number and severity of flags set the score.

How it works

Layer 2 (contextual): the correlation matrix is computed from complete-case numeric data. The largest absolute off-diagonal entry is compared against 0.99; the determinant against 0.001 and the maximum variance inflation factor against 10 (multicollinearity) and, with at least five columns, against 0.95 (independence); the eigenvalues' coefficient of variation against 0.1 (uniformity, at least five columns). Score: no flags 0; one non-extreme flag (multicollinearity or uniform eigenvalues) 2.0; one extreme flag (perfect correlation or suspicious independence) 4.0; two or more flags 5.0. Perfect-correlation is error severity, others warning. Metadata records columns_used, det, max_off_diag_r, eigenvalue_cv, max_vif, and condition_number.

Why this matters

The joint structure of several variables is far harder to fabricate convincingly than any single variable, so the correlation matrix is a sensitive place to look for invented data. Simonsohn showed fabrication can be exposed from the relationships among reported quantities, because a fabricator struggles to reproduce realistic dependence. Al-Marzouki and colleagues examined exactly this multivariate structure (variances and correlations of trial variables) to distinguish fabricated from genuine data, and Carlisle's re-analyses treat improbable joint structure as an integrity signal. Both extremes inform: a near-perfect correlation or near-singular matrix indicates variables copied or derived from one another, while implausibly independent variables with uniform eigenvalues indicate data drawn independently at random (a naive simulation) rather than measured from a real system where variables share influences.

Score thresholds

0: The correlation structure is unremarkable.
2: One moderate anomaly: severe multicollinearity or uniform eigenvalues.
4: One strong anomaly: a near-perfect correlation or implausible mutual independence.
5: Two or more correlation anomalies together.

Limitations

Requires individual-patient data, so a study reporting only a correlation table or summary statistics is out of scope. The matrix uses listwise deletion, so heavy missingness reduces the effective sample and can distort structure. The suspicious-independence and uniform-eigenvalue checks partly overlap (for a correlation matrix, uniform eigenvalues imply a near-identity high-determinant matrix), so the two can flag the same near-independent dataset and jointly reach the maximum score; this rewards concurrence but means they are not fully independent. The thresholds 0.99, 0.001, 0.95, 0.1, and a variance inflation factor of 10 are directional; because the determinant shrinks with column count, the dimension-robust maximum VIF and condition number are reported and the multicollinearity flag also fires on a factor above 10. A genuinely strong relationship between two closely related clinical measures can legitimately produce a high correlation, so a flag prompts inspection for derivation or duplication rather than proving fabrication. The reported-correlation-matrix impossibility check, including positive-semi-definiteness, is indicator T10.