D11Statistical analysisFabrication DetectionLayer 2 (Contextual)

Conditional Correlations Absent

Tests whether controlling for a third variable changes the correlation between two others, as it does in real data with genuine causal structure. When age relates to both blood pressure and cholesterol, holding age fixed should weaken the apparent link between the latter two. In data where each variable was generated independently (a signature of machine-fabricated datasets), conditioning barely changes anything, because there is no shared structure to remove. The indicator compares each pairwise correlation with its partial correlation given a third variable and flags data where conditioning has almost no effect.

Technical description

A contextual screen for the absence of conditional structure in individual-patient data. It requires at least five numeric columns with at least ten non-missing values each, and excludes constant columns (a zero-variance column has no defined correlation and would propagate NaN and collapse the analysis). It forms the correlation matrix and enumerates triples; for each triple with all three pairwise |r| above 0.05 it computes the partial correlation of the first two given the third (r_XY.Z = (r_XY - r_XZ*r_YZ)/sqrt((1-r_XZ^2)(1-r_YZ^2))) and records the absolute change (delta) from the unconditional correlation. It summarises the mean delta and the proportion of triples with delta above 0.15. A small mean delta means conditioning does almost nothing (the independence signature) and drives the score; a low proportion of meaningfully-changed triples adds to it.

How it works

Layer 2 (contextual): the correlation matrix is computed over complete-case numeric data and every ordered triple of distinct variables is considered. Triples with any pairwise |r| at or below 0.05 are skipped, as are those whose partial-correlation denominator collapses. For each remaining triple the delta is the absolute difference between unconditional and partial correlation. With at least three valid triples, the mean delta maps to a base score: below 0.03 gives 4.0, below 0.07 gives 3.0, below 0.12 gives 2.0, below 0.18 gives 1.0, else 0. A proportion of triples with delta above 0.15 below five percent adds 1.0, capped at 5.0. The pairwise formula ignores the sample size, so Fisher's z-transform brings it in (SE = 1/sqrt(n-4) for one conditioning variable), giving a two-sided p that each partial correlation differs from zero and an n-aware test of whether each conditioning change exceeds twice that SE. Metadata records n_triples, mean_delta, median_delta (robust to a few extreme triples), proportion_delta_gt_015, columns_tested, mean_partial_pvalue, prop_partial_significant, and prop_significant_conditioning.

Why this matters

Real multivariate data is shaped by shared causes, so relationships are entangled: a correlation generally changes, often substantially, once a third related variable is held constant, because part of the association ran through it. Independently sampled data (the default of a language model asked to produce a dataset) has no such entanglement, so partial correlations nearly equal their unconditional counterparts. Taloni and colleagues showed a model can fabricate a clinical dataset lacking realistic dependence, and absent conditional structure is a sharper form of that defect. The principle predates language models: Al-Marzouki and colleagues used correlation structure to separate real from invented trials, and Simonsohn exposed fabrication from relationships a fabricator cannot reproduce. Conditional correlations are powerful because reproducing a believable web of confounding, not just a few marginal correlations, is beyond a naive generator.

Score thresholds

0-1: Conditioning changes correlations as expected of data with real structure.
2-3: Conditioning has little effect, suggesting weak or absent confounding structure.
4-5: Conditioning has almost no effect across triples, consistent with independently generated variables.

Limitations

Requires individual-patient data with at least five qualifying numeric variables, so smaller or summary-only studies are out of scope. It uses listwise deletion, so heavy missingness shrinks the effective sample. Partial correlation here controls for one variable at a time and assumes linear relationships, so non-linear dependence or higher-order confounding is not captured, and a real dataset genuinely lacking confounders could show small deltas without being fabricated. The triple enumeration is over ordered triples, so the same pair conditioned on the same third variable is counted in both orderings, which does not bias the mean or proportion but inflates the raw count. The thresholds and the 0.15 delta cutoff are heuristic. The unconditional correlation structure (flat matrices, near-perfect correlations) is assessed by D1 and S17.