The guide · Part III · Chapter 10 of 14
Chapter 10 - When Clinical Data Becomes AI Data
From Clinical Meaning in the Age of Health Data & AI - a guide by Dr Balsam Khalil, MD. First edition, 2026.
Part III of four, chapter one of three. Part III sets out what artificial intelligence inherits from the record it is given. It follows Part I, which is also published here; between them sits Part II, which examines the standards themselves and is not published. Part III is written to be read without it.
"What has to be true of clinical data before a model can learn clinical reality from it - and why is 'is our data AI-ready?' the wrong question?"
The Clinical Problem
A hospital deploys a deterioration-prediction model. It scores every inpatient continuously from vital signs and raises an alert when risk crosses a threshold. It was developed on a large dataset, validated carefully, and performs well by every published measure.
Within a month the reports start to diverge. On the surgical wards it fires constantly, and nurses are describing alert fatigue within weeks. On the medical wards it is almost silent - including for two patients who deteriorated overnight without an alert.
The vendor examines the deployment and finds no fault. The model is running correctly, on correct data, exactly as specified.
The explanation, when somebody finally finds it, is not technical. The two wards chart observations differently. One records vital signs at fixed intervals for every patient. The other records them at intervals set by clinical concern - more often when a nurse is worried, less often when a patient is stable and settled.
The model had learned that observation frequency predicts deterioration. And it does. Powerfully. Because on the wards where it was developed, frequent observations were the trace of somebody who was already worried. The model was not detecting physiological decline at all. It was detecting nursing judgement - a signal of remarkable predictive quality, produced by people who already knew the answer.
Move it to a ward where observations are scheduled rather than judged and the signal disappears. Move it to a ward where they are judged more conservatively and it saturates.
Nothing was wrong with the data. The vital signs were accurate, timely, correctly coded, correctly exchanged, complete. The model learned something true. It just learned the wrong true thing, and no examination of data quality would ever have found it.
What you should see differently - by the end of this chapter
- You should understand why a model has no capacity to repair an ambiguity that a physician repairs without noticing.
- You should be able to explain why missingness in clinical data is a clinical decision, and why a model treats it as information.
- You should be able to name the mechanism that has been recurring since Chapter 4, and spot it in a new situation.
- You should be able to ask where a model's ground truth came from, and understand why that matters more than its architecture.
- You should be able to explain why AI-ready data is a category error, and what the right question is instead.
Chapter 10 · Scientific foundation
The Scientific Foundation
The Receiver With No Repair Capacity
When a physician reads a record she is continuously reconstructing, without being aware of it. She knows this consultant's notes are terse. She knows a blank field on this form usually means the question was not asked. She knows this problem list has not been cleaned in three years, and what the local abbreviation means. She fills in what is missing and discounts what is stale, silently and for free.
A model does none of that. It receives what is there and treats it as what is true.
And it is worse than simply lacking the repair. A human reader who cannot interpret something asks, or hedges, or writes unclear. A model has no hedging mechanism that was not deliberately built into it. It produces an output regardless.
Missingness Is a Clinical Decision
In most datasets outside medicine a missing value is an accident. A sensor failed. A respondent skipped a question. In clinical data it is almost never an accident.
A laboratory value is absent because nobody ordered the test. Nobody ordered it because nobody was concerned, or because the patient was too unwell for it to be relevant, or because the department does not routinely order it, or because it was ordered and the sample was rejected.
Every one of those is a clinical or organisational decision, and every one carries information about the patient and about the system.
Which means the pattern of what is missing is frequently more predictive than the values that are present. A model given only which tests were performed, and none of the results, can often predict outcomes well - because the ordering pattern encodes what physicians believed.
Add to that the kinds of absence catalogued in Part II - never recorded, not asked, sent and dropped, received and not rendered, code unmapped and therefore invisible - and a model receives a single undifferentiated blank standing in for at least six different situations. Several of them are strongly informative. It can distinguish none of them. Imputing that blank, which is standard practice, replaces six distinct clinical situations with one plausible number.
The Mechanism, Named
"Every property of the process that produced the data is available to be learned, and gets learned preferentially whenever it predicts the outcome better than the clinical signal does."
And here is the part physicians grasp immediately and data scientists often do not. The process very often predicts better, because the process was driven by people who already knew the answer.
Clinical data is not a neutral recording of physiology. It is the residue of decisions made by people exercising judgement about these specific patients.
Who was scanned, how often, with which machine, in which department, coded by whom under which rules - all of it sits downstream of clinical reasoning, and all of it therefore leaks the outcome.
In a study across three institutions, models trained to screen for pneumonia on chest radiographs generalised poorly to external sites - and the same models identified which hospital system a radiograph came from with essentially perfect accuracy, above 99.9% at two of them.
The authors noted that a model could learn that portable radiographs are more likely to contain pneumonia, which is true and reflects ordinary practice, and has very little to do with reading the image. A model that identifies the hospital with 99.9% accuracy is, to a substantial degree, a hospital detector. And a hospital is an excellent predictor of disease prevalence.
Where Did the Ground Truth Come From?
The mechanism is now named. The question that follows is: what was the model told the right answer was?
Coded diagnoses
Abundant, cheap and structured - and statistical artefacts produced under coding rules built for aggregation. A model trained on them learns what the coding system asserted, including the uncertainties it resolved on the way.
Extraction from narrative
Available at scale, and it reintroduces every modifier problem from Chapter 3: negation, hedged certainty, temporal reference to prior events, subject of record. This is how much imaging AI is labelled - the most structured data in medicine, annotated from the least structured part of it.
Clinician adjudication
The only origin that produces genuine clinical ground truth, and it is the most expensive by a wide margin. Which is why it is the rarest.
So the physician's question is not how the model works. It is what it was told the right answer was, and who decided that.
Chapter 10 · The claim examined
Why 'AI-Ready' Is a Category Error
This chapter has no standard to examine. The same discipline applies to the claim instead.
The claim: That clinical data can be prepared, standardised and cleaned to a state in which it is suitable for AI.
What genuine data preparation delivers
- Consistent representation, so the same clinical meaning is not fragmented across variants
- Explicit units and observation identity, so values are comparable
- Documented provenance of the extraction, so the dataset's boundaries are known
- Distinguished kinds of absence, where the source systems carried the distinction
- A stated population and time window
What it does NOT deliver
- It does not make the data fit for a question it was never asked about. Fitness is a pairing, not a property.
- It does not remove process signal. Standardising a code does not remove the fact that somebody chose to order the test.
- It does not repair labels. A cleaned dataset with labels derived from coded diagnoses still has labels derived from coded diagnoses.
- It does not restore what was never captured. No preparation recovers a physician's diagnostic reasoning if it was never represented.
- It does not survive a workflow change. The data describes how the work was done at the time.
- It does not substitute for prospective evaluation in the setting where the model will actually run.
"Data preparation makes a dataset legible. It does not make it appropriate. Legibility is a property of the data. Appropriateness is a judgement about a specific clinical question, and it needs somebody who understands that question."
The Coupling Rule - The Governance Finding
A hospital deploys a model that flags patients at risk of an adverse drug event. It performs well and is well received. Eighteen months later, the hospital completes a documentation improvement programme. Allergy capture moves from free text to a structured, coded field. Clinicians are required to record whether a condition is confirmed or suspected. Medication reconciliation is standardised at admission. Every one of these changes is an improvement. Each addresses a specific failure described in Part II of this guide. The model's performance degrades.
The investigation finds no defect. The model is receiving better data than it has ever received. But it was fitted to the old world - to a distribution in which allergy information was frequently absent, in which absence correlated with a particular admission route, and in which the presence of certain free-text patterns carried information that the structured field no longer generates. The model had learned the shape of the old workflow. The workflow changed. The model did not.
"A deployed model is coupled to the workflow that produced its training data. Changing that workflow is a change to the model, even though nobody touched the model."
No one in the documentation improvement programme knew that they were modifying a deployed clinical tool. No one in the AI governance process was told the programme was happening. Both groups were doing good work.
One question for your institution
Who is required to be informed when documentation practice changes, and is any deployed model on that list?
Readiness is not a property data can possess on its own; it is a governance outcome. The AI-Ready Clinical Data Framework describes conditions to be imposed, not a state to be discovered.
Chapter 10 · Clinical perspective & questions
The Physician's Apparatus
Physicians are routinely asked whether they trust AI. It is the wrong question, and it produces two equally useless postures - deference and refusal. The useful posture is neither, and it rests on something we already have. We are the process.
When somebody tells a physician that a deterioration model relies heavily on observation frequency, she does not need statistics to understand what has happened. She has ordered more frequent observations herself, on a patient she was worried about. She recognises the feature instantly as a trace of her own judgement.
A data scientist looking at the same feature sees a strong predictor with a plausible physiological story attached to it. There is nothing in the data that says otherwise.
The physician is the only person in the room who can look at a feature list and say: that one is us, not them.
This is not a claim to superior technical understanding. It is a claim about a specific and narrow expertise - knowing which marks in the record were made by somebody acting on a belief about this patient, rather than by the patient's body. Nobody outside clinical practice has that knowledge, and it cannot be acquired from the data.
There is something less flattering to us in this too. When a model underperforms, the clinical reflex is to blame the model. Often the model is faithfully reproducing something true about how we work - that we investigate more when worried, document more when concerned, code differently under different incentives. The model is a mirror before it is an instrument, and the reflection is frequently accurate and unwelcome.
Six Questions to Ask About Any Clinical Model
- What were this model's labels, where did they come from, and who decided the right answer?
- Which features does it rely on most, and are any of them traces of clinical decisions rather than patient states? (Ask for the feature list and read it as a clinician, not as a statistician. The features that are us rather than them will be visible to you and invisible to almost everyone else in the room.)
- Was it evaluated prospectively in a setting like ours, or only on held-out data from where it was built?
- How does it treat a missing value, and does it distinguish kinds of absence?
- What documentation or workflow assumptions is it making, and would they survive a change in our practice?
- Who is informed when our documentation practice changes, and is this model on that list?
Further Reading
- Zech JR et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs. PLOS Medicine 2018;15(11):e1002683.
- Reich C, Ostropolets A, Blacketer C, et al. OHDSI Standardized Vocabularies. Journal of the American Medical Informatics Association 2024;31(3):583-590.
- OHDSI - The OMOP Common Data Model specification and The Book of OHDSI.
- Literature on informative missingness in EHR data, dataset shift, external validation, and label provenance in clinical AI.
What to Reconsider
Recall the last time you were shown a model's performance figure - an accuracy, a sensitivity, an AUC. You accepted it as a statement about patients. It was a statement about agreement with a reference standard, and that standard was assembled by the same processes this chapter has traced: coded, classified, and shaped by what was recordable. You have been reading agreement with a record as agreement with a disease.