AI in medical devices · Medium · 13 min lesson
Data splitting, leakage, privacy and data management records for audit
GMLP principle 4, EU AI Act Article 10, GDPR Articles 4, 5, 9 and 35, 45 CFR 164.514(b), ISO 13485:2016 clauses 4.2.5 and 7.3.6
What you will learn
- Design train, validation and test splits that are independent by patient, site, device and time, and lock the test set.
- Detect and prevent data leakage, and explain how leakage inflates performance claims.
- Apply privacy controls and keep data management records, including manifests and versions, that an auditor can trace.
Key takeaways
- Split at the level that creates dependence, usually the patient, stratify, and freeze a manifest with its seed (GMLP principle 4).
- Lock the test set, use it once, hold out external sites and devices, and disclose every model evaluated on it (clause 7.3.6).
- Hunt leakage on purpose: overlap, duplicates, metadata-only models, shortcuts and preprocessing order.
- Document the lawful basis, minimisation, impact assessment and de-identification method; pseudonymised data stay personal (GDPR Articles 4(5), 5, 9, 35).
- Lineage records connect each reported result to exact data, split, code and model versions (clause 4.2.5).
Free practice question
Brindle splits 26,000 skin photos randomly by image. Patients contributed between 1 and 14 photos each, from several angles. The report states the test AUC is 0.97.
What is the main problem with this test result?
Photos of the same patient or lesion can be on both sides of the split, so the test is not independent and the result is inflated.
Correct. GMLP principle 4 asks that patient and acquisition sources of dependence be considered and addressed.
The AUC is too high to be reported in any technical file.
Not quite. There is no maximum; the issue under GMLP principle 4 is that independence of the test is not shown.
Random splits are forbidden by the AI Act.
Not quite. Article 10 does not forbid random splits; it expects governance of data, and independence comes from GMLP principle 4.
The test set should be larger than the training set.
Not quite. Set proportions are not the problem; dependence between the sets is, according to GMLP principle 4.
Key terms
- data leakage: Information that would not be available in real use entering training or the test, inflating performance.
- test set: Data kept independent of development to give one unbiased estimate of performance.
- split manifest: A frozen record of which file and patient belongs to which set, with seed and script version.
- stratification: Splitting so each set has a similar mix of classes and subgroups.
- external validation: Testing on sites, devices or periods that never appear in training.
- pseudonymisation: Processing so data cannot be attributed to a person without separate additional information; still personal data (GDPR Article 4(5)).
- impact assessment: A documented assessment of high risk personal data processing (GDPR Article 35).
- lineage record: A record linking a result to the exact data, split, code and model versions that produced it.
Official sources
- Good Machine Learning Practice for Medical Device Development: Guiding Principles (Health Canada, with FDA and MHRA)
- Regulation (EU) 2024/1689 (Artificial Intelligence Act), Official Journal
- Regulation (EU) 2016/679 (General Data Protection Regulation), Official Journal
- HHS: Guidance regarding methods for de-identification of protected health information under the HIPAA Privacy Rule
- ISO 13485:2016 Medical devices - Quality management systems - Requirements for regulatory purposes
Go further
The full lesson has narrated slides, more scenario questions with explained answers and a simulation.
Open the AcademyWatch on YouTubeIllustrative example. Educational only, not legal advice.