The Core Challenges: Data Usage, Provenance, and Consent

The fundamental issues surrounding AI and data privacy in healthcare can be categorized into three key areas: determining when and how data can be used to train AI models, tracking the origins of data used in AI development (provenance), and empowering patients to control the use of their data in AI-driven decisions.

Can Patient Data Be Used to Train AI Models?

The question isn’t simply whether data can be used, but rather, how to establish clear rules governing its use. Some data, by its nature or legal restrictions, should be excluded from AI training sets. Implementing these restrictions requires a multi-faceted approach. At the organizational level, policies must define which datasets, such as Electronic Health Records (EHRs), are permissible for AI training and which subsets are off-limits. Crucially, these policies must also extend to the individual patient level, allowing individuals to opt-out of having their data included in AI training.

Further details on distinguishing between data suitable for AI training and data requiring protection can be found here.

Tracking Data Provenance in AI Development

Once an AI model is created, it’s vital to maintain a detailed record of the data used in its training. This “data provenance” is essential for accountability and risk management. If concerns arise about an AI model’s performance or bias, knowing the source data allows for targeted investigation and mitigation. Provenance isn’t merely a record of what data was used, but also how it was used and when.

Learn more about the importance of provenance in AI applications here.

Controlling Data Usage in AI-Driven Clinical Decisions

Patients deserve control over how their data influences their care. This control extends to AI-driven clinical decisions. A key mechanism for achieving this is through the use of “Purpose of Use” codes. These codes define the specific reason for accessing patient data, enabling policies to permit or deny access based on the intended application. For example, AI used to assist with payment decisions would utilize a different Purpose of Use code (PMTDS) than AI supporting clinical treatment (TREATDS).

These Purpose of Use codes can be incorporated into patient consent forms, allowing individuals to specify their preferences. Organizations can also establish overarching policies, but these policies must be transparent and allow patients to override them when desired. A robust system requires both organizational governance and individual patient control.

Identifying AI-Generated Data

As AI becomes more integrated into healthcare workflows, it’s crucial to distinguish between data generated by AI and data originating from clinicians or other healthcare professionals. This is where data provenance again plays a critical role. By tagging data as having been produced by AI, healthcare providers can maintain transparency and ensure appropriate oversight. Tagging can be implemented at various levels – from the entire data resource to specific data elements – using security tags or comprehensive provenance records.

Crucially, provenance records should include details about the AI model version, the specific model used, and the portion of the patient’s chart that served as input. Further information on tagging AI-generated data can be found here.

Frequently Asked Questions

• Can AI models be trained on de-identified patient data? Yes, but de-identification is not foolproof. Re-identification risks remain, and careful consideration must be given to the potential for privacy breaches.
• What is the role of HL7 in AI and data privacy? HL7 is a standards organization that develops messaging protocols and data formats for healthcare. Its standards, particularly FHIR, are being adapted to support AI applications while addressing privacy concerns.
• How can patients access information about how their data is being used by AI? Patients should have the right to request information about the AI models being used, the data used to train those models, and how their data is being utilized in AI-driven decisions.
• What are the potential risks of using biased data to train AI models? Biased data can lead to AI models that perpetuate and amplify existing health disparities, resulting in unfair or inaccurate outcomes for certain patient populations.
• Is there a legal framework governing the use of AI in healthcare? The legal landscape surrounding AI in healthcare is still evolving. Existing regulations, such as HIPAA, apply, but new legislation may be needed to address the unique challenges posed by AI.