Sunday, November 10, 2024
Home Latest Health News Generative AI ‘drift’ and ‘nondeterminism’ inconsistences are important considerations in health care applications

Generative AI ‘drift’ and ‘nondeterminism’ inconsistences are important considerations in health care applications

by Medical Xpress
0 comment


Credit: Unsplash/CC0 Public Domain

Samuel (Sandy) Aronson, ALM, MA, executive director of IT and AI Solutions for Mass General Brigham Personalized Medicine and senior director of IT and AI Solutions for the Accelerator for Clinical Transformation, is the corresponding author of a paper published in NEJM AI that looked at whether generative AI could hold promise for improving scientific literature review of variants in clinical genetic testing. Their findings could have a wide impact beyond this use case.

How would you summarize your study for a lay audience?

We tested whether generative AI can be used to identify whether contain information that can help geneticists determine whether genetic variants are harmful to patients. While testing this work, we identified inconsistencies in generative AI that could present a risk for patients if not adequately addressed. We suggest forms of testing and monitoring that could improve safety.

What question were you investigating?

We investigated whether generative AI can be used to determine: 1) whether a scientific article contains evidence about a variant that could help a geneticist’s assessment of a genetic variant and 2) whether any evidence found about the supports a benign, pathogenic, intermediate or inconclusive conclusion.

What methods or approach did you use?

We tested a generative AI strategy based on GPT-4 using a labeled dataset of 72 articles and compared generative AI to assessments from expert geneticists.

What did you find?

Generative AI performed relatively well, but more improvement is needed for most use cases. However, as we ran our tests repeatedly, we observed a phenomenon we deemed important: running the same dataset repeatedly produced different results. Through repeated running of the test set over time, we characterized the variability. We found that both drift (changes in model performance over time) and nondeterminism (inconsistency between consecutive runs) were present. We developed visualizations that demonstrate the nature of these problems.

What are the implications?

If a clinical tool developer is not aware that can exhibit significant drift and nondeterminism, they may run their test set once and use the results to determine whether their tool can be introduced into practice. This could be unsafe.

What are the next steps?

Our results show that it could be important to run a test set multiple times to demonstrate the degree of variability (nondeterminism) present. Our results also show that it is important to monitor for changes in performance (drift) over .

More information:
Samuel J. Aronson et al, GPT-4 Performance, Nondeterminism, and Drift in Genetic Literature Review, NEJM AI (2024). DOI: 10.1056/AIcs2400245

Citation:
Q&A: Generative AI ‘drift’ and ‘nondeterminism’ inconsistences are important considerations in health care applications (2024, August 12)
retrieved 12 August 2024
from https://medicalxpress.com/news/2024-08-qa-generative-ai-drift-nondeterminism.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.





Source link

You may also like

Leave a Comment

Edtior's Picks

Latest Articles

All Rights reserved, site designed by Yellohost.co.za