Radiology AI Quality Assurance

Radiology AI quality assurance is the set of checks that keep a deployed imaging model trustworthy over time. Vendor validation and FDA clearance describe how a model performed on the manufacturer's test data. They do not tell a department how the model performs on its own scanners, protocols, and patient mix, or whether that performance holds a year later. QA and model management fill that gap.

This article sits under the AI Radiology Workflow pillar and connects to the AI Governance hub. It is written for radiologists, imaging physicists, informatics teams, and quality leaders.

Before Go-Live: Local Acceptance Testing

• Representative sample. Assemble local studies that span the scanners, protocols, and patient population the model will see, including harder cases and known-negative cases.
• Reference standard. Define how ground truth is established for the test set, whether by expert consensus, follow-up, or pathology.
• Performance check. Compare model output against the reference standard and against the performance the vendor reported. Investigate gaps.
• Failure review. Look specifically at false negatives and false positives to understand the clinical pattern of errors.
• Workflow test. Confirm output timing, display, and worklist behavior in a test environment before real studies flow through.

After Go-Live: Ongoing Monitoring

• Discrepancy review. Sample cases where AI output, the preliminary read, and the final report disagree. Feed patterns back to the governance committee.
• Real-world performance. Track sensitivity, false-positive rate, and positive predictive value on production data, not just the launch test set.
• Drift detection. Watch for shifts in input data, such as a new scanner or protocol, and for shifts in output distribution over time.
• Reader interaction. Monitor override rates and whether readers are becoming over-reliant on AI marks or dismissing them wholesale.
• Downstream workload. Measure whether the tool changed follow-up volume, additional imaging, or report addenda.

Model Management

Every deployed model needs an owner, a version record, and a change process. When a vendor ships an update, the department should know what changed, decide whether local re-testing is required, and record the decision. A predetermined change control plan from the vendor makes this easier by describing in advance which modifications are expected and how they will be validated. Retirement is part of management too: a model that no longer performs, or whose clinical use has ended, should be removed from production and from the worklist.

Where Registries and Standards Fit

Imaging QA programs and national efforts such as ACR Assess-AI give departments a structured way to collect real-world performance data and compare it across sites. The ACR imaging AI practice parameter and the NIST AI Risk Management Framework provide the operating structure that local QA plugs into.

QA Program Checklist

• Is there a documented acceptance test for every model before clinical use?
• Who owns each deployed model, and where is the version history?
• What production metrics are reviewed, how often, and by whom?
• How is drift defined, detected, and escalated?
• What triggers local re-validation after a vendor update?
• How is a failing or obsolete model removed from production?

Related AI Medicine Now Topics

• AI Radiology Workflow
• Monitoring and Observability
• AI Model Inventory
• Monitoring Clinical AI After Deployment
• Imaging Clinical Validation
• Radiology Workflow Orchestration

Reviewed: August 30, 2026. Next review: November 30, 2026.