Turn AI adoption into a governed operating capability

Hospital leaders should manage artificial intelligence as an enterprise portfolio, not as a loose collection of demonstrations. The first requirement is a complete inventory of clinical, operational, administrative, cybersecurity, research, and patient-facing uses. That inventory should identify the business owner, technical owner, vendor, data used, intended users, decision supported, level of human review, regulatory status where applicable, and the consequences of failure. A use case that summarizes internal policy presents a different risk profile from a system that influences diagnosis, prioritizes patients, forecasts staffing, drafts patient communication, or recommends a denial. Governance should reflect those differences.

The NIST AI Risk Management Framework gives executives a practical vocabulary for this work through four connected functions: Govern, Map, Measure, and Manage. For a health system, governance defines accountability and acceptable use. Mapping clarifies the people, workflow, data, and context affected by a tool. Measurement tests performance, reliability, fairness, privacy, and security. Management determines whether the organization will deploy, restrict, redesign, monitor, or retire the system. These activities should continue after go-live because models, data, workflows, and vendor services can change.

Set decision rights before procurement or deployment

A multidisciplinary AI governance group should have a written charter and a clear route for escalation. Membership may include clinical leadership, nursing, operations, information technology, information security, privacy, compliance, legal counsel, quality, patient safety, finance, procurement, human resources, and patient or community representatives when appropriate. The group does not need to approve every low-risk automation. It does need authority to classify risk, require evidence, impose conditions, pause a use, and identify who accepts residual risk.

Procurement should require vendors to describe the product’s intended use, training and validation approach, known limitations, supported populations, security controls, data retention, subcontractors, model-update practices, audit support, incident notification, and exit provisions. Claims should be tied to evidence that matches the local use. Contract language cannot replace validation. A product may perform differently when the population, equipment, documentation practices, staffing model, or workflow differs from the environment in which it was tested.

Executives should also distinguish an AI-enabled medical device from a general administrative or analytic tool. The FDA’s AI-Enabled Medical Device List helps organizations identify devices authorized for marketing in the United States and links to public regulatory records. Presence on that list does not eliminate the need for local implementation controls, training, cybersecurity review, workflow design, or post-deployment monitoring.

Test the system in the workflow where it will be used

Local validation should begin with an explicit question: what decision or task is the tool expected to improve, and what harm could occur if it is wrong, unavailable, biased, delayed, or misunderstood? Evaluation should use representative local data when lawful and feasible, include meaningful subgroups, and compare results with the current process. Measures should address more than average accuracy. Leaders should consider false positives, false negatives, calibration, timeliness, override patterns, downstream workload, accessibility, patient communication, and effects on care teams.

Human oversight must be designed rather than assumed. The organization should specify who reviews an output, what information that person receives, how disagreement is documented, and when the system must be bypassed. Training should explain limitations and common failure modes. Interfaces should make uncertainty and source information visible when those details are available. Staff need a simple way to report unsafe, misleading, or unexpected behavior without fear that raising a concern will be treated as resistance to innovation.

After deployment, an accountable owner should review a small set of decision-grade measures on a defined cadence. Monitoring should include performance drift, changes in input data, user adoption, overrides, incidents, patient complaints, security events, vendor updates, and unintended workflow consequences. Material changes should trigger reassessment. Retirement criteria should be established before the tool becomes operationally indispensable.

A 90-day sequence for responsible scale

• Days 1–30: establish the inventory, governance charter, risk tiers, approved-use policy, and an escalation path for incidents or unapproved tools.
• Days 31–60: select a small number of high-value use cases, document intended outcomes and harms, complete security and privacy review, and define local validation plans.
• Days 61–90: conduct controlled pilots, train affected teams, confirm monitoring dashboards, test downtime procedures, and make an explicit deploy, revise, restrict, or stop decision.

Generative AI requires additional attention because outputs can be fluent without being reliable. The NIST Generative AI Profile is a companion to the AI RMF that addresses risks specific to generative systems. Health systems should prohibit unsupported clinical or legal conclusions, protect sensitive information, evaluate retrieval and citation behavior, and maintain review proportional to the consequence of the task.

Board reporting should stay concise. A quarterly view can show the number of active and proposed uses by risk tier, validation status, incidents, material model or vendor changes, overdue controls, and measurable operational or clinical outcomes. The board should be able to see where AI creates dependency as well as value. It should also know whether leaders can stop a system safely, recover the affected workflow, and communicate with staff and patients. Clear reporting turns oversight into a repeatable management discipline instead of a technology presentation.