What Is AI Governance, and Why Does It Matter for Regulated Industries?

By Jean McCabe, AI Strategist, Abacus · Published November 5, 2026 · Updated November 5, 2026

Short answer: AI governance is the set of policies, controls, and records guiding how an organization uses AI, including continuous monitoring, internal accountability standards, and data protection requirements. In financial services and healthcare, it is important because existing compliance standards are already starting to include AI use. No regulator waits for an AI-specific law before asking how a tool used the private data of a patient or client.

Even for organizations that did not choose to adopt AI, it has arrived within tools they’re already using, like their email platform, documentation assistant, or CRM. Employees may also be adopting unapproved AI apps or assistants outside of leadership or IT’s view, known as Shadow AI. Frequently, AI Governance is the step that comes after discovery – which is the wrong order, but unfortunately the common one.

What is an AI governance framework?

An AI governance framework is a structured way to manage AI risk across its whole life: deciding what AI is allowed, assessing each use before it goes live, testing outputs, assigning a named owner, managing who has access to tools, identifying and controlling data access, and keeping records. Two formal frameworks come up most often: the NIST AI Risk Management Framework, which is voluntary guidance, and ISO/IEC 42001, which is a standard an organization can be certified against.

The NIST AI Risk Management Framework is a widely used starting point in the United States. NIST released version 1.0 on January 26, 2023. It organizes work into four functions: Govern, Map, Measure, and Manage. NIST added a Generative AI Profile, NIST-AI-600-1, on July 26, 2024 to cover risks specific to generative tools.

ISO 42001, published in 2023, is the international management system standard for AI, though relatively few vendors hold it so far. An external auditor can certify an organization against it, and certification matters most to companies that build or sell AI products. For a firm that uses AI, the standard is most useful during vendor due diligence. Ask AI vendors whether they are certified or working toward certification, and what the certification covers.

Neither framework is law across any global jurisdiction. Both are ways of organizing the work so that when a regulator asks how a decision was made, there is an answer on file.

What rules apply to AI in healthcare?

In the United States, existing healthcare law governs how organizations use AI. HIPAA applies in full to any tool that handles protected health information (PHI). If a tool creates, receives, maintains, or transmits PHI on the organization’s behalf, the vendor is a business associate and a business associate agreement is required. The Security Rule safeguards apply exactly as they would to any other system.

One federal rule addresses providers’ use of AI directly. Under Section 1557 of the Affordable Care Act, covered entities, which include most providers that receive federal financial assistance, must make reasonable efforts to identify patient care decision support tools that use race, color, national origin, sex, age, or disability as inputs. For each of those tools, they must make reasonable efforts to reduce the risk of discrimination from its use. This requirement has applied since May 1, 2025. It covers AI-based clinical decision support as well as simpler scoring tools.

The most detailed AI rule in US healthcare applies to health IT developers. The HHS Assistant Secretary for Technology Policy and Office of the National Coordinator for Health IT (ASTP/ONC) published the HTI-1 final rule on January 9, 2024, and it took effect on March 11, 2024. Developers had to meet its decision support requirements by December 31, 2024. For predictive decision support tools, certified health IT developers must make 31 source attributes available to users. These attributes describe how a model was trained, validated, and tested for fairness, and how it is maintained.

HTI-1 also requires those developers to run intervention risk management, covering risk analysis, risk mitigation, and governance. The risks named are validity, reliability, robustness, fairness, intelligibility, safety, security, and privacy.

These requirements may change. In December 2025, ASTP/ONC proposed the HTI-5 rule, which would remove the source attribute and risk management requirements for predictive tools. The proposal had not been finalized as of October 2026, so the HTI-1 requirements still apply.

There is an important limit. HTI-1 binds certified health IT developers. It does not reach a model a provider organization builds itself, or one a third-party supplies outside a certified module. A practice using an AI scribe or an AI triage tool cannot assume someone else has done this work.

What rules apply to AI in financial services?

The UK takes a principles-based, regulator-led model. The FCA, PRA, and ICO each have their own expectations, and the PRA’s SS1/23 model risk management standard is beginning to mold how UK-regulated firms govern AI.

In the UAE, Abu Dhabi and Dubai are working to orient the region as a global AI hub while introducing layered compliance regimes. The DIFC’s AI-specific Regulation 10 took effect in January 2026, and the UAE’s federal PDPL becomes fully enforceable on January 1, 2027.

In the United States, the SEC has no AI-specific rule for advisers. It proposed one in July 2023, covering conflicts arising from predictive data analytics, and withdrew it in June 2025 as part of a withdrawal of fourteen proposed rules. No replacement has been adopted.

Existing rules still reach AI use directly. An adviser’s fiduciary duty applies to advice generated with AI. Antifraud provisions apply to how AI use is described. With the SEC, the compliance rule at Rule 206(4)-7 requires written policies reasonably designed to prevent violations, and AI is now part of what those policies must cover.

AI note-takers need specific attention. Recordings, transcripts, and summaries of client or internal meetings can be records a firm must retain, and they can capture material non-public information. Firms should decide which meetings AI note-takers may join, where the output is stored, and who can access it.

Enforcement has focused on claims rather than models. On March 18, 2024 the SEC announced its first AI-washing actions, against Delphia (USA) Inc. and Global Predictions Inc., with civil penalties of $225,000 and $175,000. Global Predictions had described itself as the “first regulated AI financial advisor.”

In the US, enforcement so far has targeted what firms said about AI, not how well the models worked, making it clear that marketing language is a governance surface.

What does the EU AI Act require, and when?

The EU AI Act reaches beyond the EU and across industries. It can apply to organizations outside Europe that place AI systems on the EU market, so a US firm with EU clients should check its exposure rather than assume it is out of scope. The Act entered into force on August 1, 2024 and phases in over several years. Some of those phases moved in 2026.

Prohibited practices and the AI literacy duty have applied since February 2, 2025. The EU Digital Omnibus on AI changed the literacy duty: providers and deployers must now take measures that support AI literacy among their staff in accordance with their specific roles. The original text required them to ensure a sufficient level of AI literacy. Obligations for general-purpose AI models have applied since August 2, 2025.

The high-risk deadlines were postponed. The Digital Omnibus on AI, Regulation (EU) 2026/1744, was published in the Official Journal on July 24, 2026 and entered into force on July 27, 2026. Stand-alone high-risk systems under Annex III moved from August 2, 2026 to December 2, 2027. Embedded high-risk systems under Annex I moved from August 2, 2027 to August 2, 2028.

Not everything moved. The Article 50 transparency obligations, which require telling people they are interacting with an AI system, stayed on the original August 2, 2026 date. Watermarking has a grace period for already-marketed systems that runs to December 2, 2026.

The practical read for firms is simple. The postponement buys time on the heaviest obligations. It does not remove the transparency duty, and it does not change anything a regulator outside of the EU expects.

How AI governance compares across the two sectors

The obligations differ more than the technology does. The table below sets the two verticals side by side.

How do you build an AI governance policy?

Start with an inventory before writing the policy. List every AI tool in use, including features switched on inside existing software. Then assign each one an owner, a permitted use, a data boundary, and a review date. A policy written before the inventory tends to govern tools nobody uses and miss the ones staff rely on.

Six elements cover most of what a regulated organization needs:

1. An approved-use list. What AI may be used for, and what it may not touch. Name the prohibited uses explicitly.
2. A data boundary. What information may be entered into which tool. For healthcare, that means saying plainly where protected health information may go.
3. A named owner per use. One person accountable for each deployed tool.
4. Human review requirements. Which outputs need a person to check them before they reach a patient, a client, or a filing.
5. Vendor due diligence. What the vendor does with inputs, whether data trains a model, where it is stored, and what contract terms apply.
6. A record of decisions. What was approved, by whom, on what basis, and when it will be reviewed.

The last one is what turns a policy into evidence. A regulator asking about AI use will ask to see the decisions the organization made and the records behind them.

What goes wrong with AI governance?

The most common failure is governing the pilot and missing the changes that happen over time. Organizations write careful rules and set boundaries for AI projects and tools at a point in time, not accounting for updates that put a tool out of policy, changes in regulations, or employees taking matters into their own hands and using unapproved AI tools.

Five problems appear repeatedly:

• Shadow AI. Staff paste sensitive material into personal accounts because the sanctioned tool is slower. The control is an approved tool that is easy to use, plus a technical block on unapproved ones.
• No inventory. An organization that cannot list its AI tools cannot govern them. Features enabled inside existing platforms are the usual blind spot.
• Treating the vendor’s assurance as the control. The evidence is in the contract terms and the data flow.
• Overstating AI use in marketing. This is the behavior the SEC has actually penalized. Claims about AI belong under the same review as any other performance claim.
• No human review on outputs that reach a client or patient. Accountability stays with the person who acted on the output.

The security dimension is measurable. IBM’s Cost of a Data Breach Report 2026 found that one in four malicious breaches were AI-enabled, up 56% year over year, averaging $6 million — about $1 million above the global average. More than 20% of organizations reported breaches targeting AI models or applications. The two leading routes in were compromised APIs, applications or plug-ins, and cloud misconfigurations affecting AI workloads, each at 27%.

Where should a regulated organization start?

Start with three steps. Inventory the AI already in use, classify each use by the harm it could cause, and write the data boundary. Those three steps address the most immediate risk and produce the record a regulator would ask for.

Map to a framework once the inventory exists. The NIST AI Risk Management Framework is a reasonable default in the United States, and its Govern, Map, Measure, Manage structure gives a shape to the work without requiring certification.

Then decide what needs human review. Not every AI output carries the same weight. A drafted meeting summary is not a clinical recommendation or a suitability determination, and treating them identically usually means the important reviews get skipped.

About Abacus

Abacus is a managed IT and cybersecurity provider serving financial services and healthcare organizations, with infrastructure and processes designed to support compliance requirements. Abacus’ Governance, Risk, and Compliance (GRC) services are purpose-built for regulated firms navigating the complexity of global AI regulation. Learn how our AI Risk & Readiness Assessment can help you build a secure, compliant, and scalable AI strategy before risks become challenges by connecting with our team.

Next step

Get in touch with our experts to build an AI strategy you can trust with Abacus AI Governance, designed to bring structure, oversight, and accountability to AI adoption. Book an AI Risk & Readiness Assessment for a structured, evidence-based picture of where you stand and how ready you are to adopt generative AI tools, along with a prioritized plan for what to do next.