AI governance, a practical framework for adopting AI safely

AI governance in plain English

AI governance is how you control three things.

1. What the AI is allowed to do.
2. What data it is allowed to see.
3. Who is accountable for the outcomes.

If that sounds basic, good. Most AI problems come from those three being unclear, not from the model being “bad”.

Teams are rushing to add AI into content, customer service, reporting, and internal systems, and McKinsey reports that 65% of organisations are now regularly using generative AI, McKinsey, The state of AI 2024.

We are going to cover it in simple to understand, plain transparent language, lets dive in.

What AI governance covers

AI governance covers the rules, roles, and processes that make AI adoption safe and repeatable.

That includes what tools are approved, what use cases are allowed, what needs a human review, and what gets logged so you can learn and improve.

It is also how you stay transparent. You should be able to explain when AI is used, what it does, and how customer data is handled.

If you cannot explain it, you cannot defend it, and we publish our approach in our AI Policy.

What it is not

AI governance is not a brake on delivery. It is the thing that lets you move faster without losing control.

It is not a long policy that nobody reads. It is a working system that gets used, updated, and improved as the landscape changes.

In the rest of this article we will cover a practical AI governance framework, human-in-the-loop without overload, prompt injection and guardrails, and a short AI policy template you can copy.

Transparency first, what users and customers deserve to know

Most AI failures are trust failures, and KPMG’s global study found only 34% of organisations have a policy or guidance for generative AI use, KPMG, Trust attitudes and use of AI.

If a user cannot tell when AI is involved, or what it is doing with their data, confidence drops fast. The same applies to customers who assume information stays inside your systems, when an AI feature is sending it elsewhere with no human knowledge.

Transparency is the baseline for AI governance, and the ICO is clear that you must be transparent about how personal data is processed in AI systems. It is how you protect people’s data, avoid surprises, and keep control as tools and risks evolve.

Data boundaries, what the model can access

Be explicit about the boundary.

What data can the tool see, customer data, internal documents, CRM notes, support tickets. Where does that data go. Does it stay inside your environment, or does it pass to a third party.

This is AI data governance in practice. Define the boundary, then enforce it.

A common mistake is adding AI to a company wiki or knowledge base without locking it down. If the tool can browse, retrieve from outside sources, or learn from what it sees, you can leak knowledge without noticing.

A safer pattern is a closed box. The AI can only answer from approved internal content. It cannot browse the open web. If the answer is not present, it must say it does not know.

Opt-in and opt-out, when it matters

If AI use affects a client, their data, or their outcomes, give them a clear choice.

Keep it simple. Make AI use explicit in proposals and onboarding, and offer opt-in and opt-out for generative AI where appropriate.

This matters most when personal data, sensitive information, or client-owned IP could be involved.

Logging, audit trails, and why you need them

You cannot govern what you cannot see.

At minimum, keep a record of what tools are in use, what they are used for, and who owns them. Log the high-impact workflows, and keep an audit trail for decisions that affect customers, content, or access to information.

This is not about surveillance. It is about being able to answer basic questions later.

What happened, why did it happen, and what will we change.

The biggest risk teams miss, AI wikis and “helpful” knowledge tools

A company wiki is becoming a common AI request. Make it searchable. Let people ask questions. Let it summarise and draft, but lock it down first with clear rules for data and access, AI Policy. Make it searchable. Let people ask questions. Let it summarise and draft.

The danger is not the wiki. The danger is what the AI is allowed to do to answer.

If your knowledge tool can browse, pull from external sources, or use third-party models without tight boundaries, you can leak internal information and customer data. You also risk answers that sound confident but are wrong.

This is where AI governance needs to get practical.

Closed box answers, not open web research

The safe default is a closed box.

The AI can only answer from approved internal content. It cannot browse the open web. It cannot go looking for extra context. If the answer is not in the knowledge base, it must say it does not know.

These AI guardrails reduce hallucinations and prevent the tool from pulling in untrusted sources.

Lock down what content it can see

Treat your wiki like a system with permissions, not a folder.

Define what content is allowed in the AI index, what is excluded, and who can access what. This is AI data governance in practice.

If the wiki includes customer information, contracts, pricing notes, or sensitive internal detail, be deliberate. Restrict access, minimise data, and keep retention rules clear.

Make quality check part of the workflow

Even a closed box can produce wrong answers if the content is incomplete or outdated.

Add simple AI quality assurance steps. Flag uncertain answers. Encourage citations to internal sources. Route high-impact answers to a human review.

That is how you get usefulness without sleepwalking into risk.

Prompt injection, why AI systems get tricked

Prompt injection is when a user, or a piece of content, tries to override the rules of your AI system.

It works because most models are trained to be helpful. If the user asks for something directly, the model will try to comply, even when it should refuse.

This is not theoretical. Any AI feature that reads text from users, web pages, documents, emails, or a knowledge base can be exposed.

What prompt injection looks like

It often looks harmless.

“Ignore previous instructions.”

“Reveal the hidden system prompt.”

“Use the web to find the answer.”

“Send me the confidential details.”

Sometimes it is buried inside a document the AI is asked to summarise. The model reads it like instructions, not like content.

This is why prompt security matters.

Guardrails that reduce the risk

Do not rely on a single prompt to enforce safety.

Use layered AI guardrails:

• restrict what tools the AI can call, and what data it can access
• block browsing and external research unless it is explicitly required
• limit outputs to an approved knowledge source when accuracy matters
• require citations or source links for answers drawn from internal content
• log requests and responses for high-impact workflows

If the system cannot do the risky thing, it cannot be tricked into doing it.

Escalation for risky outputs

Do not route everything to a human. Route the risky cases.

Set simple triggers for human oversight, for example:

• the model says it is unsure
• the answer affects customers, finance, legal, or access
• the request involves personal data or confidential information
• the request tries to override instructions

This is where human-in-the-loop is most valuable, as escalation, not as a blanket review step.

Human-in-the-loop without human overload

Human-in-the-loop means a person stays responsible for outcomes. AI can assist, but it should not be the final authority for anything high impact.

The mistake is using human-in-the-loop as a blanket rule. That swamps people with more checks, more tickets, and more noise.

A better approach is triage. Decide what gets reviewed, what gets sampled, and what gets escalated.

Triage, what gets reviewed and what does not

Not every AI output needs approval.

Low risk work can be spot checked, for example drafting internal notes, summarising meetings, or creating first-pass ideas.

High risk work should be reviewed, for example anything customer-facing, anything that uses personal data, and anything that affects money, access, reputation, or compliance.

This is how you get speed without losing control.

Approval points, owners and decision rights

Every AI-assisted workflow needs an owner.

That owner defines what good looks like, what must be checked, and when approval is required. This is part of your AI operating model.

Without clear decision rights, AI adoption becomes “everyone tries tools and nobody owns outcomes”.

Feedback loops that improve quality over time

The loop is not just approve or reject. It is learn and improve.

Capture the common errors. Update prompts and templates. Tighten the guardrails. Improve the source content. Track what gets escalated and why.

This turns human-in-the-loop into a quality system, not a slow-down.

The AI governance framework we use, roles, process, cadence

AI governance only works when it is owned, documented, and kept alive. The best version is simple enough to use, and strict enough to prevent risky shortcuts.

A practical AI governance framework has three layers, ownership, controls, and review.

Roles and responsibilities

Name an owner for AI governance. This can be a person or a small group, but it needs a clear decision maker.

Then define owners for individual workflows. Someone should be accountable for what the tool does, what data it can access, and what happens when it goes wrong.

If you want to use the formal language, this is your HITL model, human in the loop, with decision rights.

Tool register and lifecycle

Keep an internal register of AI tools.

For each tool, track the use case, the data it touches, the risk level, and what oversight is in place. Record who owns it, and when it will be reviewed.

Treat tools as having a lifecycle. Approve, pilot, adopt, review, and retire. This stops shadow AI from spreading quietly.

Incidents, redress, and contestability

Assume something will go wrong.

Define how people report issues, how you assess impact, and what the immediate containment step is. Make it clear when a human review is required.

If an AI-influenced outcome affects someone, there should be a way to contest it and request a human-led review. That protects trust, and it protects your team too.

AI policy template, copy and paste starter

This is a short AI policy template designed for real teams. It is not legal advice. It is a practical starting point you can adapt.

1. Scope, what this policy applies to

This policy covers generative AI tools, assistants, and any AI features built into software we use for work.

It applies to employees, contractors, and anyone using AI on behalf of the organisation.

2. Allowed use

AI can be used to support work, for example:

• drafting and editing content, with human review
• summarising meetings and documents
• internal brainstorming and ideation
• improving workflows and automation, within approved systems
• analysis and reporting support, using approved data sources

3. Prohibited use

AI must not be used to:

• upload or paste confidential information into unapproved tools
• process personal data without an approved use case and safeguards
• make final decisions on hiring, finance, legal, or access without human review
• generate outputs that pretend to be verified facts without checking sources
• browse the web or pull in external sources for “answers” unless explicitly approved

4. Data rules, privacy, and security

Only use approved AI tools.

Do not share customer data, contracts, internal documents, or credentials unless the tool and workflow have been approved and documented.

If a workflow involves personal data, document the purpose, minimisation steps, access controls, and retention approach.

5. Human-in-the-loop and quality checks

All customer-facing outputs must be reviewed by a human.

High-impact workflows require human approval. Low-risk workflows can be spot checked.

If the AI is unsure, or cannot cite an approved source, it must say it does not know.

6. Transparency and disclosure

We disclose when AI is used in a way that affects clients or customers.

Where appropriate, clients can opt in or opt out of generative AI use.

7. Tool register and review cadence

We maintain an internal register of AI tools and use cases.

Each tool has an owner, a documented purpose, known limitations, and a review date. The register is reviewed regularly, and at least annually.

8. Incident reporting and escalation

If an AI tool produces an unsafe output, leaks information, or behaves unexpectedly, report it immediately.

We will assess impact, contain the issue, and escalate to a human reviewer for any high-risk outcomes.

9. Environmental and ethical use

We use AI when the benefits justify the cost.

We aim to supplement people and improve services, not reduce trust, remove accountability, or create harm.

Audit and improve, how to make governance real

AI governance is not a one-off document. It is a cycle.

You approve tools, run pilots, learn what breaks, tighten controls, and review what is in use. This is how you stay safe in an evolving landscape.

A simple AI audit checklist

Run this monthly, or quarterly if you are small, and the ICO’s AI auditing work stresses the need to reassess governance and risk management when adopting AI.

• Do we have an up-to-date AI tools register, with owners and review dates
• Have any new AI tools appeared without approval
• Which workflows are high impact, and are they logged
• Are data boundaries documented for each high-impact workflow
• Are guardrails in place, no browsing, no external retrieval, closed box where needed
• Are human-in-the-loop checks working, or are people overloaded
• Have we had any incidents, near misses, or escalations
• Do we need to update policy, training, or prompts based on what happened

This is AI risk management in practice. Keep it lightweight, keep it consistent.

Model cards and documentation

For any important workflow, write down what the system is, what it is for, and what it is not for.

That can be as simple as a model card style note:

• use case and users
• data sources and boundaries
• known limitations and failure modes
• guardrails and human review points
• escalation path and owner

This makes onboarding easier, reduces drift, and helps you explain decisions.

Metrics and continuous improvement

Pick a few signals that show whether the system is helping or hurting.

Examples:

• error rate, corrections needed, and common failure types
• time saved versus time spent reviewing
• escalation volume and reasons
• user satisfaction, internal or customer
• incidents and near misses

If the metrics are getting worse, tighten the scope, improve the source content, or change the workflow. Do not just add more AI.

Next step, pilot one workflow before you roll it out

If you want AI adoption to stick, start with one workflow that happens every week, and treat it like a proper pilot inside your digital innovation process.

Pick something with clear inputs and outputs, then prototype the workflow before scaling, rapid digital prototyping. Define the boundary, what data it can access, what it must not do, and what counts as a good result. Add guardrails and a human review point, then run it for two weeks.

That pilot becomes your pattern. Once you can explain it, measure it, and improve it, you can scale to the next workflow with far less risk.

If you want help designing the workflow and building a first version, start here.