Most enterprise AI governance frameworks read like they were written for an organisation with fifty thousand employees, a dedicated AI ethics team, and a model risk function borrowed from a tier-one bank. Many of them were. They call for review boards, multi-stage approval gates, standing bias-audit programmes, and documentation standards that quietly assume a full-time compliance officer exists to maintain them. Drop that apparatus onto a 400-person manufacturer deploying its first three AI workflows and you have not installed governance. You have installed a reason for AI to never reach production.
The instinctive Mittelstand response — skip governance, move fast, sort it out later — is the opposite failure with the same outcome. Without a named owner, workflows drift as the underlying data shifts. Errors accumulate unseen because nobody is watching the right number. And the first time a regulator, an auditor, or a major customer asks how you control the AI touching their data, the honest answer is that you do not. That answer is expensive. The same EU AI Act framed in the press as a burden on big tech also governs the small German firm using a language model to triage emails — and its obligations land on deployers, not just the labs that build the models.
The way through is neither failure. It is a deliberately small governance model — light enough that a Geschäftsführer will actually run it, structured enough to survive contact with an auditor. Within the AI Operating System methodology, governance is not a parallel workstream bolted on at the end; it is designed into how every workflow is owned, measured, and scaled from the first deployment.
Four jobs, and nothing more
Strip away the consultancy vocabulary and AI governance does exactly four jobs, and a framework that does more than these four is selling you overhead.
The first is accountability — a single named person answerable for each workflow, never a committee. They know what the system does, how it is performing this week, and what to do the moment it stops behaving. The second is oversight: performance is watched against a baseline, degradation is caught early, and problems escalate before they become incidents your customers discover first. The third is compliance — the ability to show a regulator, auditor, or buyer that AI is deployed responsibly and in line with the DSGVO and the EU AI Act. The fourth is decision rights: it is unambiguous who may approve a new deployment, who may change an existing one, and who may switch it off. Four jobs. If a proposed control does not serve one of them, it is bureaucracy, and bureaucracy in a mid-market company is not neutral — it is the thing that kills the second and third deployment before they ship.
The one-page model
The governance structure for a Mittelstand AI portfolio should fit on a single sheet of A4 and be explicable in five minutes. If it cannot, it is built for a company you are not. Three named roles carry the whole thing.
The Workflow Owner is one person per AI workflow, almost always the person who already owns the underlying business process — the head of customer service for the support-triage model, the finance lead for the invoice-extraction one. They watch the weekly numbers that matter (accuracy, throughput, error rate), review the cases the system handled badly once a month to see whether a pattern is forming, escalate when something is off, and keep the workflow's single-page record current. This is not a hire. It is an existing team lead absorbing AI oversight into work they already do, and for a stable workflow the real commitment is well under an hour a week once the baseline is set.
The AI Sponsor is one person for the whole organisation — typically the Geschäftsführer, COO, or CTO. They approve new deployments, make the go/no-go call on a retraining or a model swap, look at aggregate performance each quarter, own the relationships with technology and implementation partners, and serve as the escalation point when a Workflow Owner raises a hand. The Sponsor does not need to follow the technical detail. They need to understand business impact and decide accordingly — which is precisely the judgement a Geschäftsführer already exercises every day.
The Compliance Contact is usually your existing data protection officer (Datenschutzbeauftragter) or legal counsel, wearing one more hat. They confirm that every workflow carries a risk classification under the EU AI Act, ensure the DSGVO basics are in place — the processing records, the data processing agreements, a data protection impact assessment where the processing warrants one — and flag regulatory change that touches live deployments. This is advisory, not a veto. The Compliance Contact reviews and advises; the Sponsor decides. Collapsing those two into one approval bottleneck is how governance turns into the delay it was meant to prevent.
The quarterly review that becomes your audit trail
Once a quarter, those three roles meet for ninety minutes, and that meeting is the entire operating rhythm. Each Workflow Owner reports their numbers — is the workflow holding its baseline, are error rates stable, were there incidents. Together you look at where edge cases cluster, because a recurring failure pattern is usually a signal to retrain or redesign the process around the model rather than tolerate it. You check whether any regulatory movement affects what is already live, and right now there is plenty to track: the AI Act phases its obligations in over years, and the high-risk timeline is actively in flux. The Commission's Digital Omnibus — provisionally agreed by Parliament and Council on 7 May 2026 — would push the full high-risk obligations for standalone Annex III systems out to 2 December 2027, but those new dates only bind once the text is published in the Official Journal, so 2 August 2026 remains a live compliance date until then. "What changed since last quarter" is a real agenda item, not a formality, and a moving target is precisely the kind of thing a named review owns rather than discovers in a customer audit. You review the pipeline — new candidates, workflows to expand or retire — and you decide: approve, defer, or reject.
The output is one page — what is working, what needs attention, what was decided and by whom. Four of those pages a year, accumulated, are your AI governance record. When an auditor or a customer's security team asks how you control your AI, you hand them a dated, signed trail of decisions instead of a promise. That artefact is worth more than any ethics charter nobody reads.
Documentation that an auditor accepts and a busy owner maintains
Each workflow gets exactly one document, and the discipline is in keeping it to a page. It names the workflow and its owner, says in two sentences what it does, records what data it touches (sources, whether personal data is involved, retention), states its EU AI Act risk classification with a one-line justification, lists the metrics watched and the thresholds that trigger a review, and describes the fallback — what happens when the AI is wrong or unavailable, which is the question auditors return to most. A last-reviewed date closes it.
For the workflows most Mittelstand companies actually deploy — document classification, data extraction, knowledge retrieval, task routing — this is genuinely sufficient, and the reason is structural. The AI Act sorts systems into tiers, and the heavy obligations attach to high-risk systems: those listed in Annex III or acting as safety components, covering domains like biometrics, critical infrastructure, employment and recruitment decisions, creditworthiness, and access to essential services. A model that routes support tickets is not in that company; it sits in the lighter tiers, where transparency duties may apply but the full high-risk regime does not. The expensive mistake runs both ways — dragging a low-risk workflow through high-risk documentation, or quietly deploying something that touches hiring or credit and assuming the light-touch page covers you. Classify first, honestly, because the classification sets everything downstream. For the detail, see our EU AI Act resource.
What enterprise governance gets wrong here
The standing AI ethics board is the clearest case. A monthly committee deliberating the ethics of AI deployments is rational for a firm making automated credit decisions across millions of consumers. For a company using AI to classify support tickets, it is delay without value, and the quarterly review already carries any ethical question worth raising. The centralised model registry is the same story one rung down: indispensable at fifty models across business units, absurd at three, where a shared spreadsheet of active workflows, owners, and review dates does the identical job. Standing bias-audit programmes are essential where AI decides about people — who gets hired, lent to, or priced how — and beside the point for a model that extracts fields from a PDF; match the mechanism to the actual risk, not the framework's worst case. And the change-management committee that must bless every model tweak is pure friction: the Owner and Sponsor should clear routine adjustments between them, reserving the quarterly review for the changes that genuinely shift the risk picture.
Build the next layer only when it bites
The one-page model carries you through roughly your first handful of workflows. Past that, structure earns its place — but only once its absence actually costs you something. A shared dashboard pays off when checking metrics one workflow at a time becomes the bottleneck; an onboarding checklist matters once you launch faster than memory keeps up; stable, long-running workflows can move from quarterly to annual review; and a dedicated AI-operations function — one person, not a department — makes sense somewhere north of ten active workflows. Build none of it in advance. Premature governance infrastructure wastes exactly as much momentum as having none, and in the Mittelstand momentum is the scarce resource.
You can stand up the starting version in an afternoon: name the three roles, write the one-page records, and put the first quarterly review in the calendar. The hard part is not the framework. It is having a clear-eyed read of which of your workflows are genuinely low-risk and which one is quietly doing something the AI Act treats as serious.
A Fit Call pressure-tests your AI governance against your actual workflows and risk profile — so the first regulator or customer question finds a dated decision trail, not a scramble.
For the broader operating context, see AI in Operations; governance maps directly to the compliance-posture and operating-model dimensions of the six-dimension diagnostic framework, and for a readiness baseline, AI Readiness for Mittelstand.
References: European Commission, "Regulatory framework on AI," digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai; "Article 6: Classification Rules for High-Risk AI Systems," artificialintelligenceact.eu/article/6; "Annex III: High-Risk AI Systems Referred to in Article 6(2)," artificialintelligenceact.eu/annex/3; "Implementation Timeline," artificialintelligenceact.eu/implementation-timeline; Gibson Dunn, "EU AI Act Omnibus Agreement — Postponed High-Risk Deadlines and Other Key Changes," 2026, gibsondunn.com.