Most AI governance programs stall in the same place. Someone senior asks a reasonable question — “what AI are we actually running?” — and nobody can answer it in under three weeks.
That question is the whole job compressed into six words. You cannot classify systems you have not found. You cannot assess risk on a list that does not exist. And when the EU AI Act obligations land, or a customer sends a security questionnaire with eleven new AI clauses in it, the inventory is the first thing anyone asks to see.
This guide covers how to build one that survives past the first quarter, where to look, what to record, and the single design decision that determines whether it stays current or quietly rots in a SharePoint folder.
Why the inventory comes first
Every framework you might be working toward assumes the inventory already exists.
NIST AI RMF’s MAP function is built around characterizing the systems in scope. ISO/IEC 42001 expects a defined set of AI systems inside the management system boundary. The EU AI Act’s obligations attach to specific systems in specific roles , you cannot determine whether you are a provider or a deployer of a high-risk system in the abstract. You determine it system by system.
So the inventory is not a preliminary step before the real work. It is the object the real work operates on. It is also the artifact with the shortest path to value. A finished inventory tells you, usually within a day of completing it, which three or four systems are going to consume most of your governance attention for the next year. That is a genuinely useful answer, and you get it before writing a single policy.
The hard part: deciding what counts
Here is where most inventories go wrong in the first hour. If you define “AI system” too narrowly — trained models your data science team deployed — you will produce a list of six things and miss the ninety that actually carry risk. If you define it too broadly, every product with a machine learning feature qualifies, your list runs to four hundred rows, and it becomes unusable.
A working definition that holds up in practice: anything that produces an output your organization acts on, where the logic is learned rather than explicitly written, and where a person cannot fully predict the output in advance.
That definition captures things people routinely forget:
- The résumé screening feature bundled into your ATS
- The fraud scoring inside your payments provider
- Sales call transcription and summarization tools
- The AI assistant embedded in your CRM
- Anything a team is running against an LLM API, including internal prototypes
- Third-party tools your vendors use to deliver services to you
It correctly excludes rules engines, deterministic automation, and the “AI-powered” label on a product that is doing regex matching. Ask the vendor what the logic is. If they cannot explain how the output is produced, treat it as in scope until proven otherwise.
Set the boundary in writing before you start collecting. Document your inclusion test in one paragraph. When someone challenges an entry six months from now — and they will — you want a written rule, not a recollection.
Where the systems actually are
Asking department heads “what AI are you using?” produces an incomplete list. People do not think of the transcription feature in their meeting tool as AI. They think of it as the meeting tool.
Better to go to the systems of record, in roughly this order:
1. Your SSO or identity provider. Application access logs are the single richest source. Export the full list of applications with active sign-ins over the last ninety days. This surfaces tools nobody remembers approving.
2. Expense and card data. Filter twelve months of transactions for SaaS vendors under the approval threshold. Individual subscriptions on personal cards are where shadow AI lives.
3. Procurement and contract records. Look for AI clauses, data processing agreements signed in the last two years, and any vendor whose product category changed after 2023.
4. Your existing vendor register. Do not build a parallel list. Add an AI flag to the register you already maintain, then filter on it.
5. Browser extension and endpoint management data. If you have MDM or endpoint tooling, extensions and installed applications reveal a great deal.
6. Code repositories and cloud accounts. Search for API calls to model providers. Check for deployed inference endpoints and model-serving infrastructure. Prototypes running on someone’s cloud budget count.
7. Then ask people But ask a specific question. Not “do you use AI?” Instead: “which tools in your workflow generate text, summarize documents, score or rank anything, make recommendations, or transcribe?” That phrasing gets three to four times the response rate of the generic version.
Expect the true number to be somewhere between three and ten times what leadership estimates. That gap is normal and worth reporting, because it is the most persuasive argument for the program you are trying to fund.
What to record
Resist the urge to build a forty-column register. Every field you add is a field somebody has to maintain, and unmaintained fields make the whole inventory untrustworthy.
Twelve fields carry almost all the weight:
The two fields people skip and later regret are your role and decision impact. Role determines the entire obligation set under the EU AI Act. Decision impact is the field that separates the systems needing real scrutiny from the ones needing a line in a policy.
Tiering without over-engineering
Once populated, sort the inventory. Three tiers is enough for most organizations:
Tier 1 — Elevated. Outputs materially affect individuals’ rights, access, or livelihood; or the system processes special category data; or it operates with limited meaningful human oversight. These get a full impact assessment, documented mitigations, a named accountable owner, and a review cadence.
Tier 2 — Standard. Business impact without direct effect on individuals’ rights. These get baseline controls, vendor due diligence, and annual review.
Tier 3 — Minimal. Productivity tools operating on non-sensitive data with a person reviewing every output. These are covered by your acceptable use policy and nothing more.
Most organizations land with under ten percent in Tier 1. If half your inventory is landing in Tier 1, your criteria are too loose and you have built a program nobody can staff.
Write down the criteria. A tiering decision you cannot reconstruct in twelve months is a tiering decision an auditor will not accept.
The design decision that determines whether it stays current
An inventory built as a one-time project is obsolete within a quarter. Not because anyone neglected it — because procurement kept working while you were building it. The fix is not a better reminder schedule. It is attaching the inventory to a process that already runs without your involvement.
Pick one existing gate and add three questions to it. The best candidates, in order of effectiveness:
Procurement intake or purchase requisition. Anything new gets flagged at the point of purchase.
Vendor security review. You already assess new vendors. Add AI questions to the questionnaire.
Change management or architecture review. Catches internally built systems that never touch procurement.
The three questions:
1. Does this tool generate content, make recommendations, score or rank anything, or transcribe?
2. What data will it process?
3. Will any output affect a decision about a person?
Three questions on a form somebody already fills in will keep an inventory more current than a quarterly all-hands email ever will. This is the difference between a governance program that runs and one that requires you to push it.
Pair it with a light quarterly reconciliation — re-pull the SSO application list, diff it against the register, investigate anything new. Thirty minutes, four times a year.
What good looks like
A working inventory should let you answer these in under five minutes:
- How many AI systems process personal data, and which ones?
- Which systems would be high-risk under the EU AI Act, and what is our role for each?
- Who owns the three systems with the greatest decision impact?
- What changed since last quarter?
If any of those takes longer than five minutes, the problem is usually structure rather than effort — most often too many fields, or ownership assigned to departments instead of people.
Five failure patterns worth avoiding
- Building it in a tool nobody opens. A maintained spreadsheet beats an abandoned GRC platform. Buy tooling once the process works manually, not before.
- Assigning ownership to a department. “Marketing” does not answer emails. Name a person.
- Treating pilots as out of scope. Pilots process real data and become production quietly. Include them, flag the status.
- Recording the vendor and stopping. The vendor is not the system. One vendor may supply three AI features with three different risk profiles.
Waiting for completeness before starting the analysis. An eighty percent inventory you act on beats a hundred percent inventory delivered six months late. Publish, act, refine.
Where this leads
The inventory is stage one. Once it exists, classification becomes a defined task rather than an open-ended one, and the policy set you write afterward has something concrete to point at.
The next guide in this path covers classifying those systems against EU AI Act risk tiers and confirming your role in the value chain.
Get the template
Our AI System Inventory template ships with the twelve fields above, the tiering criteria, and the three intake questions formatted for a procurement form. It is included in the AI Governance Starter Kit, tailored to your sector and size.
[Browse governance documents →](https://aigrcadvisory.com/ai-governance-toolkits-policy-documents/)
Working through this and want a second opinion on scope? [Book a free 30-minute call](https://aigrcadvisory.com/contact-2/). No slides — you leave with the two or three things worth doing first, whether or not you hire us.
About the author
References
– NIST AI Risk Management Framework 1.0 — MAP function
– ISO/IEC 42001:2023 — AI management system requirements
– Regulation (EU) 2024/1689 (EU AI Act) — Articles on provider and deployer obligations
Advisory content, not legal advice.