What Boards Should Ask About AI Model Risk
Boards at AI-native startups often default to the same governance questions they'd ask any software company, and miss the risks that are specific to building on top of models a company doesn't fully control. Data privacy, model risk, and vendor dependency all show up differently when a core product depends on a third-party model's behavior, training data, and terms of service changing without much notice.
This checklist covers what a board should actually be asking, and the specific gaps that tend to get missed until something goes wrong.
Vendors Covered in this Article
Disclosure: We may earn a commission if you buy through some links on this page. It doesn't change what we recommend.
Questions about what data actually goes into the model
- Does the company send customer data to a third-party model provider, and does the customer contract or privacy policy actually disclose that clearly?
- Is there a documented list of what data categories are sent, or does this depend on institutional memory?
- Can the company technically fulfill a customer's deletion request when the data has already been used in a model call or, worse, a fine-tuning run?
- Has anyone checked the model provider's own data retention and training-use terms recently, since these change more often than most software vendor terms?
Questions about model risk and vendor dependency
- What happens to the product if the primary model provider changes pricing, deprecates the specific model version in use, or changes its behavior through an update the company doesn't control?
- Is there a fallback plan, even a rough one, if the primary model provider became unavailable tomorrow?
- Has anyone documented which parts of the product would break, degrade, or just behave differently if model outputs shifted meaningfully?
- Is the company tracking model provider terms of service changes as an ongoing responsibility, or reactively, after something breaks?
Governance gaps that get missed until something goes wrong
The most common gap is treating model risk as an engineering concern rather than a board-level risk category, which means it never appears on a board agenda until a customer complaint or an incident forces the conversation. A close second is assuming existing security and compliance frameworks automatically cover model-specific risks like prompt injection or training data leakage, when most standard frameworks were written before those risks were common. A third is having no one specific person accountable for model risk at all, so it falls into the gap between engineering, legal, and whoever happens to notice a problem first.
What a reasonable governance cadence looks like
Put model risk and data privacy on the board agenda at a fixed cadence, at minimum reviewed alongside other risk topics, rather than only when triggered by an incident. Assign a specific internal owner, even at a small company, responsible for tracking model provider terms changes and reporting anything material. Document, even briefly, what happens to the product if the primary model dependency changes materially, since a board asking this question for the first time during an actual outage is a bad time to discover there's no answer.
Where compliance tooling fits without replacing judgment
Platforms like Vanta and Drata automate evidence collection for standard frameworks like SOC 2, which helps demonstrate baseline security practices to customers and investors, but they aren't primarily built to evaluate model-specific risks like training data provenance or prompt injection exposure, so check what AI governance features each currently offers. Use one of them to keep standard compliance evidence current and audit-ready, and treat the AI-specific governance questions above as a separate, deliberate conversation the tooling doesn't automatically cover.
What good documentation looks like in practice
A useful model risk document doesn't need to be long or technical to be effective for board purposes. A single page covering which model providers are in use, what customer data categories reach them, what the fallback plan is if a provider becomes unavailable or changes materially, and who owns updating that page, is usually enough for a board to ask informed questions. The mistake most companies make isn't writing too little detail; it's not writing anything down at all and relying on whichever engineer built the integration to remember the details months later when the board finally asks.
What Good Looks Like
Good practice means model risk and data handling appear on a fixed board agenda with a named internal owner, not only when a customer complaint or incident forces the conversation.
Building The Capability (5-Stage Skill Ladder)
How to Get Started
Disclosure: We may earn a commission if you buy through some links on this page. It doesn't change what we recommend.
Frequently Asked Questions
Does a small AI-native startup really need formal board governance around this?
Formal in proportion to size, yes. Even a small board can put model risk and data handling on a fixed agenda a few times a year and name one accountable owner internally. The goal isn't heavy process; it's making sure these questions get asked on a schedule instead of only after an incident forces them.
Is SOC 2 compliance enough to cover AI-specific risks?
No. SOC 2 and similar frameworks cover general security and availability controls well, but most weren't designed with model-specific risks like training data provenance, prompt injection, or model behavior drift in mind. Treat compliance certification as a baseline, not as coverage for AI-specific governance questions.
Who should own model risk internally at a small company?
Someone specific, even if it's a part-time responsibility layered onto an existing role like a fractional CTO or head of engineering. The key requirement is that one named person is accountable for tracking model provider changes and reporting anything material, rather than the responsibility being implicitly shared and therefore owned by no one.
About the numbers
This guide doesn't quote a sourced benchmark. Figures in it are estimates or general guidance, so check them against your own numbers.
Related Guides
Board Governance for an AI Automation Agency's Model Risk
AI and workflow automation agencies carry model and vendor risk their boards need visibility into. A pitfall-focused look at Diligent versus BoardPro.
AI Executive Advisor or Fractional Executive: When Each Fits
Compare an AI executive advisor with a fractional executive by task: what software does well, what needs a person, and how to combine both.
Selling an AI Automation Agency: Acquire.com or Flippa?
A checklist for AI and workflow automation agencies weighing Acquire.com against Flippa, and the pitfalls specific to selling young AI-services revenue.
Optery vs DeleteMe for AI Agency Founders
A pitfalls-first checklist for AI and workflow automation agency founders comparing Optery's automated removal to DeleteMe's human-run service.
Equity Tools for AI Agencies: Pulley vs Carta Compared
How an AI and workflow automation agency should compare Pulley and Carta for equity grants, dilution, and retaining scarce AI engineering talent.
Carta vs Forge When An AI Agency's Valuation Moves Fast
A worked example of how an AI and workflow automation agency chose between Carta and Forge for its first employee liquidity program.