AI Vendor Security Questions: A Due Diligence List for US Buyers

AI Vendor Security Questions: A Due Diligence List for US Buyers

If you are buying an AI product, or hiring a team to build one, the security review needs more than the standard SaaS questionnaire. Ask where your data goes, whether it is retained or used for training, who can see it, what the vendor's audit report actually covers, and what happens when the model changes underneath you. Those five areas catch most of the real risk.

This list is written for security, procurement and IT leaders in the US. It is not legal advice. Where a question touches a legal duty, such as HIPAA, check with your adviser before you rely on any vendor's answer.

Start with the data: where it goes and what happens to it

Most AI-specific risk comes from the data path. A normal SaaS app stores your data in a database. An AI feature may also send it to a third-party model provider, write it to logs, cache it, or index it in a vector store. Each hop is a place your data can sit longer than you expected.

Ask the vendor:

• What data leaves our environment, and where does it go? Ask for a data flow diagram that includes the model provider, and the vendor's own servers.
• Which model providers and sub-processors touch our data? Get the list, the purpose of each, and how you are told when it changes.
• Is our data used to train or improve any model? Get the answer in writing, for the vendor and for every upstream model provider. "Not used for training" on a marketing page is weaker than a clause in the contract.
• How long are prompts, outputs and uploaded files retained, and can we set it to zero? Ask about logs, backups, abuse-monitoring copies and support tooling separately. They often have different retention.
• In which regions is data stored and processed? This matters for customer contracts and for sector rules, even when no law requires data to stay in the US.
• Can we delete our data, and how do you prove it? Ask what deletion covers: the database, the vector index, backups, and any fine-tuned artefacts.

If the vendor builds on a private deployment or a private cloud, the answers may be simpler because fewer parties are involved. Our guide to building compliant and secure AI systems compares those options.

Access control, especially in retrieval systems

If the product answers questions over your documents (a RAG system), the most important question is whether it respects the permissions you already have. A model that can retrieve a document will happily summarise it for someone who should never have seen it.

• How are source-system permissions (SharePoint, Google Drive, Confluence and so on) carried into the index?
• Is the access check applied when documents are retrieved, or only in the prompt? Instructions in a prompt are not a security control.
• Can we use single sign-on and role-based access, and can admins see who asked what?
• What happens to an index entry when a user loses access to the source document?

The secure RAG article explains why retrieval-time filtering is the pattern to look for.

What a SOC 2 report does and does not tell you

SOC 2 is the document US buyers ask for most, so it is worth reading properly. The AICPA describes SOC as a suite of services CPAs may provide on the controls of a service organisation, and its SOC 2 publication covers an examination of controls relevant to security, availability, processing integrity, confidentiality or privacy. These are the five Trust Services Criteria categories, which the AICPA describes as outcome-based criteria for judging whether a system's controls provide reasonable assurance that management's objectives are met.

A few points follow from that:

• It is an attestation by an independent CPA firm, not a certificate you can look up on a register. Ask for the report itself, not a badge on a website.
• Scope is chosen by the vendor. Security is the baseline, but the other categories are optional. Check which ones are in scope, and which systems and locations the report covers. A report on the vendor's corporate platform may not cover the AI feature you are buying.
• Type 1 versus Type 2. An assessor firm's explainer describes Type 1 as an attestation of controls at a point in time, covering design and implementation, and Type 2 as covering a longer period and testing whether controls operated effectively. Procurement teams often prefer Type 2 for that reason. Ask for the audit period and how recent it is.
• Read the exceptions and the auditor's opinion. Listed exceptions are information, not automatically a disqualifier. Ask how each was fixed.
• Check the complementary user entity controls. These are the things the report assumes you will do on your side.
• SOC 2 does not speak to AI-specific risks unless the vendor scoped them. It will not tell you how the vendor handles prompt injection or model changes. That is what the next sections are for.

A young vendor may not have a report yet. That is not a reason to stop, but ask what they have instead: a recent penetration test summary, a completed security questionnaire, a documented policy set, and a dated plan for an audit. Be wary of phrases such as "SOC 2 compliant" or "aligned with SOC 2" with no report behind them.

AI-specific security questions

The OWASP Top 10 for LLM Applications (2025 edition) is a practical checklist of what goes wrong in these systems. Its entries include prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation and unbounded consumption. Turn the ones that matter to you into questions:

• Prompt injection. How does the system treat instructions hidden in documents, emails or web pages it reads? Has it been tested against this, and by whom?
• Excessive agency. If the AI can take actions (send email, update a CRM, run code), what is the least privilege it needs, which actions need human approval, and can it be limited to read-only?
• Output handling. Is model output ever passed straight to a browser, database or shell without validation?
• Sensitive information disclosure. Can one customer's data surface in another customer's response? How is tenant isolation tested?
• Unbounded consumption. Are there rate limits and spend caps, so an abuse case or a runaway loop does not become a bill or an outage?
• Supply chain. Which open-source models, libraries and plugins are used, and how are they vetted and updated?

Testing, incident response and model change

• Penetration testing. How often, by whom, and does scope include the AI features and integrations, not only the web app? Ask for a summary letter, and what happens to critical findings.
• Red-teaming. Has anyone tried to make the system leak data or misbehave, and is there a record of what was found?
• Incident response. Is there a written plan, and what is the contractual notification window if your data is affected? Ask for it in hours or days, not "promptly".
• Logging. Are prompts, outputs, tool calls and admin actions logged, for how long, and can you export the logs to your own monitoring tools?
• Model change management. If the provider swaps or updates the underlying model, how are you told, and is there a regression test before it reaches your users? A silent model change can alter behaviour, accuracy and data handling at once.
• Fallbacks. What happens when the model is wrong or unavailable? Human handoff, confidence thresholds and clear failure messages are signs of a mature design.

Regulated data: HIPAA and business associate agreements

If protected health information will touch the product, ask early whether the vendor will sign a business associate agreement (BAA). In general, a vendor that creates, receives, maintains or transmits PHI for a covered entity is acting as a business associate, and HIPAA expects a BAA to be in place first. Whether that applies to your use case, and what the agreement must say, is a question for your privacy counsel.

Practical checks:

• Will the vendor sign a BAA, and does it cover the AI features and the model provider in the chain? A BAA with the vendor does not automatically extend to a model API the vendor calls.
• Is the specific model service in use covered by the upstream provider's own BAA terms?
• Are there features that must be switched off for PHI, such as certain logging or analytics?
• How are breaches and subcontractors handled in the agreement?

Using a framework to structure the review

If your organisation wants a shared vocabulary, the NIST AI Risk Management Framework (AI RMF 1.0, released January 2023) is a voluntary framework for managing AI risks, organised around four functions: Govern, Map, Measure and Manage. NIST also published a Generative AI Profile in July 2024. Neither is a vendor certification, but they help you decide which questions belong in the review and who in your company owns the answers. For the internal side, see our piece on AI governance.

Red flags and how to weigh the answers

• Answers that avoid the model provider and talk only about the vendor's own infrastructure.
• "We don't train on your data" with no contract language, or exceptions buried in terms of service.
• A SOC 2 claim with no report, an old audit period, or a scope that excludes the product you are buying.
• No way to turn off retention or delete data on request.
• No human approval step for actions the AI can take on its own.
• Reluctance to share a penetration test summary or incident response commitments.

Not every vendor needs every answer. Scale the review to the data. An internal drafting assistant that never sees customer data needs a lighter check than a support agent connected to your CRM. Write down what you accepted and why, and set a date to review it again.

If you are choosing a development partner rather than a product, the same questions apply, plus who holds the keys, who owns the code and how access is removed at the end. As an example of a design choice worth asking about, our AI support agent case study describes a RAG agent that hands off to a human below a set confidence level. Futurise does not hold a SOC 2 report, so we would not claim one.

Sources

• AICPA & CIMA: SOC suite of services
• AICPA & CIMA: trust services criteria mappings
• KirkpatrickPrice: SOC 2 Type 1 vs Type 2
• NIST: AI Risk Management Framework
• OWASP GenAI Security Project: Top 10 for LLM Applications