Putting Customer Data Into AI: What You Can and Cannot Do

Customer data can usually go into an AI system. Whether it should, and whether you can defend it afterwards, comes down to four separate questions that teams tend to collapse into one. Answering them individually is what turns a nervous conversation into a decision you can write down.

Nothing here is legal advice; data protection obligations vary by jurisdiction and by what you told customers. Check with counsel and against your own notices.

Question one: what did you tell them?

Your privacy notice and contracts define the purposes you may process data for. If they describe service delivery and you are now doing product analytics with a model, that is a purpose question, not an AI question — and it has the same answer it would have had before AI existed.

This is the question most often skipped, and it is the cheapest one to get right, because the answer is already written down. The EU's data protection framework and the UK ICO's AI and data protection guidance both treat purpose limitation as the starting point rather than a detail.

Question two: does it leave your boundary?

Sending data to a third-party API is a transfer, with everything that implies: residency, sub-processor disclosure, and your contractual obligations to the customer. This is where the vendor's terms matter more than their marketing.

Two terms decide it. Whether your data is used to train their models — for business tiers the answer is normally no, but it is a contract term and not a courtesy, so get it in writing. And retention: how long inputs are held, where, and how deletion is evidenced. Both belong in the AI vendor security questionnaire and both are cheap to secure at purchase, per AI contract negotiation.

Question three: does the data need to be there at all?

Most use cases need far less than they are given. A support assistant needs the ticket text, not the customer's payment history. A summarisation tool needs the document, not the account record attached to it.

Minimisation is the strongest control available here, because it is the only one that cannot fail open. Data you did not send cannot be retained, leaked, subpoenaed or trained on. Before the governance conversation, ask what can simply be removed — pseudonymised identifiers, redacted fields, a narrower query.

Question four: what happens on the way out?

Deletion requests have to reach the AI system too. If a customer exercises a deletion right, and their data sits in your vendor's logs, in a vector index, and in an evaluation set someone built from real tickets, you need a path to all three. That third one catches most organisations — evaluation sets built from production data are almost never in the deletion inventory.

Map where customer data can come to rest before you deploy, not after the first request arrives. The same map is what an AI incident response plan runs on.

The thing that is not a legal question

Whether customers would be comfortable if they knew. That is a commercial judgement and it is frequently stricter than the legal one — which is the argument in disclosing AI use to customers. The OECD's AI Principles frame this as accountability rather than compliance, and the framing is right: the cost of getting it wrong arrives as lost trust long before it arrives as a penalty.

Frequently asked questions

Can we put customer data into an AI tool? Usually, if your privacy notice covers the purpose, your vendor contract prohibits training on your data and defines retention, you send only what the use case needs, and you can delete it on request.

Will the vendor train on our data? For business and enterprise tiers, typically not — but treat it as a contract term to confirm in writing rather than an assumption, and check it again at renewal.

What is the strongest control? Minimisation. Data you never sent cannot be retained, leaked or trained on. Most use cases need far less customer data than they are given.

What do teams most often miss? Deletion. Customer data ends up in vendor logs, vector indexes and evaluation sets built from real production records — and that last one is rarely in anyone's deletion inventory.

Further reading

• When to disclose AI use to customers
• The AI vendor security questionnaire
• Your AI incident response plan
• The one-page AI policy template
• AI governance advisory