The short answer: every document you send to an AI provider becomes data that provider stores, and a breach at that provider becomes your breach. Before documents leave your environment, send only what the task needs, remove identifiers from both the text and the page image, do that step inside your own perimeter, and treat vendor credentials like production secrets.
What happened
Uber is notifying drivers that personal information, including driver's license data, was taken in an October breach at Fireworks AI, a third-party AI vendor. According to state breach filings reviewed by reporters, thousands of drivers are affected (2,567 in Massachusetts, 3,430 in South Carolina and 1,873 in Rhode Island, with other states pending). Attackers used compromised Fireworks credentials to reach data Uber had stored with the vendor; Uber says its own systems were not breached and it has found no misuse so far. Notion separately said the same provider was behind an exposure of AI chat transcripts affecting a small share of its workspaces. (Yahoo News / TechCrunch, IBTimes)
Public reporting has not explained why the driver records were reachable through the vendor account, so this post is not about what any one company should have done. It is about a pattern every team building on LLMs now shares.
The pattern: AI vendors hold more of your data than you think
When you call a model provider, documents and prompts can end up in request logs, batch-job files, evaluation or fine-tuning datasets, caches and support tickets. Each of those copies sits behind the vendor's credentials and the vendor's security, not yours. The more providers you use (model hosts, OCR APIs, agent platforms, observability tools), the larger that surface gets.
Six controls for documents that go to AI vendors
1. Know what you send. List every pipeline that sends documents or extracted text to an outside model, and which document types flow through it: ID documents, pay stubs, bank statements, claims, medical records.
2. Send only what the task needs. If the model is classifying a document or extracting an address, it does not need the license number, date of birth or photo. Drop or mask fields the task does not use.
3. Redact the image, not just the text. ID documents and scans are images. Removing a value from the OCR text while the page image still shows it does not protect it. Remove the pixels, the text layer and the metadata, and check the output by trying to copy, search and extract the redacted areas.
4. Do the redaction inside your perimeter. If you send originals to a cloud redaction service so that you can safely send them to a model provider, you have added another vendor that holds the originals. Run detection and redaction on-premises or in your own cloud account, and send only redacted output.
5. Use tokens when you need to re-link results. Replace identifiers with consistent placeholders (for example `<DL_NUMBER_1>`) and keep the mapping in your own systems, so model output can be joined back to the record without the vendor ever seeing the value.
6. Treat vendor access like production access. Scoped keys, rotation, short retention or zero-retention settings where the vendor offers them, and deletion of batch files and datasets you no longer need. This incident began with compromised credentials.
Be clear about the limits: redaction reduces what a vendor breach can expose, but it is not an access-control decision and it does not replace the vendor's own security. On its own it is also not formal de-identification under rules such as HIPAA.
Test it on documents that look like yours
Redaction that works on a clean PDF can miss a value on a skewed phone photo of a license. Test on documents where you know every sensitive value in advance, in the capture conditions you actually receive. GritWorks' free US KYC sample pack has fictional driver's license and proof-of-address specimens, each clean and degraded, with the expected value and position of every field.
Where GritWorks fits
GritRedact finds PII in PDFs, images and audio, applies your policy with human review, removes values permanently from text and pixels, and keeps an audit trail. It runs fully offline on-premises (Docker, no network egress), so the originals never leave your environment. If documents from your pipelines go to outside AI providers, book a working session and bring a few representative, non-production examples.
