Your AI Vendor's Bad Day Is Your Regulatory Exposure

In late March 2026, Anthropic accidentally exposed 512,000 lines of internal source code via a public npm package. The full system prompt architecture. Evidence of unannounced features. The kind of thing you file a DMCA takedown over and hope the internet cooperates.

Anthropic called it "a release packaging issue caused by human error, not a security breach." They are technically right. No customer data was exposed. The takedowns were filed quickly.

They are also missing the point.

For teams building AI products in healthcare, legal, finance, or any sector where decisions have consequences beyond a refund, the story is not about the leak. It is about what the leak reveals about the risk structure you are standing in.

The problem is not security. It is quality management.

A security breach means someone got in. A process error means the internal controls failed to catch something before it went out.

Security can be hardened at a perimeter. Quality management is systemic. It lives in how a team writes procedures, trains engineers, reviews releases, and catches mistakes before they become events. When a quality management problem surfaces, it surfaces as an incident. But the problem existed long before the incident.

In a regulated sector, you cannot outsource this distinction. The Care Quality Commission, the MHRA, the ICO, and the solicitors representing harmed patients do not draw a line between your AI vendor's quality management failures and your own. You built a product on that infrastructure. You deployed it in a care setting. You made the decision.

Your vendor's process failure is your regulatory exposure.

What platform dependency actually means

Building on an AI platform is now a standard architectural choice. Reasonable, often necessary. Most teams would spend years trying to replicate what Anthropic, OpenAI, or Google have built.

But dependency is not neutral. It creates specific risk categories that most product teams have not named or assessed.

Supply chain risk. Your product's behaviour is downstream of your vendor's system prompts, safety layers, and model updates. When the model changes, your product changes. You may not be told. You may not notice for days.

Feature risk. The Anthropic leak revealed unannounced capabilities: autonomous payment systems, proactive action modes. These are being built into the infrastructure your product depends on. Were you consulted? Did you do a risk assessment on capabilities that did not exist when you signed the contract? If a capability is released and your product's behaviour changes in a way that affects a patient, the fact that it was your vendor's decision and not yours is an interesting argument. It is not a defence.

Accountability gap. When something goes wrong in a regulated product, there is a chain of accountability: the practitioner, the product, the infrastructure. Courts and regulators are actively developing their thinking on where AI vendor liability sits. In the meantime, you are in the gap.

The difference between two sentences

"I trust this tool" and "I've assessed the risk of depending on this tool" are different sentences.

The first describes a relationship. The second describes a process. In regulated sectors, only the second is defensible.

Assessing vendor dependency risk looks like this:

• What happens to my product if my vendor makes an unannounced model update?
• What notification requirements exist in my contract when new capabilities are added?
• How do I detect behavioural drift in my product's outputs after a vendor change?
• What is my fallback if my vendor has an outage, a breach, or an incident that makes headlines the week I have a CQC inspection?
• Can I produce a vendor risk assessment and keep it current?

These are not questions most product teams have answered. They are not questions most vendor contracts require you to answer. They are questions a regulator or a solicitor will ask if something goes wrong.

What to do

None of this means you should not build on AI platforms. It means you should build with the same rigour you would apply to any infrastructure choice in a regulated environment.

Three steps.

First: document your dependencies. Name every AI vendor touching your product. Map what they control. For each one, write one sentence on what changes in your product if they have a bad day. If you cannot write that sentence, you do not understand the dependency.

Second: build detection into your product. You should be able to detect when your product's behaviour shifts, even if the cause is upstream. If you cannot tell the difference between your product working normally and your product working on a degraded model, you have an audit problem.

Third: read your contracts. Most AI vendor contracts give the vendor significant latitude to update models, change terms, and modify capabilities. Know what yours says. Know what notice you get. Know what remedies exist.

None of this guarantees a clean audit. It puts you in a position to have a conversation about risk management rather than a conversation about why you did not think about it.

The Anthropic incident resolved quickly. The next one will be different. Plan accordingly.

Crox builds AI systems for regulated sectors. If your product involves healthcare, legal, or compliance decisions, we can help you think through vendor dependency before it becomes an incident.