What your AI vendor can change without telling you

You test the model before you deploy it. You give it your hard cases, watch what comes back, and sign off on something that worked.

Then it changes. Claude has shipped with hidden behaviour rules before, the Fable guidelines among them. Tests written without knowing those rules exist will measure around them. The thing you approved and the thing now running in front of customers are not the same, and no email came to tell you.

We usually skip the safety argument and read the contract instead. The provider almost always keeps the right to update how the model behaves whenever it wants. It gives no promise that outputs stay the same across versions, and it sets the disclosure rules very low. You did not buy a fixed product. You rented access to something whose rules can change at times you don’t control.

So the question for whoever owns your vendor relationships is simple and uncomfortable. Of the behaviours you now rely on, which ones are actually written into the agreement?

Probably fewer than the sales deck claimed. Better to learn that at the next renewal than after a customer does.