Outsourced AI prompt disclosure: what you change, where maintenance ends
Before signing, agree the vendor discloses the AI parts you change and that your changes end its maintenance.
Key points
- Disclose what you change, e.g. prompts
- Prompts and links: edits end maintenance
- Eval set is read-only: the pass/fail bar
- Re-verify edits to get maintenance back
Contents
Draw the disclosure line at what you change
A&A perspective
Decide what the vendor must disclose by what you will change yourself, not by what it is willing to show. Here, disclosure scope means the parts of the vendor-built system that you receive in a form you can read.
A&A perspective
Those parts include the prompts (the instructions given to the AI), the evaluation set used to decide pass or fail, and the integration settings that link the system to email or team chat. Choosing what to disclose because hidden parts feel unsafe causes trouble later.
A&A perspective
As a rule, any part you get disclosed should be a part you can change. Once you change it, it is no longer what the vendor tested, so it leaves the vendor's maintenance. What is disclosed, what you change and what leaves maintenance: keep these three on one line.
A&A perspective
If this line is not set before the contract, the cost shows up when the system breaks. Someone has to find out whether your change or the vendor's caused it, and that work is in no estimate. The two sides may start blaming each other before anyone looks.
From the sources
Gumloop, which provides tools for automating work with AI, wrote about its own support operation in February 2026. It says its support team has only two people, and that they built the system with the same building blocks available to every Gumloop user. The system grew one workflow at a time over months.
A&A perspective
Automate & Augment (A&A), which writes this article, reads one point from this about what you receive from a vendor. Even when every part is visible, how the parts are put together and how they keep being fixed do not come with them. Receiving every part does not by itself let you run the system.
A&A perspective
Some parts need no disclosure. These are the vendor's shared base that runs the AI and reports failures across its projects. You will not change them, so the vendor can keep maintaining them.
Prompts and links are yours to edit, and edits end maintenance
A&A perspective
Have the prompts and integration settings disclosed so you can change them, and agree before the contract that changing them ends their maintenance. Both change whenever the work changes.
Hypothetical example
Say you have a vendor build an AI system that does the first sort of incoming inquiry emails (a hypothetical example). The prompt text, the sorting categories, the threshold for passing a message to a person and the people notified all change when the work changes. These go on the side you change.
A&A perspective
Integration settings belong on the same side. They connect the system to the inbox or team chat: the destination, and the key (a string that works like a password) used to connect. The key is yours, so you must be able to replace it yourself, for example when the person in charge leaves.
A&A perspective
One more thing to settle. Replacing a key moves the setup away from the tested state even if the vendor did nothing. That is not your fault, so agree that the check after a key replacement is done free of charge.
A&A perspective
Treat small changes the same way, such as adding a person to the notification list. If the contract draws a line between small and large changes, that line becomes the next argument when something breaks. In return, tell the vendor about every change, however small.
A&A perspective
Writing only "touch it and we stop maintaining it" leaves everyone too afraid to touch anything. Then the disclosure is pointless. Disclosure helps only once you also agree how lapsed maintenance comes back.
A&A perspective
Fields such as medical care or finance, where "a setting changed" is no answer for a regulator, are different. It is hard to set aside parts you change yourself, so it fits better to have the whole system made non-editable.
| Part | Disclosure and your changes | Vendor maintenance |
|---|---|---|
| Prompt text, categories, thresholds, notification list | Disclosed; you change them | Ends when you change them; returns after re-verification passes |
| Integration settings (destination and key) | Disclosed; you change them | Ends when you change them; the check after a key replacement is free |
| Evaluation set | Disclosed, but edit permissions block changes | Pass or fail is judged with the vendor's copy |
| Shared base that runs the AI and reports failures | No disclosure needed; you do not change it | Stays with the vendor |
Eval set is read-only and stays the pass/fail bar
A&A perspective
Have the evaluation set disclosed, but kept where you cannot change it. An evaluation set (eval set) is the pass/fail criteria agreed at acceptance (the check when you receive a delivery), together with the real examples used to judge against them.
A&A perspective
If the yardstick changes, you can no longer tell whether the AI still passes. This is the one exception where what is disclosed and what you change do not match. Pass or fail is judged with the copy of the eval set the vendor keeps.
From the sources
Anthropic has published a customer page on Lovable, a service that turns a plain-language description into an app. According to the page, every new Claude release goes through the same evaluation Lovable has run from the start.
From the sources
That evaluation measures how often the system hits a wall and produces an app that is broken or not what the user asked for. The page also gives the reason the check matters: Lovable's users often cannot read the code themselves, so they trust the output to work.
A&A perspective
A business that buys AI from a vendor is in a similar position. Even with the prompts in hand, there is no time to read and check all of them. What you can rely on is whether the same evaluation runs after every change. That is why the eval set stays unchanged.
A&A perspective
A read-only category does not hold just because you write "please do not change this." It holds only when edit permissions block the change itself. If the system runs on a service that cannot split permissions, treat the whole system as a part you change, and judge pass or fail with the vendor's copy.
A&A perspective
How to write the pass/fail criteria themselves is covered in a separate article, "AI deliverable acceptance criteria: 4 terms for output that varies."
Re-verify edits to get maintenance back on the last verified setup
A&A perspective
Attach the vendor's maintenance to the last verified setup; after you change something, maintenance returns once the vendor's re-verification passes. Put this in the agreement. The last verified setup is the full set of settings at the point the vendor ran the eval set and confirmed a pass.
A&A perspective
Re-verification means the vendor runs the eval set again on the setup after your change and confirms it passes. If it fails, choose between putting the changed parts back to the saved setup and asking the vendor to fix them.
A&A perspective
For this to work, the vendor has to actually save that setup. At delivery and at each re-verification, have the vendor export the full settings with only the key strings removed, plus the eval set.
A&A perspective
When a fault appears, first run the eval set on the saved setup. If it passes, compare it with the setup now running. This tells you which side to look at first. It does not settle who is responsible.
A&A perspective
Have the vendor accept change notices free of charge at any time. If a notice costs money, people inside the company have a reason to change things quietly. Fold re-verification into something like the monthly check, and set how many are included per period.
Hypothetical example
For the hypothetical system, the agreement could read, for example: "Maintenance covers the setup the contractor last verified. It stops covering it once the client changes the prompts, categories, thresholds, notification list or integration settings, and covers it again once the setup passes re-verification at the scheduled check."
A&A perspective
If you pay for monthly maintenance, its contents can be written the same way: watching that it runs, free change notices, and re-verification included in the period. Because of the third, the monthly fee still buys something while maintenance is lapsed.
Summary
A&A perspective
Match what the vendor discloses to what you change yourself, such as prompts.
A&A perspective
Have prompts and integration links disclosed, and agree that your edits end their maintenance.
A&A perspective
Keep the eval set read-only so it stays the pass/fail bar.
A&A perspective
Tie maintenance to the last verified setup, and re-verify edits to get maintenance back.
Disclosure from an AI vendor works when what you see, what you change and what leaves maintenance sit on one line. What you decide next is which parts you will change yourself, and how and how often re-verification brings maintenance back.
FAQ
What if the vendor refuses to disclose the prompts?
Put those parts on the side you do not change and leave their maintenance with the vendor. Then have the vendor include the cost of requesting a change each time the work changes, and compare that with the disclosed option.
If we get every part disclosed, is it easier to switch vendors?
Switching takes less work. But how the parts are put together and how they keep being fixed do not come with them, so receiving every part does not by itself let you run the system.
Where should these terms be written so they take effect?
Have the vendor write them into the assumptions of its quote, and add one line to your purchase order: "includes the assumptions of the quote." Whether this holds up legally as written is a separate question; check with a professional before turning it into a contract.
Sources & editorial note
Primary pages read for this article. Publication dates below belong to the sources; access dates record our research.
- Supporting the world’s most AI-native companies with a 2-person team
Gumloop · 2026-02-17
Accessed 2026-10-23 - Lovable helps anyone create software 20x faster with Claude
Anthropic · no date shown on page
Accessed 2026-10-23