A support assistant starts adding a refund policy line to every reply. Nobody changed the model. Someone edited the instructions last Tuesday to fix one awkward answer, and the new sentence now affects every conversation.

Instructions, examples, and retrieval settings shape an AI workflow as much as code does. A small edit can fix one case and quietly break five others. A light change process prevents that without slowing the team down.

Record every change in one place

Keep the current instructions in a file or tool that shows history, not in someone's chat window. For each change, record:

  • What changed: the exact text before and after.
  • Why: the example or complaint that prompted it.
  • Who approved it and when it went live.
  • Which version is now in use, with a simple label such as v14.

When an answer looks wrong later, the team can check whether it started after a specific change instead of guessing.

Retest with the same examples every time

Keep a small set of saved test cases: typical questions, tricky edge cases, and requests the workflow should refuse or hand to a person. Run the full set before and after each change, not only the case you were trying to fix.

Compare the results side by side. Look for answers that became longer, lost their source links, changed tone, or started making promises the business does not make. If a change fixes the target case but breaks another, revise it before release.

Our AI quality testing guide explains how to build that first test set.

Change one thing at a time

If you edit the instructions, switch models, and adjust retrieval in the same release, you cannot tell which change caused a new problem. Separate them where practical, and note any outside change too, such as a provider updating a model version.

Keep a way back

Make the previous version easy to restore. Agree on who can roll back and when, for example if a reviewer sees an unexpected answer type twice in one day. Rolling back should be a routine step, not an emergency project.

Watch the first days after release

Review a sample of real conversations or outputs after each change. Check the cases you expected to improve and a few you did not touch. Close the change log entry only when the team agrees the new version behaves as intended.

A short change log and a repeatable test run turn prompt editing from guesswork into maintenance. To set up a change process for an existing workflow, email Storm Nova Labs.