A team tests a new reminder automation against "a few test records" in the live CRM. One of those records still has a real customer's email address. The customer receives three reminders for an invoice that does not exist.

Most automation mistakes during testing come from the same gap: the test runs somewhere that can reach real people, real money, or real records.

Decide where testing happens

For each system the automation touches, write down which option applies:

  • A sandbox or test account provided by the vendor. Many accounting, payment, CRM, and email services offer one.
  • A separate test workspace you create yourself, with its own data.
  • The live system in read-only mode, when there is no alternative and the automation only needs to look, not act.

If a system has none of these, the automation should not write to it until it has passed a shadow-mode pilot, where it proposes actions but a person carries them out.

Build a small, realistic test set

Twenty well-chosen records teach you more than a thousand random ones. Include:

  • Typical cases that should go through without issues.
  • Edge cases: missing fields, long names, unusual characters, different currencies or time zones.
  • Records that should be skipped or rejected.
  • Duplicates, to check the automation does not act twice. Our retries guide explains why this matters.

Use invented names and contact details you control, such as a shared team inbox. Do not copy live customer records into test systems unless you have removed personal details and agreed that this is allowed.

Make it impossible to reach real people by mistake

Add a guard in the automation itself, not just a note in the documentation:

  • In test mode, send all emails and messages to one internal address.
  • Use clearly fake amounts and reference numbers.
  • Label every test record, for example with "TEST" in the name, and have the automation refuse to act on unlabelled records while in test mode.

Write down expected results first

Before each test run, note what should happen for every record. Then compare. A run that "worked" but produced a different result than expected is a failure, even if no error appeared.

Keep the test set for later

Every time you change the automation, run the same test set again. When a real problem appears in production, add a matching test record so it cannot return unnoticed.

To plan safe testing for an automation you are building, email Storm Nova Labs.