Run a Receivables Tone Experiment

Summary

This guide runs a controlled A/B comparison between two tone profiles. The experiment assigns eligible customers to one profile. It keeps each customer's assignment stable. It reports recorded payment conversions by variant.

Capabilities

You can create and run experiments that randomly assign eligible customers to two different tone profiles, track their communication assignments, and measure payment conversions attributed to each variant. After evaluating results, you can set the preferred profile as the new default and update communication templates.

Prerequisites

Prerequisites

• At least two reviewed tone profiles exist.
• The workflow communication path uses a template and can resolve a template-specific or team-default tone profile.
• Autopilot Collections is explicitly saved on for the controlled send; the invoice and contact remain eligible for communication.
• The workflow has customer context and real eligible traffic.
• You set up the workspace's AI and communication providers.
• Invoice payment and recovery records are reliable.
• The team defined the hypothesis, observation window, minimum sample, and decision owner.

An Auto-chase policy that you create only through Settings → Receivables Controls is not enough evidence that the messages use a tone profile. That form does not attach a template or a tone profile.

Concepts

A tone profile controls the language and style of customer-facing payment messages. An experiment splits eligible customers into two groups, each receiving messages shaped by a chosen profile. Assignments are random but consistent for each customer. The experiment tracks assignment counts and observed payment conversions to compare performance. Outcomes depend on actual customer payments while the experiment is active.

Workflow

1. Define One Testable Hypothesis

Choose one deliberate difference, for example:

• empathetic versus neutral words
• neutral versus firm words
• lower versus higher urgency
• standard versus simplified reading level

Write down:

• expected direction of change
• target customer population
• start and end dates
• minimum assignment count per variant
• primary metric
• conditions that must stop the test

The page reports assignment and conversion rates. It does not calculate statistical significance. It does not find a safe sample size.

2. Prepare Variant A and B

1. Open Receivables → Tone Profiles.
2. Create or edit the two profiles.
3. Give each a clear experiment-safe name.
4. Examine preset, formality, empathy, urgency, reading level, and brand voice.
5. Examine blocked phrases, necessary disclaimers, and maximum length.
6. Save both profiles.

Do not make the variants differ in many unrelated ways when you must understand why one variant did better.

3. Confirm the Eligible Communication Path

Before you create the experiment, send an approved communication to a controlled test recipient through the same workflow.

Confirm:

• the workflow uses a communication template
• a template-specific or team-default tone profile is available
• customer context is present
• the AI provider transforms content when expected
• the provider sends the message
• the original template is safe if transformation fails

A tone failure falls back to the original content and does not stop the send. Content rules help you check the transformation. They do not replace your examination of the base template.

4. Create the Experiment

1. Open Receivables → Tone Experiments.
2. Select New Experiment.
3. Enter a name of 1–120 characters.
4. Add an optional description of up to 500 characters with the hypothesis and planned window.
5. Choose the profile for Variant A.
6. Choose a different profile for Variant B.
7. Set the percentage that goes to A. The slider moves in five-point increments; Eigenn gives the remainder to B.
8. Select Create experiment.
9. Confirm that the status is Draft.

Use a two-sided split such as 50/50 when both variants need comparable volume. Eigenn lets you set a 0/100 or 100/0 split. That split cannot give a useful two-variant comparison.

5. Start the Experiment

1. Select View details.
2. Recheck the variant names and traffic split.
3. Select Start experiment.
4. Confirm that the status is Running.
5. Confirm that the started time appears.

Only one experiment can run for the team. If the page names another active experiment, pause or complete that experiment first.

6. Let Assignments Accumulate

While the experiment is active, an eligible communication can assign its customer to A or B. The first assignment for that experiment is sticky. Later eligible messages to the same customer use the same variant.

This means:

• the traffic split controls assignment probability, not exact counts
• repeat messages do not create a new assignment for the same customer
• the denominator represents assigned customers, not messages
• small samples can show an unbalanced result

Open View details to examine the A and B counts and customer names, with IDs as a fallback. If no assignments appear, do not assume that the experiment works. An assignment also does not prove that its message was transformed or delivered: assignment occurs before the AI and delivery outcome is known. Template and segment overrides can remain in effect after the variant profile is selected.

7. Preserve the Payment Observation Window

Keep the experiment Running while assigned customers have a reasonable opportunity to pay. The supported automatic outcome path looks for active experiments when it records a recovery payment conversion.

If you pause or complete the experiment too early, later payments can stay unattributed. Manual invoice status changes and some alternative payment paths can also fail to populate an experiment outcome.

Before you end the test:

1. Confirm that the assigned customers got the intended eligible communication.
2. Let the planned payment window pass.
3. Reconcile recorded payments and invoice status.
4. Confirm that the experiment summary shows updated outcomes.

8. Read the Results

The details view shows total and per-variant:

• assignments
• conversions
• conversion rate

Interpret conversion rate as:

assigned customers with a qualifying true outcome ÷ assigned customers

The saved converted field takes precedence. Only when it is absent or null does the summary use paid. Thus converted: false with paid: true does not count. Outcomes belong to customers, not individual invoices; a later recorded outcome replaces the earlier one. A rate with no assignments is shown as unavailable.

It is not:

• delivered messages divided by sent messages
• amount recovered divided by amount due
• time to payment
• proof that tone caused the difference

Compare sample sizes and source records before you compare percentages. One conversion in a very small group can produce a large but unstable rate.

9. Pause or Complete Deliberately

Use Pause when the test must stop the intake of eligible traffic for a time. Automatic payment attribution can also stop during the pause. Messages already queued are not recalled. Resume with Start experiment: customer assignments stay the same, but the displayed started time changes to the restart time. Record the pause window separately.

Use Mark completed only after the observation window closes.

1. Select Mark completed.
2. Confirm the completed time.
3. Examine both variant rates again.
4. Select Variant A wins or Variant B wins only when the decision owner approves.

The winner selection is a label on the experiment. It does not change live communication behavior by itself. The UI replaces the choice buttons with a winner badge and does not offer a restart control for completed experiments.

10. Roll Out the Winner

1. Open Receivables → Tone Profiles.
2. Select Set Default on the profile that won, when it must become the fallback.
3. Update each template-specific profile that must use the winner.
4. Send a controlled message through each affected workflow.
5. Check the transformed content, the necessary language, the provider delivery, and the analytics attribution.

Keep the profile that lost until you examine all dependent templates, segment rules, and experiment records.

11. Preserve or Delete the Experiment

Keep the completed experiment when its assignment and conversion record is useful for audit or future comparison.

Use Delete only when you want permanent removal. It opens Delete this experiment?. Select Cancel to retain the record or confirm Delete to remove the experiment and all assignments. This cannot be undone. Deleting either variant profile also removes the linked experiment and assignments; preserve the profiles while retaining the experiment history.

Behavior specification

Troubleshoot

New Experiment is unavailable

The creation area shows Create tone profiles instead of New Experiment until two profiles exist. Create two different reviewed profiles, then return.

Start experiment fails

Another experiment is already active. Open it. Then pause it or complete it before you start the new one.

No assignments appear

Confirm that the status is Running. Confirm that the workflow prepares template-based communications with a tone profile and customer context. A saved experiment or a basic recovery policy does not generate traffic.

One variant has far fewer assignments

Assignment is random and sticky for each customer. Wait for more unique eligible customers. Confirm that the split you set is not near 0% or 100%.

Payments happened but conversions remain zero

Check whether Eigenn assigned the customers. Check whether the experiment was still active when Eigenn recorded the supported recovery outcome. Check whether the payment path writes an experiment outcome. A manual paid status can leave the summary unchanged.

Tone Profiles delivery counts disagree with experiment assignments

The two views use different attribution. Tone analytics group tracked communications by the profile that links to the template. The experiment view counts customer assignments and recorded outcomes.

The winner selection changed nothing

Open Tone Profiles. Set the intended default. Update the template-specific configuration. Winner selection does not deploy the profile.

Diagrams

Rendering diagram…

Screenshots

Shown with synthetic data in a local workspace.

Verification

Confirm the Result

The experiment is complete when:

• you defined the hypothesis and the observation window before launch
• A and B are different, reviewed profiles
• you checked the eligible workflow and the base template
• the experiment was the team's only active experiment
• both variants accumulated a useful assignment count
• you reconciled the payments and the invoice records
• you thought about the limits of the result
• you selected the winner manually
• you updated the default and the template configuration separately
• you checked the messages after rollout

1. Prepare at least two reviewed tone profiles and confirm a workflow with active, eligible template-driven communications.
2. Send a test message to a controlled recipient and check it uses a selected tone profile.
3. Create and start a new experiment, checking that it appears in Draft and then Running status.
4. Monitor assignment counts in experiment details after some eligible customer traffic is present; expect A and B counts to appear, with each customer assigned to one variant.
5. Complete a supported recovery payment for an assigned test customer while the experiment runs; confirm its outcome and variant count. Check other payment paths separately instead of assuming every paid invoice updates the experiment.
6. Once the observation window passes, reconcile experiment outcome counts with source payment and invoice data.
7. Use the experiment controls to mark completion and select a winner. Verify that the selected winner does not automatically update communication defaults.
8. Manually set the winner as the new default tone profile and update any template-specific assignments.
9. Send a control message through updated workflows and check successful delivery and correct attribution in analytics.
10. Cancel the deletion dialog and confirm the experiment remains. For a disposable experiment, confirm Delete and verify that the record and assignments are removed.

Related

Related Pages

• Tone Profiles and Experiments
• Receivables Controls
• Configure Receivables Controls
• Invoice Collection Workflow
• Receivables