Proofsplit

After launch: confirming the win holds

When you roll out a winner, 5% of visitors keep seeing the original for 28 days (a holdout). The rest see the winner.

  • If the winner stops beating the original, you get an "effect faded" alert suggesting a rollback.
  • After 28 days the result is confirmed and the holdout is released, so no test code keeps running for nothing.
  • Holdout visitors are re-randomised separately from the original test, so nobody's past assignment biases the check.

Are you sure?