Traditional tests
When to use a traditional test
Use a traditional test when you have a hypothesis and want to compare different versions of an experience to find out which performs best overall.
For example, you might want to know which CTA is more likely to drive trial starts.
Build your first site free
Start a free trial
A traditional test gives both versions a chance to perform with real visitors and measures them against the same goal.
If you already know that different audiences should receive different experiences, that’s a personalization instead.
What goes into a traditional test
Every traditional test has a few core parts:
Base
The original experience you’re testing against.
Variations
The alternate versions you want to compare.
Target goal
The primary outcome that determines how the test is evaluated.
Audience (optional)
Limits who is eligible to participate in the test.
Traffic split
Controls how eligible visitors are distributed across the base and variations.
Variations can be different versions of an element or experience, such as headlines, CTAs, images, or layouts.
Be intentional about how many variations you add. In a traditional test, more variations divide your traffic across more experiences, which can make it take longer to collect enough data for a useful result.
Build your test
Once you have your hypothesis, the setup follows a simple sequence:
- Create the test. Choose the page or experience you want to optimize.
- Build your variation. Start from the base and make the change your hypothesis calls for.
- Choose the target goal. Select the outcome you’ll use to evaluate the test.
- Set an audience, if needed. Limit the test to a specific group, like mobile visitors.
- Set the traffic split. Decide how eligible traffic should be divided between your versions.
Now that you know the pieces of a traditional test, use the walkthrough below to practice building one from start to finish.
Test one clear idea
When possible, change one meaningful variable at a time. If you change the headline, CTA, and layout in the same variation, you may learn that the new experience performs differently — but you won’t know which change caused it.
A focused test gives you a clearer result you can carry into the next idea on your roadmap.
QA before launch
Before your test reaches visitors, do a quick check that everything is set up the way you expect. Make sure:
- Each variation looks right across the breakpoints that matter
- The correct audience is attached, if you’re using one
- The correct target goal is selected
- Tracking is active on the domain where the test will run
A quick QA pass helps catch setup issues before they affect the visitor experience or the data you’ll use to evaluate the test.
Launch the test
Once everything checks out, it’s time to launch the test and start collecting real visitor data.
Once you launch the test, eligible visitors enter the optimization. Optimize distributes them across the base and variations according to your traffic split, then measures how each experience performs against the target goal.
- Give it time. Early results can change, so wait for enough traffic, conversions, and time before drawing conclusions.
- Keep conditions stable. Avoid unnecessary changes to the experiences you’re comparing while the test is running.
You’ll learn how to interpret traditional test results, including when there’s enough evidence to act, later in the course. For now, the important thing is to let the test run, avoid unnecessary changes, and give it time to collect meaningful data.
Ready to continue?
You now know how to turn a hypothesis into a traditional test, configure the pieces that control it, and QA it before launch.
Next, you’ll learn how to build a manual personalization, where audiences don’t just decide who participates — they determine which experience each visitor receives.