When to use a traditional test

Use a traditional test when you have a hypothesis and want to compare different versions of an experience to find out which performs best overall.

For example, you might want to know whether visitors are more likely to start a trial when your CTA says:

• Start free trial, or
• Try it free

A traditional test gives both versions a chance to perform with real visitors, then measures them against the same goal.

[IMAGE: One visitor group splitting across two CTA variations, both measured against Trial starts.]

The question you’re trying to answer is: Which experience performs better for this group of visitors?

If you already know that different audiences should receive different experiences, that’s a personalization instead.

What goes into a traditional test

Every traditional test has a few core parts:

• Base: The original experience you’re testing against.
• Variations: The alternate versions you want to compare.
• Target goal: The primary outcome that determines how the test is evaluated.
• Audience, optional: Limits who is eligible to participate in the test.
• Traffic split: Controls how eligible visitors are distributed across the base and variations.

Variations can be different versions of an element or experience, such as headlines, CTAs, images, or layouts.

[IMAGE: A test showing Base, Variation A, Variation B, target goal, and traffic split.]

Note: Be intentional about how many variations you add. In a traditional test, more variations divide your traffic across more experiences, which can make it take longer to collect enough data for a useful result.

Build your test

Once you have your hypothesis, the setup follows a simple sequence:

1. Create the test. Choose the page or experience you want to optimize.
2. Build your variation. Start from the base and make the change your hypothesis calls for.
3. Choose the target goal. Select the outcome you’ll use to evaluate the test.
4. Set an audience, if needed. Limit the test to a specific group, like mobile visitors.
5. Set the traffic split. Decide how eligible traffic should be divided between your versions.

Now that you know the pieces of a traditional test, use the walkthrough below to practice building one from start to finish.

[ARCADE: Create a traditional test, add a variation, choose a target goal, and configure traffic.]

Best practice

Test one clear idea

When possible, change one meaningful variable at a time. If you change the headline, CTA, and layout in the same variation, you may learn that the new experience performs differently — but you won’t know which change caused it.

A focused test gives you a clearer result you can carry into the next idea on your roadmap.

QA before launch

Before your test reaches visitors, do a quick check that everything is set up the way you expect. Make sure:

• Each variation looks right across the breakpoints that matter
• The correct audience is attached, if you’re using one
• The correct target goal is selected
• Tracking is active on the domain where the test will run

A quick QA pass helps catch setup issues before they affect the visitor experience or the data you’ll use to evaluate the test.

Launch the test

Once everything checks out, it’s time to launch the test and start collecting real visitor data.

[IMAGE: Publishing or launching a traditional test.]

Eligible visitors begin entering the optimization. Optimize distributes them across the base and variations according to the traffic split you set, then measures how each experience performs against the target goal.

As data comes in, you may start to see one variation pull ahead — but avoid making a call too early. Traditional tests need enough traffic, conversions, and time before the result becomes meaningful.

While the test is running, avoid unnecessary changes to the experiences you’re comparing. Keeping the conditions stable makes the result easier to interpret later.

Note: You’ll learn how to interpret traditional test results, including when there’s enough evidence to act, later in the course. For now, the important thing is to let the test run, avoid unnecessary changes, and give it time to collect meaningful data.

Ready to continue?

You now know how to turn a hypothesis into a traditional test, configure the pieces that control it, and QA it before launch.

Next, you’ll learn how to build a manual personalization, where audiences don’t just decide who participates — they determine which experience each visitor receives.