Splitlab home

Book a Demo

Splitlab home

Book a Demo

Splitlab home
A multivariate test: three sections with their options, 12 combinations with 6 served, and a predicted winner at +21.6%

November 10, 2026

By

Bishal Katuwal

·

8 min

read

How Multivariate Testing Works: A Step-by-Step Example

November 10, 2026

By

Bishal Katuwal

·

8 min

read

How Multivariate Testing Works: A Step-by-Step Example

In the ever-evolving business environment, achieving a competitive edge demands innovative solutions beyond conventional methods. Organizations require smart systems that can optimize workflows, enhance customer interactions, and reveal new avenues for growth. Enter Biotix, the cutting-edge AI-powered CRM management tool.

Three headlines, two hero images and two guarantees make 12 versions of one sales page. Follow a multivariate test from setup to winner, step by step, and see exactly how the result is worked out.

Bishal Katuwal

Bishal Katuwal

Splitlab Founder

A multivariate test: three sections with their options, 12 combinations with 6 served, and a predicted winner at +21.6%
A multivariate test: three sections with their options, 12 combinations with 6 served, and a predicted winner at +21.6%

Most explanations of multivariate testing stop at “it tests several elements at once”. That’s true, but it doesn’t tell you what actually happens between building the test and getting a winner.

So this post follows one test from start to finish. One sales page, three ideas, 30,000 visitors, and every number you'd see in the report along the way. If you're still deciding between this and a plain split test, start with A/B testing vs multivariate testing.

Key takeaways

  • A multivariate test splits a page into sections (headline, image, guarantee), gives each section a few options, and builds every combination of them.

  • The number of combinations is the options multiplied together: 3 × 2 × 2 = 12.

  • Each visitor sees one combination. Each option is then judged on everyone who saw it, across all the combinations it appears in.

  • A smart subset serves 6 of those 12 and predicts the other 6, so each served combination gets twice the traffic.

  • The report gives you three answers: the best option in each section, how much each section matters, and the predicted winning combination.

  • In our example, the winning mix beat the original page by an estimated 21.6%, and the headline did most of the work.

What is multivariate testing?

Multivariate testing (MVT) is a way to test changes to several parts of one page at the same time. Instead of comparing two whole pages, it mixes every option for every part, sends visitors to those mixes, and works out which option wins in each part and which mix earns the most.

The example: one course sales page

Say you sell an online course from a single sales page. Right now it converts at about 3.5%. You have three ideas:

Section

Control (today)

New option 1

New option 2

Headline

Current headline

Outcome: leads with the result

Question: asks about the problem

Hero image

Product mockup

Results: a student's results screenshot

Guarantee

30-day refund

60-day refund

You could run three A/B tests, one after another, and wait three times as long. Or you could test all three at once.

Step 1: Break the page into sections

A section is one part of the page you want to change. Pick parts that do different jobs: the headline, the hero image, the guarantee, the button, a block of social proof.

In Splitlab you do this in the visual editor. Click Add Section, and the section is created with a Control option already in it: the page as it is today. You don't build the control yourself.

Good sections are independent. If the new headline only makes sense with the new image, they aren't two sections; they're one idea, and belong in one option.

Step 2: Write the options

Inside each section, add the new options with Add Variant and edit the element on the page. Our example ends up with:

  • Headline: Control, Outcome, Question (3 options)

  • Hero image: Control, Results (2 options)

  • Guarantee: 30-day, 60-day (2 options)

Keep options few and different. Two or three options per section is plenty. Five shades of the same headline won’t teach you anything a visitor can feel.

Step 3: The combinations build themselves

Every time you add or change a section or option, Splitlab rebuilds the combinations automatically. The count is the options multiplied together:

3 headlines × 2 images × 2 guarantees = 12 combinations

They're listed under Combinations & Traffic as Control, Combination 1, Combination 2 and so on, up to Combination 11. The Control is the page with every original element. Splitlab allows up to 100 combinations in one test, but the fewer you have, the faster you get an answer.

Step 4: Choose all combinations or a smart subset

Under Combinations to test you pick one of two designs:

  • All combinations serves all 12. Every combination is measured directly.

  • Smart subset serves a balanced 6 of the 12 and predicts the rest.

The subset isn't random. It's picked so that every option is served, and every pair of options from two different sections appears together at least once. For our page, the six served combinations are:

Combination

Headline

Hero image

Guarantee

Control

Control

Control

30-day

3

Control

Results

60-day

4

Outcome

Control

30-day

7

Outcome

Results

60-day

9

Question

Control

60-day

10

Question

Results

30-day

Check any two columns and you’ll find every pairing once: each headline with each image, each headline with each guarantee, each image with each guarantee. That’s what makes the other six combinations predictable.

We'll use the smart subset for the rest of this example.

Step 5: Split the traffic

Next, decide how visitors are shared between the served combinations. It's the same Traffic Split setting as an A/B test:

  • Equal gives each served combination the same share. With 6 served, that's about 16.7% each.

  • Custom lets you set the shares by hand.

  • Multi Arm Bandit shifts traffic toward what's winning as results come in. On a multivariate test it scores each option, not each combination, and builds each combination's share from how its parts are doing. How a multi-armed bandit works explains the details.

Our example uses Equal. See Equal vs Custom vs Multi-Armed Bandit for when to pick which.

Whichever you choose, each visitor is assigned one combination and sees that same version when they come back. That's what keeps the data clean.

Step 6: Pick what counts as winning

The test's primary metric decides the winner, both for each option and for the whole combination. Our example uses conversion rate to keep the numbers easy to follow. On a real sales page with upsells or several price points, set it to revenue per visitor so a headline that attracts cheaper buyers can't win on clicks alone.

Step 7: Let it run

The page gets 30,000 visitors over the test. Split across 6 combinations, that's 5,000 visitors and roughly 150 to 210 sales for each. On its own, that isn't much for one combination.

But look at it by option instead:

  • Each headline appears in 2 of the 6 served combinations, so it's seen by 10,000 visitors.

  • Each image and each guarantee appears in 3 of them, so each is seen by 15,000 visitors.

This is the core of how multivariate testing works. Every visitor counts toward one option in every section at once. A 30,000-visitor test is really three tests, each on the full 30,000.

Step 8: Read the results

The numbers below come from running this example through the same calculations Splitlab's report uses. Your test will have its own numbers; the way you read them is the same.

Main effects by section

The first view pools each option across every combination that contains it, then compares it with the control in its own section:

Option

Conv. rate

Lift vs control

P(best)

Headline: Control

3.58%

19%

Headline: Outcome

3.82%

+6.7%

81%

Headline: Question

3.07%

−14.2%

0%

Hero image: Control

3.30%

4%

Hero image: Results

3.68%

+11.5%

96%

Guarantee: 30-day

3.36%

11%

Guarantee: 60-day

3.62%

+7.7%

89%

P(best) is the probability that this option is the best one in its section. Three clear readings: the Question headline hurts, the results screenshot helps, the 60-day guarantee probably helps.

Predictions and variable influence

The second view fits a simple model to the six combinations that got traffic. It treats each combination's result as the control's result plus the effect of each option it uses. That model gives you two more answers.

Variable influence shows how much each section moves the result: the headline accounts for about 61%, the hero image 27% and the guarantee 12%. If you only had time to work on one thing next, it would be the headline.

All combinations lists all 12, each marked Measured or Predicted, with a 95% range around every prediction:

Predicted conversion rate for all 12 headline · image · guarantee combinations, 6 of them served: Control · Control · 30-day 3.34% (measured 3.52%), 3 Control · Results · 60-day 3.82% (measured 3.64%), 4 Outcome · Control · 30-day 3.58% (measured 3.40%), 7 Outcome · Results · 60-day the best at 4.06% (measured 4.24%), 9 Question · Control · 60-day 2.98% (measured 2.98%), 10 Question · Results · 30-day 3.16% (measured 3.16%); predicted only: 1 3.49%, 2 3.67%, 5 3.73%, 6 3.91%, 8 2.83%, 11 3.31%

The predicted winner

Top of the list is Combination 7: Outcome headline, results screenshot, 60-day guarantee. The report predicts it at 4.06%, an estimated +21.6% against the control, with a 97% chance of beating the control.

Combination 7 happened to be one of the six served, so you can also see its measured rate: 4.24%. When the predicted winner is one that was never served, the report tells you, and you can switch it on from the combinations screen to confirm it with real visitors before you roll it out.

Step 9: Check the one assumption

The predictions assume each section adds its own effect, whatever sits next to it. Usually that's close enough. Sometimes two elements work together: a headline that says "see the results" might only land next to a results screenshot. That's called an interaction.

The check is quick. Look at the combinations marked Measured and compare each one with its prediction. In our example every measured result lands inside its 95% range, so the simple model holds. If one landed well outside, you’d confirm the winner with all combinations or a follow-up A/B test before trusting the prediction.

Common mistakes

  • Too many options. Four sections of four options is 256 combinations, more than Splitlab allows and far more than most pages can feed. Start with two or three sections.

  • Options that depend on each other. If two changes only make sense together, make them one option.

  • Tiny changes. A slightly different shade of the button won't show up in any section. Test changes a visitor would notice.

  • Stopping at the first good day. Let the test run through at least one full week so weekday and weekend buyers both count.

  • Judging on clicks. Use revenue per visitor wherever the order value can change.

FAQ

How does multivariate testing work in simple terms?

You split a page into parts, write a few versions of each part, and the tool builds every mix of them. Each visitor sees one mix. Because every option appears in many mixes, each option is judged on a large share of your traffic, and the tool works out the best option for each part and the best mix overall.

What is an example of a multivariate test?

A sales page testing three headlines, two hero images and two guarantees at once. That makes 3 × 2 × 2 = 12 combinations. In our example, the best mix was the outcome-focused headline with a results screenshot and a 60-day guarantee, an estimated 21.6% better than the original.

How many combinations should a multivariate test have?

As few as answer your question. Two or three sections with two or three options each (4 to 27 combinations) suits most pages. Splitlab allows up to 100, and a smart subset cuts the number you actually serve.

Do I need to test every combination?

No. A smart subset (also called a fractional factorial design) serves a balanced set where every option and every pair of options still appears, then predicts the rest. In our example, 6 served combinations predicted all 12.

How do I know which element made the difference?

The Main effects by section view shows each option's lift and its chance of being best within its section, and Variable influence shows how much each section moved the result. In our example, the headline drove most of the gain.

How long should a multivariate test run?

Until each option in each section has enough conversions to tell apart, and at least one full week. Since each option is judged on everyone who saw it, think in conversions per option, not per combination.

My final thoughts

Multivariate testing looks like magic in a demo: a dozen combinations, a winner, done. In real funnels it only works when you have the traffic to feed it and you judge it on money, not clicks.

If your page gets steady traffic and you have ideas for several sections, it’s a great tool. If not, run a bold A/B test first and save multivariate for a page that’s already worth fine-tuning.

Either way, check the one assumption before you trust the result: that the sections don’t depend on each other. Book a demo if you want to see a multivariate test on your own page.

More combinations need more visitors. Test what your traffic can answer.

Built for revenue

Test for revenue, not just clicks.

Run A/B, multivariate, and multi-armed bandit experiments — with real revenue tied to every variation.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.

✓

Revenue per variation

✓

A/B & MVT

✓

Multi-armed bandit

Test for revenue, not just clicks.

A/B, multivariate and multi-armed bandit experiments, with real revenue tied to every variation.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.