Splitlab home

Book a Demo

Splitlab home

Book a Demo

Splitlab home
Three variations drawn as belief curves, with the traffic split that a multi-armed bandit gives each one

November 8, 2026

By

Bishal Katuwal

·

10 min

read

How Does a Multi-Armed Bandit Work? The Idea Behind Tests That Learn

November 8, 2026

By

Bishal Katuwal

·

10 min

read

How Does a Multi-Armed Bandit Work? The Idea Behind Tests That Learn

In the ever-evolving business environment, achieving a competitive edge demands innovative solutions beyond conventional methods. Organizations require smart systems that can optimize workflows, enhance customer interactions, and reveal new avenues for growth. Enter Biotix, the cutting-edge AI-powered CRM management tool.

A multi-armed bandit moves traffic toward the winner while your test is still running. Here’s the idea behind it, the maths in plain English, and the guardrails that stop it betting on luck.

Bishal Katuwal

Bishal Katuwal

Splitlab Founder

Three variations drawn as belief curves, with the traffic split that a multi-armed bandit gives each one
Three variations drawn as belief curves, with the traffic split that a multi-armed bandit gives each one

A normal A/B test makes a deal with you. Split traffic evenly, wait, and at the end you’ll know which page wins. The price is everything the losing pages cost you while you waited.

A multi-armed bandit makes a different deal. It moves traffic toward whatever is working while the test is still running. This post explains how it actually decides, without the equations: what a bandit believes, how it turns that belief into a traffic split, and the guardrails that stop it from betting on a lucky streak.

Key takeaways

  • A multi-armed bandit solves the explore vs exploit problem: keep learning about every option, while sending most visitors to the one that looks best.

  • It doesn't track one number per variation. It tracks a range of believable values, which is wide when data is thin and narrow when data is plentiful.

  • Thompson Sampling turns those ranges into a split: each variation's share of traffic is roughly its chance of being the best.

  • Uncertainty earns traffic. A new variation with little data still gets a fair chance, because the bandit can't rule it out yet.

  • A real bandit needs guardrails: a minimum share for the control, a little permanent exploration, a stop-loss for clear losers, and a limit on how fast it moves.

Where the name comes from

A slot machine used to be called a one-armed bandit: one lever, and it takes your money. Picture a row of them, each paying out at a different rate you don't know. You have a fixed number of coins. Which machines do you play?

That’s the multi-armed bandit problem, and it’s the same problem as an A/B test. Each variation of your page is a machine. Each visitor is a coin. The payout is a sale, a lead or revenue. You want to finish with as much money as possible, not just with a nice chart.

The core problem: explore vs exploit

Every coin you spend does one of two jobs:

  • Explore: play a machine to learn how good it is.

  • Exploit: play the machine you already think is best, to make money.

Lean too far either way and you lose.

Explore only. Spread coins evenly across every machine until the very end. You'll know exactly how good each one is, and you'll have fed the bad machines as many coins as the good one. That's a classic A/B test with an equal split.

Exploit only. Play a few coins in each, then put everything into whichever paid out first. Fast, but early results are mostly luck. If the real winner had a slow start, you’ll never find out, because you stopped playing it.

A fixed mix. Send 90% to the current leader and spread 10% across the rest. Better, but blunt. It explores a clear loser just as much as a close runner-up, and it treats a leader that's ahead by a whisker exactly like one that's ahead by a mile.

What you want is something that explores in proportion to how unsure it is. Close contest: keep testing both. Clear winner: commit. That’s what a good bandit does.

Step one: hold a belief, not a number

Say Variant B has made 14 sales from 300 visitors. Its conversion rate is 4.7%, right?

Not really. 4.7% is just what happened so far. The true rate could easily be 3.5% or 6%, and with 300 visitors you can't tell those apart. A bandit keeps that whole range of believable values for every variation instead of one number.

Two things shape the range:

  • The result so far sets where the range sits. More sales per visitor moves it up.

  • The amount of data sets how wide it is. 30 visitors gives a wide, flat range. 3,000 visitors gives a tall, narrow one.

Statisticians call this range a posterior distribution. For conversions, Splitlab uses the standard choice for yes-or-no outcomes (a Beta distribution). You don’t need the name. You only need the picture: a hill for each variation, sitting where its results are and as wide as its doubt.

Step two: Thompson Sampling, or "let each variation roll the dice"

Now the bandit has a hill for each variation. How does that become a traffic split? The most popular answer is Thompson Sampling, and the idea is surprisingly simple:

  1. For each variation, pick one random plausible value from its range. A variation with a wide range can land high or low. One with a narrow range lands close to its average every time.

  2. See which variation drew the highest value. That's one "win".

  3. Repeat thousands of times. Splitlab does it 4,000 times per update.

  4. Each variation's share of the wins is its chance of being the best. That becomes its share of the traffic.

How Thompson Sampling works three days in (Control 9 sales from 300 visitors, Variant B 14 from 300, Variant C 7 from 300): step 1, a belief curve over conversion rate for each variation; step 2, draw 4,000 times (Variant B wins 3,268, Control 544, Variant C 188); step 3, split traffic by those chances: Control 15%, Variant B 79%, Variant C 6%, after a 5% exploration share and the 10% control floor

That's the whole trick. A variation that's clearly ahead wins most draws and gets most of the traffic. Two variations neck and neck split their wins, so both keep getting tested. A variation that's clearly behind rarely wins, so it fades, but not to zero while there's still a reasonable chance it's better.

A worked example

Here's a test with three variations, three days in. We used sales for simplicity; the same logic works on revenue.

Sales from visitors

Conversion rate

Chance of being best

Control

9 from 300

3.0%

14%

Variant B

14 from 300

4.7%

82%

Variant C

7 from 300

2.3%

5%

With plain Thompson Sampling and Splitlab's guardrails (more on those below), the next split comes out at about 15% / 79% / 6%. B looks good, so it gets most of the traffic. The control and C still get enough visitors to change the bandit's mind if B's lead was luck.

Two weeks later, with 1,000 visitors each:

Sales from visitors

Conversion rate

Chance of being best

Control

30 from 1,000

3.0%

2%

Variant B

48 from 1,000

4.8%

97.5%

Variant C

27 from 1,000

2.7%

under 1%

Now the split is about 10% / 88% / 2%. B is almost certainly best, so it gets nearly everything. The control sits at exactly 10% because Splitlab never lets it drop lower, which keeps B's lift measurable.

Notice what happened to Variant C in the first table. It had the worst rate, and it still got 6%. With only 300 visitors, a 2.3% rate could be a slow start. The bandit keeps paying a little to find out. The same thing happens when you add a brand-new variation to a running test: with zero sales from its first 30 visitors, Thompson still gives it around a fifth of the traffic, because 30 visitors proves almost nothing. Uncertainty earns traffic. That’s exploration done in proportion to doubt.

Optimising money, not clicks

A bandit chases whatever you tell it to. Point it at conversion rate and it will happily favour a $27 offer that converts at 5% over a $97 offer that converts at 3%, even though the second one makes more money per visitor.

That's why the primary metric matters so much. When your test's goal is revenue, Splitlab's bandit draws revenue per visitor instead: a plausible conversion rate multiplied by a plausible order value, both learned from your data. The variation that brings in the most money per visitor wins the draws, even if it converts fewer people. With an average order value goal, it draws order value alone. And for metrics where lower is better, such as refunds, it flips the comparison.

If you're unsure which metric to use, our guide to revenue per visitor vs conversion rate makes the case for money.

The guardrails a real bandit needs

Plain Thompson Sampling is elegant, but on its own it can be jumpy, especially on low traffic. Every Splitlab bandit, whichever algorithm you choose, keeps these rails:

  • Control floor: 10%. The control never drops below a tenth of the traffic, so you can still measure how much better the winner is.

  • Exploration share: 5%. A small slice is always spread across every variation, so none is starved of fresh data.

  • Stop-loss. Once a variation has 100 visitors, if it's at least 40% worse than the control with 95% confidence, it's cut from the traffic straight away. On revenue goals the test is about money, so a variation that converts less but earns more isn't cut.

  • Warm-up. The classic algorithms hold an even split until every variation has 50 visitors, so nobody wins on the first handful of sales.

Splitlab's default algorithm, Splitlab Adaptive, adds three more:

  • A head start at the average. Every variation starts as if it had already seen 20 visitors converting at the test’s overall rate. One lucky sale on five visitors can’t make a variation look like a runaway winner.

  • A confidence ramp. It follows the evidence only as far as the traffic justifies. At 200 visitors per variation it moves halfway toward what Thompson would do. With thousands, it behaves like Thompson.

  • Gentle steps. No variation's share moves more than 15 percentage points in one update, so one strange day can't swing the split.

In the three-day example above, plain Thompson jumped straight to 15/79/6. Splitlab Adaptive, starting from an even split, moves to about 29% / 48% / 23%: B clearly favoured, but nothing staked on three days of data. As visitors pile up, the two converge.

When does the split change?

A bandit doesn't update after every single visitor. Splitlab checks every running bandit test each hour, and re-splits once 20 new conversions have arrived since the last update. Busy funnels move within the hour. Quiet ones wait until there's enough new evidence to say something new, instead of reshuffling on noise.

Each update learns from all of the test's data. The exception is Recency-weighted Thompson, which counts a day's results half as much for every 7 days of age, so it follows seasonality and offers that fade or improve over time.

An update never reshuffles people who've already been in the test. A returning visitor keeps seeing the same variation, so nobody's journey changes halfway through. The new split decides which page new visitors get.

Bandits and multivariate tests

Bandits get even more useful when there are many combinations. Test three headlines and three images and you have nine combinations; add a third section and it's dozens. Waiting for each combination to prove itself separately would take forever.

So for multivariate tests, Splitlab's bandit learns per option, not per combination. Each headline learns from every combination it appears in, and so does each image. A combination's draw is built from its options' draws. Evidence is shared across the whole grid, so a strong headline gets noticed long before every one of its combinations has enough visitors on its own.

What a bandit won't do

  • Give you a precise lift. Losers get fewer visitors, so you learn less about exactly how much worse they are. If you need an exact number for a permanent decision, run an equal split. Our Equal vs Custom vs Multi-Armed Bandit guide walks through that choice.

  • Fix a bad metric. It optimises what you measure. Measure clicks and it will find you clicks.

  • Make a test with very little traffic fast. It still needs evidence. On low traffic, Splitlab Adaptive deliberately stays close to even until the data can back a winner.

How to try it in Splitlab

  1. Open your test and go to Variants & Traffic.

  2. Set Traffic Split to Multi Arm Bandit.

  3. Leave the algorithm on Splitlab Adaptive unless you have a reason to change it. Our multi-armed bandit vs A/B testing guide compares all five.

  4. Check your primary metric is the one you actually want to grow, ideally revenue per visitor, and start the test.

Splitlab app Variations step, enlarged: Traffic Split set to Multi Arm Bandit and the Algorithm set to Splitlab Adaptive, the default

The first update runs within the hour. From then on, the panel shows when the split was last updated, and the split moves on its own as new conversions come in.

Frequently asked questions

How does a multi-armed bandit work, in one sentence?

It keeps a range of believable values for each variation, works out each one's chance of being the best, and gives each variation a share of traffic that matches that chance, updating as new results arrive.

What is explore vs exploit?

Exploring means sending visitors to a variation to learn how good it is. Exploiting means sending them to the variation you already think is best, to make money. A bandit balances the two, exploring more when the results are close and exploiting more when one variation is clearly ahead.

What is Thompson Sampling?

A way to turn uncertainty into a traffic split. For each variation it draws one random plausible value from that variation's range, notes which draw is highest, and repeats thousands of times. Each variation's share of wins is its chance of being best, and that becomes its share of traffic.

Is a multi-armed bandit better than an A/B test?

It's better at earning money during the test and worse at measuring exactly how big the difference is. Use a bandit for promotions, tests with several variations and always-on optimisation. Use an equal-split A/B test when you need a precise answer for a permanent decision.

Can a multi-armed bandit pick the wrong winner?

Early on, it can favour a variation that got lucky. That's why it keeps exploring, why Splitlab keeps a 10% control floor and a 5% exploration share, and why Splitlab Adaptive moves gradually on low traffic. As more data arrives, a lucky early leader loses its draws and its traffic.

How much traffic does a multi-armed bandit need?

There's no hard minimum, but it needs real conversions to learn from. Splitlab re-splits after every 20 new conversions, so a page with a few sales a day will move slowly, while a busy funnel can move within the hour.

Does a multi-armed bandit work with revenue?

Yes. With a revenue goal, Splitlab's bandit scores each variation on revenue per visitor, so a variation that converts fewer people at a higher order value can still win.

My final thoughts

A bandit isn’t smarter than an A/B test. It asks a different question. Instead of “which page is best?”, it asks “how do I lose as little as possible while I find out?”

That makes it a great fit for paid traffic, launches and short promotions, where every visitor sent to a weak page costs money. It’s a worse fit when you need a precise, clean answer to apply to other pages.

Splitlab’s bandit can optimize for revenue, not just conversions. Book a demo and we’ll show you how it moves traffic on your own test.

Learn while you earn, as long as you know what you’re giving up.

Built for revenue

Test for revenue, not just clicks.

Run A/B, multivariate, and multi-armed bandit experiments — with real revenue tied to every variation.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.

✓

Revenue per variation

✓

A/B & MVT

✓

Multi-armed bandit

Test for revenue, not just clicks.

A/B, multivariate and multi-armed bandit experiments, with real revenue tied to every variation.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.


Start optimizing for revenue today not just conversion rates.

Run experiments, track revenue per variation, and scale what actually makes you money.

We’ll increase your conversion rate by 30% or you don’t pay. Terms apply.