Split tests
Show your old page and your new page at the same time, and count which one works better.
You shipped a change: new pricing copy, a shorter signup form. The question you'll get: did it move the needle? A split test is the impact check that gives a causal answer.
A split test is the impact check that runs a fair race between your old page and your new page. You find out if the new one really works before everyone sees it.
Think of two lemonade stands
You have two lemonade stands on the same street. Same lemonade. Only the sign is different: stand A has the old sign, stand B has a new one.
Each person walking by is sent to one stand by a coin flip. After enough people, you count the cups. If stand B sold more, the new sign is better. The coin decided who went where, so nothing else can explain the difference.
In Needlr, the stands are the two versions of your page. The cups are your KPI, for example clicks on “Book a demo”. Needlr flips the coin for you, and a visitor gets the same version every time they come back.
When to use it
- A change you haven't shown to everyone yet: words, buttons, colors, a page, a new step in signup.
- When you need proof, not a good guess.
- The change is already live for everyone: use Before vs after.
- You want to know how many people use a new feature: use Adoption.
- Very few people visit the page: only big changes can be measured.
How to start one
- 1Open your project and press New impact check.
- 2What are you shipping? Pick Split test. Click the thing on your site you want to change, and change it (how). Or pick Split test (developer) and send your developer the one line of code Needlr shows.
- 3Which needle should it move? Pick the KPI: what you want more people to do.
- 4How should we test it? Choose how many visitors see the new version and how sure you need to be. Needlr tells you how long it will take (more).
- 5Press Start measuring. Needlr publishes to your Tag Manager. You get an email when there's a result.
What the result means
The new version got more people to do it. Keep it, and ship it to everyone.
The new version made it worse. Go back to the old one.
No real difference. Keep whichever is simpler.
Something is wrong with the counting, so Needlr puts the impact check on hold and won't decide yet. The page says what to fix (more).
Common mistakes
- Peeking and stopping early. Day 2 says B wins? Wait. Early numbers jump around (why).
- Changing the impact check while it runs. That's like moving one stand to a busier street. Stop it and start a new one.
- Testing tiny changes on a quiet page. A slightly darker blue needs a huge crowd to show a difference. Test bigger ideas.
What the video says
You shipped a change: new pricing copy, a shorter signup form. The question you'll get: did it move the needle? A split test is the impact check that gives a causal answer.
Needlr shows the current version, the control, to part of your traffic, and your new version, the variant, to the rest. Same traffic, same weeks. The only difference is your change.
Assignment is random, and it sticks: a visitor sees the same version on every visit. So the two groups are comparable, and campaigns or seasonality hit both equally.
Needlr counts your KPI in each group. Say demo bookings: 3.1% on control, 3.8% on the variant, with a confidence range around the difference.
Early numbers are noisy. Needlr sizes the impact check up front, and waits for the planned sample and a full weekly cycle before it calls it. That is how you avoid shipping a false winner.
Then you get a clear call: improved, got worse, or no clear effect, with the lift and the next step. Ship it, roll it back, or iterate.