Reading results

Unusual traffic left out

When one source sends far more visitors or conversions than a real person could, Needlr leaves them out of the numbers and tells you.

In Needlr: Check page, under the A and B numbers

0:00 / 1:01

An impact check is only as good as the traffic behind it. One script hammering your page, or someone faking conversions, could hand you a false winner.

Sometimes one source sends far more visits or conversions than a real person could. Needlr leaves those visitors out of the numbers, so they can't decide your verdict. The check page tells you when it happened.

Think of a class vote

A class votes on where to go for the school trip. One kid stuffs the box with 30 papers. The teacher doesn't count those 30. Everyone else's votes still count, and the vote stays fair.

How Needlr spots it

Every visit, view and KPI hit that reaches Needlr gets a source: a scrambled code made from the internet address it came from. The code changes every day. The address itself is never stored.
For each impact check, Needlr counts how many visitors and conversions each source brought.
A source is unusual when it brings at least 25 visitors and 5% of the check's visitors, or at least 6 conversions and 5% of its conversions.
Every visitor from an unusual source is left out, from both versions.

One person usually brings one visitor, maybe a few (a phone and a laptop). A source with dozens is a script, a testing tool, or someone trying to make a version look like a winner.

Needlr doesn't judge visitors by their browser or device. It only looks at how much comes from one place. So a busy day with lots of real people, like a sale or a mention in the news, is not left out: it comes from many places.

What you'll see

Under the A and B numbers on the check page, one line:

Unusual traffic excluded

46 visitors came from a single source sending far more than normal.

The line on the check page when Needlr leaves traffic out. No line means nothing was left out.

The numbers, the needle, the verdict and the Live panel all leave out the same visitors (the check page).

Where it applies

Split tests, Before vs after and Adoption checks all use the same rule.
Not for KPIs counted by Google Analytics. Google Analytics doesn't tell Needlr where a visit came from, so nothing can be left out there (Google Analytics).
Your team's own previews are never counted at all (Preview B).

There's one more limit: one source can bring at most 50 new visitors to a project in a day. After that, new visitors from it simply see your site as it is, and aren't put in any check that day.

Good to know

A big office or school can share one internet address. If many people there visit your site on the same day, they may be left out too. Your result stays fair. It just fills up a little slower.
Someone using many addresses can stay under these limits. No website tool can fully stop that. Needlr also caps how much traffic one project can receive in a day.

Common mistakes

Worrying when you see the line. It means the protection worked. Your verdict is based on real people.
Testing your change by clicking around your live site a lot. Those clicks count like any visitor's. Use Preview B instead: preview visits are never counted.

Words to know

Source: A scrambled daily code for the internet address a visit came from.
Unusual: A source that brings far more visitors or conversions than one person could.
Left out: Not counted for either version, in any of the check's numbers.

What the video says

An impact check is only as good as the traffic behind it. One script hammering your page, or someone faking conversions, could hand you a false winner.

So Needlr tags every event with its source: a scrambled code for the visitor's internet address that changes every day. The address itself is never stored.

A source is unusual when it brings at least 25 visitors and 5% of the check's visitors, or 6 conversions and 5% of all of them. One real person never does that.

Every visitor from that source is left out, from both versions, in the verdict and the live panel alike. The check page tells you how many were removed.

The same rule covers before vs after and adoption checks too. And your team's own previews never count at all, so testing your change doesn't skew the result.

Most weeks you'll never see this line. When you do, read it as protection working: your verdict rests on real people, not on one noisy source.