The check page
One page per impact check: the needle, its band, the numbers, and when the answer comes.
Every impact check gets one page, your scoreboard for a single question: did this change move the needle on your KPI? Here is how to read it in a minute.
Every impact check has its own page. It answers one question: is your change helping your KPI, hurting it, or doing nothing? Open any impact check from your project to get here.
Think of a scoreboard
At a football game, the scoreboard shows who is ahead, how much time is left, and the final score at the end. The check page is a scoreboard for your new version against the old one.
One big difference: Needlr won't call the game early. A team can be ahead at half time and still lose.
What to look at first
- 1The label above the needle. It says where the impact check is: Collecting data, Too early to call, or the verdict: Improved, Got worse or No clear effect. On hold means a data problem.
- 2The sentence under the needle. It says the same thing in plain words, and about when the verdict will come.
- 3The colored box under it (after a verdict). It tells you what to do next, with a button for it (more).
The top of the page
- The name is the change you're testing. The line under it (→) is the KPI it should move. Point at the name and click the pencil to rename it.
- The status chip: Draft (not started), Running, Verdict ready, Stopped, Finished (stopped with a verdict), Shipped or Moved to code.
- Verdict ready means a split test has its verdict but still splits your visitors between A and B. A yellow box says so: ship the winner or stop the test.
- The buttons: Start (a draft), Preview A and Preview B (see each version on your site), Share the result, Stop and Delete. What each one does.
- What counts as success opens to show the KPI, the rate assumed at setup with version A's rate so far (or the Real rate, once the plan follows it), the smallest change this impact check can notice (Detects changes of), and how sure the verdict must be.
- What version B gets (or What changed) opens to show the change itself. For a developer split test, the line of code is there.
The needle
The needle shows how far apart the two versions are. The middle is zero: no difference. Right of zero, the new version (B) is ahead. Left of zero, it is behind.
The two dashed marks show the smallest change this impact check was built to notice. The scale is fixed for each one, so a small difference looks small.
The needle moves only when new results come in: every 5 minutes for a split test, every 30 minutes for a before vs after impact check. The top right corner says when it last updated.
The band
The colored band around the needle shows where the real difference most likely is. With few visitors it is wide: the truth could be anywhere inside it. With more visitors it gets narrower.
A verdict needs the whole band on one side of zero. If the band still crosses zero when all the data is in, the answer is No clear effect.
The numbers
- The big number (after a verdict) is the lift: how much better or worse B did, like +22%. Under it: each version's rate, and the range the real difference most likely sits in.
- Before the verdict, you see a small grey Leaning up or Leaning down instead. It's a hint, not a result.
- The progress bar: how much of the planned data is in, and about how many days are left. Planned is how many visitors each version needs (more). If one version gets a smaller share of visitors, that version sets the pace.
- The A and B cards: each version's rate, and how many visitors did your KPI out of how many saw that version.
- Shown without flicker (visual split tests only): the share of page views where Needlr changed the page in time. Views it couldn't change in time are left alone and not counted, for both versions, so it stays fair.
The timeline and the chart
- The timeline runs from the day the impact check started to the verdict. The pulsing dot is today. Add the verdict date to my calendar saves that day in your calendar.
- The road to the verdict (split tests) shows the difference over time, with its band. The grey part on the right is the data still to come. The line above the chart says where the impact check is heading. That's a forecast, not a result.
- A Before vs after impact check shows your KPI rate day by day instead, with the release day marked (more).
The live panel
While an impact check runs, the Live panel shows visitors arriving. It updates every 5 seconds: how many got A and B since the start, how many reached the KPI, and one bar for each of the last 30 minutes.
Use it to see that traffic is flowing. Don't use it to decide. After a split test's verdict, it turns into At the verdict: the numbers the verdict was decided on. Later visitors don't change it.
Common mistakes
- Acting on a lean. Leaning up on day 2 often fades by day 7 (why).
- Reading the live panel as a result. It counts visitors. It doesn't judge them.
- Looking at the needle and not the band. A big swing with a wide band means very little.
Words to know
- Lift
- How much better or worse the new version did, in percent.
- Band
- Where the real difference most likely is. Narrower means surer.
- Planned sample
- How many visitors each version needs before Needlr decides.
- Points
- The gap between two rates: 4.8% vs 4.0% is 0.8 points.
What the video says
Every impact check gets one page, your scoreboard for a single question: did this change move the needle on your KPI? Here is how to read it in a minute.
The needle is the measured difference between B and A. Right of zero, B is ahead. Left, it's behind. The dashed marks show the smallest lift the impact check was sized to detect.
The shaded band is where the real difference most likely sits. Early on it is wide. As visitors arrive it narrows, and a verdict needs it fully on one side of zero.
Under the needle sit the hard numbers: each version's conversion rate, the visitors behind it, and progress toward the planned sample, with an estimate of the days left.
The timeline shows when the verdict is due, and you can add that date to your calendar. The chart is the road to the verdict: the band tightening as data comes in.
While it runs, the live panel shows visitors arriving every few seconds. Watch it to confirm traffic is flowing, but decide only on the verdict, never on a midway swing.