Developer guide: split test in code

For the developer: how Needlr's snippet reaches the page, what impact.feature() answers, and how to ship the winner safely.

This page is for the developer. Your product manager picked Split test (developer) in Needlr. You wrap the new version in one if, and Needlr decides which visitors get it. The product manager's version of this page is Split test in code.

Think of a feature flag that Needlr flips

You know feature flags: an if around new code. Here Needlr holds the flag. For each visitor it flips a coin once, remembers the result, and counts who saw what.

You never pick who gets what. That's what makes the result fair: the coin decided, so nothing else can explain the difference.

Before you start

• Needlr runs on the site through Google Tag Manager. Someone on the team connected the site's Tag Manager in Needlr and published (how). You don't add a script tag or install a package.
• Every page loads that Tag Manager container. If a page doesn't, there's no snippet there, and your code shows the current version.
• The site's address is in the project (your website). Needlr's server only answers pages on that address. On another domain, like staging, you may still see a version, but nothing is counted.
• There's an impact check of type Split test (developer), at least as a draft. It gives you the key.

How the snippet gets on the page

Needlr writes its own tags into the site's Tag Manager and publishes them, once someone on the team approves (publishing). A tag whose name starts with Needlr - Bootstrap fires first, on Tag Manager's Consent Initialization trigger. It loads the snippet, and the snippet sets window.impact.

The same tag carries the list of running impact checks. So most visitors are decided right in the browser, with the same formula Needlr's server uses. Nobody waits for a round trip.

Counting still goes through Needlr's server, which answers a moment later. An exposure decided before that answer is kept and sent when it arrives, and so are KPI hits on that page. So a new visitor's first page counts, even when your code asks before the server has answered.

Set it up

1. 1In Needlr, under + New impact check, the product manager picks Split test (developer). The box below it shows the code, with this impact check's key in it, like new_checkout. The key comes from the name: lowercase, with underscores.
2. 2Copy the key exactly as shown. You'll also find it on the check page: open What version B gets.
3. 3Put the new version in the if and today's version in the else. Ask through the waiting helper below, so the call is safe before the snippet arrives.
4. 4Deploy. The line can go live before the impact check starts: until then it answers false for everyone, so nobody sees the new version by accident.
5. 5The product manager presses Start measuring (or Start on a saved draft). That publishes to Tag Manager, and from then on your line gets real answers.

The calls you need

• impact.feature(key) returns a Promise that resolves to true or false. True: show the new version (B). False: show the current version (A). It never rejects. It also records that this visitor saw the impact check (the exposure).
• impact.featureSync(key) gives the same answer right away, without waiting. If Needlr hasn't decided yet, it answers false. Use it only when you know the snippet has already decided.
• KPIs. A click, a page or a form needs no code: Tag Manager counts it. For something only your code knows, like a finished payment, push an event: window.dataLayer.push({ event: "payment_done" }). In the KPI step, press Advanced: my site sends a dataLayer event and type the same Event name.
• Signed-in users. After login, send your own internal user ID: (window.__impactQ = window.__impactQ || []).push(["identify", "u_829"]). It works before and after the snippet loads. A visitor who already has a version keeps it. Never send an email address (privacy).
• Consent. Nothing to code if the site uses Google Consent Mode. The tag hands the visitor's analytics consent to the snippet, and follows it when it changes.

When the answer is false

• The impact check isn't running: a draft, stopped, or not published yet.
• The key doesn't match a running impact check: a typo, or a key from an older one.
• The visitor declined analytics consent. Nothing is stored or sent for them, and they aren't in the impact check.
• No decision within 0.8 seconds. That only happens when there's no saved or local answer and Needlr's server is slow.
• Needlr left the visitor out, for example traffic far above normal from one source (unusual traffic).

One exception: after the winner is shipped, the answer is true for every visitor, also for visitors who declined consent. Nobody is counted then.

Timing

• Most calls answer at once. A returning visitor's answer is saved in their browser, and refreshed on every page load. A new visitor is decided from the list the tag carries.
• At most 0.8 seconds once the snippet is there. No answer by then: false.
• The snippet arrives through Tag Manager, a moment after your page starts. That wait is yours to cap (next section).
• A later call can answer differently. If the first call ran out of time and the answer came after, later calls get the real version. Decide once per screen and keep the answer for that screen.
• Right after an impact check starts, returning visitors get their version from their first page too: the list in the tag is checked against their saved answer. A stopped impact check is dropped the same way. If it was the last one running on the site, it's dropped once Needlr's server answers, a moment later.

Before the snippet loads

// Waits for Needlr's snippet (it comes through Tag Manager), then asks.
// No snippet in time (blocked or slow): false, the current version.
export function needlrFeature(key, waitMs = 2000) {
  return new Promise((resolve) => {
    const started = Date.now();
    const ask = () => {
      if (window.impact) {
        window.impact.feature(key).then(resolve, () => resolve(false));
      } else if (Date.now() - started > waitMs) {
        resolve(false);
      } else {
        setTimeout(ask, 50);
      }
    };
    ask();
  });
}

Using TypeScript? Declare the global once: declare global { interface Window { impact?: { feature(key: string): Promise<boolean> } } }.

In React or another single-page app

function Checkout() {
  const [isNew, setIsNew] = useState(null); // null: not decided yet
  useEffect(() => {
    let live = true;
    needlrFeature("new_checkout").then((on) => { if (live) setIsNew(on); });
    return () => { live = false; };
  }, []);
  if (isNew === null) return <CheckoutSkeleton />;
  return isNew ? <NewCheckout /> : <CurrentCheckout />;
}

• Ask where the change is shown, not once when the app starts. Only visitors who reach that screen should be in the impact check.
• Keep the answer for the screen. Don't swap versions while someone is looking.
• Moving between pages is fine. Needlr records one exposure per page address, and in the results each visitor counts once.
• Strict mode's double effects are fine too: the second call on the same page isn't counted again.

Server-side rendering

The version is decided in the visitor's browser, so your server can't know it. Render a placeholder for the part that changes, and ask in the browser once the page is there (in React: inside useEffect). Don't call impact on the server: there's no window there.

No flicker

• Show a placeholder, not the current version. Swapping today's version for the new one makes the page jump, and people notice.
• Keep the wait short. The 2-second cap in the helper is a good default.
• Needlr's own no-flicker hiding is for visual-editor changes. In code, your placeholder does that job (how Needlr avoids flicker).

Testing

The Preview A and Preview B buttons on the check page are for visual-editor changes, not for code. So test the two branches like this:

1. 1Before the impact check starts, add a switch only your team uses, like a query parameter on staging, that skips Needlr and shows the new version. Test both versions. A forced view never calls Needlr, so it's never counted.
2. 2After it starts, open the live site in a private window. In the browser's network tab, find the request to /v1/evaluate. Its answer lists your key, with treatment (version B) or control (version A).
3. 3Then look for a request to /v1/events/batch with an event of kind exposure. It goes out within about 2 seconds, or when you leave the page. On a new visitor's first page it waits for the answer from /v1/evaluate and goes out after it. If the visitor leaves before that answer, it goes out on their next page.
4. 4Want the other version? Close the private window and open a new one. Each one is a new visitor, with a new coin flip.

QA checklist

• The key in the code is exactly the key Needlr shows.
• The call happens on the screen where the change appears.
• Both branches work, and the else branch is today's version, unchanged.
• With Tag Manager blocked (try an ad blocker), the page shows the current version within your time limit.
• With analytics consent declined, the page shows the current version.
• No flash of the current version before the new one.
• The KPI fires: the click, page or form it watches, or your dataLayer event.

When there's a verdict

Moving the winner into your code

1. 1Delete the if, the else and the old version, so the new version runs on its own. Deploy.
2. 2In Needlr, on the Shipped changes tab, press Ready for verification.
3. 3Needlr can't read code changes on the page by itself, so it shows Needs a manual check. Check the live site, then press I checked it works without Needlr.
4. 4Press Stop applying through Needlr.

Common mistakes

• Calling impact.feature before the snippet exists. It throws. Use the helper.
• Typing the key yourself. Copy it from Needlr. A new impact check with the same name gets a new key, like new_checkout_2.
• Asking once at app start, for every visitor. Then people who never see the change are counted, and the result gets weaker.
• Saving the answer yourself across visits. Needlr already keeps it sticky. Your own copy goes stale when the impact check stops or ships.
• Removing the line before shipping or stopping. Then nobody gets version B, and counting stops.
• Treating false as an error. False is a normal answer: show the current version.

Words to know

Key

The impact check's name in code, like new_checkout: lowercase letters, numbers and underscores.

Snippet

Needlr's script on the site. Tag Manager loads it, and it sets window.impact.

Assignment

Which version a visitor got. Random, and it stays the same for that visitor.

Exposure

The moment a visitor enters the impact check: when your code asks for them. Only KPI hits after it count.

Fallback

What your code shows when there's no answer. Always the current version.