Adoption checks
Count how many active people use a new feature, against a target you set.
You launched a new feature to everyone. The question now isn't whether it beats the old version. It's whether people actually use it. An adoption impact check answers it.
An adoption check is the impact check that counts how many people use a new feature after you release it to everyone, and checks that against a target you set. For example: 20% of active people use it within 30 days.
Think of a new school library
A school opens a new library. The question isn't whether it's better than the old one. The question is: do students actually go?
If 1 in 5 students borrows a book in the first month, it's a hit. And you only count students. Teachers and parents visiting on open day don't tell you anything.
That's an adoption check. The students are your active people. Borrowing a book is your KPI.
When to use it
- A new feature is live for everyone, and you want to know if it's used.
- You want a clear yes or no against a goal, like 20% in 30 days.
- You want to know if a change made people do something more: use a split test or Before vs after.
- The release was more than 60 days ago. Adoption counts people one by one from raw events, which are kept 90 days.
- Your KPI comes from Google Analytics. It only gives daily totals, not people. Pick a KPI Needlr counts itself.
How to start one
- 1On your project page, press + New impact check. Under What did you ship?, give it a short name and pick Adoption.
- 2Set the Release date. It can be up to 60 days ago, or a day to come.
- 3Set the Target: the share of active people who should use it (from 1% to 95%), and in how many days (from 7 to 60). It starts at 20% within 30 days.
- 4Choose who counts as Active people (see below). At first it's everyone who visits.
- 5What counts as using it? Pick the KPI that means someone used the feature, like a click on Export.
- 6How should we test it? Choose how sure you want to be, then press Start measuring.
Who counts as active
Next to Active people, press Change. Then press + Add a condition (up to 3) and pick what people did: a KPI you already have, or Something new: build it on my site.
With more than one condition, choose:
- All or Any: people must do all of them, or at least one.
- When (with All): Any time during the check, or All in the same visit. A visit is one person on one day (UTC).
- Order (with All): Any order, or In this order (each one after the previous one).
For example: people who reached /app, then clicked “New project”, in the same visit.
Press Done when you're finished. Use everyone who visits the site goes back to counting everyone.
When every condition is a KPI you already have, Needlr shows about how many people a week match. Fewer than 30 a week is very few, and the verdict won't be reliable. Fewer conditions, or Any, match more people.
A person counts as active from the day they first did these things. Only uses of the feature from that day on count.
What you'll see
- The share so far, like 23% of 1,240 active people used the KPI, with a range it's likely in.
- A bar with your target on it.
- Active people, Used it (at least once), Came back (used it on 2+ days) and how many days are in.
- A curve of each day's share, with the target as a dashed line.
- A table, week by week: active people, who used it, the share, and who came back next week.
Needlr counts every hour. The first numbers appear after 100 active people. The verdict comes once, when the window ends.
What the result means
People use it. Keep investing. Look at the weeks: if people keep coming back, it's become a habit.
Too few people use it. Often it isn't unwanted, it's hard to find. Try a split test on how people discover it, with the same KPI.
The share is too close to the target to tell which side it's on. If the last weeks are still climbing, it's on its way.
Your KPI stopped firing. Check that it still works on your site, then publish again (more).
If the window ends with fewer than 100 active people, there's no verdict: there's too little to go on.
Common mistakes
- Counting everyone for a feature only some can use. A feature for paying users looks unused next to every blog reader. Set the active people.
- Too many conditions. Each one shrinks the group. Too few people, and the verdict won't be reliable.
- A KPI that fires for other things too. If the KPI also counts clicks on another button, the share looks better than it is (how to build one).
- Looking only at the first week. Many people try a new thing once. The weeks table shows who comes back.
Words to know
- Adoption
- How many people use a new feature.
- Active people
- The people the feature is for: everyone who visits, or people who did certain things.
- Target
- The share of active people you hope will use it, within a number of days.
- Came back
- Used the feature on two or more days.
What the video says
You launched a new feature to everyone. The question now isn't whether it beats the old version. It's whether people actually use it. An adoption impact check answers it.
Set a target: say 20% of active people use it within 30 days. Needlr counts who used it at least once, and gives one verdict when the window ends.
Who counts as active is up to you. Everyone who visits, or people who did up to three things: all of them or any, in the same visit, in a set order.
Why not just count visitors? Because a feature for paying users shouldn't be judged by every blog reader. The right crowd gives an honest share.
Watch the weeks, too. The impact check shows how many came back and used it on two or more days, and who returned the next week. Habits beat one-time clicks.
At the end you get one call: target reached, below target, or too close to tell. Below target? Often it's not missing, it's hidden. Test how people find it.