This article explains how to test whether NPS differences between brands or groups are statistically significant in DataTile. It covers two approaches: a T-test on a recoded mean and an Adjusted-Wald Z-test.
What is NPS
The Net Promoter Score (NPS) is derived from a single-response survey question, typically phrased as "How likely are you to recommend [Brand name] to a friend or colleague?"
Respondents answer on an 11-point scale ranging from 0 ("Not at all likely") to 10 ("Extremely likely"). They are then grouped into three categories:
-
Promoters - scores [9–10]
-
Passives [7–8]
-
Detractors [0–6]
The NPS is calculated as the percentage of Promoters minus the percentage of Detractors.
NPS = %Promoters -%Detractors
Why testing NPS can be tricky
It might seem that a simple Z-test could work on NPS, since it is based on shares of promoters and detractors. But several complications undermine this idea.
First, when comparing NPS for two brands, we have four shares, not two as in the classic two-proportion Z-test: promoters and detractors for each brand. Each has its own standard error and confidence interval. The variance of the difference between two NPS values depends on how these four shares combine and does not follow the usual formula for two independent proportions. Simulation work (Rocks, 2016) shows that methods that ignore this structure produce poorly calibrated confidence intervals for NPS.
Second, NPS is a difference between two linked shares (Promoters and Detractors) within each brand. Both come from the same 0–10 scale and the same sample. Each respondent belongs to exactly one of three groups — Promoters, Passives, or Detractors — and these three shares add up to 100%. Because of that, the shares are dependent: a change in one share affects at least one of the others. If we treat Promoters and Detractors as independent and use a standard Z-test for proportions, we underestimate the true variance of NPS. That leads to biased significance results: we may think a difference is significant when it is not, or vice versa.
Methods that account for the trinomial structure and the dependence between the three groups avoid these biases and give more reliable confidence intervals and significance tests (Rocks, 2016; Eskildsen & Kristensen, 2011).
NPS can be written as the difference between Promoters and Detractors, but using a standard Z-test on that difference is not recommended. Use methods designed for this structure instead.
How to test NPS significance in DataTile
DataTile offers two approaches.
Scenario 1: T-test on recoded mean
NPS is treated as the mean of a recoded scale, not as the difference between two proportions.
Each response on the 0–10 scale is recoded into numeric values:
-
−100 for Detractors [0–6],
-
0 for Passives [7–8],
-
+100 for Promoters [9–10].
The average of these recoded values gives the overall NPS score on the conventional [−100 : +100] scale. This recode allows to analyze NPS with a standard method for continuous data: T-test.
Recent research shows that this approach produces more accurate and reliable results than traditional proportion-based tests and remains statistically stable for groups of 30+ respondents (Rocks, 2016; Cazzaro et al., 2023).
Downsides: This approach requires extra data preparation: recoding the 0–10 scale into −100, 0, and +100 before analysis. That can mean a lot of coding or relying on data specialists. For analysts who only use pre-built tables, this step can be challenging or not available. In DataTile, it is straightforward: you can group variables and apply recoding in bulk using built-in tools, without any custom coding.
Scenario 2: Adjusted-Wald Z-test on dependent proportions
In DataTile, the adjusted-Wald Z-test is applied within a built-in tool for calculating NPS.
This test is designed specifically for comparing two NPS values. It accounts for the trinomial structure and dependence between Promoters, Passives, and Detractors, and provides well-calibrated confidence intervals and significance tests (Rocks, 2016).
Principle: The method treats NPS as the difference between two linked shares (Promoters and Detractors) and uses a calculation designed for that structure. To make the results more reliable, it applies a small adjustment: it adds the equivalent of a few extra respondents to the observed counts. This does two things: it pulls extreme NPS values slightly toward the middle (e.g. 100% Promoters or 0% Detractors), so that variance estimates are more realistic; and it avoids cases where a group has almost no respondents, which would make standard calculations unreliable.
From the adjusted counts, the method computes adjusted proportions and NPS per brand, then derives the standard error, Z-score, p-value, and confidence interval for the difference between two NPS values.
Why it’s convenient: The Adjusted-Wald test works directly with the counts of Promoters, Passives, and Detractors. No recoding is needed — you can run the test as soon as you have these counts per group. This suits workflows where NPS is already in shares or proportions, or where you prefer not to create recoded variables.
Which scenario to use?
For comparing groups (shares of Promoters, Passives, Detractors): use the traditional approach and compare each segment across brands with a Z-test for proportions — Promoters with Promoters, Passives with Passives, Detractors with Detractors.
For comparing the NPS index itself (e.g. Brand A vs Brand B): you can use either Scenario 1 (T-test on recoded mean) or Scenario 2 (Adjusted-Wald Z-test on dependent proportions). Both are valid and give similar results.
How to choose between Scenario 1 and Scenario 2:
Both methods give similar significance results. Choose based on your data type and table configuration — whether NPS is set up as means (recoded scale) or as shares (proportions) — and use the scenario that matches how your data and layout are structured.
|
|
T-test on recoded mean |
Adjusted-Wald Z-test |
|---|---|---|
|
Pros |
Familiar method (T-test is widely taught and used for comparing means) |
No recoding; works directly with segment shares (Promoters, Passives, Detractors) |
|
Cons |
Requires recoding and database-level setup |
Less familiar (Adjusted-Wald is NPS-specific and less common in general stats training) |
|
Best when |
NPS is set up as means (recoded scale) |
NPS is set up as shares (Promoters, Passives, Detractors) and you want to avoid recoding |
Best practice tips
-
Use samples of 30+ respondents.
-
Look at the structure of the index, not only the overall NPS. The same NPS can come from different proportions of Promoters, Passives, and Detractors, so it can tell a different story.
-
When comparing segment shares across brands, compare within the same segment only: Promoters with Promoters, Detractors with Detractors, Passives with Passives. Use a Z-test for proportions for these comparisons.
-
Avoid using a standard Z-test on the difference between Promoters and Detractors (i.e. on NPS itself), or comparing Promoters with Passives/Detractors as if they were independent groups.
-
For NPS significance testing, use either Scenario 1 (T-test on recoded mean) or Scenario 2 (Adjusted-Wald Z-test on dependent proportions), depending on your analysis needs.
References
-
Cazzaro, M., Chiodini, P., & Capecchi, S. (2023). Statistical validation of critical aspects of the Net Promoter Score. The TQM Journal, 35(7), 1–16.
-
Eskildsen, J. K., & Kristensen, K. (2011). The accuracy of the Net Promoter Score. IEEE ICQR2MSE.
-
Rocks, B. (2016). Interval estimation for the Net Promoter Score. The American Statistician, 70(4), 365–372.