How We Automated Our NPS Program in Five Steps
Scale your customer insights with a fully automated NPS survey process to rival programs at tech titans like Facebook or Google
Automated insight-gathering programs were once a super power that tech titans like Facebook and Google leveraged en masse to grow their businesses.
When I worked at Instagram, for example, we had a fully automated customer survey program that hummed along like a well-oiled engine. NPS surveys were sent daily, responses collected, data weighted for representativeness and results displayed on a dashboard. Amazing!
This ensured business leaders always had the latest data and could act on customer insights quickly.
Unfortunately, automated analytics capabilities like this had been out-of-reach for emerging businesses and startups that didn’t have either the technical expertise to implement, the data science specialists to analyze or the funds to outsource their NPS to expensive research firms.
Until now!
We recently built a fully-automated weekly NPS program at Spaceship, a fin-tech startup, and thought we’d share the process in case it helps other emerging businesses and startups trying to scale their customer insights with limited resources.
Our NPS program at Spaceship now runs weekly, on auto-pilot, without any human involvement at all.
Results are collected, weighted and visualized on a dashboard for us to track.
The only time required is the initial setup. The entire process took two weeks to setup — so it’s a small investment for the valuable return of regular customer feedback.
You’ll see I’ve listed the tools we use so you can also understand a little more how we set-up our data flow for this project.
Insert caveat here — this is certainly not the only way you can do this, and there may be better ways for your business depending on the tools you have in-place. My goal in this article is to showcase one way an NPS program can be automated, not necessarily the best way.
Ok, let’s jump in. Here’s how we automated our NPS program in five steps:
Step 1 — Pull and Store the Weekly Sample of Customers
First step in the process — a BigQuery SQL script automatically pulls a sample* of customers to be surveyed.
The script runs every Monday morning and pulls a new weekly sample of customers that will eventually receive our NPS survey email.
I use the “scheduled query” function in BigQuery to automate this process.
The sample list is stored in a table of sampled users. This table has two purposes — (1)keep a list of users that will be emailed, and (2) track of the ongoing list of sampled customers and the date they were selected for a survey.
When pulling each week’s sample, the scheduled query script checks against this table and excludes anyone who has been sampled for a customer survey in the last six months.
This is important to make sure we aren’t spamming our customers with too many surveys.
*NB: You’ll need to calculate the most appropriate sample size for your business and error appetite. I’d recommend a research or data scientist runs this step for you. Reach out to me if you need help.
Step 2 — Move the Sample to the Email Tool
Next, we use Segment to move our sample list to our email tool Klaviyo, right after the BigQuery sample script runs.
There are lots of ways to automate this step —e.g. Segment, Zapier, CSV auto-transfers — and how you do this step will depend on your specific business environment.
We chose to use the “Personas” function within Segment to perform this function for us because we already use this with some other functions. But there are lots of other ways to do this depending on what tools your business has installed.
Personas is essentially a way to write a single SQL script that Segments runs on a regular basis and looks for new rows of data. In our case, it’s looking for a new set of customers in the sample table from step 1.
When new customers are found (every Monday, after my scheduled query runs in step 1), the new customer details are sent to our email tool, Klaviyo, as a new email list.
Step 3 — Survey Setup and Email Send
Klaviyo then sends out the NPS survey email to the new weekly sample of customers from step 2, every Monday afternoon.
The NPS survey email is a standardized email with an introduction and link to an online NPS survey hosted by Typeform. We use the same email text and survey link every week.
Typeform is our survey tool of choice for two reasons: (1) It has a beautiful interface for our customers, and (2) It has an extensive integration library, linking to a number of different platforms and apps. The integration library means we can do a lot of clever, automated projects with a Typeform survey.
Customers click on the link in the email, take the survey in Typeform and responses are collected.
We then use a Zapier “zap” to send data directly from Typeform back into our BigQuery databases where NPS responses are stored in a table.
Bonus note: Zapier has been a huge help in helping me scale and automate a ton of analytical programs here at Spaceship. If you’re a startup or low on Engineering resource, invest a little time in a tool like Zapier — it can do wonders for your automation dreams. I’m not affiliated with Zapier in any way, just a fan.
Step 4 — Weighting
Once survey responses are in, I need to weight them to remove things like response bias that creep into survey research. Weighting* ensures your survey responses more closely represent the customer base that you’re trying to study.
This is probably the most statistical bit in the whole process and requires someone with a data science background to set up.
The tricky bit here is that you can’t simply assign a static set of weights and call it a day.
Customer demographics and profiles change over time. They change with new products and strategies and different marketing approaches. So your weighting also needs to change.
The way I solve this is to run a weekly weighting query at the same time I pull the sample. My weights are calculated on variables like age and account balance.
As the responses land in the BigQuery table from Typeform, weights are automatically joined to each customer’s responses. Then it’s a case of tallying up customers in these groupings, comparing it to my customer base and weighting* accordingly to adjust for the difference.
The weights are applied to the raw data and saved in a new table to create a dataset of weighted NPS response data. “Scheduled queries” in BigQuery can easily handle this step and save the weighted table automatically.
*Weighting can get very tricky for some businesses, while it can be easy for others. Weighting is very much dependent on your business and the research strategy so I recommend it is setup by a data scientist. Reach out to me if you need help here.
Step 5 — Visualize
Now that I have weighted data for each week of NPS, I need to visualize it in a dashboard for everyone to consume.
I use the magic of a “saved view” in BigQuery to run calculations and produce the weekly, weighted NPS scores from my BigQuery tables. Using “saved view” function ensures latest data is always calculated.
The “saved view” with the NPS scores is linked directly to DataStudio, our visualization and dashboard tool. Linking the two means that NPS data is automatically charted and refreshed each week with any work from me.
One bonus tidbit here — NPS programs can sometimes launch to a lot of fanfare only to be taken for granted after a time, despite the magic found in each NPS survey.
I get around any potential NPS “blindness” with a few prompts — I’ve set up alerts when an NPS score declines by a significant margin, I publish raw customer comments in our slack channels to highlight a trend I’m seeing and I advocate for the NPS score to be a company-wide KPI (when it is appropriate and a proven key driver of business outcomes).
And voila, we now have a fully automated NPS program. We started with a sample pull, moved the sample to the email tool, sent the survey, weighted the data to be representative and visualized the data.
I’d recommend getting some input by a friendly data scientist on steps one and four. If your business doesn’t have access to internal data science resources, you could reach to a freelancer as a quick and cost-effective option.
Each piece here is a one-time setup with some light check-ins on an infrequent basis. Once the process is setup, it should largely take care of itself (touch wood).
You’ll note there is some upfront investment to all of this like any good automated process. It took us takes us two weeks to build this out from end-to-end. But the payoff has been more-than-worth the small, upfront investment.
Here at Spaceship, we took the time to build out our NPS functionality and now get granular, weekly, automated data for less than the cost of a few coffees.
Now, we track and (most importantly) can act on NPS customer feedback every week without lifting a finger. Which means we are closer to the customer as a result.
All this granular, regular, automated NPS feedback is now helping us produce better products and services. #DataFTW.
Tech titans have long tapped into the power of automated insight-gathering programs like this. Now, thanks to a host of cost-effective tools on the market and a library of integrations, emerging businesses and startups can also take advantage of the power of automation to level the playing field with big tech.
Good luck with your NPS automation journey and please reach out if you have any questions!
