Customer Sentiment Metrics Make Ash Cry
Part one in a series about KPIs / metrics in the modern Customer Experience world.
If we haven’t met yet, hi. I am Ash, and I fix broken CX functions for a living.
When I come into the various organisations with which I work, one of the things that I am commonly struck by is the near-universal relationship with KPIs (Key Performance Indicators, which I am using interchangeably in this article with “metrics”) that most folks have. Not just within whatever Customer-facing department I may be working with, but throughout the company as a whole.
Sidenote: I am telling you all of this because it is only fair that I be clear throughout this series where I am coming from. Biases and all that, yah?
Time and again when I am speaking with new teams or leaders at a company and being asked to turn around a set of metrics (because turning around metrics is literally always part of my initial remit), when I ask WHY these metrics are important at all or why it’s important that they be a certain number, the answer is . . . worrying.
Most often it’s in the realm of “because that’s what upper management told us to track” from team members. And “because it’s industry standard” from upper management. Occasionally it’s “because our investors / parent company want(s) to see these numbers”, which is comparatively refreshing, if frustrating in its own ways. I have only once or twice had someone give me a thought-out reason for the metrics they were tracking, and while I didn’t agree with them, at least this person had put in the time and effort.
I am well aware that this is a HOT hot take, but I am going to go out on a limb and say that 90%+ of all KPIs and metrics that exist within CX organisations globally are what I like to call “vanity metrics”.
The Vanity Metric
What is a vanity metric? Well, the name is pretty much on the tin.
It is a metric that is being tracked (or probably more accurately reported upon) solely to make the person, the team, the department, or the organisation as a whole look good. It’s being reported out of pure vanity.
Now I am drawing a line of difference between tracking and reporting on purpose. Because tracking things can allow us to make sure that we’re not slipping into some abyss (and more on that later in the series!), but if you are even able to brag about a metric that is and has continued to stay high then you are probably focusing on the wrong things, my friends.
As a perfect example of this, if you have a problem with your customer contacts going hours or even days without being answered, you will want to resolve that. That’s when you start tracking “Time to First Response” (TFR) and start implementing fixes to bring that down. As you make progress, report on it. Sing it to the world! Great job.
But! If you’ve never had an issue with TFR OR if your struggles with it are in the past, why is this even a discussion point? I’ll tell you: vanity.
The Three Flavours
There are, broadly, two to three types of metrics that a customer-facing team leader is going to have to deal with.
You have productivity metrics at the team level. You have productivity metrics at the individual level. And then you have customer sentiment metrics.
I’m going to spend the rest of this article on the sentiment ones, because the productivity ones are a whole separate conversation (and they are coming, no worries). But I want to name all three first, because I see people treating all three as interchangeable and they’re distinctly not. Not only do they measure entirely different things, they serve entirely different purposes, and what you can DO with each of them is completely different.
There are a ton of different thought pieces out there that like to pretend that there are a variety of niche sentiment scoring profiles that are “up and coming” or something, but in the CX world, the three scoring systems that you’re going to encounter everywhere are:
CSAT (customer satisfaction score)
NPS (net promoter score)
CES (customer effort score)
You are probably familiar with all three (though CES still has some work to do in getting attention. I’ll chat that one up first), and you probably track at least one of them. Sadly, there is also a better-than-average chance that you are either being held accountable for one of them as a leader, or are holding your team or team members accountable for one of them.
And if that’s the case: Stop it. Bad leader. No cookie.
Customer Effort Score: AKA, Ash’s “Favourite”
Let’s start with the one I have the least problem with, which is CES.
Customer effort score asks the customer to rate how difficult it was to get their answer or to get their problem solved. It can be on a scale of one to five or one to seven, depending on who built or is providing your survey software. But that’s it. That is the entirety of the question.
But here is why I find it useful: it’s something that we as CX professionals actually have a stake in. If effort scores are high, it means customers are working too hard to get help. Maybe the knowledge base is a maze or missing key articles. Maybe the ticket deflection system is directing people into the void. Maybe the answer to a genuinely simple question requires three transfers and a form. Those are things a CX leader can do something about. On paper. Knowledge bases can be improved. Deflection flows can be double-checked or redesigned. Common requests can be engineered to require significantly less . . . well . . . effort.
CES gives you a signal you can act on that, depending on your organisation, allows you to move the needle properly and take ownership.
Do not, however, confuse “useful signal” with “something to bonus your reps on.” It is not that either. I’ll get to why in a moment.
CSAT: The Astrology of Customer Metrics
Now. CSAT.
Customer satisfaction score. The metric that nearly every CX leader I know is responsible for, reports on, agonises over, and in many cases, has their team members’ variable compensation tied to (voluntarily or otherwise).
I will try to put this in a way that won’t shock the delicate sensibilities of too many people:
CSAT as a metric is hot garbage.
Sorry! Sorry. That clearly didn’t hit the right tone. Let me try again:
CSAT as a metric is an attempt to put numbers to a customer’s feeling and then apply it to people who had nothing to do with that feeling, thus rendering it almost entirely void of quantitative value. Also, it’s hot garbage.
I am, genuinely, a great big hippie. I have nothing against intuition, against the soft and unmeasurable, against things that resist quantification (ask me about astrology sometime. I have Opinions). But when we start getting into uncontrolled environments with questionably written surveys and the goal of capturing how someone feels without being able to even be sure they understand what they’re being asked, we aren’t rising to the level of discussing astrology. We’re stuck in the mud of Magic 8 Ball levels of reliability.
The tragedy of CSAT is that any particular representative could do absolutely everything right. The entire TEAM could do everything right. The customer could be getting fast, accurate, genuinely kind resolutions to genuinely complex problems that make me want to write poetry in praise of the answers.
But if that customer, I dunno:
Had a fight with their partner that morning
Got stuck in traffic on the way to work
Got their coffee order wrong (or spilled it!)
Is fundamentally the sort of person who finds the “completely satisfied” option on a five-point scale to be philosophically irresponsible
. . . then everyone is getting a low score. Not because anything went wrong. Because life is full of noise that has nothing to do with your team.
And it runs the other way too! A customer who has warm feelings about your brand, or just had a really good day, or has a natural tendency toward generosity, is going to give you a high score even when things were objectively mediocre. That’s me, BTW. I am the guy who is like “Service was slow, food was cold, prices were insulting. 5 out of 5 stars.” Too much time in the service industry, I suppose!
So you end up with a metric that captures, in roughly equal parts, the actual quality of the interaction, the customer’s emotional state before the interaction, the weather, and the general chaos of human existence. And all of that is BEFORE we even start talking about the bad news that reps and teams sometimes have to deliver to customers that is rarely their fault but that nonetheless upsets the customer.
So. Which one of those is your team responsible for?
Right.
And yet, and this is the part that makes me want to *gently* put my head through a wall some days, we attach it to individual reps as either a bonus item or a bonus disqualifier. I’ve seen representatives PIPed because of CSAT. We make people’s livelihoods dependent on a number that is substantially determined by factors that have nothing to do with them.
NPS: More like NYP. Not Your Problem, am I right?
Net promoter score asks a customer: “On a scale of zero to ten, how likely are you to recommend this company to a friend or colleague?”
Notice what that question is asking. It is asking about the COMPANY. The entire company. Whether the product delivers on its promise, whether the pricing feels fair, whether the last three features released were actually useful, whether the CEO said something embarrassing on a podcast last month, whether a competitor just launched something better. All of it.
And then, in many organisations I have worked with, this score lands in the CX leader’s lap.
As the voice of the customer, I genuinely understand it. CX sits closest to the customer, so there is a logic to saying “you should be gathering this data, quantifying it, making sure the business understands what customers are actually saying.” That makes complete sense, and I’m not challenging it.
But being responsible for gathering it and being responsible for the results are two entirely different things, and treating them as the same thing is one of the most consistent structural mistakes I see in how companies think about their customer teams.
Whether a customer would recommend a product to their network is a company-wide result. It reflects almost every decision that is made in product, sales, engineering, marketing, and in leadership. A CX department, team, and cadre of reps can be extraordinary (patient, knowledgeable, warm, effective) and if the product has a bug that has been open for six months, or the refund policy is restrictive, or the function of the product was entirely oversold . . . that NPS number is going to suffer.
So What Would You Say We Should Actually Do Here?
Weellllllll . . . I’m not gonna lie, that part is coming.
First I have to tell you about the insanity that is the other two kinds of metrics first!
But as a spoiler (because I DO love spoilers), the short answer is that I don’t fuckin’ know.
Not because I don’t KNOW know. But because the right answer is going to be specific to you. And you. And you.
But we’ll figure it out together, yah?