Debugging the Pulse Check Survey
When was the last time you participated in a pulse check survey?
Between bites of lunch, you dash off multiple-choice answers to “Does your work feel meaningful?” and “How heavy was your workload this week?”
Possibly the answers are shown at the next team standup. Possibly they disappear into the void of HR, never to be seen again.
Initial completion rates are over 90%, a number that was proudly announced by leadership. Eventually completion drops, leaving only the voices of the most enthusiastic or disillusioned employees.
No ideas have been incorporated by management. You begin to recognize the same voices each week, making the same comments. You and your teammate make side bets: “Will Fred complain about the coffee machine again?”
There is a $10 gift card given to a randomly drawn participant. You have never won the $10 gift card.
——————————————–
The question at hand: Do pulse check surveys have a place in the modern workplace?
It is easy to understand why Brennan McEachran felt moved to write an article called Startups: Pulse Surveys are dumb. Yet I will outline several important roles I believe pulse check surveys can play. Please share your thoughts with me in the comments!
Fix #1: Use for confirmation, not information.
The results of a pulse-check survey should feel obvious to a strong and highly engaged leadership team. In fact, a workplace with strong norms around two-way feedback and psychological safety might never experience a single surprise on their survey. This makes it useful for confirmation, but not for gathering new information. More direct channels of employee feedback will always outperform pulse check surveys because they are less prone to time delay and interpretation error.
In light of this, pulse check surveys can also function as an early warning system. When uncomfortable truths are revealed for the first time through the survey, it is a sign that employees lack the security or channels to raise issues directly.
Fix #2: Use to give, not to get.
Management often uses pulse check surveys to collect ideas from employees. Although employees may be the authority on their own state of mind, there is often much more that could be accomplished by subject matter experts. Communication issues? Hire a process management consultant. Structure issues? Have executives revisit the team org chart. Morale issues? HR surely knows some best practices.
While employees might proactively raise excellent ideas, expecting them to be the sole source of solutions to their complaints will lead to frustration and disengagement.
Fix #3: Use for individuals, not averages.
Consider the two organizations shown below. Which are you more concerned about? Your answer might be “it depends”. Is the unhappy team in Scenario 1 mission-critical? Is the senior leader in Scenario 2 going to eventually diminish the happiness of her entire org structure?
Source: Hacker News
While averages are tempting, they have no place in pulse check surveys. Distributions and individual dynamics are the proper application, allowing employees to see the broader organization sentiment and situate themselves within it.
Unlike Brennan McEachran, I’m not willing to throw out the concept of pulse check surveys entirely. But a responsible management team will insist on clarity on the goals and limitations of this format before deciding it is worthy of precious employee time.
Can’t help but chuckle at the opening descriptions – they are spot on! I love the idea of using these pulse checks sort of as a speedometer to gauge reactions to proactive interventions, such a refreshing take as I hated that “What could we do better to improve our situation?” question with fervor. A bit of concern with point #3 though, as individualized insights will require a certain level of breaking that anonymity veil. Depending on the culture set by leadership, the knowledge that the surveys can be traced back to a person or a unit can elicit less-than-truthful responses.
Really enjoyed this – I also agree that they can still be valuable and have a place, but that they can absolutely be a box checking exercise that doesn’t add any value if not managed actively! Inclined to agree with Dio that there needs to be a balance between open and transparent individual results, but also some form of providing anonymous feedback in case there isn’t a culture that inspires people to be open.
One thing we found helpful in pulse checks was to get rid of the standard 1-10 scales for questions like “how are you feeling”, which started to feel like a box check. We got 15 images off the internet for each pulse check (without an obvious best/worst) and asked people which one they felt like and why, which seemed to provoke more honest, in depth answers
My team used the image approach at our weekly standups! Thanks for reminding me of this. It was incredibly effective and lighthearted (e.g. for Halloween we could choose between 9 pictures of carved pumpkin faces).
I’m very conflicted about this issue. On one hand, I agree with the author that pulse check surveys are ineffective, inefficient, and usually misinforming ways of understanding the workplace environment; yet on the other hand I don’t think his solution of “empowering your managers to be amazing” is a realistic solution that can be implemented across every team and organization.
One problem I see is the format of said surveys. We need to develop a better way of collecting more precise employee sentiment data, without turning it into a chore or formality. A possible solution is employing data collection methods that do not require employee feedback but instead analyze employee interactions, movements, tone, and similar behavioral variables. Yet as we’ve covered in several cases, this would most certainly lead to privacy issues as well as potential misuse of said data.
While I have no clear solution, creating an environment of trust and transparency would certainly be a good start. Establishing strong cultural norms at the startup phase could then potentially eliminate the need for pulse check surveys, or replace them with more ad-hoc feedback sessions, even as the company grows.
This is a super interesting question! I personally have had varying experiences with pulse check surveys on various consulting projects. While the data can be more or less useful, I think what’s most relevant is the discussions they are driving. For me, the most effective pulse check surveys were the ones that were then discussed in person, with the entire team (leadership included) present. These conversations were often action-oriented, focusing on how we as individuals, and as a team could foster improvements. The least effective were those where no actions followed. If you’ve ever been on a team that has consistently negative pulse check results, but nothing ever really changes, you surely can recall how painful that feels. For me, then, it’s not a question of whether a pulse check survey is the right tool or not, but how we can better integrate the results of pulse check surveys into our team conversations, and actionable paths for improvement.
Great topic! now I am tempted to ask how should we do the pulse check surveys or which alternatives are out there. Maybe for a second post from @Laura??
I’m using a large pulse survey – the Census’ Household Pulse Survey – for a final project in Managing the Future of Work! (Sidebar: they started it about a year ago and are tracking people’s experience through the pandemic. There is a LOT of cool stuff in there if you want to learn more about it.)
Putting that aside, one sentence jumped out to me as a call to action: “No ideas have been incorporated by management.” It feels like an easy win would be to use pulse surveys to test ideas that are ready to be implemented but just need more results. Beyond getting validation from employees, it also draws a clear link: the survey DOES inform workplace decisions, so we should contribute to create change! I like the quick cycle time for these surveys compared to bigger engagement questionnaires, and I worry that without some quantitative tool, it will be hard to track what’s happening without some structure.
Also, I agree that averages throw out a lot of information – we build a robust tool and then compress all of the data into a few numbers that are easily skewed! One fix: we need to start talking and using dispersion stats more (histograms and standard deviations could be a good startpoint!). I read a book called “The End of Average” that dives into alternatives to central-tendency analysis that you might enjoy; I remember it focused on finding ways to capture the “jaggedness” of data.
We discussed Pulse surveys in Inclusive Leadership, so this was a great read. I think the question around “which team are you more concerned about” is a good one. We discussed looking for the standard deviation within a team versus the average NPS score in terms of which terms should cause the most concern, but overall agree with the premise that pulse surveys should not be thrown out, but re-imagined.