Sentiment Analysis is a future of manager-employee relationship…or is it?
Sentiment analysis and NLP is a hot topic for employee engagement, and sometimes potential flaws of the sentiment analysis are overlooked. What matters most for employee engagement is apparently “just open up our ears and listen.”
Sentiment Analysis and NLP are becoming the hot topic for Human Resources, ranging from the recruiting diversified talents, retention improvement, to more efficient hiring process. For example, this article (https://www.synergita.com/blog/sentiment-analysis-future-of-manager-employee-relationship/) claims that Sentiment Analysis could be the great measure to improve the employee engagement level across many organizations globally.
According to Gallup’s research, 85% of employees are not engaged in the workplace (Gallup; “State of the Global Workplace”). Another study suggests that, based on 2017 survey, 81% of employees consider leaving their jobs (reference: https://blog.smarp.com/employee-engagement-8-statistics-you-need-to-know). There is a study about the cost of disengagement of employees: approximately $450-500 billion each year is the total cost coming from the disengaged employees (source: The Conference Board). Not only just based on these figures but even intuitively, it is becoming bigger and bigger issue for human resource team how to build more close-knit team in the organization. But, how do we measure the engagement level of each employee? For both employees and HR team, it is a huge burden to conduct an employee survey, and gather and process the information. Furthermore, there is no guarantee that employees disclose their true motivation or engagement level in these surveys. Even under anonymous survey, employees could still get discouraged to provide the genuine feedback to HR and management (after all, why would I care about employee survey if I am disengaged and plan to leave the organization?)
Synergita, the author of the article, argues that AI-powered sentiment analysis could resolve these issues. For example, an AI-based sentiment analysis software can digest the text contents of continuous check-ins between managers and employees, performance review feedback, 360-degree feedback etc. The AI analyzes the text and grabs insights on whether the conversation was negative, neutral, or positive, and by how much level. The author adds that with the qualitative data, the organization can plan the training programs for both managers and employees, such as leadership training and skill training. The point for using the sentiment analysis is that i) the sentiment analysis encourages the employees and managers to give constructive feedback, and ii) the concerns are heard by the management. However, is it really the future of HR with no caveats?
For me, there are a couple of considerations when applying the sentiment analysis to employees’ daily check-ins or other day-to-day communication with the management.
- Validity of the Sentiment Analysis
If you simply apply the sentiment analysis to all the employees’ check-ins, it’ll give you the scores (let’s say, 1-100 ranging from negative to positive comments). But, it does not tell you “how meaningful the score XX is for employee A vs. employee B). If an employee A gives relatively optimistic comments, 40 could be a really terrible score, while for employee B, 40 might mean an excellent feedback considering his unique personality. The score provided by the sentiment analysis is not necessarily a good measure to be applied for each individual.
Further, sentiment analysis simply takes the average of the score. Employee C might provide 20 positive comments and 20 negative comments, and the score would get neutralized by taking the simple average of all the sentiment in a single conversation. Would this be the same as employee D’s 10 comments about very neutral things? Maybe a manager should listen to employee C’s negative comments much more carefully. However, it is difficult to identify this “employee C” by just watching the averaged sentiment score.
- Quality of Gathered Data
Another issue not fully addressed by the article is a privacy issue. HR department should carefully consider the trade-offs between the engagement transparency via sentiment analysis and the potential loss of genuine conversation between managers and employees. Obviously, we count on managers’ judgement when we have a closed 1on1 conversation with out managers. What if these conversations are ALL scripted and analyzed by the human resources? Could this be a genuine conversation?
So, the employee survey is dead?
Probably not. Below blog posted by Joshbersin (https://joshbersin.com/2020/08/is-the-employee-survey-dead-nope-its-becoming-smarter-by-the-minute/), gives some answers to above question. “Someday we’ll have enough AI to tell you what people at work. But right now I suggest it’s time to just open up our ears and listen.”
Interesting topic! Definitely agree with the considerations you brought up regarding how firms should ago about when tackling the issue of people’s sentiments. I think beyond validity of the analysis, one potential problem is the biased baked into the algorithm that generate the output in the analysis. As we’ve seen with many cases from the class, the algorithm is as biased as the ones who make them. Thus, without knowing which variables go into the algorithm and how bias-resistant these variables are, the results might actually become meaningless.
One more thought on data gathering part. As we’ve discussed throughout the course, can people opt-out of being a test subject for this? It’s just concerning that all of your conversations with the managers are potentially recorded. Without knowing exactly how the data would be used or even stored makes me even more skeptical about this approach.
Thank you for covering the topic, Kaz! I agree with the considerations you had for an implementation of Sentiment Analysis. I believe the aspect of a “buy-in” from the employees is not quite covered in the article and in essence, it is purely focused on the benefits for the organizations at large, while they are targeting employees.
One research-based argument to potentially strengthen the usage of this analytical tool is the Harvard experiment we discussed in class on “The Hawthorne Effect” that refers to a type of reactivity in which individuals modify an aspect of their behavior in response to their awareness of being observed.
Following the discussion of this effect, there was a suggestion from Professor Polzer that particularly stuck in my mind about people potentially being motivated to improve performance was because “they felt seen”. It would be an interesting assumption to explore. If there is a narrative that companies can create from this perspective, people would be more willing to cooperate and that, in of itself could positively reflect on employee motivation and engagement.
Thanks Aliya for really eye-opening comments, it is so true that we (at least some, probably including me,) are motivated by the notion that we are cared, considered, and appreciated by our supervisors and peers.
That said, I think the feedback loop is very important when a company wants to gather sentiment analysis results.
If employees like me get the feedback and a company/supervisor tries to address below issues proactively and concretely, I may be positive for this:
a) overall sentiment of the company – how the company plans to react?
b) my own sentiment, probably this time and historical results – how supervisor felt for the results, how we can improve?
But thanks again, cannot agree with you more.
This is a really interesting post! I was especially struck by your discussion of quality, and I think your points of caution are really well articulated. It makes me wonder whether this tool is better thought of as a complement for managers rather than a replacement or standalone tool. For example, this kind of sentiment analysis could be positioned as a starting point for a discussion between managers and employees, or a way for shy or new managers to begin to get to get a sense of their teams. I also wonder if this could be useful for surfacing toxic teams led by high performers (the Uber example comes to mind) and thus forcing accountability within organizations. It seems like this would come with its own set of training requirements, so I wonder how this would impact the positioning and value proposition of the tool.
Very interesting post !
I agree that it doesn’t reveal the quality of the judgement. But as other classmates are commenting, I wonder whether it at least provide incentive to employees to do better, simply giving them the feeling that they are “looked and measured”.
The only thing that I am worried about of the product is, that I feel people gives bad scores (as well as good scores) regardless of the quality of what you are trying to measure. This concern could only be taken away if there are enough samples, but if not, it might be a problem.