Nowadays, there is little question that deeply understanding and addressing issues of employee engagement is crucial to driving business profit and performance.   Unfortunately for companies tackling engagement issues, processing the requisite data poses a serious challenge. While surveys still remain as the most effective method of measuring employee engagement , the structure of the data collected poses a challenge, namely in that it is typically unstructured text.
Survey comments are a treasure trove of data to a people analytics team, providing the depth and color that leaders need to not only flag that a particular engagement measure is low, but more importantly understand why that measure is low. Unfortunately for these practitioners, these (often lengthy) comments are generated in extremely high volumes and are messy with grammar errors, spelling mistakes, and colloquialisms. Only a few companies have the resources to devote the tens (if not hundreds) of hours required to manually read and sort these comments into quantifiable buckets. For those lacking that luxury, turning this feedback into a utilizable dataset proves to be an insurmountable challenge. 
This need gap is where companies like Ultimate Perception enter. Perception, formerly known as Kanjoya, pairs an employee survey and performance feedback collection platform with a proprietary natural language processing (NLP) algorithm that makes the process of turning massive amounts of unstructured text into quantifiable insights almost instantaneous. Their machine learning technology focuses on two tasks:
- First, its models analyze each employee-submitted comment and assign a probability of how likely that comment is discussing one of over 70 specific themes (e.g., work-life balance, senior leadership, communication skills, etc.)
- Second, the models also determine the probability each of 100 potential sentiments is expressed in the writer’s tone (e.g., confusion, excitement, frustration, etc.). 
This coupling of two major fields of NLP – theme clustering and sentiment analysis – is where Perception’s machine learning technology shines, specifying which themes employees are unhappy about, and which specific percentages and subgroups of employees are unhappy. Not only does this process save time and money: it also reduces risk of an analyst introducing bias in the way they categorize and interpret comments. 
Long term, Perception is seeking to build out its NLP software to extend into the myriad of other potential applications across the HR and employee lifecycle (see Exhibit 1).  The company already applies its algorithms to analyzing performance reviews, a process they claim can reduce biases of language impacting promotions, especially when combined with feedback recipients’ demographic data like gender and ethnicity.  In fact, this combining of NLP with other datasets or even other algorithms holds great potential for Kanjoya, as the company could combine both quantitative and qualitative survey data to develop robust sentiment predictions, or even craft custom individually-tailored engagement action plans for leaders based on the analysis . The technology need not only be applied to volunteered comments; technically, sentiments and themes could be drawn from passively-collected employee-generated text as well (e.g., emails and messenger chats) .
Exhibit 1: Potential NLP applications in HR
With such potential, it’s no wonder Kanjoya was acquired by long-standing HR industry stalwart Ultimate Software, with similar attention devoted to its competitors by the likes of venture capitalists  and LinkedIn . So should companies go all-in on investing in an HR NLP vendor? Not quite: unlike other applications of NLP technology such as customer sentiment analysis or text data mining, employee-generated data provides its own unique challenges.
To conclude, I pose three potential challenges to consider:
- Organizational culture-specific vernacular: Consider cases in which a phrase specific to that company’s culture (e.g., Salesforce’s discussion of “Ohana”) is used pervasively in comments. How can Perception’s technology be developed in a scalable way across companies to tag those term clusters and sentiments accurately?
- Tops-down ontology vs. bottoms-up clustering: Related to the above, Perception’s theme clustering is only as accurate as the training data it’s based off of, i.e, historical text data sets and data from other companies. What happens if a new theme trend emerges that isn’t currently captured by the Perception out-of-the-box ontology? Can this issue be addressed given machine learning algorithms for bottoms-up clustering (i.e., self-generated custom clusters) has not caught up yet?
- Levels of accuracy: While Perception claims accuracy can top 95% for some companies, it acknowledges actual levels may be much lower.  For users of the product, though, how can they make an informed decision of how to use the product without knowing how “much lower” accuracy rates are? How can they check this accuracy without reverting to their old comment-coding methods? And even if accuracy is at 95%, how does a leader handle potentially explaining to the other 1 out of 20 employees that their comments may have been mistagged, and thus ignored?
 Naz Beheshti, “Our Approach to Employee Engagement is Not Working,” Forbes, Sep 30, 2018, https://www.forbes.com/sites/nazbeheshti/2018/09/30/our-approach-to-employee-engagement-is-not-working/#2416e7517274, accessed November 2018.
 Jacob Morgan, “Why the Millions We Spend on Employee Engagement Buy Us So Little,” Harvard Business Review, Mar 10, 2017, https://hbr.org/2017/03/why-the-millions-we-spend-on-employee-engagement-buy-us-so-little, accessed November 2018.
 Scott Judd, O’Rourke, and Grant, “Employee Surveys Are Still One of the Best Ways to Measure Engagement,” Harvard Business Review, Mar 14, 2018. https://hbr.org/2018/03/employee-surveys-are-still-one-of-the-best-ways-to-measure-engagement, accessed November 2018
 Luminoso, “Employee Feedback and Artificial Intelligence: A guide to using AI to understand employee engagement” (PDF file), downloaded from Luminoso website, https://luminoso.com/writable/files/White-Paper-Employee-Feedback-and-AI.pdf, accessed November 2018.
 Adam Rogers. “How Unified Employee-Feedback Tools are Revolutionizing HR”. Ultimate Software’s Blog, Feb 7, 2017. https://www.ultimatesoftware.com/blog/employee-feedback-perception/, accessed November 2018.
 Dan Ring. “Machine learning drives Kanjoya performance review software”. Tech Target, Apr 2016. https://searchhrsoftware.techtarget.com/feature/Machine-learning-drives-Kanjoya-performance-review-software, accessed November 2018.
 Raja Sengupta, “How Natural Language Processing can Revolutionize Human Resources”. AIHR Blog & Academy. https://www.analyticsinhr.com/blog/natural-language-processing-revolutionize-human-resources/, accessed November 2018.
 Cyrus Sanati, “How big data can take the pain out of performance reviews,” Fortune, Oct 9, 2015, http://fortune.com/2015/10/09/big-data-performance-review/, accessed November 2018.
 Dave Zielinski, “Artificial Intelligence and Employee Feedback”, Society for Human Resource Management, May 15 2017, https://www.shrm.org/resourcesandtools/hr-topics/technology/pages/-artificial-intelligence-and-employee-feedback.aspx, accessed November 2018.
 Frank Partnoy, “What Your Boss Could Learn by Reading the Whole Company’s Emails”. The Atlantic, Sep 2018, https://www.theatlantic.com/magazine/archive/2018/09/the-secrets-in-your-inbox/565745/, accessed November 2018.
 Michael Rochelle and Friedman, “LinkedIn Acquires Glint, Bolstering Its Position as an HCM Market-Maker”, Human Resources Today, Oct 15, 2018. http://www.humanresourcestoday.com/data/glint/survey/?open-article-id=9072138&article-title=linkedin-and-glint—potential-hcm-technology-powerhouse-in-the-making, accessed November 2018.
 Seth Grimes, “Where are the text analytics unicorns?” VentureBeat, May 3, 2015. https://venturebeat.com/2015/05/03/where-are-the-text-analytics-unicorns/, accessed November 2018.
 Sanati, “How big data can take the pain out of performance reviews”.