Study claims that when a company’s tech support adopted ChatGPT its staff got better at their jobs and became less likely to quit

With the meteoric rise of generative AI tools like ChatGPT, Midjourney, and Dall-E, workers and companies are concerned, for different reasons, about how AI will affect them. While some organizations are banning them (opens in new tab), others have added AI tools to their workflows, and in at least one case, the addition has reportedly improved productivity and morale.

Researchers from MIT and Stanford (via NPR (opens in new tab)) report that when a Fortune 500 enterprise company (which chose to remain nameless in the study) slowly implemented a customized version of ChatGPT to assist its IT support workers in late 2020/early 2021, not only did performance improve, but so did employee retention. And with a job that reports an over 60% turnover rate every year, it was a massive boon for the company. 

The process involved the representative interacting with the customer through an online chat in one window with the chatbot open in another. When a customer required help with software issues, the representative could feed the chatbot the question and use its output to troubleshoot the problem.

“At a technical level, customer support is well-suited for current generative AI tools,” reads the report. “From an AI’s perspective, customer-agent conversations can be thought of as a series of pattern-matching problems in which one is looking for an optimal sequence of actions.”

The chatbot was trained on the company’s highly-rated support call conversations and could feed workers solutions and specific phrases that had worked in the past for more experienced employees. In theory, then, individuals with little to no IT experience could respond to customer inquiries as quickly and effectively as those with years of experience.

According to the report, there is “suggestive evidence that AI recommendations can make low-skilled workers communicate more like high-skilled workers.” New workers could handle more calls per hour while providing more satisfactory customer responses. The study also reported that there was a decrease in requests to “speak to a manager,” which is a dreaded phrase in any customer-facing environment.

Veteran and high-skill employees found the chatbot tool less helpful because the AI’s training data consisted of their five-star conversations—they already had the answers.

Erik Brynjolfsson, an economist at Stanford and one of the study’s researchers, told NPR that this system “took people with just two months of experience and had them performing at the level of people with six months of experience.”

In this case, AI seems to make a job easier rather than replace it. NPR suggests that even if they don’t replace workers, AI tools could “de-skill” some jobs, potentially lowering wages since prior knowledge may not be necessary. (Although one wonders whether, on a large scale, we won’t see lots of erroneous advice from support workers who don’t know whether an AI is giving them genuine answers.)

There at least need to be some experts around to train the bot, so maybe that’s good news? It may or may not console you that OpenAI published a study listing jobs that a chatbot could never replace (opens in new tab).