5 best practices for deploying an employee or HR chatbot
1. Start with a narrow domain
Don’t try to build Siri. Unless you have a massive budget, and a team of hundreds of engineers, you’re not going to be able to keep up with the likes of Microsoft Cortana, Google Assistant or Siri. Even those teams don’t always get it right.
2. Have one killer feature
What answer does your chatbot have that your employees can’t live without? When your chatbot has a feature that keeps employees coming back again and again, they’re going to turn to it as a resource for other information as well.
At Bupa, now that Cyan (the Bupa chatbot) answers questions about day-to-day work, the number one question it gets asked is “What’s the guest wifi password?” Since the password changes frequently, many employees find that the easiest way to get the password for a guest is to ask Cyan.
3. Design failure carefully
You need to intentionally design what failure looks like, because a chatbot is not always going to be able to give a user the answer they are looking for. That happens for several reasons:
- The chatbot might need more examples to delineate related questions
- The chatbot might need more training on unexpected terms the employee is using
- The chatbot doesn’t contain any answers related to the employee’s question
Every response the chatbot gives will fall into one of the four categories on a tool known as the confusion matrix:
- True positive: the chatbot knows the right answer and delivers it
- False positive: the chatbot knows the right answer, but delivers an incorrect answer
- True negative: the chatbot doesn’t know the answer, and says it doesn’t know
- False negative: the chatbot knows the right answer, but says it doesn’t know
The Tangowork Chatbot Accelerator reduces false positive answers by using a confidence threshold: it only returns an answer if the chatbot is at least 40% sure that it has the correct answer. For answers where the Tangowork Chatbot Accelerator is 40 to 60% confident, it delivers the answer but then asks the user to confirm whether their question was correctly understood.
It’s good to consider each of these scenarios for your bot, and analyze what the chatbot will do in each case. Because failures (incorrect or unknown answers) are going to occur, designing for failure will result in the best possible outcome when it happens. Make sure that the chatbot is giving the best answer possible in each scenario, and steer the user back to supported tasks when needed.
If a user is asking the chatbot for information that it is not designed to provide, redirecting a user back to supported tasks helps the user know what the chatbot can do for them.
Sorry, I don’t understand. Ask me something else.
Sorry, I don’t understand. I know things about Acme Human Resources policies and benefits. Try "benefits", "payroll" or "time off".
4. Grow your pilot gradually
At the beginning, a small team of stakeholders and subject-matter experts brainstorms questions and answers that the chatbot will be fielding. Starting the pilot with 10 or 20 people allows the team to review the conversation transcripts and see questions that weren’t anticipated or that the bot is misunderstanding. They can then teach the bot to handle those questions, expand the pilot by another 10 or 20 people, and repeat the process.
As the pilot grows, the percentage of successful responses climbs higher and higher. If you launch at the very beginning, the number of unsuccessful responses will result in user frustration and failed adoption. Once the success rate is in the 90-95% range, the chatbot is ready to launch to the entire organization.
5. Review transcripts constantly
Reviewing chatbot conversation transcripts is especially important during the pilot period for your chatbot, but it continues to be an important part of general maintenance. Transcript review allows you to see when the chatbot doesn’t understand a message, or doesn’t have the answer a user is looking for. Fine-tuning the bot by adding more information on a topic, or training it to understand a user’s intention in a particular message allows for continuous improvement.
Apply these best practices to find success as you enhance your employees’ digital workplace with an informed, responsive chatbot.
Summer Chatbot Webinar Series
In July, August and September, sign up for 3 free webinars that explore 3 types of chatbots: Intranet Chatbots, Event Chatbots, and HR Chatbots.