Why a slow-growing pilot is vital for chatbot success

by Chris McGrath | Oct 12, 2017

When you’re developing an enterprise chatbot, the first time it understands you and responds correctly is like magic! And when it answers you perfectly dozens of times in a row, there’s a tendency to think that you’re ready to launch. But don’t! Launching a chatbot is not like launching a website or intranet.

On an intranet project, a small team can traverse the navigation, click all the links, try a few different browsers and be reasonably sure that it’s ready for launch.

On a chatbot project, it’s different. A small team can’t possibly anticipate all the questions an employee could potentially ask the bot, or all the different ways an employee could express themselves.

To ensure success with your chatbot, you need a slow-growing pilot. It should look something like this:

Initially, a small team of stakeholders and subject-matter experts establishes an initial body of questions and answers. Then they roll out the chatbot to a friendly pilot group of 10 or 20 people, and they watch the transcripts coming in from the chatbot conversations. They’ll see dozens of queries that weren’t anticipated or that the bot is misunderstanding. They’ll teach the bot to properly handle those queries, and then grow the pilot to another 10 to 20 people. They’ll watch again as the transcripts come in, teach the bot more, and keep on growing and growing.

As the pilot grows, the percentage of successful responses climbs higher and higher. Once the success rate is in the 90-95% range, you’re ready to launch the chatbot to the entire organization. But you need to reach that point by growing the pilot gradually. Training the chatbot will continue after launch, of course; but it’s the pilot period that gets the chatbot to a reasonable level of performance.

Download the 40-page workbook, “How to Buy an Employee Chatbot in 10 Simple Steps“.