How to tell if your website support bot is working

You added a chat to your website a few weeks ago. Is it earning its place? Most small business owners cannot say, because the tool's home screen shows a message count and little else. A message count tells you people typed. It does not tell you whether they got what they came for. This guide gives you four things to look at instead, a short weekly check, and what to change when the signs are poor.

Why the message count misleads

A busy chat can be a failing chat. Picture a visitor who asks about your Saturday hours, gets a vague reply, rephrases twice and leaves. That is three messages and no result. Now picture a visitor who asks once, gets the hours and books. One message, one customer. If you only watch the total, the first visitor looks three times as good as the second.

So skip the total. Look at what happened inside the conversations.

Four things worth looking at

Questions it could not answer

This is the most useful list you have. Each entry is a real customer telling you, in their own words, what they could not find. Most tools keep some record of these. In Botsmith, questions it could not answer are collected for you automatically.

A short list is not always good news. If the bot never admits a gap, check that it is not guessing. Ask it something you know is not on your site and see what it says.

People who left contact details

Count the visitors who gave a name, phone number or email. These are the conversations that can become paid work. With Botsmith, when a visitor leaves a name, phone or email, you get an email within seconds with the full transcript attached, so the count is already sitting in your inbox. Search for those emails and you have your number for the week.

Read what came just before the details were given. That shows you which answers move people to get in touch.

Real conversations

Take out your own tests, your staff's tests and the one-word messages that go nowhere. What is left is real use. If almost nothing is left, the problem is probably not the answers. The chat may be hard to spot, missing from the pages people land on, or opening with a greeting that gives no hint of what it can do.

Questions asked again and again

When the same question turns up every week, your customers are telling you what matters. Put that answer where nobody has to ask: on the home page, in the page heading, in the booking confirmation. A good chat should make itself slightly less necessary over time.

A ten-minute weekly check

Pick a fixed time. Monday morning with a coffee works. Then go through the same steps each week.

1. Open the unanswered questions. Read every one. Mark each as "we should answer this" or "not our business".
2. Fix the top few. Write the answer, or add it to the right page of your website. In Botsmith you can answer a collected question once and the bot knows it from the next message on.
3. Count the leads. Write the number down next to last week's. A notebook is fine.
4. Read a handful of full conversations. Not summaries. Whole conversations, start to finish. Look for a reply that was correct but unhelpful, or a visitor who had to ask twice.
5. Note one repeated question. Decide where on your site that answer should live.

That is the whole routine. Do not build a spreadsheet with a dozen columns. You will stop filling it in by week three.

What each sign is asking you to change

There are only two levers: what the bot knows, and how it behaves. Most problems need the first one.

• The bot says it does not know, and you do offer the thing. Add knowledge. Write the answer in plain words and include the phrases customers used. See what to put in a chatbot knowledge base.
• The bot gives a correct answer and stops. Change the instructions. Tell it to end price and booking answers with how to book.
• The bot answers something it should leave alone. Change the instructions. Add a rule about what to refuse and where to send the person. In Botsmith, rules you write apply to every reply immediately, so you can test straight away.
• Visitors ask for a person and nothing happens. Check that the handover works. Ask for a call back yourself and see whether the request reaches your inbox.
• Plenty of conversations, no contact details. Read where they end. Often the visitor got the answer and had no reason to go further, which is fine. Sometimes the bot never offered a next step.
• Hardly any conversations. Look at placement before you touch the answers.

For help wording a rule, read how to write instructions for your support bot. To check the bot refuses the right things, use the test in what a website support bot should answer.

After a month, ask one bigger question

Four weekly checks give you enough to judge. Line up your four lead counts and your four unanswered lists. Are the same gaps still appearing, or new ones? New gaps mean the old ones are fixed and customers are going deeper. The same gaps mean the fixes did not land, usually because the answer was written in your words and not the customer's.

Then think about your phone and inbox. Are you still answering the same routine questions by hand that the chat was meant to cover? If so, the chat has not reached those customers yet. If those questions have gone quiet and the enquiries that do arrive are more specific, it is working.

If you are still on a trial and weighing up whether to continue, the pricing page lists what each plan includes.