Many companies already have an AI chatbot, and many are quietly unhappy with it. It invents answers that are not in the documentation, takes five seconds to reply, and the monthly API bill keeps growing. The usual reaction is to switch models or rewrite the prompt again. In my experience the model is rarely the cause. The problem is almost always in retrieval, in the prompt contract, or in how the system calls the model.
Three symptoms, one system
Take a simple question: “What is your refund window?” A broken bot answers “Refunds are possible within 90 days, no questions asked” in 4.8 seconds. It sounds confident and appears nowhere in your policy. A fixed bot answers “Refunds are available within 60 days of purchase”, cites Refund-Policy.pdf, page 2, and replies in 1.4 seconds. Same model, same documents. The difference is engineering: what the model receives, what it is allowed to say, and how much work happens on every request.
Wrong answers are usually a retrieval problem
When a bot makes things up, the model usually did not receive the right passage, or received it buried among irrelevant ones. I check how documents are parsed and split into chunks, whether search combines keyword and semantic matching, whether a reranker puts the best passage first, and whether outdated documents are still in the index. Then the prompt makes grounding a rule: answer only from the provided sources, cite them, and say so or hand over to a person when the evidence is missing. Citations let users verify an answer and show you exactly which document caused a mistake.
Slow replies: where the seconds go
Latency is rarely one big problem; it is several small ones added together. Sequential calls that could run in parallel, a large model used for a simple classification step, thirty retrieved chunks where five would do, and no streaming, so the user stares at an empty bubble. I measure each step separately and fix the biggest one first: stream the response, run independent steps in parallel, cache what repeats, and route easy requests to a smaller, faster model.
Huge API bills: paying for tokens nobody needs
Cost follows the same pattern. Most bills are inflated by context that does not need to be there: full chat histories, oversized retrieval results, verbose system prompts, and the most expensive model used for every request. Prompt caching for the stable part of the prompt, trimmed or summarised history, smaller retrieval results and routing by difficulty often cut the bill substantially without hurting quality. Cost is logged per conversation and per feature, so savings are visible and regressions are caught.
How I fix it: audit, retrieval, prompts, cost, report
| Step | What I do | What you get |
|---|---|---|
| Audit | Collect real conversations, build an evaluation set, measure accuracy, latency and cost | A baseline to compare against |
| Retrieval | Fix parsing, chunking, hybrid search and reranking; add citations | Answers grounded in your documents |
| Prompts | Rewrite the system prompt as a behaviour contract with fallbacks and human handoff | Fewer invented answers and an honest “I don’t know” |
| Cost | Prompt caching, model routing, context limits, cost logging | A lower, predictable API bill |
| Report | Before-and-after numbers on the same test set | Proof of improvement and a reusable test suite |
Nothing changes before the audit. Without a baseline every fix is an opinion. With one, each change to the prompt, retrieval or model is checked against the same set of real questions in minutes, and you keep that test suite after the work is done.
Hire me to fix your chatbot
I am a Senior Full-Stack AI Engineer with 11+ years in software, a 100% Job Success score and more than $200K earned on Upwork. I work on the whole system, not just the prompt: data pipelines, backend, frontend, deployment and monitoring. And I stay after the fix. I prefer long-term retainers, because a chatbot needs ongoing care as your documents, products and models change.
If your chatbot gives wrong answers, replies slowly or costs too much, send me a few example conversations and a short description of your stack. I will tell you where the problem most likely is and what fixing it would take.
Give the next product someone who stays.
Ready to get started? Send me a message - I'm available to start now and to stay. Long-term retainers, not a two-week handoff. Let's discuss your project, requirements, and the best way to move forward.