Docs Contradiction Toolkit$39 DIY · $199 Async Report
DIY Toolkit — $39 Async Report — $199

Guide · AI support bots & help-center docs

Chatbot giving outdated answers? Find where the old copy lives

Docs Contradiction Toolkit · Updated · Examples are illustrative, not customer stories.

You shipped a redesign. The billing page moved. You updated the help article the same afternoon. A week later a customer writes in, annoyed: your support bot told them to open a menu that doesn't exist anymore.

The article is right. The bot is wrong. That gap is what this guide is about.

An outdated bot answer almost never means the model "forgot" something. It means an old version of the fact still exists somewhere the bot can read, and that version won. Your job is to find that copy and retire it, then make sure it doesn't come back after the next release.

If your problem is two current pages that disagree with each other, start with why your AI chatbot gives wrong answers. This guide is about time: something was true, then it changed, and the bot didn't notice.

Three places an old answer can live

When the bot repeats something you already fixed, the old text is in at least one of these places:

  1. The live help center. The article you updated is correct, but another article, a translation, or an old troubleshooting page still shows the previous steps.
  2. The bot's own copy. Most docs bots don't read your live pages when they answer. They search an index built from copies of your content, taken when each source was last synced. Owlish, a bot vendor, puts it this way: "the article is up to date" and "the agent's answer is up to date" are "two different facts."
  3. Leftover sources. A PDF you uploaded during setup, a pasted snippet, a crawled URL that now redirects. Removing a page from your site doesn't always remove it from the bot. HelpCenter.io's checklist makes the same point: a redirect helps people following old links, "but it is not proof that an old copy has disappeared from an AI index."

Each place needs a different fix, so find out which one you're dealing with before you change anything.

Diagnose an outdated answer

Step 1: Pin down the old fact

Write down exactly what the bot said that's out of date: the old menu path, the old limit, the old plan name. You'll search for this exact string, so copy it precisely.

Step 2: Search the live help center for it

Search for the old string, not the topic. If the bot said "Settings → Billing," search "Settings → Billing," "Settings > Billing," and "Billing tab." Include:

  • Articles in every language you publish
  • Troubleshooting and "common errors" pages
  • Release notes and changelog posts
  • Pages you meant to archive but only unlisted

If you find the old text here, it's a live-content problem. Skip to "Fix it" below.

Step 3: Compare dates

If the live help center is clean, compare two dates:

  • When the correct article was last edited.
  • When the bot last synced or crawled that source.

Most bot dashboards show a last-synced or last-crawled time per source. If the sync is older than the edit, the bot is still reading the old copy. Mind which date your help desk gives you. Zendesk's Help Center API, for example, has both updated_at and edited_at, and documents edited_at as "the time the article was last edited in its displayed locale." That's the one that reflects a real content change.

Step 4: Check the bot's source list

Open the list of sources the bot uses and look for anything that could hold the old fact: uploaded files, manual snippets, extra website crawls, a second help center. Owlish calls this "orphaned content": a new document replaces an old one, but the old one never gets removed, so "two chunks answer the same question differently."

Step 5: Look for undated content

If an old page has no visible date and nothing in its metadata says it's old, a bot has no way to prefer the newer page. Unrag's guide to retrieval makes the same point: "The model has no way to know which is current unless you tell it."

Worked example: the menu that moved

This example is made up to show the method.

Say your app used to put billing under Settings → Billing. The redesign moved it to Workspace → Plans & invoices. You updated the "How to download an invoice" article on release day.

A customer asks the bot, "Where do I find my invoices?" The bot replies: "Go to Settings → Billing and click Download."

Running the steps:

  • Old fact: "Settings → Billing."
  • Live search: the invoice article is correct. But "How to change your payment card" still says "Settings → Billing," and so does the Spanish version of the invoice article, which nobody re-translated.
  • Dates: the bot's help center sync ran after the release, so the index is current. That rules out a sync problem.
  • Source list: there's also an "Admin guide" PDF uploaded during setup. It describes the old menu on page 4.
  • Result: three old copies. Two live pages and one leftover file.

Updating the one article you remembered was never going to be enough. The old path lived in three places, and the bot only needed one of them.

A second example: the limit that changed

Also illustrative. Say your API docs used to list a rate limit, and you raised it in a new plan. The API reference page shows the new limit. But a "Getting started with the API" tutorial still states the old one in passing, in a sentence about "staying under the limit." The bot pulls the tutorial because it matches beginner phrasing better than the reference page does.

This is the sneaky kind of stale content. The number isn't in a policy page, where you'd look. It's a side remark inside a how-to. Searching for the old number, not the topic, is what finds it.

Fix it

If the old text is in the live help center

  • Update every page that states the old fact, including translations.
  • Where a page only mentions the fact in passing, replace the value with a link to the page that owns it ("see current API limits"), so the next change only needs one edit.
  • If a page is obsolete, unpublish it rather than just hiding it from navigation.

If the bot's copy is behind

  • Trigger a manual re-sync or re-crawl of the changed source.
  • Check the last-synced time afterward and confirm it moved.
  • If your platform supports scheduled syncing, set a schedule that matches how often that content changes.

If there's a leftover source

  • Remove old PDFs, snippets, and extra crawls from the bot's source list.
  • If you upload a replacement file, delete the old one. Don't assume the new upload replaces it.
  • Re-sync after removing.

Then re-test

Ask the original question again, plus a few rephrasings: "where are my invoices," "download receipt," "billing history." Check which source the bot cites each time. If any answer still shows the old fact, there's another copy. Go back to Step 2.

Stop it from happening after every release

Stale answers tend to follow product changes. A few habits keep them from reaching customers:

  • Add a docs step to the release checklist. "Search the help center for the old label or value, update every hit, then re-sync the bot."
  • Search for the old string, not the feature name. Old values hide in side remarks and tutorials.
  • Keep the bot's source list short and known. Every extra upload is one more place an old fact can survive.
  • Show dates on pages. A visible "last updated" line helps readers, and gives retrieval systems that use dates something real to work with.
  • Re-translate or flag translations. Zendesk, for example, has an outdated_locales field listing locales where an article was marked outdated. Whatever tool you use, translations need their own check.

For a full pre-launch pass across all of this, see the help center AI readiness audit.

FAQ

Why does my chatbot still give the old answer after I updated the article?

Most docs bots answer from an indexed copy of your content, not the live page. Until the bot re-syncs, it keeps reading the old version. If it has already re-synced, another copy of the old text still exists: another article, a translation, or an uploaded file.

Does uploading a new file replace the old one?

Don't assume so. Check your platform's source list after uploading and remove the old file yourself if both are listed.

How often should the bot re-sync?

Match it to how often the content changes. Content tied to releases, pricing, or policy should be re-synced whenever it changes. Stable content can sync less often. If your platform only offers manual syncing, make it part of the release checklist.

Will telling the bot to "prefer the newest source" fix it?

It can help when both versions have clear dates. It won't help if pages are undated, and it leaves the old copy live for readers and search engines. Remove or update the old copy first.

How do I find every page that mentions an old value?

Search your help center for the exact old string, including variations in punctuation and spelling. Then search the bot's source list and any uploaded files. Check translations separately.

A kit for the recurring part

Stale answers come back with every release, so it helps to have the search list and probe questions ready. The Docs Contradiction Toolkit packages them: a scorecard, an expanded checklist with inventory steps, worked examples including stale-vs-current policy, bot-probe prompts, and capture templates. It's a $39 one-time download you run on your own public docs. It isn't a bot or a sync tool, and it doesn't re-index anything for you. Everything in this guide works without it.

See what’s in the Docs Contradiction Toolkit

Sources