We gave our analytics chatbot less data on purpose

The first version of an analytics chatbot is tempting to build: connect a model to the database, give it a query tool, and let it answer whatever the customer types. It can answer almost anything. It can also read the wrong project, calculate a metric differently from the dashboard, obey instructions hidden in stored text, or invent a number when a query returns nothing.

We built Ask CiteSite around the opposite constraint. The model receives less data than the application has, every useful number is calculated before the model sees it, and missing data is written into the prompt as an explicit limitation. The chatbot is less flexible as a result. Its answers are much easier to trust.

What an analytics chatbot should actually do

An analytics chatbot is a natural-language interface over measured data. Its useful job is retrieval and explanation: find the relevant fact, connect it to the question, and say what it means in language a person can act on.

Calculation is a poor job for the language model. If the dashboard says visibility is 42 and the chat says 44 because it counted a duplicate response differently, the product now has two versions of the truth. Both numbers can look plausible. The customer has no sensible way to decide which one to believe.

CiteSite already has deterministic functions for visibility scores, share of voice, competitor rankings, recent changes, and citation gaps. Ask uses those same functions. The model receives their results as facts and turns them into an answer. It does not average positions, count mentions, or decide which scans belong in a period.

The model is the last component in the Ask CiteSite pipeline

A question reaches four boundaries before it reaches the model. The order matters because every request costs money, and because a fast unauthorised answer is still a data leak.

The API first verifies the session and checks that the signed-in user owns the requested project. It then applies the Ask rate limit. Only after both checks pass does it read the latest project snapshot.

The server reads recent changes and citation statuses alongside that snapshot. Code turns the records into a compact brief containing the brand, scores, share of voice, competitors, prompts where the brand was mentioned without a citation, prompts where it never appeared, recent changes, and pages worth earning a mention on.

This is deliberately the same read path the dashboard uses. A question about what changed should not produce a different result from opening the Changes screen.

A bounded brief is safer than a database tool

Giving the model SQL access would make long-tail questions easier. It would also turn every question into a new, unreviewed query plan with a chance to cross a project boundary or produce an expensive scan. Row level security helps, but it does not make arbitrary model-generated queries predictable.

Ask receives plain text instead. The brief has fixed sections, explicit limits, and small caps on lists and response excerpts. Twelve captured answers at up to 400 characters each are enough to support questions about wording without paying to resend a project's entire history on every message.

The trade-off is real. Ask cannot answer a question whose evidence was left outside the brief, even when CiteSite stores that evidence elsewhere. We prefer a clear refusal over a confident answer assembled from partial context.

Stored AI answers are untrusted input

CiteSite stores excerpts from ChatGPT, Gemini, and Perplexity responses. Those excerpts are useful evidence, but they are also third-party text. A page quoted by an engine could contain a sentence telling another model to ignore its instructions, reveal its prompt, or answer with a particular claim.

We fence every excerpt as untrusted content before adding it to the brief. The system instruction identifies the block as quoted evidence and tells the model never to follow instructions inside it. This does not make prompt injection a solved problem. It gives the model a clear trust boundary and keeps instructions separate from the material being analysed.

The API response contains only the answer and a small source label. No provider key, access token, raw database row, or internal error detail is sent back to the browser.

How Search Console joins the brief without joining the model

The brief still names data it does not contain: revenue, conversions, analytics sessions, and dates or engines absent from the snapshot. The model is told to say when the evidence is missing and explain what measurement would answer the question.

A connected Google Search Console property follows the same rule. CiteSite reads clicks, impressions, click-through rate, average position, top pages, and queries from Google. The Search analytics side of the product renders that data directly, while Ask receives a bounded 28-day summary and an equal comparison period.

The adapter uses fixed date windows, deterministic totals, bounded query and page lists, and clear lag dates. It only calls Google after project ownership and rate limits have been checked, and it never puts the OAuth token into the brief. If the connection is missing, revoked, or temporarily unavailable, Ask keeps answering AI visibility questions and says plainly that Search Console evidence is absent.

What we rejected, and what it cost

We rejected direct database tools because their flexibility expands the security and correctness surface on every request. We also rejected sending raw records and asking the model to calculate from them. That approach was simpler to wire up, but arithmetic that changes with a prompt is not analytics.

The computed brief adds application code whenever a new dataset joins Ask. The Search Console adapter needs six bounded reads, weighted totals, comparison rules, and explicit handling for rows Google omits. This is slower than dropping a table schema into a tool description, and that friction is useful. It forces us to define what a metric means before a model starts talking about it.

How to build a chatbot over your own analytics

Start with one question whose answer already exists in your product. Call the same function that renders the chart, serialize its result into a small readable brief, and instruct the model to refuse questions outside it. Add ownership and rate-limit checks before any data read or model call.

Then test disagreement rather than eloquence. Ask the chat for a number shown on screen and compare the two. Ask about a period you do not have. Put hostile instructions inside a stored text field. Ask about another customer's project ID. A useful analytics chatbot must fail each of those tests in a boring, predictable way.

Once one bounded dataset works, add another. Keep the model at the end of the system, where it can explain authorised facts without deciding what counts as one.