How GroundedBI keeps its answers honest.
The planner can only use six read-only tools, every number in an answer must match a query result, and when a question is unclear it asks instead of guessing.
What happens when you ask
Every question takes the same path. The work itself takes around 10 milliseconds, because it is ordinary code and SQL.
Check the question
Plain text, 1 to 500 characters, with a request ID for the logs.
Look up the data
Which months, products, regions and categories exist.
Plan
Pick a tool and its arguments, or decide to ask or decline.
Validate
Arguments must match strict schemas: real months, known dimensions, limits.
Query
Fixed, parameterized SQL on a read-only database connection.
Write
The answer quotes numbers from the query results, never calculates them.
Verify
Every number in the answer must exactly match a query result.
Ask for a detailAn unknown name like "Europe", or a vague period like "last month", gets a clarifying question.
DeclineProfit, forecasts, unrelated requests and commands like "delete" are refused before any query runs.
WithholdIf any number fails verification, no answer is shown at all.
Five layers of protection
Each layer works on its own, so one layer's mistake doesn't become a wrong answer or a changed database.
-
Fixed tools with validated arguments
The planner can only name one of six tools. Unknown fields, wrong types and invalid months are rejected before anything runs.
-
Parameterized, read-only SQL
Queries are written in advance with placeholders for values, and the database connection is opened read-only, so nothing can change the data.
-
Limits on every question
At most 4 tool calls, 6 planning steps and 10 seconds. Each query also has its own time budget and is stopped if it runs over.
-
A grounding check on every answer
Amounts and percentages must exactly match a value from that question's query results. No rounding, no "close enough".
-
Ask instead of guess
When the planner isn't sure what's being asked, it asks a question rather than picking an answer that looks plausible.
How well it works
Questions were tested against expected tools, arguments and numbers. Held-out questions were written separately and committed to git before their first run, so they couldn't be tuned to the results.
| Stage | Questions | Answered correctly | Confident wrong answers |
|---|---|---|---|
| First version | 10 held-out, unseen | 3 of 10 | 2 |
| After generalizing the rules | 10 new held-out, unseen | 7 of 10 | 1 |
| After a second round of fixes | 71, all seen before | 71 of 71 | 0 |
The honest figure is 7 of 10: the last time the planner met questions it had never seen. The 71 of 71 includes questions that were studied while fixing failures, so it shows the fixes work, not how well it handles new wording.
A confident wrong answer is the dangerous kind: a verified, real number that answers a different question. A clarifying question or a refusal is a safe failure; these are not.
Limitations
- The planner is rule-based. It understands the phrasings it has rules for; unusual wording gets a clarifying question instead of an answer.
- The grounding check verifies numbers, not words. Words like "rose" and "fell" are chosen in code from the sign of the change, and that rule is tested separately.
- Amounts are shown in whole dollars, so rows may not add up exactly to a total. The exact cents are kept in the data.
- The data is synthetic: an invented electronics retailer from January to June 2026, with patterns planted on purpose.
- It hasn't been run with a large language model. The planner sits behind an interface that one could plug into.
Built with
- Python
- FastAPI
- Pydantic
- SQLite
- pytest
- HTML, CSS and JavaScript, no frameworks