Interactive tool
doc-rooter
How little of a document does a model need to read to know what it is?
No document yet
How much Jev reads · choose a row to see it
| Jev reads | Doc type | Section accuracy | Doc type tokens | Section tokens | Cost |
|---|
Choose a document
Pick a sample, or choose a file of your own.
Jev's answer · how likely each document type is
What Jev saw
Each section, as Jev saw it, and its tag
- Doc type tokens
- –naming the document type
- Section tag tokens
- –tagging every section
- Total cost
- –at Jev's list price
- Section accuracy
- –labelled sections
Benchmark
The headings alone were enough.
| Jev reads | Doc type accuracy | Doc type cost | Section accuracy | Total cost |
|---|
| Model | Reads | Doc type accuracy | Cost per document | Per 1,000 documents |
|---|
Sections need more than the outline. The headings alone are enough to classify a whole document. To tag its sections, the model also needs a little of each section: adding the opening lines raised section accuracy from 62% to 75%.
A decision model or an LLM. doc-rooter works with both. Jev, a decision model, costs less per token and nothing for its answers, but takes one question at a time, so tagging a long document means many calls. An LLM generally costs more per token but answers many sections in one call (a batch), so it needs fewer calls.
About the numbers. 113 public documents of 8 types. The sections' tags to score against were labelled by Claude, for evaluation only.
How it works
Outline first. doc-rooter turns a PDF or Markdown file into its outline of headings and sections. It then sends the model only as much as you choose: the headings, then key terms, then each section's opening lines, then the full text.
Type, then tags. The model names the document's type, then tags each section with a standard label, such as balance sheet or auditor's report, so the same sections can be found and checked across documents.
Why
Most tools send a model the whole document. A 200-page filing is about 100,000 tokens, paid for before the model even starts on the question. doc-rooter reads the document's outline first and sends only what the question needs.
To name a document's type, the headings were enough: Jev's doc type accuracy was 98%, for about a seventh of what the full text costs. To tag each section (balance sheet, risk factors, auditor's report), a few opening lines from the section help.
The section tags also make data extraction more accurate and cheaper. To pull figures from a balance sheet, an extraction tool sends the model just the section tagged balance sheet, not the whole document, and doesn't have to search the document again for every field. Extraction can be set up once per section type, which keeps prompts small and every call cheap.
doc-rooter is the next evolution of my original document classifier, made into a package. It tags every section as well as the document, and works with a decision model like Jev or an LLM like Claude.