Where repetition hides
A document workflow can look like five small tasks: title, tags, correspondent, document type, and date. But each request may carry the same full document. The repeated context can be larger than the instructions that differ.
The dependency matters
In Paperless-GPT, later metadata tasks use the generated title. Combining everything into one request would change that behavior. Our opt-in path generates the title first, then requests the remaining four fields together in a structured response.
What the example showed
One fixed synthetic invoice comparison used the real application orchestration with simulated provider responses. Application calls fell from five to two, and prompt bytes fell from 4,986 to 2,347 while metadata matched. Prompt bytes describe the payload; they are not billed tokens.
What makes grouping trustworthy
Validate every required field before accepting the grouped response. Preserve the original route for excluded configurations and custom prompts. Then evaluate on representative documents with a live model before using the synthetic reduction as a production forecast.
About the evidence
Fixed synthetic workload. Tests and proof collected locally.
Project detail and source links ↗