The AI read your PDF in the wrong order

The demo was going well until page seven. The assistant had answered four questions about the master services agreement correctly, and then someone asked about the liability cap, and it quoted a number with complete confidence: the wrong number, from the right page. On screen, side by side, you could see exactly what had happened. The PDF had two columns. The extraction had read straight across both, stitching the end of a sentence in column one to the middle of a sentence in column two, and the model had faithfully answered from the resulting word salad.

Nobody had ever looked at what the model was actually being handed. The pipeline went PDF in, chunks out, answers back, and every stage trusted the one before it. The model was fine. Its reading material was shredded.

This failure has a price tag with two lines on it. You pay per query, in tokens, for answers generated from scrambled context. Then you pay again, in reviewer hours, catching the answers that came back wrong, without ever knowing the wrongness was decided before the model saw a single token.

The default chunking strategy in most retrieval pipelines is brutally simple: slice the text every N tokens, maybe with some overlap. It treats a document as a ribbon of words, and it works acceptably on the documents nobody actually sends: clean single-column prose with no structure worth preserving.

Real documents are not ribbons. They are layouts. A contract has numbered clauses that refer to each other. A financial report has tables whose meaning lives entirely in the pairing between a header row and the cells four rows down. A twocolumn academic PDF has a reading order that no left-to-right scan will ever reconstruct. Slice these every five hundred tokens and you routinely get chunks that begin mid-clause, tables amputated from their headers, and column soup like the liability cap on page seven.

Retrieval then does its honest best over broken pieces, and the model answers from what retrieval found. Every downstream dollar, embedding, storage, generation, review, is spent processing damage done in the first hundred milliseconds of the pipeline.

The method is layout-aware chunking, and it means doing the unfashionable work before the fashionable work: parse the document's structure first, then cut along it.

Concretely: run extraction that recovers layout, not just characters: reading order across columns, heading hierarchy, table boundaries, list structure. Then chunk at semantic boundaries, a clause, a section, a subsection, never mid-thought, and never through a table. Tables travel as units, header row welded to body, ideally with a one-line generated description of what the table shows. And every chunk carries its breadcrumb: the path of headings above it, so "Section 9.2, Limitation of Liability" arrives attached to the text, and retrieval can distinguish the indemnity clause from a passing mention of the word.

We rebuilt exactly this layer for a document platform whose accuracy complaints all traced upstream: same model, same prompts, same retrieval, and the answer quality moved from two escalations a day to two a month, because the chunks stopped lying about the documents. The model upgrade everyone had budgeted for turned out to be unnecessary. (It usually does.)

Retrieval cannot find what chunking destroyed, and the model cannot answer from what retrieval never found.

The counterpoint worth respecting: layout-aware parsing is slower and costs more per document than naive slicing, sometimes several times more at ingestion. But ingestion happens once per document; queries happen forever. Paying cents more at write time to stop paying wrong-answer costs at every read is the kind of trade that only looks expensive on the day you implement it.

The diagnostic costs ten minutes: pull the actual chunks your pipeline produced for your ugliest important document, the scanned one, the two-column one, the one with the pricing table, and read them as the model reads them. Most teams have never done this once. It is reliably radicalizing.

TensorLabs builds document systems from that layer up, because the answer to "why is the AI wrong" is so often "look what it was given to read." The model gets the blame. The chunker earned it.