LlamaParse is a well-regarded service that reads PDFs and hands back clean text — at 10 credits a page, on documents you upload to them. We studied how it works, built our own — NicNacParse — and ran both on the same seven documents, against tests written before either one ran. Same documents. Same scorer. Here is exactly what happened.
Mechanical checks passed across all seven document classes.
The paid service's score on the identical tests, same day, same scorer.
NicNacParse runs on our own machines. No meter. No per-page rent.
Observed at the tier we tested. 400 credits bought this whole study — then the meter would keep running.
The corpus is the paperwork of real work: a book, a church flyer, a multi-column layout, a worksheet, a scanned page set, a hymn score, and a sixteen-page report. Each class carried its own checks — headings, reading order, tables, completeness against the source, and a hard rule: invent nothing.
Same seven documents. Same fixed mechanical scorer. Criteria written before either system ran.
Methodology: criteria written before any run · one fixed mechanical scorer for both systems · whole results published, including the rows marked BEHIND.
Speed, for the record: on text documents NicNacParse was faster on every class (0.2–1.1 seconds per page, against 1.2–3.5 for the upload-and-parse round trip). On heavy scans the picture flips — about 68 seconds a page for our three-pass reading versus roughly 2.4 seconds a page for theirs. Accuracy is a tie there. Throughput is not, and we say so plainly in the section below.
We don't rent the tools our clients' businesses run on.
Every page a client's business sends through a metered parser is rent paid on their own paperwork — and the document leaves their walls to be read. NicNacParse reads on our own machines, at $0 a page, forever, and the document never travels. The study itself cost 400 of their credits, total: 160 for an earlier proof run and 240 for this one. That was tuition. We don't pay it anymore.
A scorecard that only lists our wins isn't proof — it's advertising. Three real advantages remain on their side. Each one has a closing plan, and the plan is on the record, so you can hold us to it.
LlamaParse looks at an image and writes what it sees — "photo: stained glass window showing a cross and diverse people," or reads a QR code's purpose off the page. NicNacParse marks that an image is there, with its size, and moves on. Accounted for — not described.
Closing plan: an optional describing pass using a vision model running on our own machines, written into the same places theirs appear. Then we re-run the flyer, score, and report tests.
On scanned pages our accuracy ties (0.87% vs. 0.69% character error), but our careful three-pass reading takes about 68 seconds a page on our current hardware. Theirs takes about 2.4. For a shoebox of scanned records, that difference is real.
Closing plan: run the extra passes only when the first pass looks weak, instead of on every page — and revisit the number on faster hardware.
When a table cell wraps onto two lines, their output merges it back into one clean cell. Ours sometimes splits it — the words are all there, in the right order, but the table needs a human glance. Recovered and readable; not pretty. We wrote that in our own verdict before writing this page.
Closing plan: join wrapped lines back into their cells using the page geometry we already measure, then re-score the multi-column test.
The tests came first. Every acceptance check on this page was written down before either system touched the corpus. Nobody got to move a goalpost after seeing a result — including us.
One fixed scorer. A mechanical checker program scored both systems' output against the sources. Along the way it caught three bugs in itself — each found, fixed, and verified against the raw output before any score was published. Our verdict, verbatim: the scorer was wrong more often than either parser.
Our failures are in the log too. NicNacParse took four recorded passes to get here. One change made scanned pages worse (error rate 5.54% up to 6.87%) — it's written in the record, and it was reverted, not buried. The final scan score came from studying which lines the reader was dropping, not from turning a knob until the number looked good.
Two fixtures are synthetic — labeled. The scanned set and the hymn score were generated for this study, because a test with exact ground truth measures reading error better than a found document ever can. The other five documents are real work product.
The stack is ordinary on purpose. NicNacParse is built from open-source tools — PyMuPDF, pdfplumber, and RapidOCR — assembled and tuned by us. No per-page fees, no quota, no network call. It runs the same way on the thousandth page as the first.
Study the best tool. Build our own. Measure both. Publish the score — wins, ties, and gaps alike.
A person answers. That's rather the point.
Prefer email? nic@nicpattersonstudio.com