Methodology + resultsLast updated September 22, 2026

Benchmarks

We run public document-conversion benchmarks against the live production pipeline at markovo.net — not a special build — and publish the method and the scores. Every number on this page is produced by the benchmark authors' own scoring code, on the benchmark's full published set.

Run 1: OpenDataLoader-Bench — 200 documents

Method

Dataset: opendataloader-project/opendataloader-bench — the complete published set of 200 documents with gold Markdown references, converted through the production API in fast mode (200/200 succeeded).
Scoring: the benchmark's own evaluator modules — evaluate_reading_order (NID), evaluate_table (TEDS), evaluate_heading_level (MHS). _s variants strip tables before comparing.
Coverage note: TEDS applies to the 42 documents that contain tables; MHS to the 107 with detectable headings. Metrics with no applicable content are not averaged in.

Metric	Score (0–1)	Documents	What it measures
Reading order · NID	0.925	200/200	text-sequence fidelity vs gold reference
Reading order · NID_s	0.930	200/200	same, tables stripped first
Tables · TEDS	0.897	42/200	table cell structure tree-edit distance
Tables · TEDS_s	0.914	42/200	structure only, cell text ignored
Headings · MHS	0.724	107/200	heading level/hierarchy match
Headings · MHS_s	0.814	107/200	same, tables stripped first

Run 2: DP-Bench — 200 documents

Method

Dataset: upstage/dp-bench (Upstage) — 200 documents from Library of Congress, open educational resources and Upstage's internal set; the benchmark that introduced the NID reading-order metric.
Scoring: docling-project/docling-eval, markdown_text modality — our output Markdown scored directly against the gold Markdown: F1 token overlap, precision/recall, normalized edit distance, BLEU, METEOR.
Converter: markovo.net production API, fast mode, one document per job.

Metric	Score (0–1)	Documents	What it measures
Token F1	0.877	200/200	word-level overlap with the gold Markdown (median 0.938)
Recall	0.953	200/200	share of gold text present in our output
METEOR	0.866	200/200	alignment tolerant of word order and stemming
Precision	0.847	200/200	share of our tokens that appear in the gold reference
BLEU	0.685	200/200	n-gram sequence overlap — stricter about exact ordering
Normalized edit distance	0.271 ↓	200/200	character distance to gold — lower is better

Where these scores sit

OpenDataLoader-Bench ships its own leaderboard: twelve engines scored on the same 200 documents with the same metric code. Placing our run on that table — same dataset, same scorer, same overall aggregation (mean of the three metrics), scores rounded to three decimals as published:

Rank	Engine	Overall	Reading order · NID	Tables · TEDS	Headings · MHS
1	opendataloader [hybrid]	0.907	0.934	0.928	0.821
2	nutrient	0.885	0.925	0.708	0.819
3	docling	0.882	0.898	0.887	0.824
4	Markovo (this run)	0.872	0.925	0.897	0.724
5	marker	0.861	0.890	0.808	0.796
6	unstructured [hi_res]	0.841	0.904	0.588	0.749
7	edgeparse	0.837	0.894	0.717	0.706
8	opendataloader	0.831	0.902	0.489	0.739
8	mineru	0.831	0.857	0.873	0.743
10	pymupdf4llm	0.732	0.885	0.401	0.412
11	unstructured	0.686	0.882	0.000	0.388
12	markitdown	0.589	0.844	0.273	0.000
13	liteparse	0.576	0.866	0.000	0.000

Overall 0.872 — 4th of 13 engines on the authors' own aggregation, behind hybrid (0.907), nutrient (0.885) and docling (0.882).
Reading order 0.925: ties nutrient for the second-best published NID, behind only the benchmark authors' own hybrid pipeline (0.934).
Tables 0.897: second-best published TEDS, behind opendataloader-hybrid (0.928) and ahead of docling (0.887).
Headings 0.724: mid-table — our weakest of the three metrics and the clearest improvement target the benchmark surfaced.

Baseline scores are the benchmark authors' published numbers, not re-run by us; our row uses the same 200 documents and the same scoring code. DP-Bench has no published leaderboard for the markdown_text modality — that metric set stands on its own.

Production limits we publish too

Upload size: 50 MB per job on Pro — a 193-page mixed corpus (41 MB) fit, an 81 MB bundle did not.
Job runtime: 12 minutes hard ceiling. A 105-page scan-heavy document estimated at 3.5 min in fast mode actually timed out — estimate-vs-reality on degraded scans is a known gap we're working on.
Estimate preflight: accurate mode rejects jobs estimated over the ceiling before they start (105 p accurate → est. 17.5 min → refused, no Credits spent).

What this does not claim

Both runs above are the benchmarks' full published sets — no cherry-picked subset.
Leaderboard context comes only from the benchmark repo's own published scores; we did not re-run other engines, and engines evolve — check the repo for current baselines.
A benchmark score is not the same as "quality" for your file — that's why every job also ships a reviewable bundle.