GuideBy MarkovoPublished October 27, 2026

Extract text with pdfplumber: Fix gaps and order

Use pdfplumber to extract usable encoded text, separate known columns before joining their words, and send image-only pages to OCR. The failures fall into five cases: readable text with spacing problems, interleaved columns, tables with broken field relationships, pages without characters, and existing text layers with wrong symbols. A nonempty result can still omit an attachment or join unrelated sentences. ¹³

Classify the page by what survived

Choose the extraction route from the failure visible on each page.

Classify the pages individually. Our mixed document retained its digital prose and table through the default local conversions, but its scanned page contributed no words. ³¹⁴

An image on a page does not prove its text is inaccessible: a logo can sit beside encoded paragraphs. An empty character list plus a bitmap containing the visible words identifies a recognition problem. A genuinely blank page needs no OCR. ¹¹⁴

Extract text with pdfplumber without hiding gaps

Extract text with pdfplumber page by page, and keep an empty page visible in your processing record.

This script writes plain text into a Markdown file and reports each page's state. It keeps the per-page values available before joining them, so an empty page can be routed separately rather than disappearing inside a successful document result. ¹⁴

from pathlib import Path
        import pdfplumber
        
        with pdfplumber.open("input.pdf") as pdf:
            pages = [page.extract_text() for page in pdf.pages]
        
        for number, text in enumerate(pages, start=1):
            print(f"page {number}: {'text' if text else 'empty'}")
        
        Path("output.md").write_text(
            "\n\n".join(text or "" for text in pages), encoding="utf-8"
        )
        

The empty state is a routing signal, not a diagnosis. Inspect the corresponding page's character objects and visible content before sending it to recognition. For batch processing, retain the page number and state alongside the converted document.

Spacing and reading order need different interventions. The text extractor groups positioned characters into words and lines. A spacing threshold changes those boundaries. It does not define which column should be read first. In our column sample, unrelated sentences occupy the same extracted line. ¹²³

Output from our narrow-column sample, unedited:

Invoice reconciliation starts with the export file, not Demand forecasts fail upward more often than they
        the ledger. Teams that reconcile weekly pull the fail downward. Optimistic sales inputs push safety
        payment provider's settlement report first, match it stock up for slow movers, and the excess sits in the
        against issued invoices by number, and only then warehouse for two quarters before anyone reopens
        

Use x_tolerance for a visible word-boundary error and y_tolerance for unwanted line grouping. Compare the same line before and after changing a setting. The experimental layout=True option aims to reproduce page spacing, while use_text_flow=True follows the underlying character stream. Neither option promises a semantic reading sequence. The source stream itself can be ordered differently from the visible paragraphs. ¹²

Separate columns before joining their text

Separate column regions before choosing the sequence in which their text is joined.

For a document whose pages all share a fixed central gutter, this recipe reads the left region before the right. The midpoint is a layout assumption, not a detected boundary. ⁴

from pathlib import Path
        import pdfplumber
        
        parts = []
        with pdfplumber.open("input.pdf") as pdf:
            for page in pdf.pages:
                midpoint = page.width / 2
                left = page.crop((0, 0, midpoint, page.height))
                right = page.crop((midpoint, 0, page.width, page.height))
                parts.extend([left.extract_text() or "", right.extract_text() or ""])
        
        Path("output.md").write_text("\n\n".join(parts), encoding="utf-8")
        

Output from our wide-column sample after cropping, unedited:

Invoice reconciliation starts with the export file, not the ledger.
        Teams that reconcile weekly pull the payment provider's
        settlement report first, match it against issued invoices by
        number, and only then open the accounting system. Skipping
        the export step leaves ghost receivables that nobody can trace
        back to a real transaction.
        

Cropping restored the sentence sequence on our column fixtures. Applied blindly to ordinary full-width prose, the same recipe split sentences at an imaginary gutter. Full-width headings, changing column boundaries, and tables crossing the midpoint need their own regions. In a mixed report, apply the column operation only to the pages and regions identified as columns. ³⁴

A Markdown wrapper can change extraction paths even when the page itself stays unchanged. We converted our wide-column page alone with MarkItDown, then appended a separate table page and converted the combined document. The isolated page retained its sentences. In the combined version, the column text became interleaved. The original narrower column fixture failed even on its own, so appending a table is not required for every column failure. ³⁴⁷

from pathlib import Path
        from markitdown import MarkItDown
        
        text = MarkItDown(enable_plugins=False).convert_local("input.pdf").markdown
        Path("output.md").write_text(text, encoding="utf-8")
        

Output from our wide-column sample alone, unedited:

Invoice reconciliation starts with the export file, not the ledger.
        Teams that reconcile weekly pull the payment provider's
        settlement report first, match it against issued invoices by
        number, and only then open the accounting system. Skipping
        the export step leaves ghost receivables that nobody can trace
        back to a real transaction.
        

Output from our unchanged wide-column page with a table appended, unedited:

Invoice reconciliation starts with the export file, not the ledger. Demand forecasts fail upward more often than they fail
        Teams that reconcile weekly pull the payment provider's downward. Optimistic sales inputs push safety stock up for slow
        settlement report first, match it against issued invoices by movers, and the excess sits in the warehouse for two quarters
        number, and only then open the accounting system. Skipping before anyone reopens the spreadsheet. A weekly report on
        the export step leaves ghost receivables that nobody can trace stock cover by SKU surfaces the overcorrection before the
        back to a real transaction. capital is committed.
        

MarkItDown's installed PDF converter selects the prose extractor at document scope. When it detects no form-like pages, MarkItDown runs pdfminer over the whole document. Once a form-like page is present, it joins the content collected during its page loop instead. Non-form pages in that loop were read through pdfplumber's default text extraction. Adding the table therefore changed the column page's extraction path without changing its contents. A project issue reports the same failure class. Our controlled fixture is separate from that report's attachment. ⁵⁶

When a page works alone but breaks inside a larger PDF, compare those contexts before tuning its spacing.

PyMuPDF4LLM preserved the column prose on both versions of the wide-column document in this run. Its default Markdown conversion offers a layout-based alternative when a fixed crop is unsuitable. The same local call left our scan without text and changed some of the simple formula strings. ³⁸

from pathlib import Path
        import pymupdf4llm
        
        text = pymupdf4llm.to_markdown("input.pdf")
        Path("output.md").write_text(text, encoding="utf-8")
        

Output from our wide-column sample through the layout converter, unedited:

Invoice reconciliation starts with the export file, not the ledger. Teams that reconcile weekly pull the payment provider's settlement report first, match it against issued invoices by number, and only then open the accounting system. Skipping the export step leaves ghost receivables that nobody can trace back to a real transaction. 
        
        Returns processing needs a cutoff rule before it needs software. A warehouse that accepts returns within thirty days must decide whether the clock starts at shipment, delivery, or carrier scan, because each choice shifts which orders qualify by several days. The rule has to be written down before the tooling question matters. 
        

Recover table fields before serializing rows

Extract cells for a table, then map its heading rows to the fields your application expects.

Plain text can contain every data value while losing the labels that give them meaning. Our grouped-heading fixture placed the group label above its subordinate labels, and default text extraction emitted those headings on separate lines. Treating the first heading line as a complete schema would leave an amount column unnamed. The rotated-heading fixture also retained data rows without a usable heading line. ³

Output from our grouped-heading sample, unedited:

Item Qty Price
        Unit Total
        Consulting hours 12 85.00 1020.00
        Data export fee 3 40.00 120.00
        

pdfplumber's table API returns row and cell lists, which are a different output contract from the plain-text method. Map layered headings to explicit application fields before exporting them. For this invoice, the intended fields include Unit Price and Total. Copying a parent label into every empty heading cell would invent relationships on a differently structured form. ¹

Table boundaries need a separate diagnosis when no cells are found. The project's merged-cell discussion includes a maintainer's explanation involving hidden ruling lines. That historical attachment was not rerun here. Our own fixture has visible vertical rulings through its upper heading, so it demonstrates layered-heading interpretation, not general recovery of merged cells. ⁹⁴

When inserting recovered cells into surrounding prose, exclude the table region from text extraction so its values are not duplicated.

Route scans and wrong symbols separately

Send missing image text to OCR, but treat wrong encoded symbols as a separate failure.

pdfplumber reads existing PDF character objects. It cannot recover words that exist only in an image. A project discussion describes the distinction between an image saved as PDF and a PDF containing a text layer. Our scanned attachment remained absent from the default local outputs even while the surrounding digital pages survived. ¹³¹⁰

Markovo's retained PDF to Markdown conversion result included the scanned passage from our mixed document. The retained response records OCR involvement, although its page-count fields disagree. The output uses an HTML table, so its table needs a renderer that accepts embedded HTML. Its formula output also changed expressions that pdfplumber retained as text. ³¹²¹³

Output from our mixed text-and-scan sample, unedited:

<table><tr><td>Item</td><td>Qty</td><td>Unit Price</td><td>Total</td></tr><tr><td>Consulting hours</td><td>12</td><td>85.00</td><td>1020.00</td></tr><tr><td>Data export fee</td><td>3</td><td>40.00</td><td>120.00</td></tr><tr><td>Template license</td><td>1</td><td>190.00</td><td>190.00</td></tr><tr><td>Support retainer</td><td>6</td><td>55.00</td><td>330.00</td></tr><tr></tr></table>
        
        This page exists only as a picture. A scanner produced it, the PDF embeds it as a bitmap, and no amount of text extraction will recover these words without OCR.
        

A pre-existing OCR layer can already contain wrong letters. In pdfplumber's math discussion, a reply attributes the reported errors to a scanned source with an imperfect recognized text layer. Copying such symbols reproduces the recognition errors. Compare the copied expression with its visible source to distinguish that case from misplaced but otherwise correct characters. The discussion's document was not rerun for this guide. ¹¹

Our formula fixture contains simple ASCII expressions, not a representative mathematical recognition test. Retaining those strings says nothing about recovering fractions, integral layouts, or superscripts from images. ³

Retain page identity, separate column regions, and preserve table fields before joining the document. When an isolated page and the complete PDF behave differently, inspect the document-level extraction branch.

How we tested

These controlled fixtures support specific failure diagnoses, not accuracy or speed claims across arbitrary PDFs.

We executed the local paths on 2026-10-27 with Python 3.11.14, pdfplumber 0.11.10, pdfminer.six 20260107, MarkItDown 0.1.8, PyMuPDF4LLM 1.28.2 and PyMuPDF 1.28.2. Online documentation may describe other versions. The installed source and executed calls define our test. No packages or recognition models were installed during this run. ¹⁵

Markovo is the service we build. It ran through the same corpus and scorer as the other paths; its row is reported in full, including where it scored lower.

Markovo's original PDF outputs came from public API requests on 2026-10-06 and were supplied upstream. We rescored them without making conversion requests. Its rate_card_version, 2026-09-19.v3-media-unified, is not a converter build identifier. Retained zero elapsed values represent reused outputs, not instantaneous conversion. Markovo was not executed on the added controlled fixtures, so there are no Markovo scores for those fixtures. ¹²¹³¹⁵

Each PDF has a same-name .truth.json. The combined-fixture truths deliberately score only the column paragraphs, not the appended table. This isolates the effect on the original text. It does not measure the added page's quality. The construction and retained source excerpts are in lab/experiment.py and lab/source-provenance.json. ⁴¹⁵

We ran these commands from the repository root:

PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python runs/2026-10-27-markovo-pdfplumber-extract-text/lab/experiment.py prepare
        PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python tools/lab_run.py --lab runs/2026-10-27-markovo-pdfplumber-extract-text/lab --tools pdfplumber,markitdown,pymupdf4llm --samples prose-single,table-simple,table-merged-header,two-column,scan-image,mixed,equation,rotated-header,columns-with-table,wide-columns,wide-columns-with-table --heavy-cache '' --markovo-cache ''
        PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python runs/2026-10-27-markovo-pdfplumber-extract-text/lab/experiment.py recipes
        PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python tools/lab_score.py --lab runs/2026-10-27-markovo-pdfplumber-extract-text/lab --out runs/2026-10-27-markovo-pdfplumber-extract-text/lab/scores.json --markdown runs/2026-10-27-markovo-pdfplumber-extract-text/lab/scores.md
        PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python runs/2026-10-27-markovo-pdfplumber-extract-text/lab/summarize.py
        PYTHONDONTWRITEBYTECODE=1 tools/lab-venv/bin/python runs/2026-10-27-markovo-pdfplumber-extract-text/lab/check_snippets.py
        

Raw outputs follow lab/<path>/<fixture-stem>.md. Versions and call durations are in lab/runs.json. The pdfplumber-columns path applies the fixed midpoint crop deliberately to every selected fixture, including unsuitable controls. Every displayed Python example also ran unchanged against the supplied lab/sample.pdf and each selected PDF in isolated working directories using literal input.pdf and output.md names. Their scripts, outputs, timings and first-line previews are recorded in lab/snippet-checks.json. ³⁴¹⁷

The table is a direct subset of lab/scores.json, produced by lab/summarize.py, including all selected tools' wins and losses. Other upstream tools and non-PDF fixtures remain in the full score artifact but are outside this guide. n/a means the truth does not define that metric. There were no execution errors in the selected cells. ³

P is prose_recall: normalized truth sentences with at least four words found in output. O is reading_order: adjacent paragraph anchors, using their first six words, appearing in increasing positions. R is table_rows_intact: rows whose cell strings all occur on an output line. H is table_header_intact: leaf-heading labels found together on a line. Pipe detects a Markdown pipe-table pattern. S applies sentence recall to scan text. Pages measures expected mixed-document anchors. Math is equation_recall, a substring match after normalizing mathematical delimiters, font wrappers, escapes and whitespace. Seconds is the runner's recorded duration. The definitions and exact scorer snapshot are retained. ³¹⁶

These are limited proxies. O does not validate every word's order. H does not validate a grouped-heading hierarchy, and an HTML table on a single line can satisfy R and H without proving semantic cell structure. Math does not test mathematical equivalence. No duration is a controlled speed comparison.

For the mixed Markovo response, job.pages_total=3, job.pages_ocr=3, and job.billing_metrics.pagesOcr=1. The conversion response separately records jobs[0].billing_metrics.ocrPageIndexes=[2]. The image-only response records pages_ocr=1, billing pagesOcr=1 and index [0]. These snapshots support recognition involvement but disagree on the mixed-document count. They do not establish an unambiguous page-by-page trace. ¹²¹⁸

The installed source excerpts identify Page.extract_text, Page._get_textmap, chars_to_textmap, and WordMap.to_textmap in pdfplumber, plus _extract_form_content_from_words and PdfConverter.convert in MarkItDown. In the latter, form_page_count == 0 selects whole-document pdfminer. The other branch joins page-loop output. Exceptions and empty results have additional fallbacks. We inspected these installed files rather than assuming that a floating online branch matches them. Source paths, line numbers and SHA-256 hashes are in lab/source-provenance.json. Historical issue attachments were not executed. ²⁵⁶¹⁵

Sources

1. pdfplumber pinned README and API
2. Installed pdfplumber WordMap.to_textmap source
3. Complete local score output
4. Controlled fixture construction and crop intervention
5. Installed MarkItDown PdfConverter.convert source
6. MarkItDown report of form-triggered column interleaving
7. MarkItDown pinned README
8. PyMuPDF4LLM official API
9. Merged-cell report and hidden-line diagnosis
10. Scanned PDF and missing text layer discussion
11. Math extraction and an imperfect OCR layer
12. Retained mixed-document public API response
13. Retained formula public API response
14. Page character, image and form-detection inventory
15. Installed versions, source locations and hashes
16. Exact scorer source used in this run
17. Executed article examples and output previews
18. Retained image-only public API response