GuideBy MarkovoPublished November 6, 2026

Transcribe Video to Text: Captions or Speech

Reuse a text caption track when it exists; otherwise transcribe the audible soundtrack and record screen-only details separately. To transcribe video to text, distinguish three sources: captions already stored as text, spoken words that need recognition, and information visible only in the picture. FFmpeg can extract the first, a speech service such as Markovo can process the second, and neither operation captures an unspoken instruction on a slide. ²⁵⁶

Choose the text source before you transcribe video to text

Choose the method by where the words exist in the recording, rather than by its video extension.

YouTube's Show transcript opens existing caption text when captions are available. Selecting a line jumps to that part of the video. Its absence does not establish whether the recording contains speech. For a local file, inspect its streams instead of assuming that the captions visible in a player belong to the file. ¹

A caption switch that removes the displayed words suggests a separate caption source. Words that remain on the image can be burned into the picture. A player can also load an external subtitle file, so a visible caption switch alone does not prove that the downloaded video includes that text track.

Extract existing captions with FFmpeg

FFmpeg can recover an existing text subtitle track without running speech recognition.

ffmpeg -hide_banner -i input.mp4 -f null -
        ffmpeg -hide_banner -y -i input.mp4 -map 0:s:0 -c:s srt output.srt
        

The subtitle stream must contain text that the selected encoder can handle. A bitmap subtitle requires OCR rather than a text-only format conversion. FFmpeg's explicit stream mapping chooses an existing stream. It does not create captions from sound. ²

Our captioned control used authored captions made from the narration script. The extraction returned their wording and cue times. Those times were assigned for the fixture, so the output demonstrates subtitle extraction, not automatic alignment to speech. ¹⁰

Output from our authored-caption sample, unedited:

1
        00:00:00,000 --> 00:00:08,000
        Route Video Contract Sample 01. Markovo Quality Lab is verifying a real media conversion route.
        
        2
        00:00:08,000 --> 00:00:18,000
        This is sample 1 of eight. The target number is one hundred and twenty six. The speaker names Shanghai, OpenAI, and the Quality Lab.
        

The original narrated recording had an audio track but no subtitle track. Its subtitle extraction failed. The captioned derivative kept the same audio and video, then added a text track. Extraction then succeeded. ¹¹¹²

Use Markovo for a timestamped soundtrack transcript

Markovo provides a Markdown transcript of the soundtrack when speech must be recognized rather than extracted from captions.

Markovo's video-to-Markdown path is documented as Signed-in Beta and transcribes the audio track. It does not analyze video frames or identify speakers. The sample result contains a language label and a time range around the recognized passage. A passage-level range does not give every word its own subtitle timing. ⁵⁶⁷

Use standard-library Python to submit once, poll the returned job, and download its Markdown. Read the required request values from the environment.

The protocol example was checked by offline replay against the supplied response snapshots. The transcript below comes from an earlier successful public-API job, not from that replay. We did not execute a new API conversion for this guide. The CLI was not executed for this guide. ⁶¹⁴

Output from our narrated-video sample, unedited:

## Time-coded transcript
        
        **[00:00:00.000 → 00:00:23.640]** Route video contract sample 01. Markovo Quality Lab is verifying a real media conversion route. This is sample 1 of 8. The target number is 126. The speaker names Shanghai, OpenAI, and the Quality Lab. The transcript begins at second zero and the route contract ends after this checklist statement.
        

The service returned usable timestamped speech text for the video. Its spoken-audio companion also exposed a limitation: the supplied script's “Route Audio Contract” became “Root audio contract.” The job still succeeded. Numeric spelling changed too, but writing a spoken number as digits is a different event from substituting a word. ⁷⁸

Reject empty output from a partial MarkItDown install

The tested MarkItDown installation accepted the narrated video but returned empty Markdown because its optional speech dependencies were missing.

MarkItDown routed the file to its audio converter. The installed environment lacked the speech-recognition dependencies. Conversion returned without an exception, and the raw Markdown output was empty. The diagnostic runner recorded the empty return separately from the converter's Markdown. ⁹

Output from our narrated-video diagnostic sample, unedited:

Conversion returned without an exception.
        Returned Markdown was empty.
        No audio transcript heading was present.
        

The audio converter accepts the video extension before attempting speech recognition. If the helper raises a missing-dependency exception, the converter catches it and returns whatever metadata it collected. The helper's configured recognition branch uses Google's speech service, so this implementation is not an offline speech model merely because its wrapper runs locally. ³⁴

A batch job that treats “no exception” as its only success condition can store an empty document. For this converter, reject output with no passage after the audio-transcript heading. Metadata by itself does not satisfy a transcription request.

A historical audio-failure issue reports an error message, but it supplies too little context to equate it with this blank-return branch. Our claim is limited to the installed package and dependency state recorded below. A fully configured local speech model was not executed for this guide. ¹⁵

Keep screen-only information separate from speech

Unspoken screen content needs its own visual record, even when a speech transcript is available.

Our visual control removed the soundtrack and added the on-screen label “Build target staging.” It had no audio or subtitle track. Subtitle extraction could not retrieve the label, and the partial MarkItDown installation still returned no text. We did not run OCR on this control, so the experiment establishes the input boundary rather than an OCR success rate. ¹¹¹²

For a narrated tutorial, listen for an instruction such as “select this option” while watching the selected item. If the option's name never enters the audio, add it to a separate visual note with its playback location. Keep the spoken passage intact.

Extract existing caption text when that is the text you need. Recognize speech when no usable caption text exists. Supplement the result when the picture carries unspoken information. An empty conversion result belongs in the failure queue until the missing stage is identified.

How we tested

These tests distinguish input routes and missing-stage failures. They do not rank speech-recognition accuracy.

Markovo is the service we build. It ran through the same corpus and scorer as the other paths; its row is reported in full, including where it scored lower.

Only the original narration files have Markovo receipts. The captioned and silent derivatives were local controls and were not submitted to that service. MarkItDown ran in the supplied environment without speech_recognition or pydub. Its results describe that installation, not a configured recognition engine. FFmpeg was used as a subtitle extractor, not an ASR system.

Corpus and execution dates

Ground truth is the synthetic source script, not independently adjudicated real-world speech. Original source hashes and revision metadata are retained in the supplied provenance. The input and command record is media-experiment.json. Source media remain in the editorial run. The linked evidence contains publishable text, JSON, and scripts rather than the MP4 or MP3 files.

Local execution and scoring took place on 2026-11-05 with Python 3.11.14, MarkItDown 0.1.8, and FFmpeg 7.1. Markovo's supplied jobs were created on 2026-10-27 UTC and reused for this guide. Their receipt reports rate-card identifier 2026-09-19.v3-media-unified. It is not a model version. Cached runner durations were set to null, so they supply no latency measurement.

Run from the repository root, using an existing FFmpeg binary in FFMPEG_BIN:

tools/lab-venv/bin/python runs/2026-11-05-markovo-transcribe-video-to-text/lab/run_media.py --ffmpeg "$FFMPEG_BIN"
        tools/lab-venv/bin/python tools/lab_score.py --lab runs/2026-11-05-markovo-transcribe-video-to-text/lab --out runs/2026-11-05-markovo-transcribe-video-to-text/lab/scores.json --markdown runs/2026-11-05-markovo-transcribe-video-to-text/lab/scores.md
        tools/lab-venv/bin/python runs/2026-11-05-markovo-transcribe-video-to-text/lab/score_media.py
        tools/lab-venv/bin/python runs/2026-11-05-markovo-transcribe-video-to-text/lab/replay_api_example.py
        

The offline runner records complete subprocess commands, versions, elapsed times, and errors. score_media.py invokes the canonical tools/lab_score.py, then adds stream-presence observations, output character counts, and extracted subtitle-cue counts separately. It does not change the canonical scoring formula. ¹²

Score definition and complete relevant results

prose_recall splits the truth paragraphs on sentence boundaries, normalizes case, whitespace, and selected Markdown marks, and counts exact sentence substrings with at least four words. It is not word error rate or semantic accuracy. Spelling “eight” as “8” can lower it without changing the amount. The visual label is shorter than the sentence threshold, so its cells are null, not perfect scores.

The table below is copied from scores.json. elapsed_s is local wrapper or extraction time, not recognized-audio throughput. The empty MarkItDown rows have no exception. The subtitle-extraction errors indicate absent subtitle streams. No PDF page, table, scan, or formula metric applies here. ¹¹

The Markovo video's literal sentence score is 0.6667, while the audio companion's is 0.1667. Both receipts report quality_score=1 and pages_ocr=0. Those fields do not establish verbatim accuracy or frame analysis. Numeric normalization explains some literal mismatches. The audio's “Root” substitution is a word error visible in the raw output. We measured neither speaker attribution nor word-level timing. ⁷⁸¹³

Raw outputs are lab/ffmpeg/captioned-recording.md, lab/markitdown/screen-recording.md (empty), lab/markitdown/screen-recording-diagnostic.md (runner diagnostics), lab/markovo/screen-recording.md, and lab/markovo/speech.md. The remaining relevant output names and exceptions are recorded in runs.json. Markovo's missing derivative results are unexecuted cases, not zero scores.

Standard-library Python protocol example

This example follows the supplied Markovo developer snapshot. Set MARKOVO_API_KEY and a positive integer MARKOVO_MAX_CREDIT_UNITS. It submits once, polls the returned job, and downloads Markdown. After a timeout, retain the job identifier and resume polling rather than submitting the same input again. ⁶

import json
        import os
        from pathlib import Path
        import time
        from urllib.request import Request, urlopen
        import uuid
        
        base = "https://markovo.net"
        auth = {"Authorization": "Bearer " + os.environ["MARKOVO_API_KEY"]}
        boundary = uuid.uuid4().hex
        fields = {"capability_id": "video-to-markdown",
                  "max_credit_units": str(int(os.environ["MARKOVO_MAX_CREDIT_UNITS"]))}
        body = b""
        for name, value in fields.items():
            body += (f'--{boundary}\r\nContent-Disposition: form-data; name="{name}"\r\n\r\n{value}\r\n').encode()
        body += (f'--{boundary}\r\nContent-Disposition: form-data; name="file"; filename="input.mp4"\r\nContent-Type: video/mp4\r\n\r\n').encode()
        body += Path("input.mp4").read_bytes() + f"\r\n--{boundary}--\r\n".encode()
        request = Request(base + "/v1/convert", data=body, method="POST",
                          headers={**auth, "Content-Type": "multipart/form-data; boundary=" + boundary})
        with urlopen(request, timeout=60) as response:
            job_id = json.load(response)["job_id"]
        
        deadline = time.monotonic() + 300
        while True:
            with urlopen(Request(base + "/v1/jobs/" + job_id, headers=auth), timeout=60) as response:
                payload = json.load(response)
                job = payload.get("job", payload)
            if job["status"] == "succeeded":
                break
            if job["status"] in {"failed", "cancelled"}:
                raise RuntimeError(job)
            if time.monotonic() >= deadline:
                raise TimeoutError("Job still pending; retain job_id and resume polling")
            time.sleep(2)
        
        with urlopen(Request(base + "/v1/jobs/" + job_id + "/download?format=md", headers=auth), timeout=60) as response:
            Path("output.md").write_bytes(response.read())
        

Offline replay checked request construction, a pending-to-success transition, and download bytes against recorded response shapes. It did not verify fresh credentials, a current live response envelope, or a new conversion. ¹⁴

Sources

1. YouTube Help: View video transcripts
2. FFmpeg 7.1: stream selection and mapping
3. MarkItDown 0.1.8: AudioConverter source
4. MarkItDown 0.1.8: transcription helper source
5. Supplied Markovo capability and page-ownership snapshot
6. Supplied Markovo developer API and CLI snapshot
7. Original Markovo video job receipt
8. Original Markovo audio companion output
9. Observed local result and speech-dependency record
10. Extracted authored-caption output
11. Canonical scores and separate media observations
12. Offline media experiment runner
13. Original Markovo audio companion job receipt
14. Offline protocol replay record
15. Historical audio-transcription failure report