Web-to-Markdown for RAG: extraction quality and cost benchmark
RAG retrieval depends on what enters the index. Missing tables or code can remove the answer; navigation and cookie text can dilute useful context. We compare extracted Markdown against hand-written reference content.
Download the free offline RAG evaluation sample (ZIP): two owned fixtures, labeled synthetic extractions and a reproducible JSON scorecard. Python 3.9+ standard library only; no keys or network calls. See the included README for matching rules and MIT license.
Choosing between two providers? Read our Firecrawl vs Jina Reader fixture comparison with raw-output contrasts and explicit denominators.
Extraction results
| Extractor | F1 | Recall | Precision | Code blocks preserved | Tables preserved (pages) | Boilerplate leakage | Median latency | Runs (page attempts) | Last run (UTC) |
|---|---|---|---|---|---|---|---|---|---|
| Firecrawl | 97.6% | 96.2% | 99.2% | 100.0% | 0.0% | 0.0% | 1.134 s | 10 (0 failed) | 2026-10-19T09:27:01.233619+00:00 |
| Jina Reader | 97.7% | 95.6% | 100.0% | 96.7% | 100.0% | 0.0% | 2.888 s | 10 (0 failed) | 2026-10-19T09:27:29.728997+00:00 |
| DIY Trafilatura | 87.9% | 86.5% | 91.6% | 90.0% | 0.0% | 10.0% | 0.017 s | 10 (0 failed) | 2026-10-19T09:28:15.199606+00:00 |
Metrics are averages over successful page attempts across recorded live runs; failures are shown separately. Table preservation is the share of eligible pages retaining every reference table. Code preservation averages each page's share of exact fenced block bodies. — means unavailable.
Key findings
- Firecrawl: all reference tables preserved on 0.0% of scored pages; code block preservation 100.0%; known boilerplate leakage 0.0%.
- Jina Reader: all reference tables preserved on 100.0% of scored pages; code block preservation 96.7%; known boilerplate leakage 0.0%.
- DIY Trafilatura: all reference tables preserved on 0.0% of scored pages; code block preservation 90.0%; known boilerplate leakage 10.0%.
- DIY Trafilatura had the lowest median latency (0.017 seconds). Jina latency includes request pacing; runs occurred at different times.
These results describe this small controlled fixture set, rather than the wider web. Structural Markdown checks can fail even when table or code text remains present.
Methodology
We use 10 fixture documentation pages hosted on our own domain, with hand-written ground truth. They cover prose, code and tables, including a JavaScript-rendered page and SDK tabs with an initially hidden panel. Inspect the JavaScript status fixture and SDK tabs fixture. DIY fetches static HTML without executing JavaScript.
Token-level scoring lowercases words and punctuation, ignores whitespace and compares token multisets. Recall measures reference-token coverage; precision measures the fraction of extracted tokens matching the reference; F1 is their harmonic mean. It does not measure ordering or semantic correctness. Code checks require identical fenced bodies; table checks compare Markdown cells and row order. Leakage counts known boilerplate phrase spans, allowing occurrences already in the reference, and may miss rewritten boilerplate. Latency covers the adapter call; no automatic retries are used.
Monthly cost calculator
Pricing as published on 2026-10-19. USD estimates from our saved pricing snapshot.
| Tool | Estimated monthly cost (USD) |
|---|---|
| Firecrawl | |
| Jina Reader | |
| DIY Trafilatura |
Firecrawl uses one credit per basic page and the least expensive listed annual-billing monthly equivalent plus whole $5 top-ups where available; annual subscriptions are billed yearly. The free allocation is modeled as listed in the snapshot. Jina is free without a key within its request limit; a paid estimate needs an output-token rate that is unavailable in the snapshot. DIY uses an assumed VM budget, without a throughput guarantee, engineering time or extra bandwidth.
Some links are affiliate links. We may earn a commission from the Firecrawl link at no extra cost to you; results are computed from benchmark data.