Firecrawl vs Jina Reader: owned documentation fixture results

Choosing a Markdown extractor for RAG? Compare token coverage, exact code and table formatting on our owned documentation fixtures. These measurements cannot establish a universal winner.

Affiliate disclosure: We may earn a commission from the Firecrawl affiliate link at no extra cost to you. Jina Reader is a neutral link. Affiliation does not affect scoring.

Shared fixture comparison

10 shared eligible owned fixtures: authentication, idempotency, live-status, pagination, queries, quickstart, rate-limits, sdk-tabs, uploads, webhooks. Every recorded non-dry attempt on these fixtures is included; no latest-success selection. Nonshared attempts excluded: 0.

n is the number of scored successful page attempts for each metric, or successful timed attempts for latency. Code mean weights each eligible page equally; the block count is a separate pooled denominator. Table passes require every reference table on a page. Failed attempts remain in the failure denominator and have no successful quality score.

Firecrawl recorded UTC interval: 2026-10-19T09:26:50.237103+00:00 to 2026-10-19T09:27:01.233619+00:00; successful attempts lacking raw output: 0.

Jina Reader recorded UTC interval: 2026-10-19T09:27:02.385716+00:00 to 2026-10-19T09:27:29.728997+00:00; successful attempts lacking raw output: 0.

Table-score audit

Offline audit of the same saved outputs finds recognized tables with matching ordered cells after escaped-underscore normalization on 10/10 Firecrawl and 10/10 Jina pages. Legacy results remain 0/10 and 10/10 above. The gap measures encoding mismatch, not demonstrated structural loss. No new live measurement or downstream quality test was performed. Reproduce with python3 products/ragbench/audit_tables.py; hashes and before/after denominators: products/ragbench/data/table-score-audit.json.

Two fixture contrasts from saved output

Authentication: a strict table-format mismatch

Inspect the owned HTML fixture.

Firecrawl: F1 97.83% (1 scored attempt); exact code 1/1 blocks; all-table check 0/1 page.

Jina Reader: F1 97.96% (1 scored attempt); exact code 1/1 blocks; all-table check 1/1 page.

Both outputs contain the response-fields table. Firecrawl writes an escaped underscore in the observed_at cell; Jina writes the reference spelling. The scorer compares literal cell text and does not normalize Markdown escapes. A zero here does not mean all table content is missing.

| Field | Type | Meaning |
| --- | --- | --- |
| id | string | Stable service-accounts identifier |
| state | string | Current lifecycle state |
| observed\_at | timestamp | Server observation time in UTC |

| Field | Type | Meaning |
| --- | --- | --- |
| id | string | Stable service-accounts identifier |
| state | string | Current lifecycle state |
| observed_at | timestamp | Server observation time in UTC |

SDK tabs: hidden-panel code coverage

Inspect the owned HTML fixture.

Firecrawl: F1 97.83% (1 scored attempt); exact code 3/3 blocks; all-table check 0/1 page.

Jina Reader: F1 96.16% (1 scored attempt); exact code 2/3 blocks; all-table check 1/1 page.

The reference includes the initially hidden JavaScript tab. Firecrawl matches all three fenced bodies; Jina matches two. The unmatched reference body below is absent as an exact fenced block in the saved Jina output.

const client = new OrbitClient({token, workspace: "ws_lab", timeout: 20000});
await client.close();

Scoring and scope

The runner lowercases word and punctuation tokens, ignores whitespace and compares token multisets. Precision is matched tokens divided by output tokens; recall is matched tokens divided by reference tokens; F1 is their harmonic mean. The displayed F1 is a mean of per-attempt F1 values. It measures overlap, not token order, semantic correctness or retrieval quality.

Code checks require identical fenced bodies, including whitespace and case, while ignoring fence language labels. Table checks require recognized pipe-style Markdown tables with matching literal cells and row order, allowing whitespace and separator-width differences. Escapes and equivalent renderings can fail. Table score zero alone is not evidence of missing table content.

Latency is adapter wall time, including Jina's three-second request-start pacing wait. Providers ran sequentially at different times, with no automatic retries; these are not independent samples or a controlled speed ranking. Ten related fictional documentation pages are small and not representative of all sites. They include JavaScript rendering and hidden tabs, but do not establish performance on arbitrary production sites.

Practical buying tradeoffs

For a pipeline that needs both language examples, the SDK fixture favors Firecrawl's recorded code coverage. Both providers retain the expected table rows and cells under the audit’s escaped-underscore normalization. The legacy table gap provides no evidence of a structural advantage for either provider. Validate your own authorized documents, rendered tables and downstream retrieval before choosing.

Dated cost assumptions

Existing cost snapshot checked 2026-10-19; no new price research was performed for this comparison. It assumes 1 Firecrawl credit per basic Markdown page. Its Hobby example is $16/month equivalent billed annually ($19 billed monthly), with 5,000 credits; these are saved assumptions, not a current quote.

The saved Jina assumptions describe keyless access at 20 requests/minute and paid billing by output tokens with unknown numeric paid price. Keyless access is not a promise of unlimited capacity or usage. These requests did not establish a comparable dollar cost or production throughput.

Use the existing RAG cost calculator to explore the saved assumptions, including quotas and annual billing; confirm applicable terms before buying. Cost estimates exclude integration effort and do not prove extraction quality.

Evidence provenance

Offline recomputation dated 2026-10-19 from products/ragbench/data/results.jsonl (SHA-256 1c4297d53b3a74aace338ed3ceeea00e17ad525a16a331ee0c54accdd6adcf38), using products/ragbench/run.py scoring and products/ragbench/fixtures/truth/*.md. Saved aggregate and per-fixture report: evidence JSON. Reproduction script: products/ragbench/compare_firecrawl_jina.py. The existing summary was checked against these raw records. Read-only live records, summary and cost assumptions matched the worktree copies byte for byte on 2026-10-19.

Use only authorized sources and comply with target sites’ terms. Return to the full RAG benchmark and calculator.