GitPedia
Home/us/crw/Changelog
us

us/crw

Fast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.

30 Releases
Latest: today
v0.17.0Latest
usus·today·June 21, 2026
GitHub

Features

  • renderer: add opt-in Camoufox stealth renderer tier ([744beda](https://github.com/us/crw/commit/744beda540fad3e17b6d9c1d2127cad30ab83767))
  • renderer: conditional hedge + event-driven readiness for p90 ([9065007](https://github.com/us/crw/commit/90650079d8f34e65cf67f22aad5bf96f62c580ef))
  • scrape: add screenshot output format via CDP capture ([61e03e7](https://github.com/us/crw/commit/61e03e71b8dda137dffa730bcb864660009b270d)), closes [#161](https://github.com/us/crw/issues/161)
  • sdk: add Research API methods to TS + Python SDKs ([3a2f710](https://github.com/us/crw/commit/3a2f71062b00acd0c67092fc5da7bb4caa1cf527))
  • search: add Firecrawl-compatible research API engine layer ([ba1a87c](https://github.com/us/crw/commit/ba1a87c568da7f152f33cef63b188180ec210a29))
  • search: overlap query-expansion scrape with original (C1) ([4f6147e](https://github.com/us/crw/commit/4f6147e8ef7ab3146387bad4cf0902e587d1660c))
  • skills: add crw agent skill set ([0ddbe01](https://github.com/us/crw/commit/0ddbe010fdfe484f2c58db773569d85989441372))
  • skills: publish crw-research agent skill + docs install command ([4638715](https://github.com/us/crw/commit/4638715bed7d34db0c3a08ec49be6b4a80f39791))

🐛 Bug Fixes

  • docs: render :::tabs and :::callouts in prerendered pages ([f3a495a](https://github.com/us/crw/commit/f3a495ac745598eddc83926466dede60fd7d4fa0))
  • map: render SPA shells during URL discovery ([0ec4bf9](https://github.com/us/crw/commit/0ec4bf9b7c405e98def0b6f7ab3f73bf6320c275)), closes [#166](https://github.com/us/crw/issues/166)
  • mcp,sdk: drop phantom search country param, export CrwApiError ([58b8e5c](https://github.com/us/crw/commit/58b8e5cd67347c3f4f7c8ebd341236873ba201b9))
  • pdf: bound sandbox child address space to prevent false pdf_too_large ([06acb83](https://github.com/us/crw/commit/06acb8331490713967cf95bd40005c5af839373b))
  • proxy: normalize empty CRW_CRAWLER__PROXY to None ([#154](https://github.com/us/crw/issues/154)) ([b3d0fe9](https://github.com/us/crw/commit/b3d0fe996a2d0e2b8f3231c55f6ed86f0b14552b))
  • scrape: capture screenshot outside the nav-budget race ([4021b50](https://github.com/us/crw/commit/4021b5084db3b24b1f4c143f7729aa268006a039))
  • search: resolve arXiv inspect via Semantic Scholar ([70126c6](https://github.com/us/crw/commit/70126c6e5e3c07fb7ad05400424adbdfdf129563))

Performance

  • search: research concurrency 4->8, cache cap 20k->3k ([e3a6ac3](https://github.com/us/crw/commit/e3a6ac3bef2a901a9f87b1cb710d594154ebe00f))
v0.16.0
usus·1w ago·June 14, 2026
GitHub

Features

  • mcp: optimize MCP server for context, weight, and conformance ([aac7999](https://github.com/us/crw/commit/aac7999b9379fd8b6ef818ce37f78634416f79c1))
  • proxy: add proxy list + rotation primitives and HTTP-path rotation ([776e9fb](https://github.com/us/crw/commit/776e9fbad4ee509ec5201020772169d151e0587f))
  • proxy: rotate the JS/Chrome (CDP) path per request ([422ac09](https://github.com/us/crw/commit/422ac09c1f4dc2549a54e1079cec18330ccf0ea0))
  • proxy: v2 BYOP plumbing + honest docs + verification harness ([0983ba3](https://github.com/us/crw/commit/0983ba38ad1537b21ecfdc658bdda7a79c0b5437))

🐛 Bug Fixes

  • extract: stop doubling /v1 in structured-extraction chat URL ([d8b8ebc](https://github.com/us/crw/commit/d8b8ebc887eca97ca75ae1eb1893429bc547e80e))
  • extract: unify Anthropic structured URL, stop /v1/messages doubling ([90ce3dd](https://github.com/us/crw/commit/90ce3dd5479060eee0a3319e78d3d7653984e53c))
  • proxy: accept snake_case proxy_list alias on v1 ScrapeRequest/CrawlRequest ([a8e7b71](https://github.com/us/crw/commit/a8e7b71d891355912dbfc1ad23a1d0e6f6bfd865))
  • proxy: CLI crawl/map --proxy reaches the JS/CDP tier (round-4 review) ([6ee7175](https://github.com/us/crw/commit/6ee71755ce656b6ff32e4c1f4847e35655854035))
  • proxy: resolve review findings (IP-leak/correctness hardening) ([72e0486](https://github.com/us/crw/commit/72e048683eceba19b676a4bca2c8186ad762d3e1))
  • proxy: route /map discovery through the rotator (round-2 review) ([5ee06cf](https://github.com/us/crw/commit/5ee06cfb7e0c7e910a0ce0590597a277c6f49332))
  • proxy: route crawl robots/sitemap egress through the rotator ([4835e7d](https://github.com/us/crw/commit/4835e7dc02263350ae7bbcf591a353258e308032))
v0.15.2
usus·1w ago·June 12, 2026
GitHub

🐛 Bug Fixes

  • mcp: npm launcher downloads binary when platform pkg missing ([4ae1aa6](https://github.com/us/crw/commit/4ae1aa684f93c4c6a36fe157ce013700c5a9e6ea))
v0.15.1
usus·1w ago·June 11, 2026
GitHub

🐛 Bug Fixes

  • ci: poll npm in verify_npm_sdk + expose ./package.json export ([ed185dd](https://github.com/us/crw/commit/ed185dd3fd34313e5babf1c9b71e08faab3918c5))
v0.15.0
usus·1w ago·June 10, 2026
GitHub

💥 ⚠ BREAKING CHANGES

  • CrwClient() with no API key now targets the cloud and raises if unauthenticated, instead of running locally. Set CRW_LOCAL=1 for the previous zero-config local behavior.

Features

  • add TypeScript SDK (crw-sdk) ([1dd96f7](https://github.com/us/crw/commit/1dd96f7037933f933773ef92d97b68af9134906a))
  • cloud-first default + full client SDK parity ([21e819e](https://github.com/us/crw/commit/21e819ed8a863929bb67323c8b8c4b37ac5f712f))
  • fold langchain/crewai adapters into crw.integrations extras ([cd49120](https://github.com/us/crw/commit/cd49120f957d23e06899d4628388e1aa8cd11842))
  • release: publish crw-cli (and crw-browse) to crates.io ([b3d8004](https://github.com/us/crw/commit/b3d80046be72f1495985f6f21485c6fbdae4da7c))
  • search: accept native SearXNG categories as passthrough ([b3bcc5d](https://github.com/us/crw/commit/b3bcc5d439fb37f214b76b96e053053765714e39))

🐛 Bug Fixes

  • antibot: detect Google rate-limit/bot-wall pages served with HTTP 200 ([08bb46e](https://github.com/us/crw/commit/08bb46e5e41d9933e1c9603239938dbd500323fe))
  • ci: compile TS tests to JS so the SDK suite runs on Node 18/20 ([e1b7546](https://github.com/us/crw/commit/e1b75469af88cad612f93c64333508850864a981))
  • extract: avoid doubling chat/completions in structured base_url ([3816bb6](https://github.com/us/crw/commit/3816bb6713dc0932d0b06179f2e37927dba95dfb))
  • map: fold hyphen/underscore param spellings, add compare actions ([f34ae12](https://github.com/us/crw/commit/f34ae12844082db68d052b132c524eb5a4a4c9c1)), closes [#128](https://github.com/us/crw/issues/128)
  • npm: declare node>=18 engines on crw-mcp packages ([3c953fe](https://github.com/us/crw/commit/3c953feaa3c01d18f0dc05f36be339cb68d31470))
  • renderer: reap CDP target + context on PoolGuard cancellation drop ([0eebadc](https://github.com/us/crw/commit/0eebadcb1a4360a0dad5b682301c181a0c328259))
v0.14.0
usus·1w ago·June 8, 2026
GitHub

Features

  • onboarding: cloud-default messaging + api.fastcrw.com base URL ([4ccc92c](https://github.com/us/crw/commit/4ccc92c065027d900f0bd67e0273216bd76aa065))
  • pdf: PDF→markdown via pdf-inspector with Firecrawl-compatible parsers + /v2/parse ([196b153](https://github.com/us/crw/commit/196b153db011807a80a2b19155ef0605a3ca692b))
  • search: query-relevance rerank, list answers, multi-round latency guard ([ebafe83](https://github.com/us/crw/commit/ebafe8358283260df75a0d9629c59c8a1fcdad4a))

🐛 Bug Fixes

  • accept string env vars for auth.api_keys ([534f932](https://github.com/us/crw/commit/534f9320653c37aaba79e0db43ef9400b74b749b))
  • pdf: vendor test fixture into each crate (preflight: no cross-crate include) ([2c7bfe9](https://github.com/us/crw/commit/2c7bfe93bbd38a384e84dab6daa5ae0fd2e55c65))
  • release: scope out-of-crate include check to src/ only ([cd61fbf](https://github.com/us/crw/commit/cd61fbfd24046f6ef0a72e70e764189c39463fb6))
  • release: verify apt/homebrew by artifact, not flaky status-ack ([7a572fd](https://github.com/us/crw/commit/7a572fd6e5b39117d316377919387e9e7ec8ad3b))
v0.13.4
usus·2w ago·June 7, 2026
GitHub

🐛 Bug Fixes

  • docker: install aarch64 libc headers for cross-compile + CI guard ([8cd31ac](https://github.com/us/crw/commit/8cd31ac016177d3d2621e8ecfa03a027b1621a3a))
v0.13.3
usus·2w ago·June 7, 2026
GitHub

🐛 Bug Fixes

  • release: stop verify-publish reporting false failures ([d57c8c6](https://github.com/us/crw/commit/d57c8c6285a014031bcf310386e83493c7dcd2f2))

Performance

  • docker: cross-compile arm64 instead of QEMU (2h -> ~3min) ([a7cab42](https://github.com/us/crw/commit/a7cab423d11fd8a7474c587796f86eacf2c32df9))
v0.13.2
usus·2w ago·June 6, 2026
GitHub

🐛 Bug Fixes

  • server: embed openapi spec inside the crate so it can publish ([e606cb9](https://github.com/us/crw/commit/e606cb9303cec3716b7dbffd4d848164486368ea))
v0.13.1
usus·2w ago·June 6, 2026
GitHub

🐛 Bug Fixes

  • release: correct publish tier ordering + guard topology ([c84deed](https://github.com/us/crw/commit/c84deedeeeb2698f50d94247b770560cd6f3d0df))
v0.13.0
usus·2w ago·June 6, 2026
GitHub

Features

  • search: deterministic Wikidata entity-relation lookup (W3) ([aa96e3e](https://github.com/us/crw/commit/aa96e3e182fad6d8e2caf03628461515ca5aab7f))

🐛 Bug Fixes

  • release: sync Cargo.lock internal crate versions to 0.12.1 ([b5fc8a5](https://github.com/us/crw/commit/b5fc8a5b988401bb2ecca6fdd3be328d1ccd683a))
v0.12.1
github-actions[bot]github-actions[bot]·2w ago·June 5, 2026
GitHub

🐛 Bug Fixes

  • release: bump internal dep pins and track them in release config ([0139ec2](https://github.com/us/crw/commit/0139ec247c7f328ef172646eac3de5e0f287c42a))
  • release: sync Cargo.lock with bumped internal dep versions ([5c89d60](https://github.com/us/crw/commit/5c89d6037e32435a49cfd4fbdf1c2ba05ad83ae8))
v0.12.0
github-actions[bot]github-actions[bot]·2w ago·June 5, 2026
GitHub

Features

  • answer: gated moat-hardening abstention (answer_guarded) ([7ef7f32](https://github.com/us/crw/commit/7ef7f32c085f4e01429b4afa6794c52836cdd4e6))
  • mcp: emit structuredContent for crw_search; bump protocol to 2025-06-18 ([0cd9a4f](https://github.com/us/crw/commit/0cd9a4fa0e338683b75c5719bc3c54cca3b2dba6)), closes [#89](https://github.com/us/crw/issues/89)
  • search: diagnose search config and name unreachable host ([#90](https://github.com/us/crw/issues/90)) ([25f9441](https://github.com/us/crw/commit/25f94410869e24cb79a7835e2f05627d0eb07351))
  • search: pin SearXNG infoboxes/answers as structured sources (W0) ([554f18c](https://github.com/us/crw/commit/554f18ce4747057a04ec64f9f983faf46a48dee2))

🐛 Bug Fixes

  • search: use resolvable searxng host in docker config ([#90](https://github.com/us/crw/issues/90)) ([d966021](https://github.com/us/crw/commit/d9660219d23bdd4364940c97c9911dd31e73567b))
v0.11.0
github-actions[bot]github-actions[bot]·2w ago·June 3, 2026
GitHub

Features

  • api: serve /openapi.json and /openapi-3.0.json from crw-server ([3dc79b4](https://github.com/us/crw/commit/3dc79b443b6a41c3f325865aaa6bccdf562fda49))
  • docs: ship OpenAPI spec, SKILL.md, and agent-shell ([2145c97](https://github.com/us/crw/commit/2145c977463b8d5577e5cf81d3c4106ea20eddb4))
  • docs: wave 1 — API surface unblock for AI agent citations ([9b28090](https://github.com/us/crw/commit/9b2809028cf8c1c666fd94cd9b11dcab50daa633))
  • docs: wave 2 — 15-page glossary cluster for AIO citations ([14cd7e1](https://github.com/us/crw/commit/14cd7e1db3f70cbd9dd7da08818b25d1e8688c71))
  • extract: wave 2 cache token telemetry + DeepSeek provider tag fix ([1b4f1aa](https://github.com/us/crw/commit/1b4f1aa4eec70657964864f8c7d134150b6e2354))
  • mcp,cli,docs: snippet alias + agent-shell polish for benchmark wins ([3472d13](https://github.com/us/crw/commit/3472d1333dd5520a481e226543e6271d940c145f))
  • monitor: add feature-gated self-host crw-monitor mode (M6) ([ff732f3](https://github.com/us/crw/commit/ff732f3a31a66cf97746e79f7e4017788841f94a))
  • monitor: add stateless change-tracking diff engine + LLM judge ([a078081](https://github.com/us/crw/commit/a07808127762b6d2acf248de65f0cb9d17aad2d6))
  • + 16 more

🐛 Bug Fixes

  • antibot: strip inline data-URIs before classifier deep scan ([6f1cbd2](https://github.com/us/crw/commit/6f1cbd20b98f73df099f72b7ad08e41acada2a7e))
  • browse: address outbound hardening review feedback ([a3c2076](https://github.com/us/crw/commit/a3c2076ffdd8024579bf7ab97653816cb5bd3881))
  • browse: harden outbound URL handling ([c370972](https://github.com/us/crw/commit/c37097253c7df89a5cedb3ddb79e5c9386f75b1b))
  • cli: swap SearXNG default to 127.0.0.1 + actionable error hint ([618d41b](https://github.com/us/crw/commit/618d41b52c3fa0c971eb6c34ea98922f06ea094f))
  • docker: include OpenAPI specs in crw-api build context ([f70c7af](https://github.com/us/crw/commit/f70c7af6486a82d6bd86c9a7c1054c410f045877))
  • docker: include OpenAPI specs in crw-api build context ([0238138](https://github.com/us/crw/commit/0238138f832a507a70ed411d96ec0143cbb7333e))
  • docs: add 2 missing Firecrawl-shape shims caught by sapient ([22b3d54](https://github.com/us/crw/commit/22b3d5482ca224d0eebd76a795ed8428f108d080))
  • extract: accept deepseek/openai-compatible providers in structured extract ([2f0d86d](https://github.com/us/crw/commit/2f0d86d5245c24d388671299beb5acd76458c4ca))
  • + 4 more

Performance

  • search: calibrated answer top_n 8->5 (4x faster, accuracy holds) ([#80](https://github.com/us/crw/issues/80)) ([751f8ee](https://github.com/us/crw/commit/751f8ee160d28b84d352e533864ca4043793d294))
v0.10.0
github-actions[bot]github-actions[bot]·1mo ago·May 20, 2026
GitHub

Features

  • detector: add vendor-specific anti-bot block markers ([c88c508](https://github.com/us/crw/commit/c88c508fb90b166dfe3727fd5dfb4f1597e43667))
  • renderer: add chrome_proxy as 4th fallback tier ([b4da4f7](https://github.com/us/crw/commit/b4da4f79bb4d0ed71c25f14aaae5137d00f8b26b))
  • renderer: per-request country via CDP proxy auth ([11b4d32](https://github.com/us/crw/commit/11b4d32285ed8a4e6bee8f390c264f9fc0be1b1a))

🐛 Bug Fixes

  • release: harden npm publish + fix mcp-registry verifier ([9d4076f](https://github.com/us/crw/commit/9d4076fadd252a33e7887ee6e4925be8e6aa7d8e))
  • renderer: detect CloudFront/WAF 403 as bot-wall ([7e058b2](https://github.com/us/crw/commit/7e058b2915eff8b36d1186013e0810b2290492f4))
  • renderer: escalate JS tier on 4xx/5xx and vendor-detected blocks ([648c372](https://github.com/us/crw/commit/648c372ee5d52aed1459c22725be2e6d34d95afb))
v0.9.1
github-actions[bot]github-actions[bot]·1mo ago·May 16, 2026
GitHub

🐛 Bug Fixes

  • release: sync crw-cli internal dep versions with workspace ([26c528e](https://github.com/us/crw/commit/26c528e737c5ed136d3c6e72da36b9324363805f))
v0.9.0
github-actions[bot]github-actions[bot]·1mo ago·May 16, 2026
GitHub

Features

  • cli: add AI extraction flags and `crw setup --reset` ([912eea0](https://github.com/us/crw/commit/912eea0a1fa0db5a838560f2c25da44b3ee33d44))
v0.8.3
github-actions[bot]github-actions[bot]·1mo ago·May 15, 2026
GitHub

Features

  • cli: two-phase auto-fallback for `crw <url>` scrape ([a871e54](https://github.com/us/crw/commit/a871e544443673751974a8dc5ebb4f2b0eafd59f))
  • setup: make config.toml the canonical source for `crw setup` ([b07c154](https://github.com/us/crw/commit/b07c1549932399a355d30e15632d424f0ad28b85))

📦 Miscellaneous

  • release 0.8.2 ([38ae764](https://github.com/us/crw/commit/38ae7641b44d321150febc38eda530db3d572046))
  • release 0.8.3 ([efba1b3](https://github.com/us/crw/commit/efba1b308d6ac3a5e7a5bbd3f132514bf631e86b))
v0.8.2
github-actions[bot]github-actions[bot]·1mo ago·May 14, 2026
GitHub

🐛 Bug Fixes

  • release: move crw-cli to unpublished and update dep versions ([7f121f6](https://github.com/us/crw/commit/7f121f6f731e7bfa311be48fa442e0049bbda16d))
v0.8.1
github-actions[bot]github-actions[bot]·1mo ago·May 14, 2026
GitHub

🐛 Bug Fixes

  • cli: mark crw-cli as publish=false to fix release ([3104cc5](https://github.com/us/crw/commit/3104cc5acec6d6a0307adbb5cc9897ce04417a16))
v0.7.1
github-actions[bot]github-actions[bot]·1mo ago·May 12, 2026
GitHub

🐛 Bug Fixes

  • bump stale internal version pins to 0.7.0 ([#48](https://github.com/us/crw/issues/48)) ([0bec22a](https://github.com/us/crw/commit/0bec22a76e03f428e28281b30bb34364bf5e5edd))
v0.7.0
github-actions[bot]github-actions[bot]·1mo ago·May 12, 2026
GitHub

Features

  • LLM summary and search answer (BYOK) ([#45](https://github.com/us/crw/issues/45)) ([ffcc2a5](https://github.com/us/crw/commit/ffcc2a52cd273dc334dd46b663be24fd45df4711))
v0.6.4
github-actions[bot]github-actions[bot]·1mo ago·May 12, 2026
GitHub

Features

  • renderer: add bounded browser-context pool for Chrome tier ([#43](https://github.com/us/crw/issues/43)) ([69b4861](https://github.com/us/crw/commit/69b48610d269c37fa043bcd58855b3970f554f94))
v0.6.3
github-actions[bot]github-actions[bot]·1mo ago·May 12, 2026
GitHub

Features

  • map: drop action URLs and strip tracking params (closes [#40](https://github.com/us/crw/issues/40)) ([#41](https://github.com/us/crw/issues/41)) ([6d9ed39](https://github.com/us/crw/commit/6d9ed39f67e4d92cc51ca16ccc21e451c4bb0373))
v0.6.2
github-actions[bot]github-actions[bot]·1mo ago·May 10, 2026
GitHub

Features

  • search: add /v1/search endpoint backed by bundled SearXNG sidecar ([f4bd7f4](https://github.com/us/crw/commit/f4bd7f46db9f286e3c49be95a968951802a90710))

🐛 Bug Fixes

  • antibot: drop bare 'captcha'/'access denied' markers — false positives ([fae6c09](https://github.com/us/crw/commit/fae6c09537cf286bfb08cb9ebefab0c723c4160f))
  • crawl: drop redundant `.into_iter()` for clippy 1.95 ([#39](https://github.com/us/crw/issues/39)) ([fb4032b](https://github.com/us/crw/commit/fb4032b86a5c0095e7e69b198ac2017aa7003000))
  • deadline: auto-extend implicit deadline to fit CDP ladder (closes [#35](https://github.com/us/crw/issues/35)) ([#38](https://github.com/us/crw/issues/38)) ([068193f](https://github.com/us/crw/commit/068193facc742088825a5a850800d4fd9b659eda))
  • map: WordPress sitemap-index timeout (closes [#33](https://github.com/us/crw/issues/33)) ([c3dfd6c](https://github.com/us/crw/commit/c3dfd6c66ff6bdbbaf3d2ce1646dbb9d7ac6dd5a))
  • release: register crw-search crate in release manifest ([9074761](https://github.com/us/crw/commit/907476163c69c5fd8e421e5063686a750c10ce24))
  • search: codex iteration-1 hardening — error mapping, resource bounds, container ([5acba7b](https://github.com/us/crw/commit/5acba7bbf1be2cc147e5c25ad9fca80e9bce757d))
  • search: codex iteration-2 — error-body cap, per-source row budget, doc ([a440d6e](https://github.com/us/crw/commit/a440d6e4ae3b6ac0dc8decdead3efe04ad33bc43))
  • search: codex iteration-3 — predicate-based well-formed filter ([4b4df3a](https://github.com/us/crw/commit/4b4df3a521cba31cd2a82f2d3595d726cb4b2c16))
  • + 1 more
v0.6.1
github-actions[bot]github-actions[bot]·1mo ago·May 9, 2026
GitHub

Features

  • metrics: cdp_pending_requests, cdp_live_connections, ([b5f7bec](https://github.com/us/crw/commit/b5f7bec28308e4f0094b7bbcbbecc5d2f734e385))
  • renderer: live-connection registry + 60s telemetry sampler ([b5f7bec](https://github.com/us/crw/commit/b5f7bec28308e4f0094b7bbcbbecc5d2f734e385))
  • renderer: target lifecycle metric + leaked detection ([b5f7bec](https://github.com/us/crw/commit/b5f7bec28308e4f0094b7bbcbbecc5d2f734e385))
  • server: /ready endpoint with deep status code ([b5f7bec](https://github.com/us/crw/commit/b5f7bec28308e4f0094b7bbcbbecc5d2f734e385))

🐛 Bug Fixes

  • release: bulletproof publish pipeline and drop pdf feature ([8fcf2f6](https://github.com/us/crw/commit/8fcf2f656aabe1a8a05db7d6c21011e06959e184))
  • renderer: invalidate cached chrome WS URL on connect failure ([b5f7bec](https://github.com/us/crw/commit/b5f7bec28308e4f0094b7bbcbbecc5d2f734e385))
v0.6.0
github-actions[bot]github-actions[bot]·1mo ago·May 9, 2026
GitHub

Features

  • extract: scale recall to 63.74% on 1000-URL benchmark ([5b85555](https://github.com/us/crw/commit/5b855554c5f7ba16981fbe2060e25cca4ba81686))
  • renderer: add browserless/chromium opt-in stealth profile (+2.5pt) ([d2414c9](https://github.com/us/crw/commit/d2414c9cd89dc01447b9e52501aa26180ce7d326))
  • renderer: chrome-stealth wiring + CDP discovery improvements ([6b2e77c](https://github.com/us/crw/commit/6b2e77c2a356ef8fc453560870985819ce75483a))
  • server,core,crawl: plumb tier timeouts and recall pipeline ([7cbee43](https://github.com/us/crw/commit/7cbee43e5db319f0af39dddea07faccbf0cd25ee))

📦 Miscellaneous

  • release 0.6.0 ([bd03a35](https://github.com/us/crw/commit/bd03a352922b293431e49722c052f90c945f1c56))
v0.5.0
github-actions[bot]github-actions[bot]·1mo ago·May 4, 2026
GitHub

Features

  • core: add deadline module and request/renderer config scaffolding ([5a4e69a](https://github.com/us/crw/commit/5a4e69ae605d15c0090f3d866db0f8f4fa23a715))
  • core: thread end-to-end Deadline through scrape pipeline ([5991986](https://github.com/us/crw/commit/5991986cdac9756500dab40b8bf05ad454dbd21c))
  • crawl: key per-domain rate limiter by eTLD+1 ([39c7954](https://github.com/us/crw/commit/39c7954881cfde47e34dad2eaa4141f1f10b1156))
  • crawl: per-host concurrency cap on the eTLD+1 limiter ([274f462](https://github.com/us/crw/commit/274f462b2755a02fc2485bddc7ac8ad3fd11c0e3))
  • renderer: add browserless/chromium opt-in stealth profile ([236f626](https://github.com/us/crw/commit/236f62682f29011a959bdef5a9770475a809f0a9))
  • renderer: chrome nav-budget cap + truncated/deadline_exceeded flags ([c57cef8](https://github.com/us/crw/commit/c57cef8c6ad2ba2fefce7f4110685bc779359378))
  • renderer: chrome request-paused interception pump (T27) ([13fcaa4](https://github.com/us/crw/commit/13fcaa4c5560f254d67682bff96ba24e39cdf13e))
  • renderer: leak-through fallback when global breaker open & host clean ([86a9e36](https://github.com/us/crw/commit/86a9e36880f2dddcb3d7b7bd6c993825559cf487))
  • + 3 more

🐛 Bug Fixes

  • compose: auto-restart and bound memory for renderer containers ([dd610cc](https://github.com/us/crw/commit/dd610ccae2579138d5438795e1d5ac441a0fafc3))
  • core: emit meaningful Timeout value when deadline already expired ([607bb27](https://github.com/us/crw/commit/607bb27692686f3563af52ea721d7dfb800d0405))
  • crawl: prioritize anti-bot detection over placeholder warning ([05aa933](https://github.com/us/crw/commit/05aa93358f3fa9826eb97db114bef06d1754dae3))
  • escalate to JS renderer on HTTP failure and empty markdown ([9fc7934](https://github.com/us/crw/commit/9fc79344702e30be0555e63a02aa5377f15cca93))
  • mcp: apply per-endpoint timeouts to proxy client ([741f1b2](https://github.com/us/crw/commit/741f1b245e064b267b4fb0dfb5487099bc86e2e4))
  • renderer: enforce Deadline in HttpFetcher via tokio::time::timeout ([b1c4058](https://github.com/us/crw/commit/b1c4058f47eed204d413a51c56d8ae43f547ff63))
  • renderer: keep larger thin-result HTML when stitching attempts ([8147236](https://github.com/us/crw/commit/8147236cc6b94d3c2db34f1128a687d9e110dc35))
  • renderer: rescue 39 bench failures via UA, retry, and thin-content escalation ([ddacb49](https://github.com/us/crw/commit/ddacb49e92688c3a20c7f7fe32da58d83c620f31))
  • + 1 more

Performance

  • renderer: drop fixed 2s JS wait, rely on SPA selector poll ([cb043f7](https://github.com/us/crw/commit/cb043f7754f870df67a3a56e41a552ba7f7867f4))
  • renderer: tighten tier timeouts and bump LP retry threshold ([3f93d60](https://github.com/us/crw/commit/3f93d6052251eb72abee20ea4992ca3cdfc7ddb4))
  • renderer: widen breaker tolerance to 20 failures / 10s cooldown ([6525a84](https://github.com/us/crw/commit/6525a84c18e2c4fafa92cc0d29203310755d3ef1))

📦 Miscellaneous

  • release 0.5.0 ([3987de1](https://github.com/us/crw/commit/3987de1b15b5d7605cc26645d14b74020c8eb7a9))
v0.4.2
github-actions[bot]github-actions[bot]·1mo ago·April 29, 2026
GitHub

Features

  • core: add render decision types and prometheus metrics scaffold ([e08682b](https://github.com/us/crw/commit/e08682b761822a7100e0f40cffe4cd4f3dcf2a5c))
  • renderer: add per-host renderer preference cache ([21e41d1](https://github.com/us/crw/commit/21e41d1330bed255854824f55c3419a590a86411))
  • renderer: track HTTP routing and warn on pinned-renderer failure ([3208d27](https://github.com/us/crw/commit/3208d277aa4c480c1257648dc86a7151dcfb8976))
  • renderer: wire host preferences, circuit breakers, and CF detection ([0c53c64](https://github.com/us/crw/commit/0c53c645562c43a4f62aa22f1e7f603c42b3b3f3))

🐛 Bug Fixes

  • core,renderer: surface render metadata and harden host normalization ([ee4130b](https://github.com/us/crw/commit/ee4130b62467defb61c5b85bf267c767b3bd909a))
  • renderer: correct failure classification and routing decisions ([4d684bd](https://github.com/us/crw/commit/4d684bdaa0cd1a27f011d73322099258a0f713be))
  • renderer: probe lifecycle, RAII guard, breaker counter ([02044f5](https://github.com/us/crw/commit/02044f573cd6274231b3856cd799d7e74d61f9ba))
v0.4.1
github-actions[bot]github-actions[bot]·1mo ago·April 29, 2026
GitHub

Features

  • add per-request renderer field for scrape and crawl APIs ([#29](https://github.com/us/crw/issues/29)) ([f1e0b63](https://github.com/us/crw/commit/f1e0b63fd28be0ceb38342086a309f92bbbc1e53))
  • crw-browse: add interactive browser MCP server with phase-2 tools ([e78879d](https://github.com/us/crw/commit/e78879db18c7c4b3df2a4984349a65b4493b1cda))
  • honor renderer mode and force_js in config (fixes [#28](https://github.com/us/crw/issues/28)) ([b76e473](https://github.com/us/crw/commit/b76e473facbce08a841ef8bd9fdfac97a552a8fd))

🐛 Bug Fixes

  • detect failed JS renders and fail over to next renderer ([fca8fd5](https://github.com/us/crw/commit/fca8fd5cadb4fa3c96bf5a315f96ab6d1e63989c))
  • docs: use absolute logo paths in site.config.js ([c5c9321](https://github.com/us/crw/commit/c5c93215561094a35038ab6af2b21e91c16199f4))
  • docs: use absolute paths for logo and favicon assets ([cdb1451](https://github.com/us/crw/commit/cdb14517da6425c105b95749dec35bbc9e977f5e))