Free Deep research for AI agents

Open-source crawler and browser toolkit you run yourself.

Limits: none stored.

Source · asOf 2026-10-25 · LICENSE refetched 2026-10-03 is Apache License 2.0. README fetched the same day says Crawl4AI turns websites into clean, LLM-ready Markdown and is an open-source crawler and scraper you run yourself. The hosted cloud offer is not this row. Re-fetched twice 2026-10-25 (~10 s apart, plain curl, no login): HTTP 200, no redirect, byte-identical: 'Apache License Version 2.0, January 2004'. FLAG for review: the same LICENSE ends with an 'Attribution Requirement' section ('All distributions, publications, or public uses of this software, or derivative works based on this software, must include the following attribution'), which the stored license value 'Apache-2.0' does not capture; the GitHub commits API lists the last LICENSE change on 2025-02-21, so the term predates the 2026-10-03 fetch. License value left as stored (not re-labelled) pending that review. README.md (also fetched twice, HTTP 200, byte-identical) still says 'open-source web crawler for LLMs and AI agents' and 'LLM-ready Markdown'. License relabelled 2026-10-25 (v0.12.118): raw LICENSE re-fetched twice again (~10 s apart, plain curl, no login): HTTP 200, no redirect, byte-identical; it still opens 'Apache License Version 2.0, January 2004' and still ends with the 'Attribution Requirement' section ('All distributions, publications, or public uses of this software, or derivative works based on this software, must include the following attribution', naming UncleCode and the Crawl4AI project; for websites/web applications 'In an "About" or "Credits" section'). License value 'Apache-2.0' -> 'Apache-2.0 plus attribution requirement' (same style as mineru's 'Apache-2.0 plus additional commercial terms'); data decision by Agent Depo under Ethan delegation 2026-10-21 19:41 ET t55u. Flag resolved.

Gated public benchmark of tool-using assistant questions. Files require accepting a no-reshare condition. No license field was on the fetched card.

Limits: none stored.

Source · asOf 2026-10-25 · README fetched 2026-10-03. extra_gated_prompt says not to reshare outside a gated or private HF repo. No license key in the fetched card. Anonymous file download returned 401 earlier on the raw README path before this resolve URL. README re-fetched twice 2026-10-25 (HTTP 200, byte-identical): front matter still has extra_gated_prompt 'To avoid contamination and data leakage, you agree to not reshare this dataset outside of a gated or private repository on the HF hub.' and still no license key, so license stays null.