Cloudflare launches a web search API for AI agents in partnership with Exa and Ceramic
IA en un minuto newsroom · Editor: Jon Elgezabal

In 30 seconds
Web Search API, Cloudflare's new service, lets AI agents look up current information online instead of guessing a page's address. It is in open beta inside AI Gateway with Ceramic.ai, Exa and Linkup, which charge their list price with no Cloudflare markup. In return, their crawlers must identify themselves, respect robots.txt and always link to the page each result comes from.
On October 2, Cloudflare introduced Web Search API, a service that lets AI agents search the internet for current information instead of guessing the address of a page. It comes through AI Gateway, the Cloudflare service that brings together control, logs and billing for calls to AI models in one place, and it starts with three search providers: Ceramic.ai, Exa and Linkup.
It is in open beta: it can be tried today, but it is not the final version yet. Searches are paid with the AI Gateway credit balance, at each provider's list price and with no markup, or with the customer's own key for each provider. It works through a REST API or from Workers, the Cloudflare platform for running code, and Cloudflare says it will soon add it as a built-in tool in AI Gateway.
Cloudflare sets a condition for the partners: their crawlers must meet Cloudflare's rules for verified bots, and every result must include a link to the page the content comes from. According to Cloudflare, these providers identify their crawlers, respect the robots.txt file (where each website states what can be crawled) and give the source of the results.
Why it matters · analysis and opinion
Website owners win, at least on paper: Cloudflare only accepts as partners search providers that identify themselves, respect robots.txt and link to the source page in every result. That is the most interesting part of the announcement, because it puts the conditions on the side of those who publish. For people who build agents, the change is practical: current searches without the agent making up addresses, with billing and logs in the same place as their model calls and no markup on each provider's price. It is still in beta and has only a handful of providers, so the sensible move is to compare their results with your own queries before choosing one. And for any business with a website, this is a good moment to check what its robots.txt says.
Official source: Cloudflare · Written with the help of AI: how we make the news


