Cloudflare

Generates robots.txt from your dashboard AI crawler policy instead of a static file.

Cloudflare launched Adaptive Intelligence, which reads meta-signals from live traffic to mint short-lived bot-detection rules instead of relying on stable signatures. The stated aim is raising the per-attack tuning cost; the false-positive picture for API and non-browser clients is not disclosed her

Cloudflare made Python Workers generally available, giving Python first-class, fully supported status alongside JavaScript on its Developer Platform.

Cloudflare's Vinext 1.0 moves from AI experiment to a production-ready way to run Next.js apps on Vite, adding advanced cache warming, broader compatibility, an automated test pipeline and a project compatibility check.

Cloudflare's Monetization Gateway lets content owners, API providers and MCP servers charge agents per request using x402, and enterprises building on AWS AgentCore Payments can draw on that supply now.

Cloudflare opened a closed beta of its Monetization Gateway, letting sellers charge agents per request using HTTP 402 and x402 payments settled in USDC on Base. Four production uses are shown, including AI Gateway inference. Beta access is limited to US-based sellers and buyers.

Cloudflare adds a Disallow AI Training setting that blocks training use by mixed-use crawlers while keeping search indexing. It details the Accountable designation for Apple, Google and Microsoft, changed Block semantics, deprecations, migration mappings, and Bing's early-2027 robots.txt gap.

Cloudflare reports early results from its pilot with OpenAI: network signals helped ChatGPT's search index hundreds of millions of previously unknown pages from a ~1% cohort. It covers filtering bot-invented URLs, change signals to cut recrawls, and page-level harm flags. Results are early and self-

Cloudflare expanded its Clef model family with Clef-omni, a natively multimodal model, cut Clef-flash pricing, and sped up Clef inference by up to 2.0x. Details are in the company's blog post.

Cloudflare and VoidZero open-sourced Void: a Vite plugin SDK, an agent-friendly CLI, and a self-hosted platform on Workers for Platforms. Void infers and provisions resources on deploy and gives admins access and resource controls for vibe-coded internal apps.

Cloudflare releases Clef and Clef-flash, open-weight decision models on Workers AI that return typed, calibrated classifications. The post covers Jev-compatible API, image input, 64k context, benchmark and latency comparisons, training method, and a new RL fine-tuning service.

Cloudflare's Auto Router entered public beta in AI Gateway. A classifier scores task type and difficulty, then picks a model by quality minus an adaptive cost penalty that accounts for cache rewrite costs. Internal benchmark: 86.6% success at 35% of Opus 5.5's cost.

Cloudflare releases cf, an agent-oriented command-line tool that mirrors its whole API and supports TypeScript configuration, and open-sources Forge, the internal generator behind its SDKs.

Cloudflare opened invite-only early access to Vulnerability Discovery and Remediation in Managed Defense. Using OpenAI's GPT-5.6 Cyber, it hunts bugs in authorized code, ranks them by route traffic and WAF coverage, and proposes patches and WAF rules for human review.

Cloudflare now lets Cache Rules act on the Vary header directly: normalize known negotiation headers, forward exact values to origin, or bypass caching when variation is unpredictable, on every plan.

Cloudflare's Automatic Key Exchange scans origins and picks each one's preferred TLS 1.3 key agreement, favoring post-quantum hybrids. Rollout cut HelloRetryRequests from 52% to 3.7%, trimmed p90 latency by 150 ms, and moved hundreds of thousands of domains to post-quantum origin connections.

Cloudflare's bot and agent directory now has an operator-facing dashboard: submit a crawler or agent, track and edit submissions, and declare a behavior model describing how the bot uses the content it fetches — the mechanism that determines how sites treat your traffic.

Cloudflare added task, model, turn and user breakdowns to AI Gateway User Insights, including a model-overkill view that flags requests a cheaper model could handle. Classification runs asynchronously, lags about a day, and is free for AI Gateway users.

Cloudflare's AI Search reaches general availability with image-pixel embeddings for visual search, OCR for scanned PDFs, 10 MiB file support and compatibility with any chat model.

Cloudflare open-sourced Forge, a pluggable CI pipeline that generates SDKs, CLIs and documentation from API definitions. Generation moves upstream into individual team repositories, so developer tooling stays continuously synchronized with the API.

Article URL: https://developers.cloudflare.com/changelog/post/2026-10-02-introducing-web-search-api/ Comments URL: https://news.ycombinator.com/item?id=49963171 Points: 272 # Comments: 137

Cloudflare releases EmDash 1.0, an MIT-licensed Astro CMS with a built-in MCP server and agent skills. Plugins run sandboxed with declared capabilities, and an AT Protocol registry verifies signed releases. It also open-sources the EmDash Build AI site builder alpha.