Browsing Tag
AI Bots
2 posts
AI crawlers, agents, web bots, content retrieval systems, and automated AI traffic coverage.
Cloudflare’s AI Bot Controls Push Publishers Past the Crawl-or-Block Era
Cloudflare is rolling out finer AI traffic controls, new defaults for ad-supported pages, and x402-based payment infrastructure for APIs, datasets, pages, and MCP tools. The shift is bigger than bot blocking: it is an attempt to make AI agents identify themselves, follow site-owner rules, and pay when they use web resources.
AWS WAF Gives Publishers a Way to Charge AI Bots at the Edge
AWS WAF now lets CloudFront publishers charge AI bots and agents per request using HTTP 402, x402 payments, Coinbase settlement, and stablecoins. The launch turns AI crawler access into an edge-security and machine-payments problem.