Hacker News new | past | comments | ask | show | jobs | submit
lathiat | on: Cloudflare crawl endpoint
They are one step ahead of you: https://blog.cloudflare.com/introducing-pay-per-crawl/

Sort of though. Still private beta since July 2025.