Hacker News new | past | comments | ask | show | jobs | submit
carloslfu | on: Cloudflare crawl endpoint
They have a Pay Per Crawl option for owners. This plus a /crawl endpoint is genius.