Hacker News new | past | comments | ask | show | jobs | submit
To clarify the two questions raised:

First, the Cloudflare Crawl endpoint does not require the target site to use Cloudflare. It spins up a headless Chrome instance (via the Browser Rendering API) that fetches and renders any publicly accessible URL. You could crawl a site hosted on Hetzner or a bare VPS with the same call.

Second on pricing: Browser Rendering is only available on the Workers Paid plan ($5/month). It is not part of the free tier. Usage is billed per invocation beyond the included quota - the exact limits are in the Cloudflare docs under Browser Rendering pricing, but for archival use cases with moderate crawl rates you are very unlikely to run into meaningful costs.

The practical gotcha for forum archival is pagination and authentication-gated content. If the forum requires a login to see older posts, a headless browser session with saved cookies would help, but that is more complex to orchestrate than a single-shot fetch.

> Browser Rendering is only available on the Workers Paid plan ($5/month). It is not part of the free tier.

The post says it's available for both free and paid plans. According to the pricing page of the Browser Rendering, the free plan will have 10 minutes/day browsing time.

[0] seems to suggest even paid plans are effectively limited to 500 web pages per day, right?

    Crawl jobs per day 5 per day
    Maximum pages per crawl 100 pages
[0] https://developers.cloudflare.com/browser-rendering/limits/#...