Hacker News new | past | comments | ask | show | jobs | submit
>absurd amount of effort every website is pouring into anti-scraping tech.

Maybe some people do not want to provide their entire website as free training material to [insert LLM lab here]?

You... Do know that people scrape websites for non-AI reasons, right?
Why should the site owners be interested in allowing them to do that?