Both scale and purpose make them completely different things. You're acting as if they're the same when they're not.

I won't comment about dl but ads are trackers and spyware for me. I don't spy on websites' owners, I have my human rights to stop those trackers.

Zuck serves ads/spywares to other users, he deserves to taste his own medicines, not me.

Yes, it's a god's gift when the average user can do it, and satan's curse what a hated fucking mega-corp is doing it.

Where's the contradiction?

You can see this pattern in many different topics: updoots are highly correlated with a positive answer to "do I personally get to profit"?

I would love to pay for content. I'm _paying_ for YouTube Premium.

But heck. Do I hate the YouTube interface, it degraded far past usability.

So you’re that Hal Jordan then? Why would a Green Lantern feel the need to defend either? I feel like the Guardians would not accept your arguments as soon as you got to Oa, poozer. I guess what I am saying is don’t have a famous name. Seems obvious.

You conflate web crawling for inference with web crawling for training.

Web crawling for training is when you ingest content on a mass scale, usually indiscriminately, usually with a dumb crawler for scale's sake, for the purposes of training an LLM. You don't really care whether one particular website is in the dataset (unless it's the size of Reddit), you just want a large, diverse, high-quality data mix.

Web crawling for inference is when a user asks a targeted question, you do a web search, and fetch exactly those resources that are likely to be relevant to that search. Nothing ends up in the training data, it's just context enrichment.

People have a much larger issue with crawling for training than for inference (though I personally think both are equally ok).