Hacker News new | past | comments | ask | show | jobs | submit

  keeping Search profitable without enshittifying it
I don't think Google had much to do with it. It has way more to do with the internet concentrating into major communities where all the content is private. Even Reddit content is going this way. Most FB and Instagram content are private. All of Discord is unreachable by crawlers.

Prior to this, just about everything on the web was open for crawlers since content wanted to be found.

Google caused this by pushing people towards "brands" rather than individual pages. Those platforms only exposed the data necessary for SEO and Google refused to punish them for it.