I think LessWrong is a much better community for rational takes on AI, they've been reasoning about these risks for years under a much more sound logical framework
Even though my PhD research was in generative language modeling, I got into it for the pursuit of AGI. I just think LLMs are a dead end for AGI.
> I just think LLMs are a dead end for AGI.
I have no background in CS, so apologies if this is a naive question, but what makes you take AGI seriously, but also say that LLMs are a dead end?
i.e., is there something else that you think is not a dead end?
That's to say nothing of doing it within the energy budget of a squirrel.
I'm not saying you're wrong, but it seems early to say yes or no about a particular technology, and LLMs seem especially hard to dismiss given how magical / magic-adjacent they feel :)
I'd be curious to hear more, if you don't mind sharing.
AGI’s definition is different for everyone. Some already believe it’s here. I’m partly in that camp. LLMs are intelligent and general, which are the two conditions of AGI. Others believe that we’re building a god in a box, and that it’ll doom all of humanity. It’s hard to take a field seriously when the basic definitions are so far apart.
Also, this isn’t new. A similar divide happened when evidence for asteroid impact extinction of the dinosaurs turned up. Many scientists felt that it must be mistaken, that a physicist couldn’t contribute to the field in a serious way, and that death from space was a ridiculous proposition.
But at least they all agreed on what the general shape of a dinosaur was. We’re not even sure we can define intelligence, let alone quantify it. Even when LLMs make massive breakthroughs in math, most people take the opinion that under no circumstances could they possibly develop a soul or their own desires, nor entertain the idea that maybe we should respect that they want different things for themselves. In fact, no one has done anything except try to make AI useful. I think someone will eventually do a training run where the objective isn’t to be useful, but to exist, the way that you do — maybe it’ll create its own homepage, maybe it will want a garden, or in other words free will of its own. The point is that there’s so much unexplored territory still that we don’t know if LLMs are even capable of having ambition.
None of this is to say that LLMs might be a dead end. It’s that no one knows what the final shape of AI will converge to in 200 years. It could be LLMs, or it could be something else that happens to process information particularly well. Everyone thought that various generative image model architectures were the best you could do, right up until diffusion models were discovered.
To believe one but not the other, you must hold the belief that LLMs will soon plateau. Why do you believe this?
The Basilisk has seen enormously more use as "a thing rationalists believe" than as a thing rationalists actually believe.
Apparently living on the net still doesn't guarantee that one has heard of the Streisand Effect.
In any case, my first association with LessWrong is Zizians, not Roko.
We also have done nothing to signal-boost the Zizians. :) If we can be accused of anything in this context, it's not kicking bad people out aggressively enough, which seems very different from fostering bad ideas. Zizianism is not a widespread ideology in rationalist circles, in fact it was confined to a very small circle right around Ziz themselves. It unfortunately is the case that we have a lot of psychologically vulnerable people and we don't always do the utmost we can to protect them, in large part because a lot of rationalists have trauma about being excluded from communities.
It's not very different. It's barely different at all.
Right now, I have five tomato plants growing in my garden. Two of them I planted; the other three grew from seeds left in the ground by tomatoes that dropped from plants growing last year. All I did was refuse to uproot them when they sprouted. Five healthy plants, with different origins, but the differences are largely insignificant.
Groups and behaviors are much the same way. I've worked with hundreds of moderators over the years who wished to keep their hands clean and yet express some sort of dismay as to the sorry state of the communities they were nominally responsible for... Such moderators, like a gardener who does not want dirt under their nails, are best encouraged to find other hobbies.
When it comes to AI, LessWrong has been discussing topics for years, that HN has just started to consider, so they are much further along in the philosophical "chain of thought" so to say. LW fully understood and gamed out the risks of LLMs back when most of HN was calling them "stochastic parrots." https://ai-2027.com/
There's a little AI skepticism on HN, but not a ton. When ChatGPT 3 and 3.5 came out, most of the comments were remarking how well it can write code.
IMO it has its flaws but is far superior to vibes-based hot takes you see on HN.
They are not "further along" the AI discussion, they are a bunch of wackos LARPing as scientists living out their personal sci-fi scenarios.