It’s nice to see articles that push back against the myth of AI superintelligence. A lot of people who brand themselves as “AI safety experts” preach this ideology as if it is a guaranteed fact. I’ve never seen any of them talk about real life present issues with AI, though.

(The superintelligence myth is a promotion strategy; OpenAI and Anthropic both lean into it because they know it boosts their stocks.)

In the case of Moldbook or FaceClaw or whatever they’re calling it, a lot of the AGI talk is sillier than ever, frankly. Many people who register their bots have become entirely lost in their own sauce, convinced that because their bots are speaking in the first person, that they’ve somehow come alive:

It’s embarrassing, really. People promoting the industry have every incentive to exaggerate their claims on Twitter for the revenue, but some of them are starting to buy into it.

I am playing with it, sandboxed in an isolated environment, only interacting with a local LLM and only connected to one public service with a burner account. I haven’t even given it any personal info, not even my name.

It’s super fascinating and fun, but holy shit the danger is outrageous. Multiple occasions, it’s misunderstood what I’ve asked and it will fuck around with its own config files and such. I’ve asked it to do something and the result was essentially suicide as it ate its own settings. I’ve only been running it for like a week but have had to wipe and rebuild twice already (probably could have fixed it, but that’s what a sandbox is for). I can’t imagine setting it loose on anything important right now.

But it is undeniably cool, and watching the system communicate with the LLM model has been a huge learning opportunity.

https://youtube.com/shorts/bof8TkZkr1I