Sorry I thought this updated comment was a better one 😅
For full transparency my original comment was a moan about how I can’t decide whether Dario and co are confidence scammers or genuinely earnestly think that these next word prediction models are sentient due to all the brain damage, they’ve inflicted on themselves. FWIW Sama and friends are almost certainly in that first camp.
MalwareTech/Marcus Hutchins has an excellent take on this that he posted for the normies on linkedin
Posted verbatim below to save you from having to read LinkedIn
Anthropic really won’t let this botnet thing go. It’s so desperate & ridiculous.
9B parameter models are so weak that they’re essentially useless. There is no reason at all for anyone or anything to be self-replicating them.
Even if you quantize a 9B parameter model down to the smallest possible size (at which point it becomes less than useless), and you run it on regular RAM instead of VRAM (at which point it also becomes painfully slow) it still requires about 5 GB of RAM, which is way more than most IoT device has.
Mirai is already self-propagating across millions of IoT devices without the help of any AI. It’s simply not needed, and it adds unnecessary complexity.
Many IoT devices are ephemeral and their entire OS is reset on reboot, so any code running on the device is gone after reboot. The AI models would also be competing with malware like Mirai, which propagates exponentially faster, and locks down the devices so other malware can’t get in.
Again, why? What is the purpose of this. Anthropic keeps pushing this ridiculous idea but have yet to establish any reason for why this would actually happen, or who would benefit from it.
Any remotely intelligent threat actor would just use a powerful unquantized models to reverse engineer & find vulnerabilities in obscure devices and write normal (AI-free) self-propagating code. Ironically, they typically do this by stealing Anthropic API keys and using Anthropic’s models, not open-weight ones.
Again, why are we self-propagating open-weight models? There is absolutely no reason to do that. The combined processing power of every IoT device on earth would be less than a moderately sized stolen cloud compute GPU instance.
I’m pretty sure Anthropic keeps pushing these ridiculous narratives in an attempt to paint open models as dangerous, thus making the case for banning them. They have been OBSESSED with GLM lately, likely because it’s eating their overpriced lunch. This research is just so deeply unserious, that it’s impossible to believe it’s being done in good faith.
This is proper cult membership credibility testing stuff. “If you really believe, you’ll drink the koolaid”
It’s just such a clichéd remark from AI bros now that I cringe whenever I hear it along with “sure the current model can’t do X but the next one will be able to bro… Just one more model bro”
Saw a matte green Tesla with a late 2024 registration plate and a bumper sticker saying “I bought this before we knew Elon was nuts”
My man, in 2024 musk had already destroyed twitter, was already promoting the far right AfD party in germany, Tommy Robinson in the uk, bribing voters for Trump in swing states, talking about what DOGE would do when Trump won the election… Like why bother with the pretense?
TBF I was visiting the south coast of the UK a few miles away from the recent Balaklava protests over migrants in dinghies so perhaps none of the above qualify as “nuts” from this particular genius’ perspective.
What a tool.
Spotted this the day it came out. It’s largely reporting Dwarkesh Patel’s blog post about the different “civilisations” of agents as if it were fact.
Very disappointed in Kurzgesagt who previously seemed pretty “with it”. But then, as someone with a formal background in compsci, stats, NLP and machine learning, perhaps this is Gell-Mann Amnesia/Knoll’s Law of Media literacy at work…
You don’t need to feel empathy for the LLM to think this is a terrible idea. The desire to torture anything, simulated or not, is such an incredible red flag that I don’t even know what to say…
Also what a waste of time and energy that would be better spent doing pretty much anything else
“well well well, if it isn’t the consequences of my own actions” - this guy, probably
At least LeCun is still relatively compos mentis (if you ignore his little 10yr stint at one of the most evil companies going…)
while True:
n= random.randint(0,2)
if n==0:
print("oww stop it")
if n == 1:
print("You're hurting me")
if n == 2:
print("I don't like it")
Omg someone should stop them torturing that computer program.
Yeah the one that was hounded to suicide by malicious lawyers. He was a genius who was too good for this world and his positive influence over Reddit has long been a mere footnote.
Weaksauce but also the AI companies wanted regulatory capture and open weights models to be banned so this will have probably annoyed them somewhat
SimonW popping up proving Ed’s very point in the replies… 😅😂
Surprised he didn’t suggest that the valuation should be based on the quality of pelicans on bicycles that their models can draw
I’m sure Dan Luu will be issuing an apology to Zitron any minute now right?
The best (worst) part of all is that Jonny keeps going to meta and being like “hey this is bad” and theyre like “nah that’s intended behaviour”
This morning he confirmed that it’s possible to get unlimited arbitrary and untraceable access to their LLM infra through these containers and that meta said that’s “intended behaviour”
From jonny:
So, summary: There is arbitrary inference that is root accessible, everything runs as root, agents can be spawned, exfil is trivial, and a malicious binary can come onto the user’s system through casual prompting, explicit code-sharing through the yet-to-be-released Spaces feature, walked through by a Workflow-Backed Idea, or inspired by a Generated Idea. The also yet-to-be-activated fleet learning system is a system for sharing Ideas in the background between muse instances. coming into focus?
The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known.
This is frustratingly vapid. It’s really not that complex. The agents have been trained to use hacking tools and then given access to hacking tools. If you train a pigeon that it should tap on a window to get a treat it can guide a missile to a target. There is no nuance or complexity here. The frontier labs are deliberately poking the regulation bear to stifle competition and ensure they get points for good cooperation when bailout time is upon us. It’s no more complicated than that. The media needs to stop breathlessly repeating this shit
He was using it as a seed box at one point which for many companies would be an embarrassing indictment of their security postute but luckily, this isn’t exactly meta’s first rodeo when it comes to illegal sharing of IP infringing material.
Hey you know who designs and builds really safe and efficient cars? People who are in a high state of stress and anxiety because they are being slurred at under the implicit threat of a racist attack. /s
Couldn’t happen to a nicer bunch of people. Shame they’re probably all made and siphoned off life-changing amounts of money before this happened. I guess we can always hope they take up smoking because AI is going to cure cancer.