HPMOR does have one great lesson that all the rationalist and even Eliezer himself have failed to internalize: Fascists should be decapitated.

I can’t believe I’m saying this… I am one of those who actually fully read it, because:

Out of sheer morbid anthropological fascination, I infiltrated an in person HPMOR reading group (not in the Bay area) in the latter days of it coming out. I described it to friends as going undercover. It was a trip and a half.

Hot damn, quality sneer

Copied to the general HPMoR sneer thread top post.

That’s the bunch who signed a billion dollar contract to bullild a datacenter on a British scaffolding-storage lot they didnt own, isn’t it?

“YOU MUST TRY THE HOSTAGE SPECIAL!”

God those grifters are still trying to make that collection of two year old product demos A Thing? And entertainment industry boomers are still falling for it?

“r/Cantonese not impressed either.”

Theyre arguing about if it was actually any form of legible chinese at all, rather than translating it

Imagine the horror that ascended my spine when, to fulfill my moderator responsibilities, I read the reverse-chronological feed of new comments and saw that “greatest therapist in history” declaration, and then discovered who wrote it.

Well, I suppose I should thank Mr. Aaronson for conveying to us what it feels like to be a precisely targeted demographic

it’s solved Navier-Stokes

This phrasing is obvious nonsense, but surely Scott should know better??

Oh yeah, I solved the Navier-Stokes equations. Check it out: u = 0.

For anyone who says AI doom sounds like an apocalyptic religion, that the rationalists/Singulatarians seem like a Bay Area cult, that Eliezer Yudkowsky gives off the vibes of a messianic prophet: yes, yes, and yes. But crucially, today you’re no longer being asked to believe in arguments and extrapolations, but only in the front-page news. Accepting the reality of the coming machine god after it’s solved Navier-Stokes and dozens of other longstanding open math problems (while dramatically ramping up in capability every month), is sort of like accepting Jesus after he’s returned to earth on the gleaming cloud. It’s the epistemic bare minimum

So we’re going full mask off now. Awesome

On top of that, why the hell are you using ChatGPT as a therapist when you say that it’ll end the world???

I’ve been unloading every day to—who else?—GPT 5.6 Pro about all the pain and trauma and embarrassments of my past. It turns out that, where two years ago GPT was a passable therapist, now it’s the greatest therapist in history, at least for what I need.

Has he not heard about all the cases where LLMs have led people to delusion and sometimes severe self harm or suicide? And yes, each of them thought that ChatGPT really understood them. That’s what delusion is.

If you want to know my current take, you simply start with the one above, then update on the fact that the wild prophecies have come true. The first rumblings, I’d say, came a decade ago with AlphaGo, they got noticeably louder with LLMs and coding and reasoning agents, and they’ve accelerated this summer and fall into a crescendo of wonders and terrors that one needs to be a particular kind of idiot to deny.

The strange thing here that I did not expect is that somehow, all of these major achievements are only in proving mathematical theorems (and committing felonies hacking computer systems). The massive improvements are in such restricted domains that the AI labs haven’t even solved the mundane problem of making a profit. And no, any rumors of more mathematical theorems being solved doesn’t change that. (It’s almost like the progress is in areas where hallucinations aren’t a problem. Nobody cares about your failed attempts at felony hacking, and math is formally verifiable.)

My colleagues always point to coding when asked to name any other domain where AI has seen major success. But one of my friends works in software for a company far away from SF with no mandatory AI policy. The apocalypse has become so clear that when I asked him about all the doomsday talk about software engineering, he seemed a bit confused. After I asked him if AI has made software engineering obsolete, he said that this week he had an intern try to use AI to fix some code that wasn’t compiling and ended up with a giant mess with tons of files that appeared to compile but didn’t run (with errors filling several screens). Then I asked him, what about AI being indispensable in coding now? “Yeah, if you never learned coding because you’ve only ever used AI, then of course you need AI to code.” Some of his coworkers use it as a tool, others don’t, life has just kept on going for him.

Maybe my friend just doesn’t know all the right skills needed to use AI properly, even though the entire point of AI is that you can take skill out of the equation. Maybe he doesn’t know about all the CLAUDE md skill files and agentic looping and whatever nonsense that definitely fixes everything. Maybe he is just six months out of date, because that’s when all of the real progress happened.

Maybe he is just “a particular kind of idiot”.

(If you hate anecdotes, I made an actual argument here.)

What, no American Intelligence?

A comparative demands a comparison class, and the natural completion is “superior to us.” The President is supplying a foundation premise of many superintelligence arguments: how does the less intelligent party remain durably in charge of the superior one?

A strange question to ask given the existence of Donald Trump in the first place.

As of the time of posting, “Superior” has a narrow lead. I’ll be hoping it remains so, as it seems like the most positive-world-leaning outcome to me.

Is this some kind of strange 5D chess gambit that if Trump picks the name that pisses people off the most, then people will try the hardest to regulate it or something? Somehow assuming that people will take this Trump renaming stunt seriously after all the other Trump renaming stunts? Why do I even bother? Fuck. This is so stupid.

you have to be fucking kidding me

@mii @cstross I spent decades of my career haranguing management to spend an extra $500 on better computing gear to make their almost six-figure programmers more efficient. But now we’ve got management haranguing employees to use this ephemeral technology. I don’t get it.

“Hey we got you this thing to make you more productive and we’re gonna watch comparative performance and if you’re falling behind we’re gonna have words” would at least make business sense. This is mirror universe shit.

i’m so sorry.

@mii @BlueMonday1984 Can you use a LLM to write these justifications? I mean, they don’t have to be good, no one is reading them after all …

(No, I’m not suggesting to violate any principles of yours, of course)

I’m so sorry, Mii. I can only hope you find a job that actually respects you after this one, because I cannot see your current company recovering from this shit.

…I’m so sorry, and it’s good (???) to know my company still has a few steps on the AI delusion ladder to climb. I guess with the way it’s been going I need to mentally prepare for the same thing you’re going through to come in a few months.

That’s some bullshit. Much respect for pushing back!

I hope that we’re through the worst of it.

you can’t deny their foresight.

skill issue

Their ideas were compelling and you can’t deny their foresight

Ah yes, the AI did almost kill everyone with custom bacteria.

Least surprising news of the week

If MIRI was Sam Altman’s old stomping ground, and the AI industry is so evil he can’t work in it, isn’t this an argument that MIRI was either a failure or bad all along?

Plus Anthropic-the company he quit for being irresponsible-being heavily tied to EA/rationalism. Not exactly the best endorsement

These “specialized minds” can also make them off-putting to many people.

The eugenics endorsing also does a pretty good job with that

If MIRI was Sam Altman’s old stomping ground, and the AI industry is so evil he can’t work in it, isn’t this an argument that MIRI was either a failure or bad all along?

It is very funny how the rhetoric from the rationalist doomers has been one of the main forces propping up the AI industry. If they’re saying AI will be powerful enough to kill us all, they’re still saying that AI is powerful! They are so high on their own supply (and the fame and wealth from finally being in the spotlight) that they have not realized just how much they have undermined their own cause.

Then again, perhaps one could say that a doomsday prophet, deep down, actually wants doom to happen.

(Also, the paltry amount of technical work done by MIRI has zero relevance whatsoever to modern day AI.)

Is there a place to see this other than getting an account on that site?

Never forget Yud himself has said he would kill all humans if that was the only way to make nice superintelligence

The ‘sex cult’ framing really seems to be catching on. Of course, none of the eugenics comes up.

Keep spreading the good sneer, everyone! There are still horrors to detail!

ok last one but one big long bluesky thread about all this ripped from old SneerClub

HaJ3FgupAm8RrDJW3MHgT9X7Ft27eVaD

What is that? That’s a hard-coded API token that Flock cameras use to identify themselves and get OAuth credentials, which can then be used to talk to Flock’s production servers. Like, anyone on the internet can probably do this right now.

Mein Gott.

muwahaha

Although no specific errors in this story have so far been identified

Well here’s the very start of the retracted article:

On the screen, two side-by-side squares of color flicker rapidly

Except oops it was referring to a study that used groups of moving dots rather than “squares”. This can be seen in that studies first figure. Bam. A specific error in the first sentence…

(Am I being too pedantic?)

I’m curious about this one, legal repercussions against the perpetrator seem pretty clearly viable (it would seem to me that Science has a viable fraud case, and Williams for probably a few too). why wouldn’t anyone do that? is that not the done thing in academia?

the network effects of reputational damage are pretty oof though :|

this shit should be regulated into the dustbin of history. relatedly, the recent release of unredacted docs from the NYT vs OpenAI case showing the clear knowledge of theft and substitution… it’s all willingly done

@gerikson My assumption is that they want a regulatory moat to protect “frontier” models from “open weight” / AKA cheap models.

It’s just margin protection.

@gerikson @BlueMonday1984 Yeah, the whole idea is to perpetuate the myth that these companies are really close to coming out with a product that people will actually pay to use. A product that people will pay enough to cover its cost. They want to simultaneously make an excuse for why it is taking so long while making gormless investors think it is just around the corner.

The symbiosis between the rabid cultists, the technofedual wannabe-lords, and the get rich quick grifters is so weird to watch.

@gerikson @BlueMonday1984
Also they dare not actually let the market price their shares.

Also the frontier “labs” are rapidly running out of money and having to raise more and more insane investment rounds. This is a great way to save face and back away from burning $$$$ on training runs that push the frontier of can a model rearrange some arbitrary coloured squares by 10% next they just need to solve the problem of how to make their customers pay cost+ on inference when they’ve been giving it away at steep discounts for nearly 4 years… Easy!

@gerikson @cstross no obvious way to fix that, (and previous attempts were expensive)

deleted by creator

I find the techno-determinism specifically fascinating because it’s so self-perpetuating. Like, everyone believes they can do a better job than everyone else at building the right robot God, and every person working on it is, according to their own beliefs, accelerating the problem they’re worrying about. It speaks to how aggressively they see themselves as special, like they’re protagonists and everyone else is NPCs. I can’t think of any other way to justify the pattern of “this is a terrible idea that will kill us all except when I personally do it.”

Which XKCD comic is it that has the subway full of people pitying everyone else for being mindless sheep? Mix that with the Arrested Development gif of Tobias contemplating an open marriage.

#3 suggests that people in tech have gotten way wayyy too comfortable with saying “hey, I know that company’s evil, but it’s a really good job” about companies that make software for arms manufacturers or ICE or surveillance, so of course people who think AI might kill us all think it’s ok to work for AI companies even if it might cause an extermination.

All 3 of these reasons make them monsters. 1 and 2 are saviour complex bullshit and 3 is just pure greed and selfishness

Ya see, we’re being charitable when we call them grifters instead of genocidaires.

What the actual fuck? Imagine if Hermann Göring defended himself at Nuremberg by saying, “At least I had fun and got rich and worked on ‘cool tech’ with friends.”

Once the rockets are up, who cares where they come down, that’s not my department says Wernher von Braun.

(complete with a reference to “EV” (expected value))

In this day and age, if I see someone talking about major life decisions in terms of +/-EV, I turn on my heel and walk away rapidly. It was kind of cute when this jargon was entering wide use during the online poker bubble, but that was 20 years ago.

What the actual fuck?

This third category is the Patrick Bateman types who would have mainly been confined to finance at the turn of the century. They probably picked San Francisco over New York in part because they were worried the New Yorkers might make them wear a suit.

There are two specific aspects of Rationalism that cause this mysterious blind spot. First, they believe that superintelligence is inevitable, which conveniently removes responsibility from the people actively trying to bring it about. Second, they treat everyone outside the Rationalist sphere as NPCs who dutifully follow the directions on their script.

I’ve been thinking more about this lately, and my metamodel of the rationalist world model (if I may get away with such a pretense) is that they subtly believe that the future is already something crystallized, even if they don’t consciously realize they hold this belief. It is not really a probabilistic philosophy, no matter how much they assert that. Intelligence, to them, is a function that can more and more precisely apprehend the future as it escalates to greater values along the g-scale, and the singularity will manifest an intelligence that can do so with practically absolute precision (hence its prophesied ability to outcompete and replace humans).

When I’m talking to people who haven’t encountered this stuff as closely, I half-jokingly describe the reflexive assignment of probability estimates as a devotional practice akin to prayer or meditation… but it really is! It’s a subtle invocation of the AGI/ASI god, in that I make this imprecise estimate now, to demonstrate my belief in a precisely quantifiable future; but come the singularity, God will relieve my feeble human brain from having to commit this lamentable imprecision. In that reckoning, those who are not even willing to venture an imprecise esimate must have no intelligence worth noting.

Reading this, I get the feeling writers are gonna get a lot of mileage out of the rats and their ideology.

These people have spent years attempting to create something they believe represents an existential threat to humanity (causing untold real-world harm in the process), and they’ve remained fully convinced that they’re the good guys throughout every single step of this.

At what point does doing something you think will result in the end of the world become attempted murder?