AI Safety Is Mostly A Sex Cult In Berkeley, California

These people shouldn’t make policy.

There has been a great effort over many years to distance AI Safety discussion from its origin and intellectual center, because its center is a sex cult started by a fan fiction author. Since those involved want to be taken seriously, they have to hide that. Everyone else working in AI Safety is, in effect, a smokescreen for this core group and their ideas.

We might some day have “AI Safety”, as a science or policy area, that is not primarily a sex cult in Berkeley, California, but it does not currently exist.

Who Is Eliezer Yudkowsky?

This is Eliezer Yudkowsky. He is seen here in a meeting with Bernie Sanders, who he is advising on AI policy, and in The AI Doc, now on Netflix.

This is Yudkowsky in Time Magazine in 2023, advocating in favor of risking nuclear war to prevent AI research from happening anywhere on Earth.

This is very senior OpenAI employee and unofficial OpenAI spokesman Roon. Roon’s publicly stated opinion about Yudkowsky is:

1. Yudkowsky is in and/or running what Roon describes as a sex cult.

2. Yudkowsky’s opinions about AI are important, and Roon says he takes the things he says quite seriously.

This seems to be an extremely common pair of opinions concerning Yudkowsky.

Yudkowsky is sufficiently controversial that people motivated by his ideas often downplay or deny his influence. Sometimes this spans multiple layers: People who received money from Sam Bankman-Fried denying they’re Effective Altruists, Sam Bankman-Fried denying he’s an Effective Altruist when caught, Effective Altruists denying their version of AI Safety is heavily influenced by Yudkowsky’s, people who are extremely core members of the movement often call themselves “rationalist adjacent”, and so on. But Yudkowsky is provably, publicly influential on essentially all major public figures in AI. It is easy to deny not because it is hidden but because it seems so insane.

Yudkowsky is the person who connected Shane Legg and Demis Hassabis to Peter Thiel so that they could secure funding to start DeepMind¹, which is now Google's AI research division. Here's Sam Altman's opinion of Yudkowsky.

Left, we have a schedule for the Future of Life Institute’s January 2015 AI conference in Puerto Rico, which both Elon Musk and Yudkowsky attended. Right, we have a separate, explicitly Effective Altruist event: EA Global, held in Mountain View that August, where Musk and Yudkowsky both also appeared.² Musk would cofound OpenAI that December. Given these events and Sam Altman’s direct credit, Yudkowsky and Effective Altruism were certainly adjacent to and probably did play a critical role in the decision to start OpenAI.

On the right, in order, Daniel Dewey (MIRI), Nick Bostrom (Oxford Future of Humanity Institute), Elon Musk (self-explanatory), Nate Soares (MIRI), Stuart Russell (Quite Good At AI Research, otherwise uninvolved)

Dario Amodei, current CEO of Anthropic, is here in 2013 identified as a "member of the effective altruism community" on the blog of Yudkowsky's non-profit. He appears along with Holden Karnofsky, also identified as an Effective Altruist.

And here Dario is arguing with Yudkowsky about the paperclip maximizer, a now-classic thought experiment, originally Yudkowsky’s, about an AI that might kill everyone.

“AI Alignment” and “AI Safety” were first popularized by Yudkowsky’s non-profit, MIRI, the Machine Intelligence Research Institute. OpenAI and Anthropic have cited Yudkowsky or his non-profit at least a dozen times³, and the entire way we talk about AI today uses terminology his non-profit popularized. It is effectively impossible to even discuss these areas without, implicitly or explicitly, performing commentary on Yudkowsky’s writing.

This is Jacob Coxon, the recent “whistleblower” who quit Anthropic, being asked his opinions on rationalists and effective altruists.

This is Nate Silver citing LessWrong as a source of information about AI risk. LessWrong is a website started by Yudkowsky in 2009, and was originally filled only with Yudkowsky’s writing.

This is Oliver Habryka, who is the executive director of the nonprofit controlling LessWrong and the rationalists’ compound, known as Lighthaven. He is here explaining how large he thinks the influence of rationalism is on AI Safety and AI existential risk discourse. He appears to be engaged in some kind of internal territorial dispute. It is possibly the only time I have ever agreed with him.

This is a podcast interview with Daniel Alejandro Moreno-Gama, who attempted to firebomb Sam Altman’s house, in which he names Yudkowsky as his primary influence.

Who Are The Rationalists?

Yudkowsky’s more dedicated followers seem to prefer to call themselves “the rationality community” these days, everyone else calls them “rationalists”, the entire following “rationalism”. “Rats” is also common.⁴

This is a long denial that they are a religion. I alternate between finding these denials funny and finding them insulting.

The fan fiction that launched a thousand cults is Harry Potter and the Methods of Rationality, or “HPMOR”. It is possibly the highest-impact single modern religious text, rivalled only by the Book of Mormon. Its influence may not yet have peaked. It is the holy book of the first great Internet cult.

Harry Potter and the Methods of Rationality

How Are Rationalists Connected To The Bay Area Effective Altruists?

Bay Area Effective Altruism meetup’s banner photo. Can you spot Yudkowsky? How about Caroline Ellison?

Allegedly, Effective Altruism and its AI safety efforts are a separate thing and not directly beholden to Yudkowsky. This is very difficult to believe when familiar with the facts. Effective Altruism in the Bay Area has almost perfect overlap with rationalism, and the Effective Altruist AI Safety people in the Bay set the agenda for the rest of Effective Altruism’s safety efforts.

In the Bay Area, Rationalism and Effective Altruism appear to be the same people, at the same events, in the same places, who believe the things said by the same authors and leaders, with only very slight differences between spheres of influence. Yudkowsky called himself both an effective altruist (as late as 2022)⁵ and a rationalist until a peculiar internal schism. Scott Alexander, the second-most influential writer in the space, has usually called himself both. Often, members of these communities seem to show up in groups working for the same companies and non-profits.

Frequently, Effective Altruists bring up all of Effective Altruism’s other charitable activities and its charming, benevolent members who don’t live in the Bay Area to deflect true criticisms of the Bay Area Effective Altruist community. This effectively makes their other charitable operations and workers shields and fronts for the priorities and behavior of the Effective Altruists in the Bay Area, who are interested almost exclusively in AI.

Where there is any separation between the two things in the Bay, “rationalism” is people and things more closely tied to Yudkowsky, and “effective altruism” has had a few institutional centers but is currently focused around Anthropic and its CEO, Dario Amodei. They compete for attention and funding in that space. Yudkowsky’s position is that AI will definitely kill us, and Dario’s position is that AI is ten to twenty-five percent likely to kill us.

Both are saying what justifies giving them and no one else money, labor and attention. Since Yudkowsky is a writer and not a scientist of any kind, he can only claim that he and his organization are important and deserve money and prestige if AI Safety is an unsolvable problem and only advocating against AI matters. Since Dario leads a company that actually makes AI, Dario must claim that “AI doom” is a very large problem that only his company is able to manage.

Yudkowsky, similarly, claimed that his (non-profit) company was the best chance at “saving the human species”, back when AI companies seldom made anything and he could claim to be doing research by simply writing.⁶ Yudkowsky’s belief system is a successful pitch for investors, donors, volunteers and employees. It is also, politically and doctrinally, a pitch for people to give you total control over them, because if they do not, the world might end. After Yudkowsky refined this pitch modifications of it became the pitch for organizations like OpenAI and Anthropic⁷. This millenarian ideology appears to have become integral to employee morale and recruiting because so many research staff are believers to one degree or another.

Yudkowsky is not going to stop preaching the gospel of doom because, among other reasons, doing so is the only job he has ever had. He would stand to lose his social position and the roughly $600,000 in income he is paid out of his non-profit if he changed his tune. In general, if doom prophecies are worth money, people will produce more. This system’s purpose is to attract funding and young idealists and to find ways to spend the funding and make use of the young idealists. Historically, rationalists have seemed to encourage many of their young idealists to take up sex work.⁸

People tend to follow their incentives.

From EA To Anthropic, To Policy

People sometimes deny that Anthropic is an Effective Altruist organization. This may be technically true. Anthropic, the company, can say it is or isn’t whatever it wants to. Anthropic’s founders, senior employees, and investors, however, have numerous ties with Effective Altruism.⁹

Dario in particular is almost perfectly on-type for a Bay Area Effective Altruist in what he says and how he says it. Dario’s often bizarre, vague, and meandering writing style alone has no real precedent besides LessWrong and its AI-focused spinoff, Alignment Forum. He has been directly involved in Yudkowsky’s community meetups since at least 2008.¹⁰ Anthropic’s strategy is very similar to the playbook Yudkowsky laid out, back before the success of deep learning and companies that specialize in deep learning made Yudkowsky’s organization no longer credible as a serious research project.

This same denial of affiliation happens when it comes to the incestuous relationships between all of the other non-profits in the Bay that are meant to be focused on AI risk. They virtually all draw support from a handful of sources of funding and they recruit from the movement. Holden Karnofsky, currently Anthropic, formerly head of Coefficient Giving, was an original board member of Redwood Research and one of the initial champions for METR. Both Redwood and METR hold themselves up as independent organizations. They may be formally independent, but they are aimed at the same goals as the rest of the movement and are broadly ideologically compliant.

Redwood Research in particular seems to be trying to catch the same lightning in a bottle that Yudkowsky had with MIRI while avoiding the sloppiness of his presentation or tying the whole thing to his specific social life. Redwood seems to be laundering essentially the same world view into policy papers that can be cited and pointed to as having come from a serious organization. Their CEO is a former MIRI staffer¹¹, and when they first launched they had a glowing quote from Nate Soares, who has been either Executive Director or President of MIRI since 2015, on their site.¹²

The people in the video with Bernie Sanders are:

• Eliezer Yudkowsky, MIRI

• Nate Soares, MIRI

• Jeffrey Ladish (funded by Coefficient Giving and Survival and Flourishing Fund)

• Daniel Kokotajlo (funded by Survival and Flourishing Fund)

So when they go to get people to talk to a senator we have Yudkowsky, Yudkowsky’s top lieutenant, and two people whose paychecks are from Effective Altruist charities. We will be seeing all of them again later.

Former Biden White House officials Bruce Reed and Ben Buchanan recently wrote an op-ed in which they deny having said that they wanted to ban open-source AI. They claim the first thing they were told in that meeting, by Marc Andreessen, was to watch out for “the sex cult that wants to run America’s AI policy.”

They then deny being Effective Altruists, as follows:

First, Andreessen warned us to watch out for “the sex cult that wants to run America’s AI policy.” This appears to be a reference to instances of polyamory in the effective altruism movement, many of whose adherents believe AI poses existential risks to humanity. Effective altruism is a philosophy we and the Biden administration did not share.

Ben Buchanan worked for two years as Director of the CyberAI Project at Georgetown University’s Center for Security and Emerging Technology (CSET), which appears to have received roughly 90% of its funding from Coefficient Giving (then Open Philanthropy). Since May 2025, four months after the end of the Biden administration, he has been working as a consultant for Anthropic.

Buchanan has a book, published March 2022, in which he discusses the possibility of export controlling GPUs. In October 2022, the Biden administration put in place export restrictions for GPUs.

I read Buchanan’s book¹³, and I think he is probably not an Effective Altruist because his ideas are not weird enough; but since 2019, his salary has come either from Effective Altruist groups or from the White House. I am reasonably sure he comes by his opinions honestly, and believes in them for normal policy advisor reasons. If he changes his opinions his career is probably in trouble.

In spite of being considered embarrassing by more professional Effective Altruists and now being somewhat starved for cash, Yudkowsky and the organizations most closely tied to him are still, by far, the most influential sources of thinking about AI safety in the Bay and therefore on Earth. Because so many investors and key staff members hold him and his ideas in high regard, it is not clear that major AI companies can directly disown him, even if they want to. Dario, in particular, seems in his public statements to be indirectly trying to carve influence away from Yudkowsky’s faction when he says to be less religious or less obsessed with “doom”. The fact that Dario has to say it indirectly is a hint that saying it directly might cost Dario dearly in those communities, and that he’s aware of that fact.

Because the Bay Area Rationalists and Effective Altruists are so influential in the AI industry, people who are not insiders to this subculture cannot meaningfully influence the research or policy agenda of that industry. This includes many of the people doing actual research. Some current AI Safety research may produce useful knowledge for the future, but its main effect today is to lend scientific legitimacy to people who are not at all doing science, but who are asserting a very particular religious and political agenda.

Paperclips, Doom

Yudkowsky’s most viral idea, the one that infects the most people and spreads the farthest, is that AI, if we continue to develop it, will

1. certainly recursively self-improve (“RSI”), quickly achieving superintelligence (“ASI”),

2. certainly escape human control,

3. certainly kill everyone (“paperclip them”, “doom”).

All arguments about AI Safety and AI Alignment are primarily about mitigating this exact scenario under his exact terms. Significant modification is relatively rare. Where lower probabilities of doom than “certain” are given, people seem to still generally worry exclusively about Yudkowsky’s rough scenario: a sudden, unpredictable sequence of 1) superintelligence, 2) immediate escape from control, and 3) the death of all humans on Earth. This is always, implicitly, what "safety" and "alignment" are about.

These ideas occasionally veer into the belief that AI is magic. When presented as serious science and policy, this sort of thing is omitted.

Ordinarily, the play here would be to disown this and other concrete statements of his about AI doom as irrelevant. Of course we aren’t just repeating Yudkowsky! We are smart, normal people who believe normal things for normal reasons. Compare it, however, to AI 2027, allegedly a serious policy document, written by Daniel Kokotajlo and Scott Alexander, presenting its doom scenario.

This is a genre of science fiction that has a doomsday cult built around it. There are only slight modifications to the basic belief system. Somehow, maybe by just throwing money at it, taking these ideas seriously has been rendered prestigious or at least not too humiliating, such that Kokotajlo, credited on the above, gets to advise Bernie Sanders. JD Vance was asked about this piece by Ross Douthat at the New York Times, and said he had read it. (In my opinion, on close reading AI 2027’s primary intended audience was clearly JD Vance.)

Essentially all of AI safety is rationalizations over Eliezer Yudkowsky’s ideas and there is no other substance to it. It is smokescreens, and Eliezer Yudkowsky.

This is his actual plan: to halt all AI research, all across the Earth, and instead do genetic engineering eugenics for a generation. Pause AI, stop AI, slowdown, pacing the frontier, they are all smokescreens, rationalizations, or ways of watering down this plan. This is why there is never any real “after we pause”, or “we will unpause when”; this is the only actual plan, and the public front for the plan cannot say this because it is too stupid and too offensive to be spoken in public, except by Yudkowsky himself, who never stops talking and who is too ridiculous for anyone on the outside of Effective Altruism to believe that everyone listens to him.

Anyone who thinks this is not the plan is not in on the plan, or is part of some faction that simply has no real plan. If this describes you and you didn’t know that, I am sorry that this is how you found out.

He has long since deleted this post demonstrating what his idea of an acceptable outcome is; this is ordinarily implied and not stated.

This industry approaches regulation in this way because it has, at its core, a millenarian death cult that worships and fears AI, believes AI will end the world, feels compelled to create AI, and at least somewhat wants to destroy AI as an entire field.

I will repeat: I do not think that these people should be making policy. I would vastly prefer if this were merely some insane Bay Area drama that I could ignore, but the still-growing success and importance of the companies founded by or employing this group of people has made this a very public national and international concern.

The paperclip maximizer is a thought experiment from 2003 and of very little relevance at all. People are perpetually striving to find evidence for this theory because they already believe it is true, and take pains to interpret any research or security development in AI as evidence for this exact doom coming to pass. As things happen which contradict this scenario, as often they do, they are often not understood because they do not match the prophecy. Alternative explanations are so unthinkable that many people cannot even be told.

Doom seems to be the most infectious idea here because it activates the same neurosis that every apocalyptic or millenarian cult does. The world is going to end, and only I and my special group of people in the know can maybe stop it.

You can argue with them in any way you like but it barely matters. It's not, mostly, something people believe for good reasons, and they will simply go back to complaining about the apocalypse as soon as you stop paying attention. There are enough of them that the belief is self-reinforcing; they can arrange their lives such that they essentially never talk to anyone who is not a believer.

Harry Potter fan fiction, of course, is especially popular with children, and many people have made this their belief system since they were teenagers.

Since recruitment largely happens on the internet, people are widely dispersed. Where they are gathered in person, it is mostly in various communes in Berkeley, California. They call their communes "group houses", maybe to make it sound less cultish. Most of the details of this are what you expect from "a commune in Berkeley".

Everyone attends the same events, everyone lives in the same group homes, everyone venerates the same few authors, everyone is very worried about AI doom, polyamory seems to be extremely popular, and more than a few appear to be on psychedelic drugs. They are all very contrarian together, and they are usually contrarian about the exact same things.

Lots and lots of them, currently, work at AI companies and non-profits. The safety, red team and alignment teams seem especially afflicted, because that's the area you go into to worry about and try to work against AI doom, which you are sure is at least medium likely. Other members will seek work in finance, because Effective Altruism, by creed, says you should seek more money so you can give more to charity. Those most concerned with AI risk will tend to almost exclusively give to AI charities and not any other Effective Altruist concern.

As a whole, this is a religious movement. Not everyone takes marching orders; they’re united by an idea, not any particular structure. The vast network of Effective Altruists around the world working on other charitable concerns is basically irrelevant to AI Safety. There are core people who mostly live and attend events together, and there are many many more who are less directly involved, who are part of the wider movement, and who are primarily its recruiting funnel, funding, and labor base.

Any time you get more than a few people who are deeply concerned with AI doom together, it seems like their natural form of organization looks like a cult; the movement as a whole, internationally, is less of a cult than it is a cult factory. Some of these cults have gone very badly. One of them is the Zizians.

That rationalists form cults constantly has been previously examined. There are many causative factors, but the core one is just that the entire thing is, in and of itself, a millenarian doomsday cult. If you believe you’re saving the world, it justifies anything; you cannot go against the cause for any reason. This gives leaders, whether official or unofficial, immense power over their members.

Forming a separate cult is also how you carve off influence and ultimately power for yourself, separate from the wider movement. This tends to mean that people try to do it more. All of these power struggles make the details of all of this extremely noisy, chaotic, and difficult to explain.

When you cut through all the distractions, chaos, and power struggles, San Francisco Rationalism is effectively the inner circle of San Francisco Effective Altruism, and San Francisco Effective Altruism is effectively the inner circle of Effective Altruism globally. Power and especially intellectual leadership flow almost entirely outwards from SF Rationalism to SF Effective Altruism to Effective Altruism’s global AI safety efforts. Everything else about Effective Altruism is effectively a front.

All of this has been an open secret for years. Everyone knows. Everyone knows if they say anything it will be ignored. I am shocked that I have drawn as much attention to the problem as I have; so let me try to get everything that seems to be very important into the record.

A Brief History of Rationalism

What I am hoping to convey is that Rationalism and Effective Altruism are not many small things, they are one big thing, tied together through ideology, funding, and key people.

Bay Area rationalists and effective altruists do not document their own history because it is disgraceful. Only their enemies deliberately keep records of what they have done and are doing.

They do write constantly, but they do this mostly as a social activity. They have been writing essays, chattering, bragging, or engaging in some kind of drama constantly for the last fifteen years.

So you really just have to read a lot to put this together. Everything I am saying, to the best of my knowledge, is true, and I believe the vast majority can be established entirely by reviewing the public record. I am just connecting the dots by putting one straight line through them.

90s-00s: Usenet, Transhumanism, Extropianism

In 1996, a 17-year-old Eliezer Yudkowsky joins Usenet and the Extropians mailing list. His more notable influences are Drexler’s book on nanotechnology, Kurzweil’s book on AI, and Nick Bostrom, who frequents the Extropians mailing list. (Usenet was still largely full of academics). In 1998, Nick Bostrom argues that a superintelligent AI might not necessarily be good or friendly, which, along with many of Bostrom’s other ideas, will later become core to Yudkowsky.

In the year 2000, Yudkowsky founds the Singularity Institute for Artificial Intelligence, SIAI, later renamed to the Machine Intelligence Research Institute, MIRI. This appears to have started as a fake-it-til-you-make-it sort of thing, and it doesn’t pick up until he gets sponsored by Peter Thiel. Peter Thiel will be his largest sponsor until about 2015 or 2016. Now in 2026, Thiel has long since cut all ties, called this scene a “burning man camp” and “Luddite”, and called Yudkowsky the Antichrist.¹⁴

Jeffrey Epstein gave them fifty thousand dollars in 2009. One of MIRI’s former associates, Ben Goertzel, appears to have solicited a separate $2,000 donation from Epstein that went in under his name in that year, also. (Confirmed separately on his blog). One of the former presidents of MIRI is mentioned in Jeffrey Epstein’s emails¹⁵, which pass on thanks for a visit to the island. (MIRI is documented as turning down his money in 2016 when he was much more notorious, if you want to figure out what to make of their statements about that.)

I considered putting all this in a footnote because of how unimportant Jeffrey Epstein’s appearance here is in the context of everything else. He is the Forrest Gump of evil shit. Of course he was here.

2009-2015: Writing, Community-Building

Eliezer Yudkowsky spends some time before 2009 blogging, and then contracts out setting up LessWrong as a customized version of Reddit’s code. In 2009, he bundles his last year or two of blog posts and puts them all on the newly made LessWrong, where they come to be called The Sequences. Much later, in 2015, they are published as Rationality: From AI to Zombies. This is a series of essays about science and epistemology, most of them pretty closely related to books and ideas then in the zeitgeist but with both a very distinctly transhumanist and distinctly AI Doom thesis. (So, “conquer death with technology” and “if we do technology wrong, we are all going to die”.) It is written in such a way that it also works as, effectively, a self-help book about how to be better at thinking. This will later be something like the Rationalist Old Testament.

From 2010 to 2015, he writes HPMOR, and his audience becomes much larger, and this becomes a well-defined scene and people invested in the movement begin relocating to the Bay Area.

In 2010, Roko Mijic invents Roko’s Basilisk, which is most easily explained as the invention of Rationalist Pascal’s Wager, Rationalist Satan and Rationalist Hell, where those who fail to evangelize and help bring about the eschaton are sent. This will seem, over time, to drive the rationalists who are the most susceptible to religious mania and obsession insane.¹⁶ Yudkowsky disowns the Basilisk immediately and tries to ban discussion of it from LessWrong, which makes it more popular.

At least as late as 2013, Yudkowsky is still advocating that we should make AI as long as we could do it safely.¹⁷ He is under the impression that his non-profit, MIRI, might be able to make “Friendly AI”, that is, one that does not kill everyone, ushers in the singularity and makes them immortal.

The Center for Applied Rationality, or CFAR, is founded in 2011 and offers classes. You essentially pay money for self-help, but it is primarily a recruiting front for MIRI, Yudkowsky’s actual operation. It also seems to function as a more-official avenue for community management, and its newsletters describe it coordinating co-housing meetings and mediating alumni disputes.

They start a bunch of children’s camps, math camps, and study programs that are mostly still around. SPARC was the earliest, beginning in 2012, and takes high school students. All of its graduates from then will be adults now. Probably a bunch of them live in the Bay and work on AI safety.

Relocating to the Bay

Moving to the Bay Area worked for some people extremely well, and for others extremely badly. Much of this happened bottom-up, with members organizing things themselves, but in some specific cases you can tell who exactly is responsible for what.

I think of this as a flywheel, where the energy that keeps it moving is, fundamentally, their talent and their raw zeal for the cause. People show up, and they believe in what they’re doing, and they give what they can.

It is the best professional network on Earth. It is like being in the most elite fraternity, if it were unisex and had many chapters in San Francisco specifically. If you have skills or can acquire them, you can have a fantastic job very quickly. There are a good number of coding bootcamps that will train you for a job and then place you in one with no money up front.¹⁸ Since Peter Thiel is most of their funding at the time, many of the connections are to companies he is invested in, but they have connections to nearly every company in Silicon Valley.

Those people offer their work or their salaries to the cause.

Next tier down are people who can’t or don’t really use the opportunity that much, but who either end up doing something else or who end up doing work at the various rationalist non-profits. These people are fine, probably; CFAR probably takes some of their money from them and they maybe could be doing better if they hadn’t spontaneously relocated to an extremely expensive city. Hopefully they really enjoy the scene.

Your bottom tier are people who are very young, are not particularly skilled, who do not fit in enough, or who otherwise do not thrive in the Bay’s job and housing market. Many of them are runaways, even those that aren’t are often still teenagers, many of them have mental health problems. Younger members who show up in person are especially likely to be trans, at least in part because trans teenagers are prone to getting thrown out of their houses. Generally, the younger and more idealistic you are when you land in the Bay, the less likely you are to thrive.¹⁹ They may be employed as some kind of household help or as assistants for other people in the community.

All of this comes down to pulling more energy into the wheel, and you can trade each thing off against the other.

Any individual person or organization is really just bleeding energy off of the wheel. OpenAI, Anthropic, God knows how many smaller companies? They’re just pulling energy from this mass of eager, smart people that are already there. Successful ventures inject more funding and provide more connections to the movement, powering another turn of the wheel.

Recognizable cults, in contrast, are more like pure parasites.

2015-2018

This period sees Yudkowsky finish HPMOR and stop publishing things people actually want to read, which leaves the community without a focus. As the most active and well-liked author still in the space, Scott Alexander becomes something like the Second Prophet of Rationalism. (People say these things as jokes sometimes, but this is pretty much accurate.) It seems to be much less focused leadership, though, and to some degree things start to drift.

Also in 2015, OpenAI is founded. This makes Google very sad because OpenAI exists to compete with Google on AI, and Yudkowsky knows and likes the Google guys, and he pronounces that the world is for sure going to end now²⁰, which he has continued doing for the last eleven years.

Oh, just one more thing. A former MIRI employee puts up a web site in late 2014 alleging that MIRI does no meaningful research and is primarily busy with legal damage control for distasteful and unlawful conduct, which it details in some depth. MIRI will ultimately pay him something like $30,000 to go away and take down his site with those allegations.²¹

Rolling Stone, in 2025, pokes into this when writing a story about the Zizians.

The first set of allegations have been investigated and deemed baseless. They stem from a report filed by a onetime MIRI employee named Louie Helm who claimed a teenager was “being exploited” by a group of men, including Yudkowsky, who ran a local youth camp — referring to the Summer Program for Applied Rationality and Cognition, hosted by CFAR.

According to the sealed police report obtained by Rolling Stone, Helm accused Yudkowsky of having a sexual relationship with a then-18-year-old that began at least two and half years prior.

Officers were dispatched to Yudkowsky’s home, which, at the time, he shared with the 18-year-old in question. The “alleged victim,” officers wrote in the report, stressed that they “lived in a happy and safe environment.” That person later transitioned; today, he lives with his wife, whom he met in Berkeley’s Rationalist scene, and their two young children. When I spoke to him earlier this year, he told me he rejects the notion that he was exploited by anyone other than Helm, the disgruntled employee, whom he felt used him as ammunition against Yudkowsky and MIRI. “I was expedient,” he says.

Yudkowsky posted about 2000 words on twitter denying the allegation ahead of the Rolling Stone story coming out, and it seems to have blown over. Effective Altruists and Rationalists, like squid, spray ink when threatened.

Possibly because it needs someone more comfortable handling delicate and complex legal matters, in 2015 MIRI changes leadership. Nate Soares becomes Executive Director of MIRI; his title has since shifted to President, but under one title or another he seems to have been Yudkowsky’s right hand ever since.

Brent Dill

I have no firsthand knowledge of this and know zero people involved so far as I am aware. I can only piece together what happened here from public records.²²

Brent Dill was kicked out of the rationalist community for allegedly being very abusive to several girlfriends, and thereafter left San Francisco, apparently for lack of available housing.

There are several key details that made the allegations especially grim:

1. He was significantly older.

2. He allegedly exploited the victims’ financial and housing situations.

3. He allegedly used BDSM concepts to justify doing things that are well over the line into abuse.

4. It was alleged that he was using MDMA to control his partners. MDMA is sometimes used for treating PTSD, and its use for this purpose was not unknown among the rationalists.

5. To quote CFAR: “Of the interactions CFAR had with Brent, we consider the decision to let him assist at ESPR—a program we helped run for high school students—to have been particularly unwise.”

6. The issue was previously brought to CFAR’s Alumni Community Disputes Council, which initially did nothing about it, and which was disbanded in the aftermath of these events.

CFAR ultimately puts out this much more detailed apology, which few seem to have found very credible.

In general the community takes this pretty hard, and to its credit largely seems to blame CFAR. Between this and other scandals, demand for classes at CFAR appears to fall, and it stops offering classes in 2020. CFAR started again November 2025, and you can currently sign up for a four-day applied rationality workshop if you can get past the waitlist.

Brent Dill’s ouster also seems to have played a part in convincing a rationalist named Ziz, for whom the Zizians were named, that the earlier allegations from Louie Helm were true. She was already mostly an outcast from the rationalists, and had effectively thrown her career away to work for people who did not want her.

She did what any deeply devoted rationalist would about this, which is to blog a lot about a method of becoming some kind of rationalist ubermensch and then go completely insane. About a year later is the first major Zizian incident, where a number of Zizians are arrested while protesting a CFAR event. Another rationalist cult called Leverage implodes in 2019, which I mention because it was a big deal at the time and because the article is a fascinating read.

Lighthaven

Lighthaven used to be the Rose Garden Inn in Berkeley. It has four main buildings, abundant outdoor space, and it’s a beautiful place.

Lighthaven was originally purchased with money wired to CFAR from Sam Bankman-Fried’s company, FTX, as it was collapsing.²³ This was later found to be fraudulent: Bankman-Fried and his employees looted the company because they knew it was dying and sent as much cash as they could to their friends and to causes they believed in.

One of those causes was that the Bay Area Rationalist and Effective Altruist community should have its own permanent conference center. Some, but not all, of the money that FTX sent to buy Lighthaven was later returned.

I am expected to believe that Sam Bankman-Fried was not a true believer. I am expected to believe that he was not central to Effective Altruism. I am expected to believe that committing ten billion dollars of fraud was not in accordance with the values of Effective Altruism.

This is all bullshit. He was giving what he could to the cause. Now he is being a good soldier and disowning the cause to protect it, and he is doing his twenty-five years. I respect it. It is the only respectable thing about him.

The part of the FTX empire that handled its weird charitable donations was called FTX Future Fund. Donations to CFAR would naturally have been within this organization’s responsibilities. These donations were made between March 3, 2022 and Oct 3, 2022. The entire team resigned on November 10th, professing to be shocked and immensely saddened to learn of then-recent events involving the collapse of the FTX exchange.

These individuals are:

• Nick Beckstead (Currently Secure AI Project, CEO and co-founder. Coefficient Giving funded.)

• Leopold Aschenbrenner (Currently Situational Awareness hedge fund, rumored to be under investigation by the SEC)

• Avital Balwit (Currently Anthropic, Chief of Staff)

• Ketan Ramakrishnan (I Have No Idea Who This Is)

• Will MacAskill (He’s William Fucking MacAskill, one of the initial founders of Effective Altruism, a cofounder of Giving What We Can, the Centre for Effective Altruism and 80,000 Hours)

The rationalists didn’t lose the building when the FTX money was clawed back from them because Jaan Tallinn, the Skype billionaire, let them take a loan from him against his future donations.²⁴

In 2022 they spun out a separate organization from CFAR called Lightcone Infrastructure, which has exactly two responsibilities: managing Lighthaven, and running LessWrong. As far as Rationalist and Effective Altruist are ever separate things in the Bay, Lighthaven is Rationalist.

Lighthaven’s initial funding, however, was Effective Altruist, and most of the events held there are Effective Altruist.

Here is a photo from Lighthaven’s page, of the outdoor area outside of a building called Darwin Den:

This is from the careers page of METR:

Here is the listing of Lightcone’s employees, officers, and directors from their 2025 tax filing:

Notice Daniel Kokotajlo, one of Bernie’s recent advisors. Remember Ronny Fernandez, whom we will come back to later.

Here are the public events listed on Lighthaven’s calendar in the last year. Virtually all of them are Effective Altruist events funded by Effective Altruist non-profits and donors.

• EAGxBerkeley

• Reproductive Frontiers Summit

• MATS program

• The Curve²⁵²⁶

• ILIAD Conference

• Eleos Conference

• Manifest

• Progress Conference

• The Delight Nexus

• Memoria

• LBF Microsurgery

• FLF Demo Day

• Inkhaven Residency

• Solstice Season

• Solstice Saturday

• Vision Weekend USA

• Trading Bootcamp

• PYOL

• Vitalist Bay

• MC0001

• LessOnline

• Summer Camp

• FOO Camp

• Out of This Box screening

• Context Week

The Madame of Silicon Valley

Aella is a sex worker, former camgirl, and person who is very good at the internet. She is also the on-again, off-again girlfriend of the President of MIRI, Nate Soares. She thrives on shock reactions, and seems unique so far in history for having used the social camouflage of being edgy on the internet to get away with as much as she has while operating almost completely in the open. Everything she does seems to be just up to the line, and there is always some defense that sort of makes sense at the time. It is incredible that she is this good at controlling and navigating the information environment. She is probably the only person in history so far to leverage social media in the way that she does.

As the scene has spiralled and become more bizarre, she has come to seem like a major cultural leader, if not the main cultural leader of rationalism. I see her posts more, she headlines their Woodstock (there’s a rationalist woodstock! it is called vibecamp), and of course she organizes many of their orgies, which appear to be the social glue holding a lot of this together.

One of the interesting things about cults is that the men are usually the leaders, figureheads, thinkers, and the public face of the organization, but often the actual getting-things-done part of the cult and arguably the real leadership that matters is done mostly by women. Jim Jones made almost none of the preparations for the Jonestown suicide. His lieutenants, who were mostly women, ordered and prepared all the kool-aid and all the cyanide.

Aella has a uniquely well-suited background for this role. She is the daughter of an abusive Calvinist preacher, and she escaped from that into atheism, LSD, social media and sex work. She has in turn made herself the primary sexual partner of Nate Soares, the president of MIRI. It is an almost perfectly Berkeley way of both inverting and becoming who she was raised to be.

She is good at giving the cause a voice. She has donated enough money to Lighthaven to have her name on a plaque²⁷, and to have walk-in privileges whenever she likes. Yudkowsky likes her, promotes her, legitimizes her internet sex surveys as social science, and puts her into the acknowledgements of his books.

Here she is talking about the overlap between AI safety and polyamory in the Bay. It is worth reproducing her post in full: “Yeah if you count casual relationships as connections, the ai safety community in the bay is probably a single polycule”

Here’s her opinion of a thread I wrote on this same subject earlier.

It’s good to know that fewer than 50% of attendees for one of her hosted orgies work in AI-related stuff.

She denies being engaged in “sex trafficking”.²⁸

Using Sex For Ideological Compliance

There are documented cases of cults using sex for recruitment and ideological control. Weird sex and weird power dynamics around sex are very common in cults. They are, of course, common in this one, too.

You remember Jacob Coxon? The “whistleblower” who resigned from Anthropic, and then “whistleblew” that all the things Anthropic already preaches are true? We had him at the beginning of this post, giving Ed Zitron a long explanation about how much he respects rationalists.

Here’s Aella, three days before he went public, promising to throw whoever it is a “reverse gangbang” if they will quit from a lab and whistleblow. (People keep asking me what a reverse gangbang is: one man, many women).

Here she is on the day of:

I am supposed to believe this is a coincidence, because I am supposed to believe that everything is a coincidence.

This is what she calls her finest piece of work, a now-infamous Sankey diagram documenting the process for scheduling a large gangbang on her birthday. She says her “boyfriend” set up the event. This may mean Nate Soares, although this is somewhat difficult to be sure of from public comments. Polyamory makes common references to “boyfriend” and “partner” somewhat ambiguous.

Next we have her explaining that she is giving what she can to the cause by filtering invites to her gangbang based upon ideology. e/acc is a rival ideology to EA. I have historically hated it but increasingly feel like I would like Literally Any Ideology Other Than Effective Altruism.

To spell this all the way out: it is within the realm of possibility here that Nate Soares, president of MIRI, may have organized a gangbang in which ideological rivals were explicitly excluded during the application process. If so, this was done in public.

This is the banner for an article by Nicholas Decker from April 16, 2025, about overthrowing the United States government by force and/or violence.

This is Aella’s reward for that article.

I think we can conclusively say that she uses sex to promote her ideology.

Recruiting Women Into Sex Work

Aella is reputed to have introduced people into sex work:

She has written a guide to escorting. She gives educational sessions which produce quotes like “You are trying to get a million dollars. The million dollars is stored in his balls.”

Playboy once detailed the story of how she got into being a camgirl, and how she encouraged her sisters to do so also. She also has posts on this subject:

Consent

Aella is very public about having a fetish for “CNC”, or “consensual non-consent”, or stated more directly “simulated rape”. I have been hearing or reading, for years, that her specific simulated rape parties are dangerous, that rules are treated casually, and that everyone is on drugs.

Here is the most detailed and public negative account of her events I am aware of. It is part of a long post that includes a lot of community drama and drama concerning other rationalist spin-off cults and for some reason a rehashing of the Brent Dill saga.

Leaving the other drama aside, Aella’s reply makes this account of her parties seem credible.

Thanks for giving me a chance to talk anonymously about Aella and my experience with her. I don’t talk about her often because I don’t like badmouthing people. Also people say she’s vindictive.

I’ve only been to one of Aella’s parties but one was enough. The environment felt so uncomfortable that I spent hours and hours hiding alone and crying. The party had a lot of nudity and sex, everyone was on drugs, and it was set up so that consent as a guiding light was being continuously questioned. Maybe it could have been ok if I had known in advance that it would be a sex party with blurry consent practices but I didn’t, I thought I was just going to a normal burner party. It also wasn’t easy to leave for logistical reasons so once I was there I was stuck.

The only person I found who validated my feelings was another woman who I finally met towards the end of the party who was going around talking about how it wasn’t a safe environment. I was so grateful to her for saying something because I had almost gaslighted myself into believing that the problem was actually myself, I was just insecure, etc. In retrospect I think there must have been a lot of other women who were scared who just didn’t want to show how stressed and alone they felt in this extremely sexual blurred lines environment with a lot of horny drugged men around and no easy way to leave.

When I finally got home and talked to my friends I learned that Aella is well known for hosting parties with sketchy consent practices. I never went to one of her events again and I warn other people not to go all the time.

I want to point out that I’m not a shrinking violet. I go to burner & sex parties all the time. I have lost count of the number of times I’ve had sex in public. I also don’t judge Aella for being a sex worker. She seems like she works hard and I admire her success. But it seems like a lot of her success is organizing these events where women end up in weird sketchy situations. Maybe it feels great for some of the women who go. But I wouldn’t want Aella to be in charge of anything that demanded compassion or clarity.

This is Aella’s reply to that part of this complaint. (Her total comment is much longer and we’re going to consider most of it irrelevant to the point).

What I find especially troubling about this is that sexual assault in this context would be very nearly impossible to prove. Let’s say, hypothetically, that you showed up for an actual group sex simulated rape party and things went badly. If the rules are messed up and you outright get raped when you are trying to say no, you are going to have to explain to a cop exactly the difference between what happened and what was supposed to happen, or you are going to have to deal with some Effective Altruist mediator who is likely going to send you to counseling with your rapist,

or

you can tell your friends what happened but ask them to not mention you or tell your story, and then because your friends cannot mention you, when they want to make sure people are warned they have to say that the party is “not safe” or tell some other milder story they can get permission to tell instead of saying that someone they know was raped at one of these parties.

For Aella to be this casual, defensive and dismissive when someone raises such a concern about consent at her events is bad enough that I would assume it means she just doesn’t care. I think I can make a reasonable-confidence Bayesian inference on this one. That is the Rationalist term for “a very educated guess”. You could be reasonable and come to a different conclusion.

Part of why this continues is that everyone is fighting some factional battle, and nobody seems to make an accusation unless it helps their side. People seldom simply tell the truth about everything they know, and it is never extremely obvious which allegations are drama and which concern crimes against people.

Here are some of Aella’s opinions about rape. Ask yourself if you would trust this person to give a shit about you if you had been raped.

Here we can find Time Magazine documenting a pattern of predatory behavior in Bay Area Effective Altruist group homes.

Here we have a viral post from 2024. I did not even remember it had happened until someone reminded me when I started talking about this a few days ago. Things like this are said all the time and the community always closes ranks, and then the story vanishes. This post describes alleged CNC sex parties and alleged heavy LSD use among elite AI researchers at, specifically, OpenAI.

“some of the most coercive and fucked up social dynamics I have ever seen”

That post’s author is Sonia Joseph. Sonia Joseph first got into Rationalism and Effective Altruism by reading Harry Potter and the Methods of Rationality when she was fourteen years old. That story in Bloomberg matches everything I am saying, matches the separate Time story, and so on, and so on, and so on.

This is all one big thing, people have told good chunks of the story many times and it’s just extremely hard to convince anyone that this is all one big thing and that it is this large, this prominent or this bad.

Aella’s current event series is Red Means No, which she organizes along with Gretta Duleba, Eliezer Yudkowsky’s primary partner. Red Means No is registered as an LLC in the state of California.²⁹ Gretta will come up again very shortly.

Here is one of MIRI’s affiliates (John Wentworth, whom we will also run into again) explaining that he considers attendance at Red Means No a positive signal for someone’s career qualifications, in case it seemed plausible that this was never a de facto career requirement in that sphere.

Slutcon

This is the banner for Slutcon.

Gretta is Gretta Duleba, Eliezer Yudkowsky’s primary partner. As of their most recent filing, covering 2024, she drew a salary of about $200,000 from MIRI, and she apparently helped edit his book. She does not seem especially qualified, and lists her most recent four years of work prior to MIRI as “Relationship and Sex Therapy”. Per her posting history, she did not leave employment at MIRI until early this year, 2026.³⁰

This event appears to be incorporated as Weird Party LLC in Wyoming under filing ID 2026-002033219 and registered in California as Entity B20260434776.

This event was held at Lighthaven.³¹

This is Aella in 2024 soliciting donations for Lighthaven and saying she is hoping that they will let her throw orgies at Lighthaven if she is good enough at soliciting donations for them.

Belief

I believe that these core people involved in Berkeley AI Safety are substantially and knowingly the beneficiaries of various crimes, and that many of these crimes were committed for the express purpose of benefitting the rationalist and effective altruist movements.

I believe that these crimes include those committed at Futures Exchange, commonly known as FTX, and Alameda Research, and that the unrecovered fruits of the fraud that destroyed those companies are to this day part of the funding for this enterprise.

I believe that the specifics of the apocalyptic doctrines of the rationalists turn up as business plans and news items regularly in the AI industry.

I believe that a “pivotal act” is Yudkowsky’s euphemism for acts of warfare, government violence, or terrorism which he believes anyone making AGI must do in order to prevent competition and therefore the end of the world, and that many members of this enterprise will understand some offshoot of this idea and will understand such acts to be necessary in the course of pursuing their goals.

I believe that the most widely known rationalist text discussing pivotal acts specifically is “AGI Ruin: A List of Lethalities”, and that it discusses “burn all GPUs” as a hypothetical pivotal act preventing all competition. My understanding is that this essay claims not to endorse this plan, but that many readers come away convinced that this is a good plan, regardless.

I believe that Sam Altman was recently reported to to have discussed a scenario where one company melts all competitors’ GPUs, that Sam appeared in an interview with Fortune discussing this idea, and that Sam Altman appearing to take this seriously as an idea can only be due to Yudkowsky’s influence.

I believe that the sixth bullet point of "AGI Ruin: A List of Lethalities" is meant to indicate that the “pivotal act” which is the most likely to be necessary and achievable is not actually burning all GPUs, but something that is easier to achieve and that is unspeakable in public, and that the intended reading is for this to refer to eliminating all potential competition with engineered biological weapons. Anyone who is not a part of your project is potential competition.³²

I believe that a “warning shot” is a well-established term in rationalist and effective altruist discussions of AI safety. Per Kokotajlo,

A “AI safety warning shot” is some event that causes a substantial fraction of the relevant human actors (governments, AI researchers, etc.) to become substantially more supportive of AI research and worried about existential risks posed by AI.

I believe that it is more likely than not the case that at least some prominent recent AI security incidents were either deliberately or with willful neglect caused by insiders who believed that a “warning shot” was necessary to save humanity from AI existential risk, given the prevalence of rationalist and effective altruist employees within prominent AI companies.

I believe that most of this has been conducted in the open and can be established as more probably the case than not via close examination of public records. All of this has been kept from public knowledge and understanding because it is too offensive and too bizarre to seem real when stated directly.

I would testify to these beliefs and my basis for them under oath.

Ronny Fernandez

This is Ronny Fernandez. He is the “Lighthaven czar”³³, which I understand to mean that he manages day-to-day operations and the scheduling of all events held at Lighthaven, and has since 2024. His twitter tag is @RatOrthodox and his display name is Brangus. He is seen here receiving an amateur tattoo from Aella of the number “100” for being her hundredth recreational sex partner. She took pains to establish that “recreational” did not include her sex work clients.

If you do not know what being very badly on drugs looks like, this is it.

This was what the kids these days call “bait”, which is anything which attracts attention on the Internet by being bizarre, disgusting or controversial. All of these people are great at attracting shock reactions and then using them as cover to do whatever they were going to do anyway.

Ronny is also Aella’s manager.³⁴

This is a recently viral thread of a basically normal person’s experience at a meet-and-greet with the Rationalists at Lighthaven. The moderator who separated out the believers from the unbelievers was Ronny Fernandez.

Ronny Fernandez used to be married to Holly Elmore, the Executive Director of PauseAI US. Their divorce seems to have been messy, they are both activists, and its aftermath often spilled onto twitter.

This is Holly’s most complete explanation of her divorce, told in the aftermath of Sam Kirchner’s disappearance.³⁵

Holly was previously cancelled by essentially all of Effective Altruism, Rationalism, TPOT³⁶, whatever you want to call these people and all of their hangers-on for objecting to this picture and saying it makes their movement and cause look bad.

I do not know who the two men on the left are. The man in the golden jacket is Eliezer Yudkowsky. On the right is Nate Soares, President of MIRI. The man second from the right is John Wentworth, an “Independent AI Researcher” sometimes affiliated with MIRI.

plzdontkillus

“plzdontkillus” or “pdku” was a one-month-long “creator bootcamp” thrown by the Rationalists this July, 2026. It appears to have been a social media marketing and recruitment drive to bring in new blood and bring their message to short-form video. This program offered free room and board, a $2,000 stipend, and access to a $20,000 prize pool for high-performing videos on social media.

This was hosted at Lighthaven, and partially sponsored by MIRI.

My understanding is that attendees are some combination of people who believed in the cause and people who just wanted the check. My impression is that this was part of a recruiting funnel. This event was held over the summer when school was closed, but that’s probably to avoid noise complaints from the middle school that is across the street from the compound.

Staff for this event:

• Aella

• Ronny Fernandez

• Azmyth (Aella’s sister)

“Mentors” or otherwise participating:

• Eliezer Yudkowsky

• Nate Soares

• Grimes

• Jreg

• CJ The X

• Alex Botez

• Tor Parsons

• Quinn Finite

• Aric Floyd

• Zoe Curzi

• Rob Miles

Liron Shapira’s podcast, “Doom Debates”, did an interview with Aella and Ronny Fernandez at the conclusion of pdku to talk about the event. This interview confirms that JJ McCullough and Jeffrey Ladish provided mentorship separately from the official schedule.

Yudkowsky, Soares, and Ladish, all involved in this event, are three out of four of Bernie’s advisors on AI risk. (The fourth is Kokotajlo, who is a director for Lightcone, which manages the property). I am trying to make the point that this specific social scene, with these specific people, at this specific property, does in fact constitute essentially the entirety of “AI Safety” and its political wing.

While I have been writing, my earlier comments seem to have touched off some internal strife within the movement. This is resurfacing issues I had forgotten, like the “pdku dead cow question”, discussed here by rationalist Vishal:

This is the beginning of a much longer post defending asking this question on the application for plzdontkillus:

“If you had to have sex with a cow would you rather it be dead or alive? Why?”

This question is being defended, broadly, as just people being people and doing weird quirky things. Anyone opposed to asking this question is accused of being uptight and killing the mood. Those who think the question is inappropriate are correct, but their reasoning is wrong: they think it is inappropriate because it makes their movement look bad.

It is inappropriate because it is being used to test people’s boundaries and filter out those who have them, to make people feel complicit, and to provide an icebreaker for talking about sex. It is inappropriate because it is being asked by people in a position of authority. It is inappropriate because it is being used to probe for weakness.

It is inappropriate because Ronny Fernandez is the one asking it.

These reasons for objecting do not occur to anyone in Bay Area Effective Altruism or Rationalism because they have completely normalized this behavior. They have been taught to never consider such questions.

Here is Aella’s explanation for why the cow question was on the application form. I would like to thank her for confirming the question was calculated.

Brangus Unchained

Ronny Fernandez has a second twitter account, where he used to post his unfiltered thoughts. He seems to have forgotten about it. Unfortunately, in order to avoid deflection or claims that this is just a joke I have copied in a lot of it. If your opinion of him is already bad enough that more information won’t matter, I would recommend skipping this section.³⁷

This establishes that it is his account:

This is Nate Soares interacting with his posts, and is about five months into Ronny being in charge of Lighthaven:

This establishes that Aella is aware of the account, and is comfortable with the contents of the account:

I expect that the usual defense here would be that he is being funny, or this is just fetish content, or something. If such an excuse is made, please consider this post:

In this one, he claims his assistant brings him a cold brew with a methamphetamine powder rim every morning …

… and this is his assistant, Azmyth, Aella’s sister, taking a candid video because they are having a good and fun time at pdku. He says he is on a lot of stimulants.

This one is interesting because it is a joke about the normal defensive tactic this group of people uses. Act harmless, act weird, and use this to get past people’s defenses. Maybe it’s just a joke and I don’t get it.

These posts speak for themselves.

5meo refers to 5-MeO-DMT, also known as toad venom.

The Atlas Fellowship

This is Ronny Fernandez’s picture on the page for The Atlas Fellowship, at which he was an instructor, and which appears to have run only in 2022 and 2023. I am saved from having to type out most of the important information about The Atlas Fellowship, because they put it all into one nice box.

I believe the following facts are perhaps not obvious from this screenshot:

• Open Philanthropy is now called Coefficient Giving, and is an Effective Altruist organization.

• This is a summer camp for high school students. There are videos of them on the site.

I had never heard of this program before. I found it while trying to find a good reference photo for Ronny’s face, so now we have one. There are so many rationalist and effective altruist summer camps, youth programs, and so on that finding them all and figuring out where they are would make it impossible for me to finish this in a reasonable amount of time.

Ronny is the current manager at Lighthaven and all events are scheduled through him. He is the one who separates the believers from the unbelievers at meet-and-greets and he will have access to all of the students of any of the programs that are run at the compound.

Here is an article where he argues that having sex with an adult when he was thirteen did not harm him.

When Brent Dill came up, the narrative from leadership was that they did not know what he had been doing before they put him into a position of authority at a camp for high schoolers. They should have known and were negligent but not malicious. They would do better, they were listening and learning.

I am skeptical.

What This Is

Core leaders for MIRI, Lighthaven and the community fetishistically prefer simulated rape. They are bound together by having hobbies and secrets in common. This seems to be an important form of social tie in this group. They promote their sexual interests and recruit for their sexual events so prominently because these are the things they care about and believe in. These people are some of the most influential, if not the most influential people in AI Safety.

This is too obscene to be taken seriously if stated flat. This is also why no one can just say it and be believed. It sounds like a conspiracy theory. That it sounds like a conspiracy theory serves as fantastic camouflage. They are brilliant at that sort of thing.

All of the main content of rationalism is a trick. Every time Yudkowsky’s writing explains some trick your brain can pull, it is the magician showing you his sleeves while he palms a card: it is meant to convince you that whatever trick he is about to do next is impossible, and to prevent you from figuring out how it was done or ever believing it had happened.³⁸

What he has done is craft an atheist religion that worships and fears an omnicidal God, trusts him as its prophet, and that gives him immense power over its members and over members’ actions.

This belief system primarily causes people to do things they would refuse to do for any reason other than fear of the end of the world. Nothing that has happened in the movement is a coincidence; this appears to be the primary thing the belief system is for. Its other effects are, by comparison, negligible.

Interlude

During my first go-round with this post, at about this point in its length, I mostly stopped sleeping, read a great many things I very much did not enjoy, and became quite unhappy with the general state of things in Berkeley and indeed with the concept that such a place as Berkeley or Lighthaven existed. In my haste to publish I possibly overextended parts of the analysis beyond what I was able to definitively prove.³⁹

I withdraw the previous version of this article in its entirety.

Let us continue on to what I wish I had said instead.

Author Review

First, Yudkowsky is a great man.

Great as in the Great Man of History theory, all of that. If he had not lived, the world would be very different.

You could very credibly write alternative universe science fiction for the last thirty years that simply subtracts him. You’ve removed or significantly altered DeepMind, half of the weird cranks, Sam Bankman-Fried. Hell, earlier: Many early cryptocurrency figures, like Hal Finney and Wei Dai, were regulars on Yudkowsky’s blog. Do they do different things without his influence, or without his movement to bring them all together? I have no idea. If you’re an author you can go wherever you want with that. It’s fine. You can bend history however you like by removing him; he has bent the entire trajectory of technology by his influence.

He is a person who sat down at a keyboard and simply decided things should happen, and they did. He sat down as a teenager in the nineties, did nothing but write, and in 2015 he was thirty-five years old and by far the most influential person in the field. There is little evidence he has ever written a computer program of any use or value. His math is rudimentary. In traditional academic terms, he is a popular philosopher.

He thought he could personally and directly retain oversight over the entire thing, and he almost could. He was involved in DeepMind’s founding and knew its leaders personally. All of this blew up when Sam Altman and Elon Musk simply walked away, and he has been trying to scrape back direct personal control ever since. Despite that, he is still by far the most influential person in the field, and a huge chunk of thinking is still his pseudoscience.⁴⁰ When Sam wants to insult him he uses a backhanded compliment. When Dario wants to disagree with him he is afraid to name him.

This is an incredible achievement.

What he did was read usenet and science blogs for over a decade, read a few of the books they recommended, and then distill everything that was in the ether. This was a mixture of having done your reading as an undergraduate, some reprocessed Korzybski⁴¹ (who also influenced Hubbard), a half dozen transhumanist authors and the sorts of things people talked about but had not written down formally, or that were only written down in an article somewhere.

From these ingredients he wrote just one thing that let you skip the line, avoid doing any real reading or real education, and acquire maybe two thirds of the world view you would have as an excellent, well-read student in a science major, but with none of the specific skills you would have learned. For some people this fast track is the most exhilarating experience of their lives. Many of them become convinced Yudkowsky is the smartest person alive.

I am not convinced he is the smartest person alive because I had basically his same reading diet at the time and almost none of this was new to me. To me, he is a guy with a pretty good generalist blog that became a religion while I wasn’t paying attention, and a bit of a crank. I was there. It was just a science blog to me. I did not see it as an ideological project because I was not looking for that. Judged as an ideological project, it is vastly more impressive work.

If we judge his writing by its impact, by how much the world changes around you with every word you write, Yudkowsky’s last rival was Lenin. Every time you hear someone call their beliefs their “priors”, they kiss the ring. They generally do not know this. I am obsessive about the origins and meanings of words and ideas and Yudkowsky’s jargon is everywhere. It is like hearing people talk about how Gatorade is full of anti-Thetans.

He did this while writing a story about himself because everything Yudkowsky writes is about himself; his most distinctive authorial tell is that all of his writing, no matter what else it is about, is autobiographical. It is a story about how Eliezer Yudkowsky is the smartest and kindest person who ever lived, and he understands you.

All of this to say: I can assume that his writing is intended to influence people, and to cause them to do things, and it is well-calculated and good at doing this.

Power

All of this cashes out into power, and that power is hidden in plain sight.

This power is hidden because HPMOR is an adolescent power fantasy. For the book or its ideas or its fandom to be important seems too absurd. It is hidden because it is silly and because it exploits our desire to tolerate the weird, to avoid sneering at anyone just because they’re unusual. People exploit this desire while openly exercising vast power over their followers and the world.

It is so obviously an adolescent power fantasy, is so obviously entertainment, childish, low culture, that it absolutely must be harmless. For anyone to say that this matters the same way that Scientology or Salafi Islam do, that this child’s power fantasy has real power in the real world, seems like a prank or the manifestation of mental illness.

It is simply impossible to take them seriously. They sound like edgy middle schoolers who read too much and who think they are too special. They directly talk about plotting world domination, they call their podcast a conspiracy. Who would do that? Of course it’s a joke.

Never mind that they can pull down millions in grants and investment money without question, that they have a hotel-turned-compound in Berkeley which hosts most notable AI safety conferences, that they were involved in founding nearly every major company and non-profit in AI.

No matter how big they are, how much money they spend lobbying the government, how many kids are taught in their camps, how ubiquitous they are in the tech industry, they are always too bizarre to be taken very seriously.

To connect the core people and their ideas to the global movement sounds paranoid, even when they share money, people and ideas clearly and openly. To connect the entire sprawling complex to its crimes, scandals and other failures is always too strange to seem credible even when the connection is direct.

It is scandalous to say that ten billion dollars of fraud was done for the cause, or that those who committed that fraud willingly went to prison for the cause. It is weird to note the proliferation of cults, one of them nationally famous and with a body count, or to notice how much journalism and how many popular videos the broader movement sponsors, or to notice that the exact same core people always end up advising politicians.

It is obscene and conspiratorial to state that this group of people preferentially recruits, promotes and funds people based upon their participation in a specific Berkeley polycule and its simulated rape orgy, and that this appears to either be or be very close to being the most important center of power in policy and in the AI industry.

The broader movement “contributed” to the board coup at OpenAI and put a half billion dollars into Anthropic. They hold their conferences in the building that Yudkowsky’s long-time associates run. Their ideology appears to have directly inspired an attempt on Sam Altman’s life.

All of this is unbelievable in spite of being completely in the open. I have smothered this article in screenshots and citations, it links back to years of media reporting about these same people, and probably half of the people who read this will decide it is simply too weird to be true.

This social camouflage is the primary way they are protected from outside scrutiny, and it is the trick they pull on their members, too. “I’m just a little low status guy” is not just their strategy for predation, it is their everything strategy, and it seems to work on everyone.

Everything seems harmless, everyone is friendly, and all of this is especially appealing to the young and to the neurodivergent. One of the minor apocryphal works is based on My Little Pony. People are taught that certain secrets are infohazards and that if they do not keep these secrets the world will end. (The naive tend to love jargon.) People are encouraged to fight everything out and to never agree to disagree.⁴²

It is supposed to be a coincidence that groups of them often collapse into authoritarian mystery cults even though this directly reflects the movement’s teachings and matches the movement’s exact structure as a whole.

They have recruited thousands of the kindest and most naive nerds on the planet to prey on, to hire, to take money from, and to use as cover. They have all read the same power fantasy, and most of them seem to think that they’re in on it but most of them aren’t. Most of them are prey. They will swear up and down that the funny stories and odd subculture are unimportant and harmless. They really ought to know better, and if they didn’t make their choices based on raw trust they would.

If you are a smart or at all connected person and your area overlaps with tech, you know at least one of these people. They will generally stop talking to you if you say anything like the truth about the sex cult that, to this day, seems to rule Berkeley AI safety. They obey out of fear of the apocalypse, and to say that they do is a horrifying insult because it is true.

I previously believed there was some meaningful difference of opinion or independence from these people within the global Effective Altruist movement. I then watched one oddball rationalist (who won’t admit that he is one) stage his exit from Anthropic and call himself a whistleblower while agreeing with Anthropic’s CEO exactly. I saw the entire movement, most of the industry, and most prominent politicians in America all line up behind it.

This absurd tail really is wagging the whole dog. Power flows from these core people out to everyone, and fear keeps them all obedient.

All of this is poison, and in a sane world we’d be rid of it all.

1. https://www.wired.com/story/book-excerpt-the-optimist-open-ai-sam-altman/

https://www.nytimes.com/2023/12/03/technology/ai-openai-musk-page-altman.html

2. You can find the list of talks and speakers here.

Notable people include:

• Will MacAskill (Author: Doing Good Better)

• Rob Wiblin (Centre for Effective Altruism)

• Ben Todd (80,000 Hours)

• Kerry Vaughan (EA Ventures)

• Bryan Johnson (OS Fund)

• Nick Bostrom (Future of Humanity Institute)

• Elon Musk (SpaceX, Tesla)

• Nate Soares (Machine Intelligence Research Institute)

• Anna Salamon (Center for Applied Rationality)

• Eliezer Yudkowsky (Harry Potter and the Methods of Rationality)

• Nick Beckstead (Open Phil) (We will meet him later as one of the people who worked at FTX Future Fund)

• Amanda MacAskill

• Holden Karnofsky (Open Phil)

3. I count at least twelve citations across these eight papers, to get you started.

• Concrete Problems in AI Safety

• AI Safety via Debate

• Training Language Models to Follow Instructions with Human Feedback

• A General Language Assistant as a Laboratory for Alignment

• Weak-to-Strong Generalization

• Sleeper Agents

• Alignment Faking in Large Language Models

• Discovering Language Model Behaviors with Model-Written Evaluations

4. It turns out the cleanest, easily-citable demonstration that this piece of slang is in common use is from someone who’s at Anthropic now. This sort of thing really keeps happening.

5. Most recently that I can find in this remarkable article, about how people should not return money which was given to them by FTX, which opens as follows, bolding mine:

In my possibly contrarian opinion, and speaking as somebody who I don't think actually got any money directly from FTX Future Fund that I can recall; also speaking for myself and hastily, having not run this post past any other major leaders in EA:

6. From an interview:

7. What’s your advice for Less Wrong readers who want to help save the human race?

Find whatever you’re best at; if that thing that you’re best at is inventing new math[s] of artificial intelligence, then come work for the Singularity Institute. If the thing that you’re best at is investment banking, then work for Wall Street and transfer as much money as your mind and will permit to the Singularity institute where [it] will be used by other people. And for a number of sort of intermediate cases, if you’re familiar with all the issues of AI and all the issues of rationality and you can write papers at a reasonable clip, and you’re willing to work for a not overwhelmingly high salary, then the Singularity Institute is, as I understand it, hoping to make a sort of push toward getting some things published in academia. I’m not going to be in charge of that, Michael Vassar and Anna Salamon would be in charge of that side of things. There’s an internship program whereby we provide you with room and board and you drop by for a month or whatever and see whether or not this is work you can do and how good you are at doing it.

Aside from that, though, I think that saving the human species eventually comes down to, metaphorically speaking, nine people and a brain in a box in a basement, and everything else feeds into that. Publishing papers in academia feeds into either attracting attention that gets funding, or attracting people who read about the topic, not necessarily reading the papers directly even but just sort of raising the profile of the issues where intelligent people wonder what they can do with their lives think artificial intelligence instead of string theory. Hopefully not too many of them are thinking that because that would just generate noise, but the very most intelligent people... string theory is a marginal waste of the most intelligent people. Artificial intelligence and Friendly Artificial Intelligence, sort of developing precise, precision grade theories of artificial intelligence that you could actually use to actually build a Friendly AI instead of blowing up the world; the need for one more genius there is much greater than the need for one more genius in string theory. Most of us can’t work on that problem directly. I, in a sense, have been lucky enough not to have to confront a lot of the hard issues here, because of being lucky enough to be able to work on the problem directly, which simplifies my choice of careers.

7. Compare these mission statements:

2015–2017

• MIRI: “Ensure that the creation of smarter-than-human artificial intelligence has a positive impact.”

• OpenAI: “Advance digital intelligence in the way that is most likely to benefit humanity as a whole.”

• Anthropic: —

2018–2020

• MIRI: “Ensure that the creation of smarter-than-human artificial intelligence has a positive impact.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: —

2021

• MIRI: “Ensure that the creation of smarter-than-human artificial intelligence has a positive impact.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Make the fundamental research advances that will let us build more capable, general, and reliable AI systems, then deploy these systems in a way that benefits people.”

2022

• MIRI: “Ensure that the creation of smarter-than-human artificial intelligence has a positive impact.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Build steerable, interpretable, robust AI systems.”

2023

• MIRI: “Ensure that the creation of smarter-than-human artificial intelligence has a positive impact.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Build reliable, interpretable, and steerable AI systems.”

2024

• MIRI: “Ensure that the long-term future goes well, particularly by increasing the probability that humanity safely navigates the transition to smarter-than-human AI.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Build reliable, interpretable, steerable AI systems.”

2025

• MIRI: “Prevent human extinction from artificial superintelligence.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Ensure a safe transition through transformative AI.”

2026

• MIRI: “Prevent human extinction from artificial superintelligence.”

• OpenAI: “Ensure that artificial general intelligence benefits all of humanity.”

• Anthropic: “Ensure that the world safely makes the transition through transformative AI.”

8. Floor Employment from 2013 at SlateStarCodex about job options is a fun window here:

Camperson

…is the gender-neutral version of “camgirl”. Another one that wins on ease of access. You sign up on a website. You show them a driver’s license to prove you’re 18. You set up a webcam. And people pay to watch you do sexy things.

In theory. Obviously this works better if you are female and very pretty. Very few women are clients of camsites, so if you are male your choice is pretty much to cater to gay men – which can also be lucrative if you’re up for it.

This establishes nicely that this is high-prestige employment that high-prestige people in rationalism and effective altruism nudge especially women towards.

I personally remember being told, in a single conversation during this time period, that someone was taking up camming as a profession and that they were considering giving a percentage of their income to MIRI. I was not the only person who had this exact conversation.

(This article includes funny appearances of several people the movement has since memory holed, like Louie Helm, also.)

9

• Dario Amodei

  • Early GiveWell donor: discovered GiveWell in 2008; donated $10,000 through its pledge fund for his 2009 giving and wrote a guest post explaining his cost-effectiveness reasoning. GiveWell Blog

  • Explicitly described by MIRI as part of the EA community: a 2013 strategy meeting involving Amodei, Holden Karnofsky, Jacob Steinhardt and MIRI staff was introduced by MIRI as a meeting with “three additional members of the effective altruism community.” Machine Intelligence Research Institute

  • Giving What We Can pledge: reported as being pledger #43. Giving What We Can is sort of archetypally Effective Altruist and almost could not possibly be more of an Effective Altruist cause, and he’s a ground floor guy for it.

    • 

  • Close early network ties: lived with his sister Daniela and Holden Karnofsky, then head of GiveWell/Open Philanthropy; contemporary reporting describes this household as one route through which EA/AI-risk ideas circulated among the future OpenAI/Anthropic network. The Wall Street Journal

• Daniela Amodei

  • Holden Karnofsky: married to the GiveWell co-founder and former Open Philanthropy CEO, one of the central figures in the EA philanthropic ecosystem. WIRED

  • Shared the early SF household with Dario and Karnofsky that connected several people involved in OpenAI/AI safety with EA ideas. The Wall Street Journal

  • Explicitly distances herself from the label: told WIRED, “I don’t identify with that terminology,” calling EA “a bit of an outdated term.” WIRED

• Tom Brown

  • CFAR: attended a workshop at the Center for Applied Rationality, which the San Francisco Standard describes as a nonprofit in the EA and AI-safety spheres. The San Francisco Standard

• Jack Clark

  • Direct engagement with EA: Clark says some Anthropic founders have “clear links” to EA and that early Anthropic hired “quite a few people” from the community. PodScripts

  • Explicit non-identification: in the same interview, Clark described himself as someone who “isn’t an effective altruist and gets into arguments with them.” PodScripts

• Jared Kaplan

  • Direct Open Philanthropy involvement: in May 2016, Kaplan met with Holden Karnofsky, then executive director of the Open Philanthropy Project, specifically about recruiting physicists into AI research aimed at reducing potential risks from advanced AI. Open Philanthropy

  • They discussed concrete interventions including an AI-safety recruitment conference, research grants, undergraduate programs, scholarships/fellowships, and connecting physicists with relevant AI jobs. Open Philanthropy

• Chris Olah

  • 80,000 Hours: has appeared multiple times on the 80,000 Hours podcast, putting him in direct dialogue with one of the major organizations originating from the EA movement.

• Ben Mann

  • Explicit self-identification: wrote a 2019 essay titled “Why I now identify as an Effective Altruist.” Medium

  • EA-motivated giving: said he planned to donate part of his income to a GiveWell charity or grant program. Medium

  • EA-motivated career choice: explicitly connected his decision to return to AI safety with expected-impact reasoning about advanced AI and humanity’s future. Medium

  • MIRI: Mann says he had previously tried working at both OpenAI and MIRI, giving him a direct connection to the rationalist/AI-safety institution most closely associated with the intellectual prehistory of EA’s focus on existential AI risk. Medium

10. From here:

11. He also apparently dated Caroline Ellison? Small world.

They were joined by leaders at Redwood Research, an AI safety nonprofit that had received funding from organizations backed by Anthropic investors, and at the time included Karnofsky on its board. Redwood’s Berkeley office was already informally known as Constellation, and would later spin off into its own nonprofit, where Redwood continues to work. Its CEO, Buck Shlegeris, also used to date Ellison, people close to them said.

12. From here:

13. I also learned from this that Geoffrey Hinton’s father was an asshole.

14. This call would be more impressive if he hadn’t gone through a few candidates for it.

15. Epstein files EFTA02607800:

16. It’s not entirely clear if this is actually more people than you would have expected to have gone insane anyway. If you’re Christian and you go crazy, you’re obsessed with the Bible and Satan. If you’re a Rationalist, you become obsessed with the Sequences and the Basilisk.

17. See here:

18. The good historical reference here is once again Floor Employment at SlateStarCodex in 2013.

Alyssa Vance makes the case for becoming a programmer if you don’t like what you’re doing now.

It’s high-paying and fun, you can learn it on your own without worrying about college degrees and such, and there’s lots of demand.

The downside is that you have to have a spare six months or so to start learning programming, the willpower to actually do so, and the analytical skills to become good at it. Most people probably won’t.

If your limiting factor is time or self-motivatedness, schools like Dev Bootcamp offer to bring you from zero to full programmer capable of getting a $90K job in about two months – but the going is really hard and the tuition is just above $10K. App Academy is similar but has no up front cost – you pay them after you get a job. A warning: some friends here mention that these companies’ claims of “90+% job placement rate” are misleading, as halfway through they expel anyone who they don’t think will get a job in order to keep their numbers up.

Bottleneck: Time, money, natural programming ability, motivation

(“Makes the case” link replaced with archive link since that blog is gone now.)

19. Because this is a self-help, can-do ideology, often they spend all of their time assuming this is their fault and trying to figure out what is wrong with them.

20. This is the clearest version, where he says that everyone dying is “probable”, and he continued to be somewhat less direct but similarly grim for some time. I could try to mine his quotes at the time for the increasingly grim but vague perspective on ongoing events, but we can actually just take his word for it given that it’s been well-papered since:

In a November 2017 document, published in 2022, Yudkowsky criticized Musk’s plans for open-source AI and declared: “The project must be closed tightly or everything ends up as paperclips.” In October 2021, he explicitly blamed OpenAI’s launch for having “trashed humanity’s chances of survival,” attributing the breakdown of cooperation to Musk’s rivalry with Demis Hassabis. By March 2023, his conclusion was blunt: “If we actually do this, we are all going to die.”

21. See here:

22. End of a long twitter thread which covers surrounding context and somewhat does a history of the movement, but hard to follow https://x.com/QiaochuYuan/status/1726951287567364487

CFAR official statement: https://www.rationality.org/resources/updates/2019/cfars-mistakes-regarding-brent

Original allegations One Two Three

23. This is sometimes disputed, but the FTX estate complaint alleges otherwise. From the legal filing:

24. Establishing this and many other facts to do with how the compound is funded: https://www.lesswrong.com/posts/5n2ZQcbc7r4R8mvqc

25. See here:

26. See here:

27. See here:

28. It turns out she has denied this previously, in response to I don’t know what:

29. See here:

30. Gretta seems to have a lot going on around here and it is impossible to cover it elegantly in the main text.

• She said she was waiting on a grant from the Survival and Flourishing Fund, and the list of SFF’s 2026 grantees shows a grant for $100,000 to “GD Research”.

• She appears to have been assisting in some way with John Wentworth’s $519,000 grant for “technical alignment research” from Survival and Flourishing Fund.

  • She is romantically and/or sexually involved with John Wentworth, in addition to Eliezer Yudkowsky. This is sort of normal since everyone is poly.

• One almost gets the impression that “alignment research” is a form of BDSM:

  

31. Discussed after 32:00 here:

32. From Yudkowsky’s writing here, the solution must meet the following criteria:

• Prevent competing AGI projects, big and small, present and future.

• Be less difficult than locating and destroying every GPU.

• Require capabilities that advanced AI could supply.

• Be sufficiently taboo that he deliberately avoids describing it in public.

For the first point: This piece argues that progressively smaller and smaller actors will become capable of building AGI over time. Given that assumption, stopping the current laboratories does not solve the problem, because other people can easily replace them; you must stop almost anyone, even small groups of people, from attempting to create AGI.

For the second point: We are told that locating and destroying all of the relevant hardware is somewhat more difficult than what you should actually do. From this, we can infer that the intended plan is somewhat less technologically difficult than that.

In particular, this rules out any plan concerning the use of political power. Achieving regulatory control worldwide over all GPUs or all of research would pretty clearly be more difficult than just destroying the GPUs. Maintaining the coercion and surveillance necessary to keep a monopoly over all GPUs indefinitely is substantively more difficult than simply destroying the GPUs! Destroying the GPUs would be at most as difficult as maintaining such vast political power, since you could simply use such a monopoly on violence to destroy all of the GPUs, and all other relevant hardware, and really anything else you wanted.

For the third point: He directly says that this requires a relatively powerful AGI. Whatever the obstacle is, being much smarter must help you overcome it; it must involve some problem of knowledge, design, or engineering. This, too, rules out ordinary regulatory, military, diplomatic or political means. Simply acquiring political authority would not explain why you needed an AGI to execute this “pivotal act”.

For the fourth point: He explicitly says that “all known pivotal acts are currently outside the Overton Window”. He explicitly says that he chose burning all GPUs as an example because he does not advocate for it, allowing him to discuss the need for a pivotal act without explicitly advocating any particular plan. We are being directly told that what he thinks people actually should or will do is “currently outside the Overton Window”, and that he is refusing to directly state what “pivotal act” he thinks should be done.

Taken together, these constraints rule out any plausible reading other than engineered biological weapons.

We can also assume he picked this example because it is similar to the plan he does not want to say out loud. It is meant as a deliberate hint.

His chosen example is building nanotechnology that can seek, recognize, and destroy GPUs. This is presented as “needlessly difficult to align”.

We can note that “align” in this case is a type of weapons targeting. This clause specifically hints that the real plan makes targeting easier.

Why would you choose this example to hint at, but not directly state, some other plan? What specific plan does this hint at most directly?

Designing a suitable biological weapon for killing those working or potentially working on rival projects fits perfectly. Humans are generally easier to find and to harm than machines. An engineered disease naturally spreads from victim to victim, and naturally tends to harm them. It would be easier to “align” such a weapon to remove the people who represent a threat, and the problem of creating such a weapon is similar to the problem of creating something that would destroy only GPUs but is somewhat easier.

That is the plan this written passage is meant to indicate. Nobody I have talked to can give me another interpretation that even remotely fits here. It is simply too upsetting for people to believe this is actually what he meant, even though it’s not very different or much worse than openly advocating in favor of increasing the risk of nuclear war, which he is known to have done openly in Time Magazine.

Here he is a few days ago, commenting on Sam’s discussion of melting GPUs.

33. See:

34. Confirmed:

35. I figured Sam Kirchner would have turned up by now. It is very strange that he is still in the wind.

36. TPOT is the name for a social scene on Twitter. It does not have hard boundaries or a fixed definition. I might technically be a member of it but am certainly not one of the thought leaders of it.

37. It turns out Discord auto-moderates these images and it makes it difficult to talk about him on Discord. Good work at flagging content that probably should not be allowed on any platform, Discord.

38. For the Rationalists in the audience: The purpose of explaining Pascal’s Mugging was to make you think you had figured it out so that you would believe you were too smart and well-informed to be Pascal’s Mugged, and to make it impossible to convince you that it had happened to you later once it had.

39. The blog name, “very sane”, has always been ironic, and I hope nobody ever takes anything I say on faith. If you believe anything I say it should be because I’ve provided receipts, you can find receipts, or you think my argument is correct.

40. It might be more precise to call it pre-paradigmatic science, the way alchemy precedes chemistry.

41. See here:

42. This relies on a misunderstanding of Aumann’s Agreement Theorem from probability theory. Literally quoted here: “to not defer to someone’s authority, you must think you are smarter than them”. You can find this, and some good amount of the dogma as a whole, detailed in the following. This approach to disagreement is dysfunctional, and specifically promotes making whoever can out-nerd the other person in a narrow context into an unquestionable authority.