68 Comments

A long time ago, (an in-person chapter of) Less Wrong was my "tribe" of weirdo geeks who would do things like use neurodes and electromagnets (personally attached) and then try to see if that resulted in improving short-term memory performance. We'd meet and talk about astronomy, health foods (including one incident where a friend brought huge bags of dried kale to a coffee shop that looked like a drug-deal in progress), D&D, and life in general. They were some of the first groups I met who were openly accepting of gay and trans people. And when I was in high school / early college, they were my first exposure to a lot of new ideas about philosophy and AI.

All that to say, I hope you don't condemn weird nerds for being weird nerds. We're doing our best, and LW was simply the online community we grew up in, like others did from SA. I don't consider LW or Slate Star Codex or Robin Hanson to be something I "do" anymore, but they were parts of my growing up. It hurts my feelings a little when you make fun of crypto scammer nerds for being nerds rather than for being crypto scammers.

Expand full comment

we will be talking about Slate Star and Yudkowsky and the racism and pro-rape fetishism in their work. i don't care to mock anyone for being interested in those communities ten years ago but i do think it's worth discussing the way they reproduced cult dynamics.

per your "nerd" comment, i was also a nerd in high school. i got bullied for it. but i also think the term means something fundamentally different now. when we were kids it was an insult, and a way to mark you as weird and bad. today the most powerful people in the world proudly call themselves nerds, and nerd media utterly dominates popular culture. i think this has corresponded with some deeply toxic ideas about what it means to be nerdy, and the kind of behaviors that are positive in light of that (obsessive devotion to consumerism, myopic focus on media from 30 years ago, a refusal to consider the blinkered nature of such obsessions as a moral issue)

Expand full comment

Oddly enough i discovered slatestar through a cracked article during the David Wong era.

Expand full comment

Jason Pargin/David Wong still comments on the astral codex substack occasionally.

Expand full comment

I cannot wait to hear you talk about Yud's pro-rape, libertarian free market government baby-eating sci fi story. Only 3 people know about it. It's so god awful

Expand full comment

And probably the worst part is that it’s so specifically rooted in LW’s weird weeby early-2010s culture. Fucking ‘kiritsugu’

Expand full comment

haha I'm one of those three. I actually didn't know that rational-harry-potter was written by the baby-eating-aliens guy. I used to think that story was the most bizarre thing i would ever read. then i read that a bunch of yudkowsky fanboys managed to convince real-life people to give them multiple billions of dollars, and that took the top spot.

Expand full comment

I feel similarly.

In 2014-2015 ish, Scott's "Non-Libertarian FAQ", or whatever it was called, was what convinced me to change from being a Libertarian (formerly conservative) to a vaguely Progressive center-leftist. It was a flawed document to be sure, but I remember that blog in general with fondness. The criticisms I found at the time seemed to mostly be about Scott during his high school (from what I remember, at least), and as someone who changed a lot after high school, I assumed he had done the same and let it go.

In Fall of 2018, HPMOR is what helped me take the final step in giving up Catholicism, a decision I do not regret at all, especially given what I've learned in some BtB episodes. (I admit, I liked that this Harry chose Ravenclaw.)

In Fall of 2019 I started listening to Worst Year Ever, and then BtB; Robert Evans (my name is also Robert) was a large part of moving me away from being a huge Elizabeth Warren stan to voting for Bernie Sanders in 2020, and huge influence on me still today.

Scott and LW helped me think about the world around me in a different way than everyone else around me, which I found helpful given I live in an extremely Red state, and have a lot of conservatives in my family. Evans was a large part of continuing and stabilizing that leftward trend. I also learned a lot about psychiatric medicine thanks to Scott, which turned out to be helpful to me for a variety of unexpected reasons. Funnily enough, I found SneerClub earlier this year and found it a breath of fresh air, so I guess I was already getting frustrated and critical of the community. Then all this stuff happened with FTX.

I have complicated feeling about all of it; making fun of them hurts a bit, but on the other hand, I also realize this whole "rationalist" thing is like... specifically tailored both to help and to stroke the ego of people like me, so I feel a bit betrayed by Scott and the whole community. They were always problematic, but it seemed like the good outweighed the bad at the time. I was never into crypto thankfully, but a lot of criticisms I hear lately with regards to STEM communities in general are partially criticisms of me, of what I do for a living, of the type of people I'm around, of the way I used to think, and the way I still think sometimes out of habit. Taking criticism from people I respect (and in some ways cling to) is tough, but, I think, necessary.

That said, I'm glad to hear I'm not the only one who feels this way right now.

Expand full comment

I think the difference between a healthy nerd and a narcissistic nerd is the ability to sit with those hurt feelings and accept valid criticism with honesty and a bit of good grace. Self-awareness is what keeps us grounded.

Expand full comment

I love this entire saga. It's perfect distillation of how nobody, not even outcast nerds, should be given large amounts of power and money.

Expand full comment

Hi Roberts, love BtB, love your newborn substack. This entry has left me quite baffled, though.

When, years ago I read HPMOR, I did not feel to me like the protagonist was meant to be taken as an example of virtue. The whole arc of the story (spoilers!) is that Harry's unrestrained overconfidence and unwillingness to listen to advice bring him very, very close to killing every single human on Earth. The thing stopping the disaster is not his cunningness or intelligence, but the fact that the world's most powerful wizard (Dumbledore) had been working tirelessly to prevent it (sacrificing in the process his reputation and his own life). In the end, thanks to Dumbledore's sacrifice, Harry realizes what he is about to do one instant before setting off the disaster, epically shits his pants and finally figures out that his great, disruptive, world-reorganizing ideas maybe are not actually that great. Many of the elements in the narration (obsession with world domination, inability to listen to advice, minimizing risk, feeling smarter and superior etc.) I had read as the set up to the great he-almost-killed-everyone humbling in the end.

Compared to what I had gotten from the book this post (and Whelan's article that I discovered thanks to it) reads like someone summarizing «A Christmas Carol» as an endorsement of being a miser and hating the poors and Christmas because such is Scrooge at the beginning of the story.

Swapping the young wizard endowed with a tool (the scientific method) that gives him an edge over an establishment with a young techie endowed a tool (familiarity with novel informatic technology) that gives him an edge over an establishment; the whole book reads to me as a cautionary tale about Silicon Valley startups having a considerable destructive potential within our society. Which is an advice that I really wish were more listened to in the startup sphere.

I discover today that apparently most people (both fans and detractors) did not get this moral from HPMOR, which is... well dizzying. It feels like discovering to be the only one in a large group of friends to have ever noticed the boy hidden in the Camel logo. Was it really that well hidden?

Expand full comment

I appreciate the perspective here, thanks for sharing!

I'm also looking forward to any future podcase on Yudkowsky since reading through the article and the comments made me realize there's a LOT to unpack, and I didn't realize there was a weird cult around rationalists.

The way I recommended the book to my brother was by saying that, "it's got some good concepts about how to use rationalist thinking, but it's also got some critiques about using that type of thinking."

I too am surprised that most people did not get that moral from HPMOR.

Expand full comment

Damn, it feels good reading this... scrolling through the comments I thought I was the only one who got that message from the book.

Expand full comment

Hey Robert, I wanted to start by saying that I have a deep appreciation for all of your work. Your podcasts, books, reporting, and bird-app banter have all helped to grow my thinking tremendously over the past few years. Also, big thanks for coming to Seattle on your book tour. The Q&A was truly amazing, one of my favorite moments of the past year.

I probably have some unlearning to do and your upcoming podcast with Margaret Killjoy will likely help me to get there, but I took away a lot of valuable learnings from Yudkowsky’s Harry Potter and the Methods of Rationality. I also have a few friends and family members who enjoyed the book who I think would qualify as “normal, decent, and well-socialized human beings”. So I thought I’d share a personal perspective on the subject.

Harry Potter and the Methods of Rationality was particularly influential in getting me to a place where I’m able to confidently explore and question a set of proposed moral axioms (from the church, individuals, etc.) and to analyze difficult decisions or policy questions that involve complexity and uncertainty and will have significant impact on our communities. Previously I felt like I had no real tools for this other than the subjective norms of any chosen community and/or “my gut”. This fan fiction was a real gateway into a greater love for moral philosophy, causal inference, the scientific method, and asking hard questions about whether and how we can assess the quality of alternative choices or actions in a substantive way. At some point I think we have to be able to describe why some actions are preferred to others, and typically we rely on either religion or some subjective selection of things that “feel right”. This often leads to abundant contradiction, and gives us an excuse to not think critically about what we believe in and why. We can say that moral philosophy has always been available to us, but accessibility of the subject to large audiences is something that I think matters. The bulk of existing theory also seems to fail at giving frameworks for dealing with the reality that most decisions have outcomes which are probabilistic in nature. I found that HPMOR fills both of those gaps.

This book helped me to move left by shattering some remnants of liberal thinking still floating in my brain. It improved my ability to leverage robust critical thinking when it comes to questions about our collective responsibility to understand how and in what ways our actions ACTUALLY affect one another. It provided a foundation for moving away from woke-washed feel good behavior, and towards critical analysis and genuine empathy.

HPMOR didn't give me all of the answers but it gave me a language and a framework of thinking that helped lead me to the point where I am now. A point where I can say things like “I believe in preventing suffering, and promoting joyful experience, all done with a bias for equity”, and then be able to defend that stance with an argument that I believe is more robust than typical circular arguments given from religion, or common and fragile sentiments like “it’s just what I believe”. It led me to a point where I can use the languages of statistics and formal logic to extrapolate from those fundamental values to inform stances on decisions, actions, and policy decisions.

I haven’t consumed all of Yudkowsky’s works, and I don’t doubt that there are some well founded criticisms to be made and even some troubling skeletons to be discovered, but I also found Inadequate Equilibria (another of Yudkowsky’s [much shorter] works) to offer some innovative thinking for how we might “break away” from many of the most painful and frustrating mechanisms of modern society, from effectively dismantling the two party system to leveraging collective action in ways that are both sustainable and highly effective. That doesn’t mean Yudkowsky doesn’t belong on an episode of BtB (looking forward to discussions about cult dynamics and toxic “nerd hero” worship), but I don’t think anyone who takes value from these books should be labeled as weird or indecent.

My initial sense from both the BtB podcast on Sam Bankman-Fried and this piece on Shatter Zone is that what started with a solid and informed understanding about a few seriously troubled individuals who created some of the greatest grifts in history, eventually led to the discovery that they were “inspired by” a particular book and community… and by that time all of the knowledge about these grifters had us wearing whatever the opposite of rose colored glasses would be… which led to an understandable projection of their warped thinking onto that book and community which - though understandable - I’m not completely sure is fair.

It’s not that there aren’t plenty of valid criticisms to be made of HPMOR as well as Thinking Fast and Slow by Kahneman and Tversky (upon which much of HPMOR is based). I could speak at length. But I also found HPMOR to be an invaluable springboard for some profound modes of critical thinking and empathy, and I’m not the only person I know who shares this sentiment.

Anyways sorry for the long post, cheers and looking forward to your conversation with Margaret as well as more new content on Shatter Zone.

Expand full comment

One of my """favorite""" facts I ever learned while dating LessWrongers like Michael Vassar/Arc and Sarah Constantin was that Eliezer Yudkowsky would pick his female submissives by how good they were at solving equations. I wish I was making that up.

Also, check out SneerClub.

Expand full comment

oh this episode would not be possible without SneerClub

Expand full comment

They were also INCREDIBLY supportive of me and mine during that time, I owe SneerClub and David Gerard a lot.

Expand full comment

May I introduce you to my introduction to SneerClub after I shared my own experience with LW in the wake of Kathy Forth's suicide?

It's. A lot. https://www.reddit.com/r/SneerClub/comments/8sjxm9/serious_twitter_thread_by_someone_detailing_their/

Expand full comment

Well this is a thing I just learned about and now wish I hadn't. Jesus Sweet Christ.

Expand full comment

did he actually refer to them as "math pets"? or was that just a rumor?

Expand full comment

I remember reading a good chunk of this and thinking it was a fun absurdist satire of the too-rational types and how silly it would be to apply their ideas to a system as legendarily pulled out of the author's ass as the Harry Potter magic system. Then I looked into the author's website and realized it wasn't a joke and ran screaming from the fic.

Expand full comment

I read HPMOR as a teenager and absolutely loved it - not because I thought it was some kind of work of rational genius, but because, growing up around conservative religious types, I could relate to the feeling of insanity that comes from the people around you believing in magic without questioning it too hard.

I never thought Harry's character was supposed to be a genius - just a kid raised by Oxford professors, with all the weird social disconnection and overconfidence that entails.

So weird to see other people interpreting the story differently, and using it to justify batshit insane worldviews 😂

Expand full comment

Yeah, that's how I read the first part of the story and I like how its Harry wasn't a perfect genius, was ludicrously overconfident, and had weird social disconnections. But it turns out that the author meant for Harry to be a perfect genius and his idea of how a perfect genius should act is just deeply, deeply weird.

Expand full comment

aaaaaaaaaa god knows i read this entire fanfic for SOME reason, gotta correct u Sam, harry's explicitly a genius bc in this cannon, the horcrux piece of voldy's soul basically took over harry's identity when he was a baby, and voldy's a magical supergenius so harry's a supergenius too so they get to do a death note with supergenius dumbledore

Also i just finished reading Children of Dune, now i get to recognize how much HPMOR is obviously trying to be dune, the main cast is basically Leto Ghanima and Aria (voldy) and the plot beats line up with Children wayy to close imo, anyone else read dune series agree?

Difference is frank herbert A) knew what he was writing and JUSTIFIED his characters and B) has NO patience for plot twists, my god, tells u everything relevant to the story beats in the first chapter. makes this kind of story so much more readable and not, yknow, 600K words of anime nonsense

Expand full comment

I read the first chunk of Harry Potter and the Methods of Rationality and it does have it's moments, specifically skewering how the rules of Quidditch make no sense. One thing I like about it was that, unlike most fanfic, the author wasn't afraid to give the main character flaws and have him be a real dick at times. It took me an embarrassingly-long time to realize that the story's Harry Potter wasn't meant to be a dick at all but rather a perfect person as imagined by an asshole.

Expand full comment

No, the part of HP who wants to conquer the world is definitely presented as a bad guy, and this is reinforced toward the end of the story when we find out why he feels this way.

Expand full comment

Chapter four seems to be the origin of FTX's entire business model. It has a weird section all about exchange rates of silver and gold. In a universe where you can magic both into existence.

"Upon noticing the wizarding world uses coins of silver and gold, Hariezer asks about exchange rates, and asks the bank goblin how much it would cost to get a big chunk of silver turned into coins, the goblin says he’ll check with his superiors, Hariezer asks him to estimate, and the estimate is that the fee is about 5% of the silver.

This prompts Hariezer to realize that he could do the following:

Take gold coins and buy silver with them in the muggle world

bring the silver to Gringots and have it turned into coins

convert the silver coins to gold coins, ending up with more gold than you started with, start the loop over until the muggle prices make it not profitable"

(Taken from the scientist critique you linked above)

There are parallels, and then there are just thicker lines.

Expand full comment

HPMOR is one of my favourite books. I am not involved in anything crypto-related, and with the rationalist and effective altruism communities only in tangential ways, but this entire piece still feels like an ad hominem attack written in bad faith.

I made an account here just to say this. Happy to engage in discussion if anybody is interested.

Expand full comment

I don’t mind most marine mammals, but sea lions? I could do without sea lions.

Expand full comment

Which part was ad hominem or written in bad faith?

Expand full comment

Kinda weird how deep it cuts to see a man who I generally admire miss the point of one of my favourite books so completely.

The idea that !rationalHarry is someone to be admired or imitated makes about as much sense as saying Fear and Loathing in Las Vegas was shameless pro-Drug propaganda.

(Spoilers ahead)

Throughout the story Harry is portrayed and frequently called out by about every named character, the narration and by himself as an arrogant brat who sticks his nose in often dangerous matters he doesn't unterstand, he is completely played like a fiddle by the main villain (who is the only other "rational" character) and gets his best friend killed.

Near the end he realizes that almost the whole plot with several dead people would have been preventable if he had just decided to trust Dumbledore instead of making up clever excuses to trust his pretty clearly evil mentor.

Can't say I'm surprised that the Crypto bros didn't get the message between their whole thing being creating a new reality because they don't like the current one and the disturbing amount of Rorschach Fanboys out there, but I hope I can hold you to a higher standard than judging a book by skimming a few chapters and taking a look at a couple of idiots who didn't get the message.

Expand full comment

Gonna throw my hat into the ring with others defending HPMOR. Beyond being an entertaining introduction to a lot of useful concepts (and general mockery of JK Rowling), it's also pretty clearly miles ahead of the original series in its morality. Like one of Rationalist Potter's defining motivations is that he's a Death Eater Abolitionist and wants to harness magic to cure disease for all humanity.

Singling out HPMOR (vs. the original series, which is worse), and trying to tie it to the crimes of SBF (via his gf no less?) feels like a stretch. It's a very popular work. Some of its fans will inevitably be psychos. It's not like its perspective is inherently rotten or mostly attracts assholes, like say Atlas Shrugged. Even the Effective Altruism movement is mostly people who do lots of good charity work.

I'm sure you have much more to say about it, but some of your criticisms already come off as being in bad faith, or as cheap shots against nerds. It's too long? It's nowhere near as long as many other popular fanfics. It's also nowhere near as time consuming as many popular animes, or DnD podcasts, or sci fi shows. Nerds like long stuff. Similarly, a screenshot of a tongue-in-cheek tumblr fandom post does little to convince me that it breeds authoritarians or something.

I'm also baffled by the assertion that Yudkowsky doesn't do anything interesting with the premise. It's probably been a decade since I read it but I sure remember an incredible amount of detail going into the implications, physically and especially morally, of a world where science encounters magic. I was interested enough to read the whole thing, and I am definitely not the kind of nerd who likes long media (I can't even manage one episode of Critical Role lol).

Expand full comment

The Harry Potter series is already a Tony Blairian neoliberal wet dream; I don't even want to imagine what a self-indulgent wank off session a fan fic with a name like "Harry Potter and the Methods of Rationality" is.

Expand full comment

"He hits Voldemort out of the fucking park with a bunt while scratching his ass with his foot. "

I hate to be *that guy* but at the end of the story Harry only beats Voldermort because Harry knows one particular technique that Voldermort never would have figured out because he wasn't raised with a muggle understanding of how objects break down at the microscopic level... and this comes after Harry realizes that he's been utterly out maneuvered by Voldermort.

The story is plenty bad, but it by no means depicts Voldermort as being no threat to Harry.

Expand full comment

Agreed. One of the things I liked about the book was that the bad guy was terrifying because he seemed to be intellectually terrifying and I had no idea how Harry was going to win. Early on in the book Harry is basically unstoppable - which is fun - but also sets up his fall from confidence once he actually meets the true villain.

Voldemort represented truly unrestrained rationalist immoral evil. Harry represented someone about to walk down that path if things didn't change, and by the end he was spooked enough by his experiences to reconsider a lot of his life.

Book still had problems and earned a lot of eye rolls from me, but I didn't see it as a

Expand full comment

I think Eliezer Yudkowsky is something akin to a cult leader. LessWrong is just a website but a lot of the people involved in that community were actually living in communal "group houses" in the San Francisco bay area (this started years before the FTX polycule) and have quasi-religious ideas about AI (some of them were legitimately afraid of Roko's Basilisk) . At one point an underage girl who ran away from hom joined the Bay Area Rationalist community, and there have since been allegations of psychological and sexual abuse, statutory rape, blackmail and cover-ups.

https://fredwynne.medium.com/an-open-letter-to-vitalik-buterin-ce4681a7dbe

https://archive.ph/Kvfus

https://deluks917.wordpress.com/2019/09/20/private-post/

https://archive.ph/epyMh

https://everythingtosaveit.how/case-study-cfar/

(Sorry for dumping link like this, but part of the problem is that no journalist has really covered this yet, and I'm still trying to piece together what happened from disconnected blog posts)

I think an archived copy of Eliezer Yudkowsky's public OkCupid profile might also be worth looking at, might give you some insight into his character (among other things, he describes himself as having "a strong sexually sadistic streak"):

https://web.archive.org/web/20120829144053/http://www.okcupid.com/profile/EYudkowsky

Expand full comment

Jesus Christ I did not know it was 660K words.

Expand full comment