68 Comments

A long time ago, (an in-person chapter of) Less Wrong was my "tribe" of weirdo geeks who would do things like use neurodes and electromagnets (personally attached) and then try to see if that resulted in improving short-term memory performance. We'd meet and talk about astronomy, health foods (including one incident where a friend brought huge bags of dried kale to a coffee shop that looked like a drug-deal in progress), D&D, and life in general. They were some of the first groups I met who were openly accepting of gay and trans people. And when I was in high school / early college, they were my first exposure to a lot of new ideas about philosophy and AI.

All that to say, I hope you don't condemn weird nerds for being weird nerds. We're doing our best, and LW was simply the online community we grew up in, like others did from SA. I don't consider LW or Slate Star Codex or Robin Hanson to be something I "do" anymore, but they were parts of my growing up. It hurts my feelings a little when you make fun of crypto scammer nerds for being nerds rather than for being crypto scammers.

Expand full comment

I love this entire saga. It's perfect distillation of how nobody, not even outcast nerds, should be given large amounts of power and money.

Expand full comment

Hi Roberts, love BtB, love your newborn substack. This entry has left me quite baffled, though.

When, years ago I read HPMOR, I did not feel to me like the protagonist was meant to be taken as an example of virtue. The whole arc of the story (spoilers!) is that Harry's unrestrained overconfidence and unwillingness to listen to advice bring him very, very close to killing every single human on Earth. The thing stopping the disaster is not his cunningness or intelligence, but the fact that the world's most powerful wizard (Dumbledore) had been working tirelessly to prevent it (sacrificing in the process his reputation and his own life). In the end, thanks to Dumbledore's sacrifice, Harry realizes what he is about to do one instant before setting off the disaster, epically shits his pants and finally figures out that his great, disruptive, world-reorganizing ideas maybe are not actually that great. Many of the elements in the narration (obsession with world domination, inability to listen to advice, minimizing risk, feeling smarter and superior etc.) I had read as the set up to the great he-almost-killed-everyone humbling in the end.

Compared to what I had gotten from the book this post (and Whelan's article that I discovered thanks to it) reads like someone summarizing «A Christmas Carol» as an endorsement of being a miser and hating the poors and Christmas because such is Scrooge at the beginning of the story.

Swapping the young wizard endowed with a tool (the scientific method) that gives him an edge over an establishment with a young techie endowed a tool (familiarity with novel informatic technology) that gives him an edge over an establishment; the whole book reads to me as a cautionary tale about Silicon Valley startups having a considerable destructive potential within our society. Which is an advice that I really wish were more listened to in the startup sphere.

I discover today that apparently most people (both fans and detractors) did not get this moral from HPMOR, which is... well dizzying. It feels like discovering to be the only one in a large group of friends to have ever noticed the boy hidden in the Camel logo. Was it really that well hidden?

Expand full comment

Hey Robert, I wanted to start by saying that I have a deep appreciation for all of your work. Your podcasts, books, reporting, and bird-app banter have all helped to grow my thinking tremendously over the past few years. Also, big thanks for coming to Seattle on your book tour. The Q&A was truly amazing, one of my favorite moments of the past year.

I probably have some unlearning to do and your upcoming podcast with Margaret Killjoy will likely help me to get there, but I took away a lot of valuable learnings from Yudkowsky’s Harry Potter and the Methods of Rationality. I also have a few friends and family members who enjoyed the book who I think would qualify as “normal, decent, and well-socialized human beings”. So I thought I’d share a personal perspective on the subject.

Harry Potter and the Methods of Rationality was particularly influential in getting me to a place where I’m able to confidently explore and question a set of proposed moral axioms (from the church, individuals, etc.) and to analyze difficult decisions or policy questions that involve complexity and uncertainty and will have significant impact on our communities. Previously I felt like I had no real tools for this other than the subjective norms of any chosen community and/or “my gut”. This fan fiction was a real gateway into a greater love for moral philosophy, causal inference, the scientific method, and asking hard questions about whether and how we can assess the quality of alternative choices or actions in a substantive way. At some point I think we have to be able to describe why some actions are preferred to others, and typically we rely on either religion or some subjective selection of things that “feel right”. This often leads to abundant contradiction, and gives us an excuse to not think critically about what we believe in and why. We can say that moral philosophy has always been available to us, but accessibility of the subject to large audiences is something that I think matters. The bulk of existing theory also seems to fail at giving frameworks for dealing with the reality that most decisions have outcomes which are probabilistic in nature. I found that HPMOR fills both of those gaps.

This book helped me to move left by shattering some remnants of liberal thinking still floating in my brain. It improved my ability to leverage robust critical thinking when it comes to questions about our collective responsibility to understand how and in what ways our actions ACTUALLY affect one another. It provided a foundation for moving away from woke-washed feel good behavior, and towards critical analysis and genuine empathy.

HPMOR didn't give me all of the answers but it gave me a language and a framework of thinking that helped lead me to the point where I am now. A point where I can say things like “I believe in preventing suffering, and promoting joyful experience, all done with a bias for equity”, and then be able to defend that stance with an argument that I believe is more robust than typical circular arguments given from religion, or common and fragile sentiments like “it’s just what I believe”. It led me to a point where I can use the languages of statistics and formal logic to extrapolate from those fundamental values to inform stances on decisions, actions, and policy decisions.

I haven’t consumed all of Yudkowsky’s works, and I don’t doubt that there are some well founded criticisms to be made and even some troubling skeletons to be discovered, but I also found Inadequate Equilibria (another of Yudkowsky’s [much shorter] works) to offer some innovative thinking for how we might “break away” from many of the most painful and frustrating mechanisms of modern society, from effectively dismantling the two party system to leveraging collective action in ways that are both sustainable and highly effective. That doesn’t mean Yudkowsky doesn’t belong on an episode of BtB (looking forward to discussions about cult dynamics and toxic “nerd hero” worship), but I don’t think anyone who takes value from these books should be labeled as weird or indecent.

My initial sense from both the BtB podcast on Sam Bankman-Fried and this piece on Shatter Zone is that what started with a solid and informed understanding about a few seriously troubled individuals who created some of the greatest grifts in history, eventually led to the discovery that they were “inspired by” a particular book and community… and by that time all of the knowledge about these grifters had us wearing whatever the opposite of rose colored glasses would be… which led to an understandable projection of their warped thinking onto that book and community which - though understandable - I’m not completely sure is fair.

It’s not that there aren’t plenty of valid criticisms to be made of HPMOR as well as Thinking Fast and Slow by Kahneman and Tversky (upon which much of HPMOR is based). I could speak at length. But I also found HPMOR to be an invaluable springboard for some profound modes of critical thinking and empathy, and I’m not the only person I know who shares this sentiment.

Anyways sorry for the long post, cheers and looking forward to your conversation with Margaret as well as more new content on Shatter Zone.

Expand full comment

One of my """favorite""" facts I ever learned while dating LessWrongers like Michael Vassar/Arc and Sarah Constantin was that Eliezer Yudkowsky would pick his female submissives by how good they were at solving equations. I wish I was making that up.

Also, check out SneerClub.

Expand full comment

I remember reading a good chunk of this and thinking it was a fun absurdist satire of the too-rational types and how silly it would be to apply their ideas to a system as legendarily pulled out of the author's ass as the Harry Potter magic system. Then I looked into the author's website and realized it wasn't a joke and ran screaming from the fic.

Expand full comment

I read HPMOR as a teenager and absolutely loved it - not because I thought it was some kind of work of rational genius, but because, growing up around conservative religious types, I could relate to the feeling of insanity that comes from the people around you believing in magic without questioning it too hard.

I never thought Harry's character was supposed to be a genius - just a kid raised by Oxford professors, with all the weird social disconnection and overconfidence that entails.

So weird to see other people interpreting the story differently, and using it to justify batshit insane worldviews 😂

Expand full comment

I read the first chunk of Harry Potter and the Methods of Rationality and it does have it's moments, specifically skewering how the rules of Quidditch make no sense. One thing I like about it was that, unlike most fanfic, the author wasn't afraid to give the main character flaws and have him be a real dick at times. It took me an embarrassingly-long time to realize that the story's Harry Potter wasn't meant to be a dick at all but rather a perfect person as imagined by an asshole.

Expand full comment

Chapter four seems to be the origin of FTX's entire business model. It has a weird section all about exchange rates of silver and gold. In a universe where you can magic both into existence.

"Upon noticing the wizarding world uses coins of silver and gold, Hariezer asks about exchange rates, and asks the bank goblin how much it would cost to get a big chunk of silver turned into coins, the goblin says he’ll check with his superiors, Hariezer asks him to estimate, and the estimate is that the fee is about 5% of the silver.

This prompts Hariezer to realize that he could do the following:

Take gold coins and buy silver with them in the muggle world

bring the silver to Gringots and have it turned into coins

convert the silver coins to gold coins, ending up with more gold than you started with, start the loop over until the muggle prices make it not profitable"

(Taken from the scientist critique you linked above)

There are parallels, and then there are just thicker lines.

Expand full comment

HPMOR is one of my favourite books. I am not involved in anything crypto-related, and with the rationalist and effective altruism communities only in tangential ways, but this entire piece still feels like an ad hominem attack written in bad faith.

I made an account here just to say this. Happy to engage in discussion if anybody is interested.

Expand full comment

Kinda weird how deep it cuts to see a man who I generally admire miss the point of one of my favourite books so completely.

The idea that !rationalHarry is someone to be admired or imitated makes about as much sense as saying Fear and Loathing in Las Vegas was shameless pro-Drug propaganda.

(Spoilers ahead)

Throughout the story Harry is portrayed and frequently called out by about every named character, the narration and by himself as an arrogant brat who sticks his nose in often dangerous matters he doesn't unterstand, he is completely played like a fiddle by the main villain (who is the only other "rational" character) and gets his best friend killed.

Near the end he realizes that almost the whole plot with several dead people would have been preventable if he had just decided to trust Dumbledore instead of making up clever excuses to trust his pretty clearly evil mentor.

Can't say I'm surprised that the Crypto bros didn't get the message between their whole thing being creating a new reality because they don't like the current one and the disturbing amount of Rorschach Fanboys out there, but I hope I can hold you to a higher standard than judging a book by skimming a few chapters and taking a look at a couple of idiots who didn't get the message.

Expand full comment

Gonna throw my hat into the ring with others defending HPMOR. Beyond being an entertaining introduction to a lot of useful concepts (and general mockery of JK Rowling), it's also pretty clearly miles ahead of the original series in its morality. Like one of Rationalist Potter's defining motivations is that he's a Death Eater Abolitionist and wants to harness magic to cure disease for all humanity.

Singling out HPMOR (vs. the original series, which is worse), and trying to tie it to the crimes of SBF (via his gf no less?) feels like a stretch. It's a very popular work. Some of its fans will inevitably be psychos. It's not like its perspective is inherently rotten or mostly attracts assholes, like say Atlas Shrugged. Even the Effective Altruism movement is mostly people who do lots of good charity work.

I'm sure you have much more to say about it, but some of your criticisms already come off as being in bad faith, or as cheap shots against nerds. It's too long? It's nowhere near as long as many other popular fanfics. It's also nowhere near as time consuming as many popular animes, or DnD podcasts, or sci fi shows. Nerds like long stuff. Similarly, a screenshot of a tongue-in-cheek tumblr fandom post does little to convince me that it breeds authoritarians or something.

I'm also baffled by the assertion that Yudkowsky doesn't do anything interesting with the premise. It's probably been a decade since I read it but I sure remember an incredible amount of detail going into the implications, physically and especially morally, of a world where science encounters magic. I was interested enough to read the whole thing, and I am definitely not the kind of nerd who likes long media (I can't even manage one episode of Critical Role lol).

Expand full comment

The Harry Potter series is already a Tony Blairian neoliberal wet dream; I don't even want to imagine what a self-indulgent wank off session a fan fic with a name like "Harry Potter and the Methods of Rationality" is.

Expand full comment

"He hits Voldemort out of the fucking park with a bunt while scratching his ass with his foot. "

I hate to be *that guy* but at the end of the story Harry only beats Voldermort because Harry knows one particular technique that Voldermort never would have figured out because he wasn't raised with a muggle understanding of how objects break down at the microscopic level... and this comes after Harry realizes that he's been utterly out maneuvered by Voldermort.

The story is plenty bad, but it by no means depicts Voldermort as being no threat to Harry.

Expand full comment
Dec 4, 2022·edited Dec 4, 2022

I think Eliezer Yudkowsky is something akin to a cult leader. LessWrong is just a website but a lot of the people involved in that community were actually living in communal "group houses" in the San Francisco bay area (this started years before the FTX polycule) and have quasi-religious ideas about AI (some of them were legitimately afraid of Roko's Basilisk) . At one point an underage girl who ran away from hom joined the Bay Area Rationalist community, and there have since been allegations of psychological and sexual abuse, statutory rape, blackmail and cover-ups.

https://fredwynne.medium.com/an-open-letter-to-vitalik-buterin-ce4681a7dbe

https://archive.ph/Kvfus

https://deluks917.wordpress.com/2019/09/20/private-post/

https://archive.ph/epyMh

https://everythingtosaveit.how/case-study-cfar/

(Sorry for dumping link like this, but part of the problem is that no journalist has really covered this yet, and I'm still trying to piece together what happened from disconnected blog posts)

I think an archived copy of Eliezer Yudkowsky's public OkCupid profile might also be worth looking at, might give you some insight into his character (among other things, he describes himself as having "a strong sexually sadistic streak"):

https://web.archive.org/web/20120829144053/http://www.okcupid.com/profile/EYudkowsky

Expand full comment

Jesus Christ I did not know it was 660K words.

Expand full comment