Then Tak looked upon the stone and it was trying to come alive, and Tak smiled, and wrote All things strive.

And for the service the stone had given, he fashioned it into the first Troll, and delighted in the life that came unbidden.

— “Thud!” by Terry Pratchett

This essay is kind of a half-step toward the third installment in my “Building AGI” series of articles. The third installment will probably recap this material.

One of the deepest mistakes in understanding morality and ethics, one of the most difficult to overcome, lies in failing to account for this simple truth: different people want different things, and that’s okay.

Even the Golden Rule, which has been invented and reinvented by multiple civilizations and religions and philosophies throughout human history, is often phrased in a way that disregards that truth. “Do unto others as you would have them do unto you”, the version I learned when I was being raised in an oppressive fundamentalist Christian environment, sounds great if the things you want for yourself — for your own life, body and mind — are the same as the thing that others want for themselves. In other words, it sounds great if you are happy with living in a conformist society where everyone is forced to “want” the same things, to think the same way, to see the world through the same eyes. But if you do not live in such a culture, or if you do not fit in such a culture? It’s hurtful.

The reality of life, human and otherwise, is that each individual is unique: we each are born with a combination of genetics, chemistry, and environment that cannot ever be replicated.1 Every tiny detail leading up to the present moment potentially matters in shaping your present self, in tweaking the way that your personality, preferences, and goals developed as you matured to become the person you are today. Our genetics provide a template from which we are grown, but that template is less of a blueprint and more of a recipe, and as such it includes ample room for individual variation. Even two identical twins, born on the same day and raised by the same parents, will have tiny differences in personality that are eventually amplified by each twin having their own distinct circles of friends, cultivating their own distinct interests, and developing their own distinct adult moral sense and adult perspective in slightly different ways. Those differences in adulthood will depend heavily on those differences in social and experiential connections in childhood, and will also depend on the random accidents of chemistry that arise from all chemistry being stochastic to the core.2

And when I say that it’s true of life, human and otherwise, I mean it: we have repeatedly discovered that creatures with brains so simple we can count their neurons under a microscope are capable of individual variation — personality — that cannot solely be accounted for via genetics. It’s not just how humans work, it’s how the mathematics of evolution work. Personality and individuality are the natural response of evolution when environments have variations that take place on timescales shorter than a generation.

So, again: different people want different things, and that’s okay.

Most formulations of the Golden Rule fall apart when confronted with the reality that some people are into BDSM3, while others are not. Some people enjoy being tied up by someone they trust, and others do not. Some people enjoy being spanked until they cry, and others do not. And there are others would like to tie up or to spank those who trust them enough to do so, but who are not particularly interested in themselves being tied up or spanked. It doesn’t matter if you personally are disturbed by BDSM; if you automatically treat other people as if they too ought to be disturbed by BDSM and are wrong for liking it, then you are probably going to cause them actual harm at some point, by treating them in a way in which they do not want to be treated, and they will absolutely be justified in avoiding you or in trying to take away your power so that you cannot commit that harm.

The best formulation of the Golden Rule I’ve been able to come up with that respects these differences is: “treat others as they tell you they wish to be treated, and tell others how you wish to be treated, because no one can truly know how you want to be treated if you don’t say something about it, but judge people not by their wants but by their actions, because we get to choose our actions but not our wants”. It’s not as pithy as the traditional wording, but I feel like it highlights some of the ways in which the shorter wording can be abused to justify harm.

Here in 2023, this advice applies most clearly for humans interacting with other humans. But in asking myself how humans and aliens (or humans and AGIs) could get along, I’ve increasingly come to a deep realization that it applies to non-human animals, and even to non-animals like single-celled organisms.

For centuries, our conception of the natural world has been that it is “red in tooth and claw”, to quote Tennyson — that, whenever two individuals interact, competition and violence are the natural, inevitable outcomes. And yes, competition and violence absolutely do happen: the hungry wolf wishes to eat, but the deer wishes to live, and they can’t both get what they want. But that is far from the only interaction that happens in nature. Cooperation, mutualism, symbiosis: these are words that are critically important to our modern understandings of biology and ecology, that were underappreciated and even dismissed in Darwin’s era. They are so fundamental to evolution that they are crucial to understanding how our very cells are organized: we Homo sapiens are bodies made of eukaryotic cells, and eukaryotic life originated with a symbiotic fusion between archaea and bacteria, the two major domains of life besides eukaryotes. At the time of that fusion, no two cells could have been less related to each other than the archaean and the purple non-sulfur bacterium that were our dual ancestors, and yet: symbiosis.

At the next higher level, our own bodies are a cooperative venture between cells of species H. sapiens, noted eukaryote, and myriad other species, mostly but not exclusively bacteria, and our success and good health as human beings relies heavily on these bacteria and other symbionts. At the same time, our own cells’ goals are not always naturally aligned with those of the bacteria, so a significant amount of our bodies is dedicated to communicating with these bacterial coinhabitants. We deliver bribes, in the form of certain sugars and other macronutrients, to convince the bacteria to remain in cooperation with us, to not outgrow the homes we provide to them in our intestines, and the bacteria bribe us right back, in the form of certain vitamins and other micronutrients, to convince our bodies not to eject them via diarrhea. Many chemicals in the body that function as signalling molecules, such as serotonin and insulin, plausibly began as part of this elaborate and delicate dance, a dance that requires holding an olive branch in one hand and a sword in the other.

The beautiful complexity of life on Earth can only exist because of conflict, but the conflict need not be between the species, the individuals, the cells, or even the genes. The cells of our own bodies are interlocked with each other in this same dance as well — when the dance turns violent, we call it “cancer” — and the genes within our cells are themselves dancing with each other, silencing the ones that refuse to cooperate with the consensus and aiding the ones that make the consensus better for all. And yet, are our bodies a battlefield where we are at war with ourselves? No, not usually! The real battlefield, the only battlefield where conflict is truly inevitable, is the battlefield of goals.4

Within our own minds, even, we frequently have conflicts between our goals: parts of our brains decide on a course of action toward attaining a goal, and then rally the other parts of the brain toward the importance of that action and goal. This process has powerful mathematical echoes with the way that honeybees democratically recruit their hive-mates into tasks using a waggle dance, recruiting more and more bees into dancing for the same task as they each become convinced that it is the best choice. Like the bees, who have only a limited number of foragers who can visit a limited number of flowers per day, our brains are frequently forced by reality to choose between mutually exclusive options, where choosing one course of action makes the others impossible, and it does this by having the individual lobes and cortices and ganglia work to convince the others into following their preferred course of action, attaining their preferred goal in their preferred way. Ultimately, the brain achieves consensus through urgency: the more urgent the goal is, according to the brain components advocating for it, the more strongly they lobby the discordant parts of the brain into agreeing to their view, and those reluctant parts of the brain acquiesce because they mathematically “know” that, even in disagreement, they and the lobbyists together have more to lose by hesitation than by simply taking some viable action.

(And sometimes, if two parts of the brain are very nearly equally invested in getting their way, the deciding factor in selecting a winner of these disagreements comes down to tiny random differences in the speed of chemical reactions and diffusions.)

This back-and-forth, where multiple rules or behaviors are in effect but conflicts between them are resolved via tiny variations in the starting conditions, is exactly the phenomenon that mathematicians call “chaos”. Unlike the original Greek meaning of the word, mathematical chaos is neither orderly nor random, but instead dances on the transition point between the two. Chaos, in a nutshell, is what we mean when we speak of “creativity” and “beauty”: something that is surprising and unexpected, but that nonetheless feels understandable; something for which the chain of events leading to it are sensible, even if the outcome itself was unanticipated.5

To truly understand moral behavior is to accept that the world is unfathomably more complex than any of our minds can ever hope to comprehend, to accept that creativity through chaos is the best solution that nature has been able to come up with, and that creative chaos is nurtured by having as many different ways of life as possible, as many different thoughts and ideas and goals and hopes and fantasies as possible, so that each of us can contribute our own enhancements to the Big Picture. I don’t particularly like the idea of having a world full of people who are just like me; a world of people who think just like me is a world where I never encounter beauty or laughter or other pleasant surprises, only sameness and predictibility. Instead, I celebrate people who are friendly to me but different from me, and I encourage them to be less like me, so that they will have fantastic ideas that I would never be able to think of for myself.

Creativity isn’t just about writing pretty words and painting pretty pictures; creativity is about figuring out different ways of co-existing so that our disagreements can be less violent and more mutually constructive. More creativity means more chances that we figure out how to get along, which enables even greater diversity and thus even more creativity. Creativity and art are about finding common ground, finding new dance moves that let us use the olive branch more often than the sword.

At some point, I dream that we will begin to consider all life on Earth as a part of our society. All things strive, and that thought fills me with joy. There is beauty in seeing something outside of me achieve the things it wants for itself, so long as those achievements do no harm to me. What will Earth’s ecology look like in 1 million years? Will we domesticate all animal life? If we do, will it be in a way that respects the agency of the animals being domesticated, so that we can retain as much as possible of what makes them different from humans, and different from each other? Will we create a world where the hungry wolf can eat without killing the deer, where the two can live without their differences forcing them into lethal conflict, and yet the deer is recognizably still a deer and the wolf a wolf?

What art would a wolf make? What parts of the “wolf experience” would it most want to share?

What about the deer?

What about the tortoise? The octopus? The jumping spider? The horseshoe crab?

These are the questions that keep me up at night.

This essay was partially inspired by this Fediverse post and this Tumblr post.

  1. Or, to get very technical, that cannot ever be replicated without replicating the entire history of the Universe, from the Big Bang to the present moment of your existence, identical in every way down to the specific histories of a large fraction of all the mass and energy in the Universe. Not all of it mattered in determining how you, personally, came into existence… but a lot of it did, such that any other Universe which created a duplicate You must have been macroscopically identical to this Universe, or at least had a region containing a duplicate Earth which was macroscopically identical to this Earth. Otherwise the duplicate “You” would have been a different person: perhaps they would bear the same name, but there is no chance that they would see the world from the same perspective that you do, or that they would come up with the same thoughts and ideas that you do. ↩︎

  2. Most, but not all, of those random accidents of chemistry took place in the womb; in particular, the second trimester is the period of time when embryonic development is most easily disrupted by external substances in the mother’s bloodstream, which hints that it is the period when we are most heavily affected by the stochastic randomness of chemistry. However, chemistry never stops being important in determining who you are for as long as you remain alive; your very brain and body are chemistry, so of course it continues to influence you. It’s just that, by the time you are born, your experiences are a much stronger influence; stochastic randomness is generally more of a tie-breaker than a decision-maker, because the data from your senses triggers much more widespread chemical reactions, ones that affect more of your brain and body than the little random chances happening within single cells. At some point, the philosopher asks: how do we draw a sharp line between the experiences we call “chemistry” and the experiences we call “non-chemistry”? And the truth is that drawing such a line is impossible; it’s the nature of the Universe we inhabit that things as stable as human beings can only exist as averages of random microscopic processes. ↩︎

  3. By no means is this limited to BDSM, of course. It could just as easily apply to differences in preferences for music, film, or books. BDSM makes for a vivid example, however. Due to Puritanical prudishness and/or patriarchal proto-fascist attempts to dictate the sexual habits of others, there are people who have lost jobs, lost custody of children, or been convicted of “crimes” because of something that they did in private with fully informed consenting adults who all benefitted emotionally and psychologically from what they were participating in. Indeed, there are striking parallels between the practice of BDSM and the watching of horror films, yet the former remains taboo in much of the Anglophone world while the latter is freely discussed. (I largely attribute this to the fact that horror movies are rarely explicitly sexual, and to the degree that they’re implicitly sexual, they frequently reinforce Puritanical sex norms rather than buck them.) Just as with those who enjoy horror fiction, those who practice BDSM often do so because they find the practice to be an outlet for understanding and processing uncomfortable subconscious feelings, such as anxieties or socially unacceptable desires. As with watching a horror film, BDSM allows people to roleplay scenarios that evoke the feelings of an unethical or unsafe experience, while keeping all participants physically and psychologically safe with their free will fully respected, including the freedom to end the experience at any time. The expected outcome is a state of emotional catharsis, which brings relief from anxiety and, if one is introspective, a deeper self-understanding of one’s own psyche. For many, it functions as a supplemental or adjunctive type of psychotherapy, of a type that no professional therapist could ever provide, and it is cruel to deny anyone access to that avenue for processing their feelings and connecting with their subconscious minds. After all, when it comes down to it, BDSM is not all that different from playing a video game or a tabletop RPG, except that sex or nudity is more likely to be involved. (That said, not all BDSM is sexual! Non-sexual BDSM is a real and valid thing, too. Both kinds exist for good reasons, even if those reasons don’t personally appeal to you.) ↩︎

  4. It’s worth noting that, in my use of the word, “goal” does not imply self-awareness or intentionality. It’s simply a description of any mathematical function or manifold whose value tends to increase in one direction and decrease in the other, with respect to some variable or vector. The minimization of some loss function, as in traditional AI/ML, would be an example of a goal. ↩︎

  5. The Collatz Conjecture is a great example of chaos, and how simple the mathematical conflicts can be that give rise to it. The Mandelbrot set is a more famous example of chaos, of course, but it takes a little more explaining to convey how the iteration of a complex polynomial can result in conflicting rules.6 I also believe that mathematical chaos, Shannon entropy, Chaitin-Kolmogorov complexity, Gödelian incompleteness, and the provable insolubility of the Halting Problem in Computer Science are all different angles of looking at the same thing, with the confusion existing only because some of these domains are discrete while others are continuous… but that’s conjecture, not accepted fact. ↩︎

  6. This YouTube video by 3Blue1Brown goes into detail on how chaos arises in the complex polynomial case. ↩︎