Sci-fi: Bodhisattvas and "Understand"

DISCLAIMER: I highly recommend reading Chiang’s story, it’s fairly short, and absolutely fantastic. The worst sort of story-ruining follows. You’ve been warned.

Ted Chiang writes a relatively small amount of fiction, writing technical computer manuals as his day job. However, Chiang has been very succesful - his succinct and potent stories have garnered many accolades. His short story, Story of your Life, was made into the Denis Villeneuve film Arrival.

Chiang’s stories are completely driven by a core idea, with the narrative being a scaffolding to support the central concept. Understand offers a first-person view of what it might feel like to acquire super-human intelligence.

Leon Greco has become brain damaged after nearly drowning, and is given a compound that increases intelligence. His rapidly increased intelligence eventually leads him to a confrontation with Reynolds who has also received the drug. They both have different conceptions of how to apply their accelerating intelligence: Greco wants to pursue further intelligence gains in pursuit of “enlightenment” AKA complete understanding AKA omnipotence; the other character wishes to use his superintelligence for the benefit of humanity at large, to create a prosperous, flourishing world for all human beings.

In the end, Greco’s desire for omniscient “enlightenment” proves to be his downfall. The phrase that defeats him is “understand” - and it is his understanding of this phrase that causes him to “dissolve”.

Chiang purposefully uses the language of “enlightenment” and “meditation” to elicit comparison between the two main characters: Reynolds the bodhisattva, and Greco the arhat. Arhats are those that achieve enlightenment for themselves, while bodhisattvas are always dedicating their work for the benefit of other sentient beings.

Within Mahayana Buddhism, there is a derogatory word for arhats (or those who have achieved enlightenment without regard for other beings): sravakas. Sravakas care only about the individual liberation, while bodhisattvas care for the welfare of all beings. Patrul Rinpoche distinguishes between three types of intention that bodhisattvas possess:

1. king-like bodhicitta - to aspire to become a buddha first in order to then help sentient beings
2. boatman-like bodhicitta - to aspire to become a buddha at the same time as other sentient beings
3. shepherd-like bodhicitta - to aspire to become a buddha only after all other sentient beings have done so

Reynolds has the third type of intention: he does not want to expand his intelligence past the point of caring about human beings. For him, wisdom is the method towards a compassionate end. For Greco, intelligence is the end-in-itself.

In Tibetan Buddhism there is a harmonious interplay between dual aspects: wisdom and compassion. These two aspects are seen to be necessary for the full flourishing of Buddhahood. Without either one, the development of the practitioner will be stunted. The protagonist, Greco, pursues intelligence and knowledge for its own sake – this proves to be his own undoing. Chiang’s thesis is that the pursuit of an isolated and fragmented wisdom is not onyl immoral, but ultimately weaker than a holistic combination of wisdom and compassion.

[Reynolds] has an unmentioned plan for establishing a global network of influence, to create world prosperity. To execute it, he’ll employ a number of people, some of whom he’ll give simple heightened intelligence, some meta-self-awareness; a few of them will pose threats to him. “Why assume such a risk for the sake of the normals?”

“Your indifference toward the normals would be justified if you were enlightened; your realm wouldn’t intersect theirs. But as long as you and I can still comprehend their affairs, we can’t ignore them.”

I can measure the distance between our respective moral stances precisely, see the stress between their incompatible radiating lines. What motivates him is not simply compassion or altruism, but something that entails both those things. On the other hand, I concentrate only on understanding the sublime. “What about the beauty visible from enlightenment? Doesn’t it attract you?”

“You know what kind of structure would be required to hold an enlightened consciousness. I have no reason to wait the time it would take to establish the necessary industries.”

He considers intelligence to be a means, while I view it as an end in itself. Greater intelligence would be of little use to him. At his present level, he can find the best possible solution to any problem within the realm of human experience, and many beyond. All he’d require is sufficient time to implement his solution.”

Reynolds is not interested in leaving the world to its fate. He wishes to bring about quasi-bodhisattva’s to help in his quest to save humanity. It is not mentioned, but perhaps the final goal will be enlightened consciousness for all beings (as long as there is sufficient material to build the necessary structures).

Side-note on AI:

Eliezer Yudkowsky believes that intelligence and values are orthogonal, that is, values and intelligence aren’t necessarily related: a being can be intelligent with any set of values. Chiang’s story depicts two agents with comparable intelligence, but two very different sets of values (one values other human beings, the other values beauty and intelligence). The chances of a non-human intelligence hitting upon human-aligned values is absolutely tiny (because the space of “all values” is truly enormous). A charitable reason for Chiang to give 1 out of 2 agents human-aligned values, is because both characters are human (despite being superintelligent). Funny enough, Chiang does not think that runaway artificial-superintelligence is a danger to humanity link.