Ethical future? Science fiction and the tech billionaires

9 June this year marked the tenth anniversary of the death of the Scottish author Iain Banks. A self-declared ‘evangelical atheist’, Banks’s fiction is famous for its macabre, mind-bending narratives exploring the extremes of the human psyche; his 1984 debut novel, The Wasp Factory, concerns a psychopathic teenager who believes that the future can be ordained by the movements of wasps within a cruel maze of his own design. Despite a sizeable body of well-received fiction, it is Banks’s science fiction that he himself considered his best and most important work. He differentiated his two writing styles with the addition of a middle initial, M.

Iain M. Banks believed that science fiction is the only genre that seeks to tackle the effects of scientific and technological change on the individual and society. Banks’ vision of a utopian society is the ‘Culture’: a post-scarcity, space-faring, hedonistic, egalitarian, atheistic, communist utopia, run entirely by sentient artificial intelligences. The Culture is a ‘classic’ vision of utopia: their citizens want for nothing and live in a state of hedonist bliss, freed from labour to pursue higher human meaning. A reader would find it difficult to overlook the socialistic overtones of the Culture, where money is considered a crude form of rationing and other civilisations are frowned upon for the barbarism of their hierarchical societies and lack of empathy.

The first book of the Culture series, Consider Phlebas, is set during an intergalactic war between the Culture and a species of tripedal aliens that are on a religious crusade to proselytise the galaxy. The Culture—despite not being at threat themselves—consider it their moral imperative to intervene. Over the course of a nine-novel series, Banks fleshes out the Culture’s society. To a backdrop of vivid space-opera grandeur, spattered with lasers fights and spaceship battles, Banks confronts readers with a series of moral quandaries, including the extreme eventualities of being able to upload one’s consciousness into a computer, whether the Culture should intervene in the development of nascent alien societies to mitigate war and cruelty, and the recurrent theme of how citizens occupy their time when freed from economic servitude.

The edge-of-your-seat action and immense scope of the Culture series has long been a favourite of the tech billionaires of Silicon Valley. Mark Zuckerberg, the 39-year-old co-founder of Facebook, suggested one of Banks’s books, The Player of Games, for his book club to read. Jeff Bezos has been pushing for years to have Consider Phlebas converted into an Amazon TV series. Elon Musk, too, seems to have been so inspired by the Culture series that he considers himself to be the embodiment of its ideology: in a 2018 Tweet, Musk states ‘I am a utopian anarchist of the kind described by Iain Banks.’ Previously Musk had appropriated the names of two of Banks’s fictional spaceships for his reusable rocket landing barges: the ‘Just Read the Instructions’ and ‘Of Course I Still Love You’. The Link, Musk’s brain-computer-interface (BCI) device, is allegedly responsible for the deaths of  large number of primates and other animals. It was originally to be named the ‘Neural Lace’, the namesake of the fictional BCI technology used by the Culture.

In a 2010 interview, later posted on the literary website Strange Horizons, Banks was told that many critics and reviewers regard his utopian society as representing the libertarian ideal, to which Banks responded with astonishment, stating, ‘Have these people seriously looked at the problems of the world and thought, “Hmm, what we need here is a bit more selfishness”?’ Further, he created the Culture as ‘hippy commies with hyper-weapons and a deep distrust of both Marketolatry and Greedism.’ So how is it that the technology-billionaire-class could see themselves in his writing?

A sizeable body of literature has been written about the ideological motivations driving the billionaires of Silicon Valley. One of the better-known polemical works on this subject is still the contentious, although arguably prescient, essay The Californian Ideology (1995), written by the British media theorists Andy Cameron and Richard Barbrook. In their essay, Cameron and Barbrook consider the conflicting ideologies of the founders of Silicon Valley, speculating on the nascent internet and predicting the birth of a new virtual social class who would rule the information age. According to The Californian Ideology, the Silicon Valley of the 1990s was dominated by liberal ideals and a feeling of technological optimism, often inspired by the hi-tech utopias described in the science fiction of writers like Isaac Asimov. Many technology-literate workers sought to break free from conventional forms of labour and leveraged their software skills to find niche employment in the unfolding dot-com boom. Because of the esoteric nature of their work and skills, workers were able to negotiate high wages and favourable working conditions.

According to Cameron and Barbrook, these workers were willing to abandon their larger utopian ambitions to become workers in the free market because of their belief in ‘technological determinism’. This refers to the theory that the technology developed by a society will firstly evolve according to its own internal rules, and secondly it will dictate the social structure, economics, and cultural growth of society. For the workers of Silicon Valley, the knowledge that the wheels of technological progress were turning, whether they were directly contributing or not, assured an inevitable technological utopia and freed them to capitalise on being skilled workers in a new era of the free market. Many predictions made in The Californian Ideology were proven incorrect, such as that the internet would be used solely to perpetuate Americo-centric, libertarian, neo-liberal ideals, and the idea of a new apartheid forming between the information ‘haves’ and ‘have-nots’. However, Cameron and Barbrook successfully highlight the paradox that is the marriage of free-market libertarianism and technological-utopianism still at work in California today.

This is perhaps how the billionaires interpret Iain M. Banks’s utopia. In the far-distant future, technological determinism will bring about a post-scarcity society inevitably of its own accord, without any single human architect. As observed in a blog post by the Adam Smith Institute, effective post-scarcity may only come about because of efficiency improvements created by the free market. Following this logic, the billionaires are not misunderstanding Banks’s utopian idealism; on the contrary, they believe they are actively working towards it.

The ardent belief that free-market principles must be defended, government oversight fought off, and responsibility towards society shirked, has created a host of serious social issues over the course of the last twenty years. The world is still reeling from the rise of the internet, itself a utopian technology, and the panoply of innovations to which it has given rise. To take but one example, the operation of run-of-the-mill social media presents a threat to liberal democracy that cannot be overstated. During the run-up to the 2016 US election, misinformation from the Kremlin-linked Wagner Group’s Internet Research Agency reached some 100 million Americans a month. That is not to mention the other ills associated with social media, such as isolation and climbing suicide rates. It is not immediately apparent that the recent products from Silicon Valley are moving humanity closer to a state of utopia.

The latest innovation out of California is vastly more significant than any product that has come before. Generative Artificial Intelligence has captured the zeitgeist; triggered a new arms race between China and America; raised questions concerning consciousness, humanity, and human expression; sparked labour strikes around the world; seeded a new scientific revolution; and caused serious concern among governments and pundits of an emerging existential threat. Most people will have been exposed to the technology in the form of the popular chatbot ChatGPT.

The unleashing of AI on the world—although the technology is in reality currently nothing more than an expert plagiariser—has many researchers screaming ‘AGI!’. ‘Artificial General Intelligence’ is the theoretical God-computer: an AI programme with agency, capable of assimilating information at the speed of light, and able to bring about its own efficiency improvements – that is, to evolve. AGI was the stuff of science fiction, the cornerstone of Banks’s Culture, Asimov’s robots, Agent Smith of The Matrix, and Skynet of Terminator. Now it is a goal towards which companies, governments and universities are actively working.

If the technology that is currently emerging turns out to be as ‘revolutionary’ as is anticipated, and not just a means of, say, churning out bad poems, then that would surely be no coincidence. Today, we are facing a ‘poly-crisis’, as it has been called. The many crises include the Ukraine war, the reawakening of great power struggles between the West and China, rampant populism and ultranationalism, economic crisis, and above all the existential threat of climate change, which exacerbates each of the other crises. Market economies have, by design, overlooked the externality—a factor outside of the remit of a business model—that is the continuation of human life on Earth.

The climate change crisis, stemming from the greenhouse effect, has now spawned a series of abstruse, secondary cascading effects—each a crisis in its own right—that threatens near every facet of life on earth. Carbon offsetting efforts, amounting to little more than planting trees, are failing to account for the influence of microplastics on the ocean, for example. Previously, climate models suggested that the planet’s oceans sequestered around one third of annual human carbon emissions – but this may no longer be the case. Nor are carbon offsets able to compensate for the effects of large-scale industrial farming on top soil fungi networks, which recent studies suggest would normally sequester another third of atmospheric carbon. Artificial intelligence will influence all these crises, for better or worse.

AGI research is inexorable for several reasons. Perhaps the most immediate drive is geopolitical: Chinese researchers, under the direction of the CCP, are making significant progress in the field of artificial intelligence; whichever nation secures the technology first may become the reigning superpower. Free market principles are another driving factor. Each company will want to beat its competitors to the prize. Third, mitigating the climate crisis will require significant technological innovation and efficiency improvements for carbon capture and the cleaning up of pollution. The need for mitigating technology is immediate and perhaps AI is the only way to bring about the technological changes required for society, as we know it, to survive in the long term.

Ostensibly, GPT4, the latest AI model powering ChatGPT, cost in the region of one billion dollars to train. In the western world, the billionaires of Silicon Valley, with the aid of many poached PhD students (whose contribution is unacknowledged but likely to be substantial), may be the only group with the resources to create an AGI. Beyond cost, as with the esoteric technologies of the 1990s, the complexity of the software architectures involved have become extreme: often the processes are obscure even to the engineers themselves. Having broken this ground, however, the companies are unlikely to restrain themselves from pursuing further innovations. In this context, it is imperative that these companies invest in ‘alignment’: that is, in ensuring that an AI programme performs tasks that are in the best interest of a human user, thereby maintaining a sort of ‘moral compass’. The need for this in areas like social media is already clear. Without such investment, the rise of AI could well lead to an existential threat.

That effective artificial intelligence has come about in this way must be considered a misfortune. Lack of government oversight presents a problem, as does the concentration of technological power in the hands of a tiny, individualist, libertarian-leaning minority rather than in a democratic, multilateral effort with well enforced fail-safes. It is also unfortunate for the billionaires who must now attempt to control this technology. They can no longer sit idly and allow technological determinism to unfold of its own volition. AGI could be construed as manifest technological determinism. How the billionaires respond to this in practice will likely revolutionise societal structures in the very near future.

Enjoy this article? Subscribe to our free fortnightly newsletter for the latest updates on freethought. Or make a donation to support our work into the future.