The long read: The spectre of superintelligent machines doing us harm is not just science fiction, technologists say so how can we ensure AI remains friendly to its makers?
It began three and a half billion years ago in a pool of muck, when a molecule made a copy of itself and so became the ultimate ancestor of all earthly life. It began four million years ago, when brain volumes began climbing rapidly in the hominid line.
Fifty thousand years ago with the rise of Homo sapiens sapiens.
Ten thousand years ago with the invention of civilization.
Five hundred years ago with the invention of the printing press.
Fifty years ago with the invention of the computer.
In less than thirty years, it will end.
Jaan Tallinn stumbled across these words in 2007, in an online essay called Staring into the Singularity. The it was human civilisation. Humanity would cease to exist, predicted the essays author, with the emergence of superintelligence, or AI, that surpasses human-level intelligence in a broad array of areas.
Tallinn, an Estonia-born computer programmer, has a background in physics and a propensity to approach life like one big programming problem. In 2003, he co-founded Skype, developing the backend for the app. He cashed in his shares after eBay bought it two years later, and now he was casting about for something to do. Staring into the Singularity mashed up computer code, quantum physics and Calvin and Hobbes quotes. He was hooked.
Tallinn soon discovered that the author, Eliezer Yudkowsky, a self-taught theorist, had written more than 1,000 essays and blogposts, many of them devoted to superintelligence. He wrote a program to scrape Yudkowskys writings from the internet, order them chronologically and format them for his iPhone. Then he spent the better part of a year reading them.
The term artificial intelligence, or the simulation of intelligence in computers or machines, was coined back in 1956, only a decade after the creation of the first electronic digital computers. Hope for the field was initially high, but by the 1970s, when early predictions did not pan out, an AI winter set in. When Tallinn found Yudkowskys essays, AI was undergoing a renaissance. Scientists were developing AIs that excelled in specific areas, such as winning at chess, cleaning the kitchen floor and recognising human speech. Such narrow AIs, as they are called, have superhuman capabilities, but only in their specific areas of dominance. A chess-playing AI cannot clean the floor or take you from point A to point B. Superintelligent AI, Tallinn came to believe, will combine a wide range of skills in one entity. More darkly, it might also use data generated by smartphone-toting humans to excel at social manipulation.
Reading Yudkowskys articles, Tallinn became convinced that superintelligence could lead to an explosion or breakout of AI that could threaten human existence that ultrasmart AIs will take our place on the evolutionary ladder and dominate us the way we now dominate apes. Or, worse yet, exterminate us.
After finishing the last of the essays, Tallinn shot off an email to Yudkowsky all lowercase, as is his style. im jaan, one of the founding engineers of skype, he wrote. Eventually he got to the point: i do agree that … preparing for the event of general AI surpassing human intelligence is one of the top tasks for humanity. He wanted to help.
When Tallinn flew to the Bay Area for other meetings a week later, he met Yudkowsky, who lived nearby, at a cafe in Millbrae, California. Their get-together stretched to four hours. He actually, genuinely understood the underlying concepts and the details, Yudkowsky told me recently. This is very rare. Afterward, Tallinn wrote a check for $5,000 (3,700) to the Singularity Institute for Artificial Intelligence, the nonprofit where Yudkowsky was a research fellow. (The organisation changed its name to Machine Intelligence Research Institute, or Miri, in 2013.) Tallinn has since given the institute more than $600,000.
The encounter with Yudkowsky brought Tallinn purpose, sending him on a mission to save us from our own creations. He embarked on a life of travel, giving talks around the world on the threat posed by superintelligence. Mostly, though, he began funding research into methods that might give humanity a way out: so-called friendly AI. That doesnt mean a machine or agent is particularly skilled at chatting about the weather, or that it remembers the names of your kids although superintelligent AI might be able to do both of those things. It doesnt mean it is motivated by altruism or love. A common fallacy is assuming that AI has human urges and values. Friendly means something much more fundamental: that the machines of tomorrow will not wipe us out in their quest to attain their goals.
Last spring, I joined Tallinn for a meal in the dining hall of Cambridge Universitys Jesus College. The churchlike space is bedecked with stained-glass windows, gold moulding, and oil paintings of men in wigs. Tallinn sat at a heavy mahogany table, wearing the casual garb of Silicon Valley: black jeans, T-shirt and canvas sneakers. A vaulted timber ceiling extended high above his shock of grey-blond hair.
At 47, Tallinn is in some ways your textbook tech entrepreneur. He thinks that thanks to advances in science (and provided AI doesnt destroy us), he will live for many, many years. When out clubbing with researchers, he outlasts even the young graduate students. His concern about superintelligence is common among his cohort. PayPal co-founder Peter Thiels foundation has given $1.6m to Miri and, in 2015, Tesla founder Elon Musk donated $10m to the Future of Life Institute, a technology safety organisation in Cambridge, Massachusetts. But Tallinns entrance to this rarefied world came behind the iron curtain in the 1980s, when a classmates father with a government job gave a few bright kids access to mainframe computers. After Estonia became independent, he founded a video-game company. Today, Tallinn still lives in its capital city also called Tallinn with his wife and the youngest of his six kids. When he wants to meet with researchers, he often just flies them to the Baltic region.
His giving strategy is methodical, like almost everything else he does. He spreads his money among 11 organisations, each working on different approaches to AI safety, in the hope that one might stick. In 2012, he cofounded the Cambridge Centre for the Study of Existential Risk (CSER) with an initial outlay of close to $200,000.