[T]here would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Vinge was the first to bootstrap Good's "intelligence explosion" to something like the singularity — and the first to explain how the singularity might arrive.
>> READ:"URI’s ''Are You Ready For the Future?'' colloquium in brief" <<
He laid out four paths: the development of an artificial intelligence that "awakes" to consciousness; people and their computer networks — the Internet — waking up to a superhuman intelligence; humans plugging their brains into computers and boosting their intelligence to unseen heights; and biological sciences supercharging the human mind.
Underpinning his prediction was a concept that Kurzweil would later articulate more fully with his "law of accelerating returns": technology improves at an exponential rate.
Kurzweil likes to note, as he did in his URI talk a few weeks ago, that the smartphone in his pocket is a million times cheaper and a thousand times more powerful — a billion-fold improvement, in toto — than the half-building-sized computer he used as a student at the Massachusetts Institute of Technology in the '60s. We can expect the same kind of growth, he argues, over the next 25 years.
But if Kurzweil goes on to predict what that explosion in technological power will yield — solar panels capable of filling all of the planet's energy needs, virtual sex as everyday occurrence — Vinge does no such thing.
For him, the arrival of superintelligence will create a world as unimaginable to us as "opera is to a flatworm." It will be, as he wrote in his 1993 essay, "a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control."
It is, of course, easy to see how this gets ugly for homo sapiens — even if the artificial intelligence isn't malicious. "The AI does not hate you, nor does it love you," writes Eliezer Yudkowsky, an artificial intelligence researcher, "but you are made out of atoms which it can use for something else."
Yudkowsky imagines a machine designed to solve a complex math problem developing molecular nanotechnology in a bid to convert the entire solar system into computing power — and killing humankind in the process.
Vinge, a cheerful sort, is a bit leery of the gloom that attaches to his views. He is intrigued, for instance, by Yudkowsky's call for the development of a "friendly AI," benevolent toward humans; we cannot control the coming superhuman intelligence, he says, but perhaps we can create the conditions to survive its arrival.
And Vinge is more hopeful about the run up to the singularity than he once was. Fifteen years ago, he tells me, he had a rather dark view of what our destructive impulse might wreak on the Internet; now he is cheered by the good will that infuses, say, Wikipedia.
Still, he says the extreme compression of technological advance — the rate of progress we're realizing in 20 or 30 years once took evolution millions of years — has led to a worrisome volatility.
