Posted June 26, 2014 by Bernard J. Baars in artificial intelligence

Has the Turing test really been cracked?

It’s every headline writer’s dream: A scientific breakthrough! Named after the great mathematician Alan Turing, so it has to be important!

It was a dream nobody could resist.

Does he pass the Turing test? Does it matter?

When the The University of Reading (UK)  announced at the Royal Society in London that, 60 years to the day after Alan Turing’s death, his proposed test for human-like computer intelligence had been passed, the announcement took the media by storm.

It was science fiction come to life. A story too good to check.

But skeptical voices piped up, and a few days later another wave of headlines went viral, saying just the opposite.

Many scientists didn’t believe the Turing test had been passed.

So what’s up?

Let me start with a common confusion. Every computer science student learns about Turing Machines, which are not machines at all, but mathematical fictions that are used to prove theorems about computation. Math is full of powerful fictions like this — straight lines that have length but not breadth, a natural number series that goes to infinity, a square root of 2 that can only be expressed as a dwindling, infinitesimally small fraction of the real number line.

The Turing Machine is another useful fiction. It is Alan Turing’s greatest contribution to the mathematics of abstract automata.

But the Turing “test” is whole ‘nother ball of wax. It’s not math. It’s a hypothetical. The Turing test only suggests that computers might be said to have human-like intelligence when a human judge can no longer tell the difference between the output of a computer program and the (typed) responses of a human being

Chances are that headline writers don’t know the difference between TM’s and TT’s. So let’s put the Turing Machine on the shelf as a useful concept for proving theorems about programs. It’s been studied for half a century, and it is as useful as the Pythagorean Theorem. But it’s old news.

Millions of kids on the web are excited by robots, zombies and brain hacking, and the PR release from Reading University caused a flurry of excitement in that huge audience, until those boring old skeptics piped up.

But if you are interested in real science there’s a lot to learn from this whole brouhaha. One is that

you can’t do science if you don’t know the meanings of words. Mark Twain wrote that “The difference between the right word and the almost right word is the difference between the lightning and the lightning bug.” Turing tests have nothing to do with Turing Machines. One is the lightning, the other is the lightning bug.

A second big lesson is that in frontier science there is always a hot debate between proponents and skeptics. Everything is up for grabs. Even the recent (apparent) discovery of the Higgs boson particle is still a matter of some debate. You can’t understand frontier science if you don’t think about the arguments pro and con. In real science nobody is always right, and skeptics always get their turn.

A third great lesson is knowing the proof conditions of any proposed idea. That is: When do we know that some idea is really true?

And this is the real question about the “Turing test”.

If you think about it you’ll realize that there is not one, single TT, but literally millions of them.

That’s because:

(a) human beings can do millions of things that could be imitated by computers, and

(b) human judges who try to compare people to machines don’t use standard metrics. They are no thermometers. Some are smarter than others, some know more about computer programming or about human beings. Some are more skeptical and others are more gullible.

In general, humans can be fooled with amazing ease.

If you doubt that, look at all the stage magicians and snake-oil salesmen who make a living from human gullibility. In a way the “Turing test” is not about computer programs at all, but about the judges who are supposed to tell the difference. Since humans have an astonishing tendency to personify things like plastic dolls and planet earth, everything depends on what the judges know.

The latest set of judges might have remembered that, when it comes to solving computational problems, machines have been faster and more efficient for decades, since the early days of air conditioned rooms full of vacuum tubes. That’s what computers are FOR, to solve math and other specific problems. Headline story: Machines are smarter than people! They really are.

But that’s old news.

Things get more interesting when we try to make computers understand and produce human language. Language is the most distinctive human specialization, not shared with any other living species. Language and speech are enormously complex adaptations, some of which emerged over millions of years, while others — like our new web jargon— are still being invented  today. Language is not just one “thing.”

If language is many things, there is NOT likely to be a single “test” to see if computers can do it. There has been good progress over decades in machine processing of natural language. Apple’s Siri is a good example — an important step in speech recognition, but with poor syntactic capabilities and NO semantic (meaning) or pragmatic (purposeful) capabilities at all. Siri works with a long list of things users are likely to say, like “what’s at the movies today?” It’s very cool, but it doesn’t “solve” the Turing test, because only young children could believe there’s actually a little person inside.

And that’s the biggest fallacy about “the” Turing test. At a low level of performance, we’ve had computers that can fool people for decades. In the 1970s Professor Ken Colby ran a simulation of a paranoid personality called PARRY at UCLA Medical School, and was able to fool medical residents into believing they were talking to a real patient in the hospital. Colby’s PARRY passed the Turing test for those medical residents. PARRY was run by a list of fixed questions and answers, and it was easy to beat PARRY by asking it to do things outside of its very narrow domain. I was able to beat PARRY by asking it questions like “What did you just say?” and “What did you mean by “he” in the last sentence?” PARRY could not solve referents in language, even by referring to its own speech. But people do that all the time, without even thinking about it.

Still, PARRY’s developer, Prof. Ken Colby, did a remarkable job. Language scientists have been getting better at true language perception and comprehension over time. It is still only partly solved — as you know from playing with Apple’s Siri.

The point is that there is NO breakthrough in the “Turing test” because there IS NO SINGLE TURING TEST. There are as many TT’s as there are human skills — AND as long as there are human beings who are willing to be fooled. The Turing Test should really be called the Turing Illusion, because it tells us more about the people who fall for the stunt than about computer programs.

For headline writers the Turing Illusion is a gift that keeps on giving. Every year or two they can run the Turing Illusion again, and every time millions of innocent readers fall for the trick.

The Turing Illusion tells us more about human gullibility than it does about artificial intelligence. Human gullibility defies the Second Law of Thermodynamics, because gullibility isn’t dissipated, like heat in a closed system. Gullibility expands as a function of the number of innocent minds available. The web has a billion users, and the audience for The Great Hype Machine is now in the hundreds of millions.

Skepticism — but not rigid denial — is the mark of good science.



Reusable under Creative Commons license.

The Society for Mind-Brain Sciences is a 501 c(3) nonprofit corporation. To join the Society, please visit our website, The membership page is here. 

To receive FREE WEEKLY UPDATES of A Conscious Brain blog, please provide your email address on our frontpage. Your privacy will be protected, and you can unsubscribe at any time.



Bernard J. Baars