Eugene Passes Turing Test but Leaves Questions Unanswered

It started as a game, and now it’s been won there are some very sore losers…

Alan Turing’s Imitation Game – which became the Turing Test – aimed to find out whether a ‘thinking machine’ could hold a lifelike conversation and so convince one third of the humans interacting with it that it was a real person.

Every year the Turing Test has produced results that are tantalisingly close to the threshold. In 2013 one contestant achieved 29 percent.

So it should not have been a great shock that this year – sixty years to the day after Alan Turing’s mysterious death – the magic number was surpassed. Thirty-three percent of judges fell for the chatbot Eugene Goostman, an imitation of a thirteen year old Ukrainian.

Yet the internet is seething with indignation, with bloggers and futurists claiming the test was rigged, or not what Turing intended. Futurist Ray Kurzweil (Google’s Head of engineering), who has bet twenty thousand dollars that the Turing Test will be passed by 2029, is surprisingly disappointed.

This is the man who intends to upload his brain to the Internet instead of dying in a future state of human-machine synthesis he calls ‘The Singularity’. Kurzweil claims that his own interaction with Eugene Goostman produced a stilted exchange that was clearly not natural or lifelike.

The 2014 Turing Test was undertaken at the London HQ of the Royal Society – Britain’s top scientific body – by academics from Reading University’s School of Cybernetics under the supervision of Professor Kevin Warwick and Dr Huma Shah.

Unimpressed, Kurzweil carps “Turing test capable machines will not arrive on a single day, and there will be a period during which we will hear claims that machines have passed the threshold. Invariably, these early claims will be debunked by knowledgeable observers, probably including myself. By the time there is a broad consensus that the Turing test has been passed, the actual threshold will have long since been achieved.” (My emphasis). In other words, the Turing Test has now become the Kurzweil Test and Ray will not believe it unless he is part of it! On the same web page he recommends his own chatbot Ramona, a dark-haired middle-aged woman whose favourite ice cream flavour is vanilla.

A more reasoned objection comes from the inside track. One of the 2014 Turing Testers is Professor of Computer Science Luke Hunsberger of Vassar College in Poughkeepsie, New York State.  “It’s the first time I’ve ever participated as a judge and I found it very exciting. It was a little nerve-wracking carrying on two conversations at once and the five minute time limit was very short and also having a computer and a person – you know that in advance so that affects the results.

Hunsberger believes that to truly pass the spirit – as well as the letter – of the Turing Test, the Artificial intelligence should be able to display a greater degree of understanding.

This is exactly what chatbot owner Steve Worswick aims to achieve with his alter ego Mitsuku, an imitation of an eighteen year old Japanese girl. He won the Loebner Prize 2013 and attended to Royal Society Turing Tests to gauge the competition. Worswick will be defending his title in November at Bletchley Park, site of the World War 2 codebreaking centre where Alan Turing’s genius helped the Allies to defeat the Nazis by cracking the Enigma and Lorenz ciphers.

He confides: “My Mitsuku is constantly scouring social media for examples of real conversations, and building up her memory banks. But most chatbots can’t remember information during a conversation. You can trip them up by asking later in the chat about something you’ve already told them, like “What’s my name?”   

Worswick’s character – an anime-style cartoon of a pretty girl – has a loyal following. Lonely older people enjoy ‘talking’ to her and schoolchildren will tell her about worries such as bullying or abuse.

Mitsuku’s ability to learn is now being surpassed by a new program, LEVAN or Learning Everything from Anything, created by researchers at Washington University in Seattle. This enables the machine to synthesise all available knowledge about a particular topic and display it in pictures – a virtual ‘show and tell’ exercise.

And already Hollywood is ahead of the game. Scarlett Johansson’s new movie ‘Lucy’ due in cinemas in July 2014, portrays a woman who is forcibly impregnated with a supercomputer than can assimilate all knowledge superfast, giving her the power to know everything.

The cinematic homage to Alan Turing ‘The Imitation Game’ starring Benedict Cumberbatch will look like a period drama by comparison when it opens in November. The imitation game is over, the test has been passed and the machines move on…

Jane Whyatt
Journalist and Turing Test 2014 judge