"...Robby Garner, a.k.a. Robitron, practices the art of programming and computational behaviorism from an outpost in northwest Georgia. While some believe in a computer "science" and the need for code grinding mercenary clock watchers, Robby believes that humanity is a prerequisite for good computer software. Robby has also been called a freak by some of the best people he knows..."
- Robby Garner
What the modest Mr. Garner forgets to mention is that he is the winner of the Loebner Competition (formalized Turing Test) two years running (1998-1999)! Garner's 'conversational creations' include FRED, Albert and Albert2.
G5: The Turing Test is an area of extreme controversy. Do you believe that computers can ever be classified as intelligent? Will the Turing Test suffice for doing so?
Human intelligence is such a vague term, and we don't fully understand ourselves. Artificial Intelligence has become even more vague, and I like to say that I don't believe in the word intelligence at all because human behavior is more than just the meanings of our words and how many of them you can remember.
So it's important to try to be specific about what we are trying to duplicate here.
I define intelligence as the capacity to acquire and apply knowledge. Knowledge is familiarity, awareness, or understanding gained through experience or study. This may already exist in some forms, just like we might say a rat is intelligent enough to traverse a maze.
I believe that some kind of machine will be intelligent compared to humans eventually if we earthlings survive here long enough and don't wind up living in a post-apocalyptic war zone.
Passing the Turing test does not require intelligence. But the computer must embody familiarity if it is to make successful contact with the human.
It is possible to seem human so long as a suspension of disbelief can be maintained. The computer software must portray a character to the human that the human can recognize as one of his/her own.
G5: You've won both the '98 and '99 Loebner Prizes -- with the past and current competition, do you believe the grand prize will be won any time in the near future? Why?
It is not clear to me what the conditions for winning the 100,000 dollar grand prize are. If anyone does know, please email me.
I believe we are close to Turing's estimate of 30% success by the turn of the century. That is to say, the computer would be perceived as human 30% of the time. This would be enough to win the silver medal and 25,000 dollars. But C3P0 will have to do better than that.
At this point, I believe it would take years to refine a system to even come near to that. Eventually, computers will program other computers. That's when things will get weird.
G5: How did you approach programming your conversational 'bot'?
I think of myself as a Computational Behaviorist, using the term coined by Dr. Thomas Whalen of the CRC in Canada. Dr. Whalen won the Loebner Contest using a conversation model strategy in 1994.
We don't really understand the human mind. It is foolish to try to duplicate a system that you can't provide the specs for.
Albert is based primarily on a simple behavioral model which attempts to represent conversations as strings of Stimulus --> Response objects. It is a brute force approach that learns from past conversations. But there is no way to predict what people will say to a bot, so I designed a series of backup strategies gleaned from past winners of the Loebner Prize Contest, and some that I have developed from years of thinking about this.
The secondary methods include an algorithm which reduces the search space for a response based on the observed frequencies of the words in the input phrase, and methods for using observed responses in new conversations, and for providing templates for generic replies that may be somewhat predictable or frequently observed.
The original FRED concept was to amass everything ever said to the bot and give each input a reply. For this to be of any value, some kind of search algorithm was needed so that a person didn't have to *exactly* match an example in the file. The conversation was thus implemented by a series of searches on a large database. As the database grows large, the results got better, but search time becomes a constraining factor for a performance in real-time. Obviously, I didn't search every record in the database, but the number of similar examples for uncommon inputs begins to grow large as well.
My solution was to use the observed frequencies of the words in an attempt to estimate their significance and to base a search algorithm on the 3 most significant words in the input phrase. This yields respectable results, but still does not work every time. As a fallback, I used a program based on Barry DeFacto 0.9 to provide filler. (and sometimes it just changes topics)
Albert One uses baru.cpp in C++ and Elvis.pas, a Pascal version of Eliza, compiled as executables, as helpers when the database doesn't yield a satisfactory reply.
I've rewritten Albert in Visual DataFlex, and one of the first tasks was to replace the helper apps with an object that could provide the same kinds of replies, but has a DBMS so that unlimited numbers of templates may be employed. This is Albert2.
Albert2 can also do some data mining during the conversation using a "frame based learning" or so I believe. Tokens can be placed in response text so that the mined data can be utilized in the bot's response.
All of the data files in Albert2 are DataFlex DBMS files, so they are easily searched and edited with online indexing.
One thing about Albert that doesn't come up much in contests is another kind of learning. During conversation, if you say something that Albert has never heard before, it goes into another file like a huge queue of random utterances. Sometimes these utterances are thrown into the conversation, and whatever gets typed in as a reply to that gets saved in the stimulus/response database. Each new record has it's 3 most significant words estimated, and a character frequency key is made using the letters in the input phrase.These are alternate ways of finding a phrase and it's corresponding response.
Albert has a file that was originally intended to store online help text, but is now holding most of Probert's Electronic Encyclopedia, The Jargon File, The Devil's Dictionary, and other assorted references that may be called up as replies to certain kinds of input.
An important aspect of contest level chaterbotting is the "character". Who is the bot? Can you convey an identity of sorts within 5 minutes of typing?
I think the original Turing estimate of 30% success in appearing human based on a 5 minute conversation is more attainable than previous measurements at 10-12 minutes. The longer the conversation, the harder it is to maintain the suspension of disbelief. But it's also interesting to see how the bot fares at 15 minutes or more. Then the challenge becomes greater and greater.
G5: What tips would you give to those wanting to create a conversational program?
I think the programming language should suit your approach and your preferences. I like to use an OOP 4GL called DataFlex for programs like Albert because of its fast DBMS and robust programming language. For internet applications, I prefer Java. I like C++ too, and sometimes if speed is the key, that's the answer. It depends on the project.
One of the tricks that comes up with the Loebner Prize Contest is that it must appear that a human being is typing the responses. The machine must make the appropriate pauses to seem like a human responding. This forces a program like Albert to confine it's search activities to within reasonable time limits.
If you'd like to know all the tricks you should read Jason Hutchen's paper "How to Pass the Turing Test by Cheating." You also must keep in mind that Joseph Weintraub won the Loebner Prize Contest four times with an Eliza clone. We're not that far away from that in some respects, but there are other ways, other strategies.
G5: Natural Language Processing (NLP) is a very slow moving field within AI: what do you think NLP scientists need to crack before the field can really take off? What advances in NLP would you like to see?
I wouldn't presume to know what a scientist needs to crack. I am an artist. Trained professionals get the first crack at high technologies. Then the artists take these tools and use them in ways they weren't intended for. :)
Conquering grammar isn't enough though. That's like step 1. We must be able to define "understanding", and "meaning" and be able to integrate that with a behavior that is purposeful, and understandable. That would be a good start.
I think it's good to see people working from the cellular level up to higher organisms. I think that is the only way to build a robust understanding of what is going on in living systems.
I want to see C3P0.
G5: More generally, what advances in Artificial Intelligence would you like to see in the near future?
I'd like to see Albert sail past the 30% Turing percentage. :)
I hope to see better and more reliable speech recognition systems.
For now it still makes sense to work with text and do text-to-speech, and Vox-to-text to interact with, but as processor speeds increase and mass storage advances, it can be done with audio and video.
G5: What areas of Artificial Intelligence do you think could be most successfully applied to NLP? Why?
Why limit yourself? Apply them all! Use everything. Don't try to do things the way humans would. Solve it from the top down and exploit the computer's strengths.
I say this as my general philosophy to use the best tool you have for the job. There are many facets and many jobs required to simulate a human being.
Although I have heard of neural networks, genetic algorithms, etc., I must confess that I've never studied them and don't really know what they are. I work from first principles, and when I reach the need to understand what is known about something, I will read about it and ask questions of myself and others.
All content copyright © 1998-2007, Generation5 unless otherwise noted.