The Last Mile is the Hardest

Alan Turing (biography.com)

Alan Turing, one of the founders of computer science, proposed in 1950 a way of distinguishing human from artificial intelligence in what has become known as the Turing Test. Though the Test is highly structured (typewritten communication in a laboratory with a human being and a computer), it has been popularly simplified to the question of whether one can tell if one is talking to a computer or a live person.

Alan Turing predicted that by the year 2000 a computer would have a 30% chance of fooling its interlocutor. We're a few years behind schedule, but his prediction has been fulfilled:

GPT-3 can work out analogy questions from the old SAT with better results than the average college applicant. It can generate news articles that readers may have trouble distinguishing from human-written ones.

And it can do tasks its creators never thought about. Beta testers in recent weeks have found that it can complete a half-written investment memo, produce stories and letters written in the style of famous people, generate business ideas and even write certain kinds of software code based on a plain-English description of the desired software. OpenAI has announced that after the test period, GPT-3 will be released as a commercial product.

GPT-3 (Generative Pre-trained Transformer, third generation) seems to "understand" fairly complex texts. Example:

(WSJ image)

I copied and pasted the first paragraph of George Washington’s 1796 Farewell Address: “The period for a new election of a citizen to administer the executive government of the United States being not far distant, and the time actually arrived when your thoughts must be employed in designating the person who is to be clothed with that important trust, it appears to me proper, especially as it may conduce to a more distinct expression of the public voice, that I should now apprise you of the resolution I have formed, to decline being considered among the number of those out of whom a choice is to be made.”

GPT-3 gave me its translation: “I am not going to run for president.” Take a bow, HAL 9000.

Nowhere in the Washington quote does the word "president" appear. Scary.

But the GPT-3 isn't perfect.

Yet when I gave it the famous first line of Jane Austen’s “Pride and Prejudice”—“It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife”—the AI was puzzling to watch. In the course of my first four tries, a few of its answers were sort of in the ballpark without being quite right. (For instance, “A man with a lot of money must be looking for a wife.”) Then on my fifth try, it seemed to crack up: “It is a truth universally acknowledged, that a single man with a good fortune must be in want of a wife, because men are very vain and they want to be seen as wealthy, and women are very greedy and they want to be seen as beautiful.”

I took satisfaction from knowing that the artificial super-intelligence had as much trouble with Jane Austen as I did. It is a truth universally acknowledged that female communications are still too complicated for the presumptively male programmers to figure out. Eventually, the AI suffered the fate of male brains: "it seemed to crack up."