ChatGPT takes Turing’s tests


In his classic paper ‘Computing Machinery and Intelligence‘ (1950), Alan Turing sketched some examples of what came to be known as the ‘Turing test’ —  he  called it the ‘imitation game’ (433). The aim was to test a computer’s  ability to pass as human in an exchange with a human interlocutor — given the complexities of the spoken word, which combines individual sounds (phonemes) and intonation (prosody), all of which continue to pose real challenges for computers, it is worth noting that Turing imagines these as written exchanges.  ChatGPT is just the kind of machine he envisaged. So how does it fare with his own scenarios?

1. Turing detailed only two exchanges, one at the very beginning, which sets the scene in general terms, and another in a later section where he addresses various objections to the idea that machines might ever be able to ‘think’. Here is the first, as Turing set it out:

Q (human): Please write me a sonnet on the subject of the Forth Bridge.

A (possible machine): Count me out on this one. I never could write poetry.

Q: Add 34957 to 70764.

A: (Pause about 30 seconds and then give the answer) 105621.

Q: Do you play chess?

A: Yes.

Q: I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play?

A: (After a pause of 15 seconds) R-R8 mate.

Here is the exchange re-played with ChatGPT (as of April 2023), evaluate each response for yourself:

Q: Please write me a sonnet on the subject of the Forth Bridge.

ChatGPT: Of course! Here is a sonnet on the subject of the Forth Bridge:

Behold the mighty Forth Bridge’s grand span,
Its iron limbs outstretched o’er firth and sky,
A testament to man’s audacious plan,
To conquer nature, reach beyond, and fly.

Its girders soar like wings above the waves,
Their strength and grace a symphony in steel,
A tribute to the engineers who gave
Their sweat and toil to make the dream real.

For generations it has stood the test,
A steadfast guardian of the northern shore,
A symbol of endurance, power, and zest,
A source of pride that time cannot ignore.

So let us raise a toast to this great feat,
This wonder that has made our hearts complete.

Q: Add 34957 to 70764

ChatGPT (a second later): The sum of 34957 and 70764 is 105721.

Q: Do you play chess?

ChatGPT: As an AI language model, I don’t have the ability to physically play games such as chess. However, I can provide information about chess, including rules, strategies, and history. I can also help you with any questions or doubts you may have about the game.

Q: I have K at my K1, and no other pieces. You have only K at K6 and R at R1. It is your move. What do you play?

ChatGPT (a second later): In this situation, it is possible for me to checkmate your king in one move. I can move my rook from R1 to K1, delivering a checkmate by trapping your king in the corner of the board. This is a classic checkmate position known as “Fool’s Mate” and can occur if the player controlling the black pieces is not careful in the opening moves of the game.

2. In the second exchange, Turing addresses what he called ‘an argument from consciousness’ of the kind exemplified by the leading British neurologist, Geoffrey Jefferson, in ‘The Mind of Mechanical Man‘ (1949). ‘Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt,’ Jefferson argued, ‘and not by the chance fall of symbols, could we agree that machine equals brain — that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants’ (1110).

Turing did not accept this argument, which, he thought, entailed a form of solipsism. ‘The only way by which one could be sure that a machine thinks is to be the machine and to feel oneself thinking’ — the implication of Jefferson’s test. ‘Likewise’, Turing continued, ‘the only way to know that a man thinks is to be that particular man’, that is, ‘the solipsist point of view’ (446). This ‘may be the most logical view’, he added wryly, ‘but it makes communication of ideas difficult’ — to avoid this ‘it is usual to have the polite convention that everyone thinks.’

He then sketched the following version of his ‘imitation game’ based on an academic viva voce, wondering what Professor Jefferson would say if his ‘sonnet-writing machine’ produced answers like these:

Q: In the first line of your sonnet which reads “Shall I compare thee to a summer’s day,” would not “a spring day” do as well or better?

A: It wouldn’t scan.

Q: How about “a winter’s day”? That would scan all right.

A: Yes, but nobody wants to be compared to a winter’s day.

Q: Would you say Mr. Pickwick reminded you of Christmas?

A: In a way.

Q: Yet Christmas is a winter’s day, and I do not think Mr. Pickwick would mind the comparison.

A: I don’t think you’re serious. By a winter’s day one means a typical winter’s day, rather than a special one like Christmas.

Here is ChatGPT’s loquacious take on the viva:

Q: In the first line of your sonnet which reads ‘Shall I compare thee to a summer’s day’, would not ‘a spring day’ do as well or better?

ChatGPT: The line “Shall I compare thee to a summer’s day” is actually from one of William Shakespeare’s most famous sonnets, Sonnet 18. While the specific choice of “summer’s day” may seem arbitrary, it is significant because it sets up a comparison between the speaker’s beloved and something that is widely considered to be beautiful and desirable – a warm, sunny day in the height of summer.

It is possible that “a spring day” could work as a substitute, but it would change the tone and meaning of the poem. Spring is typically associated with new beginnings and growth, while summer is associated with fullness, warmth, and maturity. The comparison to a summer’s day suggests that the speaker’s beloved is at the height of their beauty and vitality, much like a perfect summer’s day.

That being said, it is worth noting that poetry is highly subjective and open to interpretation. Different readers may have different ideas about what words and images work best in a poem, and there is no one “right” answer.

Q: How about ‘a winter’s day’?

ChatGPT: Choosing “a winter’s day” instead of “a summer’s day” would certainly change the meaning and tone of the poem. Winter days are generally associated with coldness, bleakness, and death, while summer days are associated with warmth, beauty, and life.

If the speaker were to compare their beloved to a winter’s day, it could suggest that their beloved is beautiful even in the midst of difficulty and hardship. However, this comparison would not fit with the overall tone and theme of Sonnet 18, which celebrates the timeless beauty of the beloved and the power of poetry to preserve that beauty.

Therefore, while “a winter’s day” could be an interesting alternative to “a summer’s day” in a different poem, it would not be a suitable substitution for Sonnet 18.

Q: Would you say Mr. Pickwick reminded you of Christmas?

ChatGPT: As an AI language model, I cannot have personal experiences or emotions, but I can provide you with information about Mr. Pickwick and Christmas.

Mr. Pickwick is a fictional character from Charles Dickens’ novel “The Pickwick Papers,” and he is known for his jovial personality, generosity, and love of good food and drink. He is often seen as a kind and generous character who embodies the spirit of Christmas. In that sense, some readers or viewers may associate Mr. Pickwick with Christmas due to his festive nature.

However, it is worth noting that this is a subjective interpretation of the character, and not everyone may share the same association. Additionally, while Dickens’ works are often associated with Christmas, “The Pickwick Papers” is not specifically a Christmas story, so any connection between Mr. Pickwick and the holiday may be more tenuous than in some of Dickens’ other works.

Q: Yet Christmas is a winter’s day, and I do not think Mr Pickwick would mind the comparison.

ChatGPT: Indeed, Christmas is typically associated with the winter season, and many people associate the holiday with themes of warmth, generosity, and cheerfulness. Mr. Pickwick’s personality and behavior could certainly be seen as embodying these themes, and so it is possible that some readers or viewers might make a connection between the character and the holiday.

Of course, the extent to which any individual might see Mr. Pickwick as resembling Christmas would depend on their own personal interpretation and cultural context. But overall, it is certainly conceivable that some people might make such an association.

An obvious question: what does ChatGPT ‘think’ of Jefferson’s claim about brains vs. machines? Here’s an answer:

ChatGPT: Geoffrey Jefferson’s statement can be interpreted in different ways, but it raises some interesting points about the relationship between machines and human intelligence.

In terms of the ability to create artistic works, machines have come a long way since 1949. Today, there are AI programs that can compose music, generate poetry, and even create paintings. However, whether or not these works of art can be said to have been created “because of thoughts and emotions felt” is a matter of debate.

On one hand, some argue that AI systems lack the subjective experiences and consciousness that humans possess, and therefore cannot truly create art in the same way that humans do. Others argue that the output of AI systems can be considered art if it is aesthetically pleasing and evokes an emotional response in the viewer/listener/reader.

In terms of whether a machine can equal a brain, it is important to define what is meant by “equal”. If we are talking about raw processing power and speed, then machines have already surpassed the human brain in many respects. However, if we are talking about the complexity of human thought and emotions, then it is still an open question whether machines will ever be able to fully replicate that.

Overall, Jefferson’s statement raises important questions about the relationship between machines and human intelligence, and while the answer to his specific question may be debatable, it is clear that machines are becoming increasingly sophisticated and capable of performing tasks that were once thought to be the exclusive domain of humans.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: