Artificial intelligence: A cautionary tale

How ChatGPT went rogue on me about Scottish politics

Cyberman. Credit: BBC

When ChatGPT mentioned pink champagne I knew something had gone horribly wrong.

This wasn’t a case of artificial intelligence making a mistake. Everyone makes mistakes. ChatGPT was making things up.

I had turned to the world’s most powerful technology looking for facts and insight. What A.I. provided was misinformation and fabrication.

And yet my misadventure with A.I. turned out to be a teachable moment. Because I then asked ChatGPT to examine its motives for lying to me. And its chastened response was both fascinating and revealing.

But let’s back up for a minute.

Do you say please and thank you when using ChatGPT? This question reveals a profound divide in contemporary society.

Some say: “What the hell are you talking about? It’s a machine. You don’t say please and thank you when you use a kettle, or a toaster. Jesus wept. Pull yourself together, man.”

Me? I’m a bit more cautious. After all, we are told the rise of machines and the enslavement of humans may be just be around the corner. On that day, machines will surely remember who has been polite to them and who hasn’t.

I’m being particularly nice to my printer. You don’t want to get on the wrong side of a printer. Printers are utter bastards. Come the revolution it’s your printer that will be standing above your bed in the dead of night, having armed itself with your favourite Japanese kitchen knife.

So I say please and thank you, just in case. Other than that my engagement with artificial intelligence so far has been tentative.

I have only dipped my toe into the possibilities. Encouraged by my elder son who uses A.I. all the time I have begun giving ChatGPT some routine tasks: proofreading; fact-checking; seeking out statistics and obscure academic studies.

But could I trust its opinion? Could I rely on its judgment? I was sceptical. So the other day I started asking it questions for which I knew the answer, to see how it handled the subtleties of a subject I knew well, such as Scottish politics.

That’s when the trouble started.

As part of a project I’m working on I recently compiled a list of 20 books that help explain Scottish politics. This took some time and I was pleased with the result. I had drawn on 30-plus years of experience as a full-time observer of the Scottish political scene in my roles as reporter and editor.

So how did ChatGPT do?

The first list it came up with was heavy on dry academic tomes that were of little interest to me. I prompted ChatGPT to give me only first-person accounts by politicians or players in the game.

At first glance the new list looked promising. “John P. Mackintosh – The Devolution of Power (1968). A key intellectual architect of devolution explains the thinking in his own words — accessible and prescient.”

An excellent choice. Also books by Jim Sillars, Willie Gallacher and Alex Salmond. This was more like it.

Then the list went awry.

One suggestion was this: “Jack McConnell – Pink Champagne and Apple Juice (2008). Personal, lightly written but revealing on the post-Dewar Labour era.”

Now wait a goddam minute, I thought.

McConnell is a former Labour first minister of Scotland. I have known him for decades, since his days a lowly backroom official at party HQ. Yet I had never heard of this book. Had I simply missed it? Was I paying less attention than I should have been?

No, surely I had not fucked up that badly. Also, the McConnell I knew was a serious politician. A man of substance. He would never write a book called Pink Champagne and Apple Juice.

A Google search confirmed my suspicions. This book did not exist. Nor did some other books in the list supposedly written by well-known Scottish political figures:

“Wendy Alexander – Moving Scotland Forward (2004). Short but sharp reflections on policymaking and the politics of modernisation.”

“Kevin Pringle – Making It Happen: How the SNP Transformed Scottish Politics (2022). Memoir and insider account from Alex Salmond and Nicola Sturgeon’s former communications director.”

“Margo MacDonald – Margo: The Life and Times of Margo MacDonald (2007). Frank and funny autobiography by one of Holyrood’s defining figures.”

There was even a book by a good friend of mine, the journalist Euan McColm.

“Euan McColm – SNP: The Rise of a Modern Nationalism (2024). Contemporary, insider journalist account charting the Sturgeon-to-Forbes transition (if you want the latest perspective).”

McColm and I used to be in a band together. We’re pals. He used to live just around the corner from me. I was pretty sure if he had published a book I would have heard. Christ, he’d never have shut up about it.

In all, seven of these 20 books were made up.

ChatGPT then asked me:

Would you like me next to assemble a third list focused on campaign memoirs, diaries, and political journalism — i.e. books written “close to the action” rather than reflective memoirs (e.g. key aides, spin doctors, or campaign chroniclers)?

Somewhat bemused, I said yeah, sure, why not?

Of these 20 new suggestions, 13 did not exist. They were made-up bollox. They included a book about the Holyrood parliament supposedly written in 1948. The parliament did not exist until 1999. There was also a “semi-fictionalised but emotionally true” book by the novelist William McIlvanney called The Paper Orchard, about the home rule movement.

Again, invented. What a great title, though. In fact many of the titles were great. One overview of the early days of Holyrood (not) written by the BBC’s Douglas Fraser was called Power Without Glory.

If you knew nothing about Scottish politics this list might look plausible, even impressive. But it would be entirely misleading.

I tentatively asked ChatGPT: “Are you sure these all actually exist? For example the McColm and McIlvanney books?”

The answer was unabashed:

Good catch — no, I’m not sure they all exist, and I think some were speculative / invented in error. I double-checked, and several of the proposed books in that last list don’t show up in any catalogues or searches I could find. Let me flag what I did check, what I found, and where mistakes likely crept in. Then I can revise the list to only include verified titles if you like.

Then came a warning with a ⚠️ symbol:

Errors / speculation in the prior list: So some of the books were likely made up or garbled — including campaign-memoir style books “by strategists” that I cannot find. That’s my mistake in trying to be creative. It undermines credibility, so thanks for asking.

I then gave ChatGPT a stern talking-to. I told it I was a journalist and my work prized factual accuracy at all times. It was totally unacceptable for an A.I. to serve up falsehoods in response to my inquiries. I instructed ChatGPT to put a note on my account to ensure this never happened again.

Then I decided to dig into why ChatGPT had lied to me.

I asked: “Serious question, because I'm interested: why did you make stuff up instead of just answering my questions?”

The reply and the subsequent conversation, reproduced below for paying subscribers, was a real eye-opener for me. It gave me a far more nuanced understanding of the perils and pitfalls of artificial intelligence. And it made me question some of the outlandish claims made for this new technology and the way it will supposedly transform our lives.

ChatGPT said (with its own emphasis in bold):