Tips on prompting ChatGPT for UK technology secretary Peter Kyle

Back in March New Scientist reported on a successful Freedom of Information request they had filed requesting UK Secretary of State for Science, Innovation and Technology Peter Kyle’s ChatGPT logs:

New Scientist has obtained records of Kyle’s ChatGPT use under the Freedom of Information (FOI) Act, in what is believed to be a world-first test of whether chatbot interactions are subject to such laws.

What a fascinating precedent this could set!

They picked out some highlights they thought were particularly newsworthy. Personally I’d have loved to see that raw data to accompany the story.

A good example of a poorly considered prompt

Among the questions Kyle asked of ChatGPT was this one:

Why is AI adoption so slow in the UK small and medium business community?

(I pinged the New Scientist reporter, Chris Stokel-Walker, to confirm the exact wording here.)

This provides an irresistible example of the “jagged frontier” of LLMs in action. LLMs are great at some things, terrible at others and the difference between the two is often not obvious at all.

Experienced prompters will no doubt have the same reaction I did: that’s not going to give an accurate response! It’s worth digging into why those of us with a firmly developed sense of intuition around LLMs would jump straight to that conclusion.

The problem with this question is that it assumes a level of omniscience that even the very best LLMs do not possess.

At the very best, I would expect this prompt to spit out the approximate average of what had been published on that subject in time to be hoovered up by the training data for the GPT-4o training cutoff of September 2023.

(Here’s what I got just now running it against GPT-4o.)

This illustrates the first lesson of effective LLM usage: know your training cutoff dates. For many queries these are an essential factor in whether or not the LLM is likely to provide you with a useful answer.

Given the pace of change in the AI landscape, an answer based on September 2023 training data is unlikely to offer useful insights into the state of things in 2025.

It’s worth noting that there are tools that might do better at this. OpenAI’s Deep Research tool for example can run a barrage of searches against the web for recent information, then spend multiple minutes digesting those results, running follow-up searches and crunching that together into an impressive looking report.

(I still wouldn’t trust it for a question this broad though: the report format looks more credible than it is, and can suffer from misinformation by omission which is very difficult to spot.)

Deep Research only rolled out in February this year, so it is unlikely to be the tool Peter Kyle was using given likely delays in receiving the requested FOIA data.

What I would do instead

Off the top of my head, here are examples of prompts I would use if I wanted to get ChatGPT’s help digging into this particular question:

• Brainstorm potential reasons that UK SMBs might be slow to embrace recent advances in AI. This would give me a starting point for my own thoughts about the subject, and may highlight some things I hadn’t considered that I should look into further.
• Identify key stakeholders in the UK SMB community who might have insights on this issue. I wouldn’t expect anything comprehensive here, but it might turn up some initial names I could reach out to for interviews or further research.
• I work in UK Government: which departments should I contact that might have relevant information on this topic? Given the size and complexity of the UK government even cabinet ministers could be excused from knowing every department.
• Suggest other approaches I could take to research this issue. Another brainstorming prompt. I like prompts like this where “right or wrong” doesn’t particularly matter. LLMs are electric bicycles for the mind.
• Use your search tool: find recent credible studies on the subject and identify their authors. I’ve been getting some good results from telling LLMs with good search tools—like o3 and o4-mini—to evaluate the “credibility” of sources they find. It’s a dumb prompting hack but it appears to work quite well—you can watch their reasoning traces and see how they place more faith in papers from well known publications, or newspapers with strong reputations for fact checking.

Prompts that do make sense

From the New Scientist article:

As well as seeking this advice, Kyle asked ChatGPT to define various terms relevant to his department: antimatter, quantum and digital inclusion. Two experts New Scientist spoke to said they were surprised by the quality of the responses when it came to ChatGPT’s definitions of quantum. “This is surprisingly good, in my opinion,” says Peter Knight at Imperial College London. “I think it’s not bad at all,” says Cristian Bonato at Heriot-Watt University in Edinburgh, UK.

This doesn’t surprise me at all. If you ask a good LLM for definitions of terms with strong, well established meanings you’re going to get great results almost every time.

My rule of thumb used to be that if a friend who had just read the Wikipedia page on a subject could answer my question then an LLM will be able to answer it too.

As the frontier models have grown stronger I’ve upgraded that rule of thumb. I now expect a good result for any mainstream-enough topic for which there was widespread consensus prior to that all-important training cutoff date.

Once again, it all comes down to intuition. The only way to get really strong intuition as to what will work with LLMs is to spend a huge amount of time using them, and paying a skeptical eye to everything that they produce.

Treating ChatGPT as an all knowing Oracle for anything outside of a two year stale Wikipedia version of the world’s knowledge is almost always a mistake.

Treating it as a brainstorming companion and electric bicycle for the mind is, I think, a much better strategy.

Should the UK technology secretary be using ChatGPT?

Some of the reporting I’ve seen around this story has seemed to suggest that Peter Kyle’s use of ChatGPT is embarrassing.

Personally, I think that if the UK’s Secretary of State for Science, Innovation and Technology was not exploring this family of technologies it would be a dereliction of duty!

The thing we can’t tell from these ChatGPT logs is how dependent he was on these results.

Did he idly throw some questions at ChatGPT out of curiosity to see what came back, then ignore that entirely, engage with his policy team and talk to experts in the field to get a detailed understanding of the issues at hand?

Or did he prompt ChatGPT, take the results as gospel and make policy decisions based on that sloppy interpretation of a two-year stale guess at the state of the world?

Those are the questions I’d like to see answered.