Ten minutes of leaning on ChatGPT made people quicker to give up, says a Berkeley-led study, like a calculator you can't put down
By Brett Gavaghan, Founder of Adaptd ·
A UC Berkeley-led study found people who used ChatGPT on problems did worse and skipped more once it was taken away. Worth knowing before AI becomes your team's first move.
The short version
- Researchers found people who used ChatGPT on problems got worse and skipped more questions once the AI was taken away.
- The study ran three online trials with 1,222 US participants and was presented as a peer-reviewed paper at a conference this month.
- Let staff try hard problems themselves first, and set AI up to explain its steps rather than just hand over answers.
We all suspected that having the answer one click away might be doing something to our patience. Now there's a study that measured it.
What did the Berkeley study find?
The study found that about 10 minutes of relying on an AI assistant made people less accurate and quicker to give up on hard problems once the assistant was removed. It was led by Brian Christian of UC Berkeley's Center for Human-Compatible AI, with researchers from Carnegie Mellon, MIT, Oxford and UCLA.
In the first trial, one group solved fraction problems alone and another had ChatGPT open alongside. The AI group started out more accurate. After 12 problems the researchers took the AI away, and that group's accuracy dropped almost straight away while their rate of skipping questions jumped.
How was the study done?
The researchers ran three randomised controlled trials with 1,222 people recruited online in the US. Two used fraction problems and one used SAT reading comprehension, and the results lined up across all three.
The paper was first posted in April and the peer-reviewed version was presented this month at the Conference on Language Modeling. It tested short tasks, not skills built up over years, so treat it as an early warning rather than the final word.
What should your business do about it?
Businesses rolling out AI should decide which skills staff need to keep sharp themselves, and protect time to practise them.
My take: this doesn't mean switching AI off. It means thinking about how people use it, especially new starters who are still learning the job. Christian suggests AI could act more like a tutor than an answer machine. My advice: ask staff to have a go first, then use AI to check or explain, and tell your AI tools to show their working. Otherwise the day the internet drops out could get very quiet, very quickly.
Sources
Wondering what this means for your business?
That's exactly what the discovery call is for.