1/12 We just found something unsettling: Today's most advanced AI models - including the latest powerhouse reasoning models - can't keep track of what actually happened. Even in a simple conversation. Our ICLR'25 paper reveals why this matters 🧵
🚨 Studies are starting to show what many of us feared: AI use might lead to overreliance and human disempowerment. Below is the SHOCKING conclusion of this particular study [download for future reference]:
"1. We surveyed 319 knowledge workers who use GenAI tools (e.g., ChatGPT, Copilot) at work at least once per week, to model how they enact critical thinking when using GenAI tools, and how GenAI affects their perceived effort of thinking critically. Analysing 936 real-world GenAI tool use examples our participants shared, we find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against external sources.
2. Moreover, while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.
3. Higher confidence in GenAI’s ability to perform a task is related to less critical thinking effort.
4. When using GenAI tools, the effort invested in critical thinking shifts from information gathering to information verification; from problem-solving to AI response integration; and from task execution to task stewardship.
5. Knowledge workers face new challenges in critical thinking as they incorporate GenAI into their knowledge workflows.
6. To that end, our work suggests that GenAI tools need to be designed to support knowledge workers’ critical thinking by addressing their awareness, motivation, and ability barriers."
-
➡️ Comments:
- It would be interesting to see more studies like this, especially with larger samples and focusing on different occupations, demographics, and use patterns;
- The paper focuses on Generative AI; it will be interesting to see a similar study focused on advanced AI agents, especially as their popularity grows.
👉 Download & read the full paper "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers" by @hankhplee@AdvaitSarkar, Lev Tankelevitch, @iandrosos, @seanrintel, Richard Banks & Nicholas Wilson below.
👉 NEVER MISS my AI governance updates, including must-read research papers like this one: join 51,800+ people who subscribe to my weekly newsletter (below).
Google thinks the average IQ of a monkey is 74 which is of course ridiculous. I looked into why and it’s because it’s pulling from a website that’s AI generated. 😂
Are you interested in a #PhD position in Human-centered #Explainable#AI? We are hiring!
🗓️ Application deadline: 28 August 2024
🔗 Find out more & apply here: https://t.co/2MOu98dQKi
Repost = ❤️
One week left, you #RecSys2024 people of the future!
Please submit your applications to the doctoral symposium. This includes an extended abstract in which you outline your plans, a CV, the expected benefits of participation, and letter of recommendation:
https://t.co/xga7ME8I7f
We made our code available at: https://t.co/21qSQlu4dN
You can fully reproduce our results with just a few command lines as there is no expensive LLM training 😉
The submission deadline for the TOIS Special Issue on Pre-trained Models for Search and Recommendation is approaching! Don't miss the chance to contribute your insights and research on pre-trained models for IR! #TOIS https://t.co/eyDBu7e139
The SIGIR Student Travel Grant Program will be offering Student Travel Grants for students presenting their accepted contributions at #chiir2024. Paper notifications are due on December 12th and applications for travel grants can be made between December 13th and December 28th
Still possible to register for online #CIKM2023! You will be able to attend the keynote talks and main room talks live; all other talks will be further provided as online encore ones on request at mutually agreed times after the conference. More at https://t.co/FtOuTlIZBL
Much needed checklist to ensure ML papers are actually robust, reproducible, and don't misrepresent findings (eg because of data leakage, no external validity etc).
Web: https://t.co/fkeyXYChPY
Checklist: https://t.co/uz3M3bnknw
This took me more than expected, but I wanted to tell you about a pre-print we recently released with @ipoga and @lariviev on the evolution of citations-concentration 🧵
https://t.co/h8SXLuofis
EMNLP 2022 is seeking a limited number of student volunteers, for both online and in-person versions. In exchange for one full day equivalent of work, students receive free registration to the main conference. Details and application form below. #emnlp https://t.co/eBzXsQCygX