This is why it's so important that we call out over-reliance on AI when we can, because the students who care might respond and have a chance at actually learning how to learn. Since I teach large courses, I focus on the most egregious cases, but it's better than doing nothing.
I had something similar happen. I had a student who came to weekly office hours for help. They would ask about some concept, so I would ask about a related concept to help prompt their thinking. But they rarely knew enough to respond, so they asked ChatGPT on their laptop.
necessary counterpoint bound to get less traction, from the in-person class w/ lots of frank discussion: student told me heโd *reversed* his AI usage. previously, heโd have LLMs finish essays whenever he felt overwhelmed. but he stuck w/ it all semester, now writes independently
After a few weeks (and some failed quizzes), the student finally realized that they just weren't learning anything, so they followed my advice and stopped using AI for studying. After that, their office visits were much more productive AND their marks improved.
@GhostOfSocrates I don't? I only flag obvious AI traces which basically eliminates all false positives and all true/false negatives.
I'm not an AI cop trying to catch absolutely every single AI user.
@rainbowwnez Same reason we say "head over heels" instead of "heels over head" and "have your cake and eat it, too" instead of "eat your cake and then still have it".
Idioms are whole units, not logical constructions, so they often drift from their origins by rearrangement, reduction, etc.
@GhostOfSocrates I think you're talking about someone else. I do not (and never claimed to) use Turnitin's AI detection software. I evaluate the text on my own, aided by plagiarism flags. I repeatedly mentioned specific examples of how I determine AI use. Sorry that wasn't clear enough!
@GhostOfSocrates Yeah, a lot of my students do that too, and they fail, because my classes are mostly about skill-building (which is practiced through homework) rather than memorizing facts.
Again, I can only speak to my courses. Your method can certainly work in others.
@GhostOfSocrates They are hardly getting rewarded. They get "credit" for submitting the assignment, but they aren't learning anything, and it shows up on their final exams, which is weighted far more than their homework.
@GhostOfSocrates Research published three years ago (which means it was started even earlier) is woefully out of date when it comes to generative LLM tools. They have evolved way too fast.
@GhostOfSocrates I get a disproportionate number of native speakers flagged than ELL, so however Turnitin is working for others, it's not doing that for me.
But I'm a linguist, so maybe I'm more exacting in what linguistic evidence I use for identifying academic dishonesty.
@GhostOfSocrates I can have as many as 500 students in one class. I can't interview every student individually about their work, so yes, the false negatives and true negatives both get ignored, as do many many many of positives. I really only ask to meet with students in the most egregious cases.
@GhostOfSocrates I don't know of any neurodivergent or ELL students who write about their "last knowledge update". I'm not flagging people because they used "delve"!
Funny enough, my ELL students are often some of my best writers, because they were actually taught to write.
@GhostOfSocrates If it's done well, we aren't going to notice, so when we do notice, it's because it wasn't done well. I literally cannot count the number of times I've seen "As a language model", "As of my last knowledge update", "Let me know if you want me to refine this", and similar traces.
@GhostOfSocrates Sure, I am biased against work that copies entire pages (even names) from previous years, brings up irrelevant topics, or leaves obvious AI traces.
As for transparency, our students are informed when the plagiarism checker is being used, and they have the right to opt out.
YMMV
@ranjodhd Seriously! I understand that mistakes happen, and we can't read every line of everything we cite, but the causal anti-intellectual "oopsie, oh well" attitude is so astounding here. If I discovered such an error in my own scholarship, I'd work even harder to prevent it in future.
"Does the existence of one hallucinated reference mean the entire paper is worthless?"
Yes! I don't see how that isn't obvious.
(1) If you support claim X with a hallucinated reference, then X can no longer be taken as true, and everything derived from it is questionable.
@DarrinADurant@zenahitz I don't read the OP as being "indifferent" to hallucinated references. I'm certainly not. The issue is the magnitude of the sin. Does this existence of one hallucinated reference mean the entire paper is worthless? The OP doesn't think so. If you do, make an argument.
(5) Hallucinated references may spread through continued careless citation practices, which pollutes the literature and makes every scholar's job harder. LLM junk is particularly pernicious because it can be produced so much more quickly and convincingly than traditional junk.