@thisdudelikesAI Nice summary. I've felt this intuitively; nice to see research that attempts to measure this. I'm certain there are second- and third-order effects too: a CEO asks chatGPT for advice; directs management to do X instead of Y, and the impaired judgement trickles down
@DNAutics@testingham For many benchmarks, I also wouldn't be shocked if even the evaluation sets have slowly leaked out, making them no longer rare. Hard to audit comprehensively.