@ednewtonrex I have shown you plenty of new information today, while all you were doing was constantly making accusations of bias, bad faith and dishonesty. I wonder who really is the one with bad faith.
I have no ther words than that you should really think about what you did.
Well it WAS used to head VERY prominent witchhunt cases in publishing this year so your framing is at best naive, and in fact dismissive.
The CEO of Pengram has habit of exaggerating its accuracy, which kept adding fuel to the fire before any proper probe could take place.
Pangram provides an incredibly important service, and it's sad that so many who are generally critical of AI view it as an enemy instead of an ally.
First-party and third-party evaluations seem to agree that it has a low false positive rate. It isn't perfect, and doesn't claim to be, but if it identifies a long passage as being AI-written, the chances are that it is right.
It shouldn't be used for thoughtless witch-hunts, but IMO it is a very useful tool for helping to provide evidence of AI text generation & editing.
To offer just one story: I used it to help call out an elected politician for AI writing. They published an article in a national newspaper that I suspected was partially written by AI; I contacted the paper in question, including among my evidence Pangram's verdict; they contacted the author, who admitted the use and said they would not do it again.
For people who distrust Pangram: I would encourage you to dig out some of your own writing from Google Drive or similar that you never put online, written before 2020, and try it. There is a growing idea that Pangram has trained on the entire internet, meaning that public pre-2020 text does not make a reliable test (I think this idea is mistaken, but I won't get into that). But private writing does not suffer from this problem.
Run your private pre-2020 writing through Pangram, and see what comes back. My guess is that it will identify it as human-written.
Clearly, there are people who have a vested interest in Pangram not being trusted. Primarily, these are people who use AI to write or edit text themselves, or whose organisations do so. If people criticise Pangram without evidence, I recommend the first thing you do is run their writing, or their organisation's writing, through Pangram. Not as a witch-hunt, but to provide you with further evidence before you decide what weight to give their words.
We should absolutely demand that AI detection systems are fairly evaluated. But IMO Pangram has done a good job of this - and I think that any claim that it does not work well should be backed up by at least one study that suggests this is the case.
@ednewtonrex Look at this entire thread and see if you have ever backed any of your claims with one morsel of evidence yourself. And see if you have ever once apologized after I have responded your every claims of my dishonesty with real cases and quotes.
and then answer who was vibing.
@ednewtonrex Not only there's blatant absence of risks, acknowledgement of existence of false alarms, he is also aiming for the tool to function as future seal of approval.
Would 'he is giving highly undue weight to his confidence,' fit your bill of accuracy? Eitherway, he is lying.
@ednewtonrex Also did you actually read the article? The false positive rate is just him reporting on himself. When other studies that show higher error rate then his claim, he goes 'well its wrong because our technology is better now' (evidenced by?)
What kind of conclusion did you want?
@ednewtonrex Be it one in a thousand, one in ten thousand, if it happens to you,there is no going back. How the hell do you argue yourself back from 'the very high statistical confidence?' Because the first thing you said was 'well the publisher may have been plotting' so I really don't know
@ednewtonrex Yeah everyone starts with 'well it is not perfect' but goes 'Pangram is really better'
Then in the cases I showed you they went to 'So I am going to use it to back up my accusations that is going to tarnish a person forever.' because Pangram is really that good.
@ednewtonrex Here. https://t.co/eq6JfgQpzD
https://t.co/Djiv9THG5V
Also it is time that I tell you what you are doing right now is technically Sealioning.
@ednewtonrex And I do this purely out of care for my craft. If you write literary prose long enough, you will know using Gen AI to 'write' these things is like drinking salt water because you are parched.
@ednewtonrex people who didn't have English as the first language. We show different pattern of language use than majority of native users, which can skew towards translatese or unnatural. We are the statistical minority.
@ednewtonrex Also with 'infallible', I am reponding your question of 'so you still agree the accurasy is darn good?' because that logic is what is giving strength to people making accusations.
@ednewtonrex He got his fame by using Pangram results as the basis for accusing people left and right. He got it right with Mia Ballard, but with the Zimbabwean author he is holding on to 'well the vibe isnt right'. Is that what a CEO conscious of the possible risks would normally do?
@ednewtonrex 'It should not be used for witch-hunts' but the claims of 'it is infallible' is precisely what is fueling people's confidence when they are using it for the said witch hunts. And framing people's fear and anger against it as prejudice is very dismissive of the possible risks.
If you keep telling people that the Pangram is a magic hammer that can check your vibes and nail your accusations, you give these people with no professional stake or expertise wield it around with no care for 'false positives'.
That's not empowerment.
H.M. wolfe is another author who was accused of using AI by Atlantic journalist with Pangram. The article is largely 'well it feels like it' but also, 'Pangram agrees with me.' How the hell are authors supposed to defend against 'well I just feel like it and machine said so'?
@ednewtonrex We live in the age in which people routinely use Gen AI to send out emails and notices. Announcement is not novel at stake. What are you getting at?