obviously. not human, just slop. people like this will scream until they’re blue in the face but they have no explanation for the complete transformation of their prose/productivity post-LLM. and no, screenshots of a day of Google Docs edit history or whining about RP isn’t it.
Well, at least "my husband/RP partner can't visualize scenes well and that's why I write so descriptively" is a creative and new excuse from a slopper.
@rpondiscio 75% of farmers' wives are teachers or nurses because the school district or hospital provides health insurance and a year-round, steady paycheck that farming does not. Losing the local public school district would decimate rural communities.
@jovedaughter@ghostiesdagger It's their only real defense, because they genuinely don't believe their friends would do something like this, so OBVIOUSLY the only reason someone might accuse them is jealousy.
So, what exactly is LLM writing? How is it different from how humans write?
If you look at it closely, it's not just the groups of three, or "it's not X, it's Y"; it's the whole set of characteristics that usually go together.
To study this, I used the same algorithm that Pangram uses to train its classifier. It's called "LLM twins."
It works like this: I downloaded 200 samples of verifiably human writing across all genres, then asked a language model to create a short description of each, and then to fire up separate agents, which had to re-create the pieces from the descriptions alone and make them the same length as the originals.
As a result, I obtained 200 pairs of pieces of writing, each pair sharing a genre, a length, and a rhetorical goal.
For example, a human-written press release about a Boca Raton condominium hosting the filming of "Selling New York," and an LLM-written press release about the same event.
Pangram generates millions of those. I only had 200, but that allowed me to measure the main differences between humans and LLMs.
Here are the main differences:
1. LLMs rarely let two adjacent clauses or sentences share a noun. A human would write, "I stepped into a corridor. The corridor was dark and narrow." An LLM would say, "I stepped into a corridor. Dark. Narrow."
2. LLMs show much more diversity in vocabulary. A human would say "red" or "dark-red." An LLM would write "plum-coloured."
3. LLMs write fewer words per clause. A human would write, "We checked the reported numbers, our investigation found no discrepancies." An LLM would write, "We checked the numbers, no discrepancies found."
4. LLMs write shorter sentences. This is self-explanatory, I believe.
5. LLMs seem to avoid proper nouns (people, places) if they can. A human would write,
"A distraught displaced woman, Hajiya Mairo Abdullahi, was at the weekend seen holding the picture of her seven-year-old child."
An LLM would, among other things, omit the woman's name if it isn't needed for the narrative and doesn't recur later in the text. The LLM version would go something like:
"A displaced woman, struggling with grief, was clutching a photograph of her seven-year-old child."
Even if the person is named, an LLM is more likely to avoid using the name when attributing speech to them. A human would write, "Ms Abdullahi said," while an LLM would write, "The woman said."
6. LLMs also avoid repeated 3-word sequences, whereas humans are okay with them. A human would write "the Kenya cases"; an LLM would rephrase it as "cases from Kenya" or "the cases," or structure the sentence differently to avoid repetition.
7. LLMs operate in a narrower band of word frequencies: more often than humans, they avoid very common words (they are more likely to say "plum-coloured" instead of "red"), but they also avoid rare words (they are unlikely to say "incarnadine" instead of "red").
8. LLMs use fewer numbers. This could be specific to news articles, but LLMs prefer to say "many," "several," or "thousands" when the exact number isn't essential beyond conveying the ballpark. A human would put an actual number.
And finally, an aspect that captures the specific feeling of hollowness that encountering LLM writing brings.
9. Each new sentence in an LLM twin is more likely to introduce a new common noun ("displacement," "war," etc.). But it's less likely to introduce a new, specific, verifiable fact.
A human would write: "Parents want to know if their children are alive and when they will come home. An inquiry has been launched by the Nigerian government..."
An LLM would write: "For the parents, the question is simple. They want to know if their children are alive and when they are coming home."
This is, of course, from studying 200 samples. Pangram has millions, and its neural networks, as unexplainable as this technology is, can probably be contextualised as checking tens of thousands of such "tics."
Importantly, none of these is consequential on its own, but in LLM writing all these features tend to go together. That's why people who work a lot with LLMs can often (but not always) recognise their style.
@Slagathor946@ghostiesdagger Tbh I would expect basketball written by AI to make slightly more sense than hockey written by AI just because there is more basketball in the training data but very real humans also can’t write sports in sports romance for shit.
More like a pastel yellow flag than a red one.
@Izzycantskatefs in case someone saw this and thought "lol no way, not even Crosby is that fucking weird" there is video proof: https://t.co/gp4KQiXrmX