Andy Wheeler

Technical consulting for coding, predictive analytics, and optimization. Focused on public sector applications, mostly with police departments.

[Andy Wheeler @CrimeDecoder]

So the time has come, VerusCite is a fully functional site and ready to release to the public.

Hallucinations in peer review research are a current major problem. It signals a grave lack of care by researchers, and should trigger subsequently more scrutiny of the claims therein.

But even in articles that researchers clearly use genAI tools to write whole cloth, the majority of the citations are correct.

VerusCite only takes about 2~3 minutes to extract and verify those citations in an academic paper.

[Andy Wheeler @CrimeDecoder]

I wrote a book on AI engineering with the LLM APIs. I am begging you to read and understand it, because I need to hire more people with these skills!

https://t.co/FluejmtOd7

[Andy Wheeler @CrimeDecoder]

Can see this report at https://t.co/56baS7ddAd, 16 hallucinations that point to DOIs to other papers (and the papers as cited do not exist).

[Andy Wheeler @CrimeDecoder]

One of the recent AI hallucinations I have come across is DOIs that point to totally different papers. I honestly don't know if these are author or editor induced.

If editor, it just means the journals are publishing papers that *they know* have a pretty gross amount of errors.

[Andy Wheeler @CrimeDecoder]

@joefrancis505 I tried to Manski bound the reasonable synth space, https://t.co/FIyPCaVPlA. It is wide in those examples but honestly not as hopeless as I thought the bounds would be.

[Andy Wheeler @CrimeDecoder]

Hall of Hallucinations is up to 142 papers. Sign up, contribute a few of your own.
https://t.co/OTfgJBRxid

[Andy Wheeler @CrimeDecoder]

Can see the post at https://t.co/LrDygIIS7z

I suggests a human review after my tool has flagged. Which is of course is so much faster than going through citations yourself one by one.

[Andy Wheeler @CrimeDecoder]

New VerusCite blog post, my false positive rate is under 1% in my benchmark and likely outside of it.

I go through my Grok Bot scan yesterday to show an out of sample example. Of the 200+ citations scanned, my tool flagged 10. Only one is potentially a false positive.

[Andy Wheeler @CrimeDecoder]

Daniel vs Alexander is a big difference, but again no way to know for certain unless the author admits it (which they never will).

If you would like to see additional examples of author hallucinations, one of my scans for the day found a paper with 4 examples, https://t.co/94YgFcPoHk. Those are more egregious but better examples of hallucinations.

A not uncommon pattern is to get the first author right (like the first example) but all the secondary authors are wrong. A few of the examples in that paper though just 100% of the authors are wrong.

[Andy Wheeler @CrimeDecoder]

I will never have a smoking gun. The design philosophy behind when to identify a hallucination is a whole cloth made up word -- a name, a title, a journal, etc. Minor errors (as you can see in the linked report), like a year off, a misspelling, etc. will not be flagged as hallucination.

In my public benchmark, I have a false positive rate of under 1%, https://t.co/TOzk6YNbbP, so I do suggest humans to review anything I flag after the fact. And of course identifying minor errors is useful for copy-editors as well.

[Andy Wheeler @CrimeDecoder]

There are some legacy systems that are terrifying and I have no clue how they still manage to function (e.g. email). I am going to add 811 to that list.

Got an email from North Carolina that has a parcel map that is at least over 10+ years old.

It does work fast and gets the various companies out, but gosh that must be a tangled web to work on as a developer. (And the utilities must drop a significant amount of labor cost for something that should not be that hard.)

[Andy Wheeler @CrimeDecoder]

@ian_t_adams I asked grok comparing to lotto like powerball. To reach that volume is closer to the monthly level for some of the bigger states.

Given you can bet for pennies on these ridiculous sequences and do it for so many things I imagine it is a better kick though for gamblers.

[Andy Wheeler @CrimeDecoder]

Holy moly guys, had no idea that level of volume was happening.

[Sam McQuillan @sam_mcquill]

Kalshi trading over the weekend

Parlays:
$6.09B

Sports (excluding parlays):
$1.69B

Everything else:
$727M

The midterm elections are in 2 weeks.

[Andy Wheeler @CrimeDecoder]

https://t.co/JTzONiS6Ez

[Andy Wheeler @CrimeDecoder]

Damien Charlotin's AI hallucination tracker in court cases has flattened. Possibly via not collecting as many, but also just as likely the models are getting better. (I am skeptical individuals are using them more effectively!)

[Andy Wheeler @CrimeDecoder]

Can see the report at https://t.co/Ao6VovV3Lx

[Andy Wheeler @CrimeDecoder]

Hallucination of the day, there is basically a cottage industry of junk review articles now.

[Andy Wheeler @CrimeDecoder]

@CharlesFLehman I have never used these, is this just a UI pattern? Like they just give a pop-up "Win 10 million dollars" for these random events coinciding. (Which does not go against your point, but just genuinely curious.)

Or are are people going in and making their own combinations?

[Andy Wheeler @CrimeDecoder]

https://t.co/CKXm688JWc

PILLAGE! Is Not a Management Strategy
DOGE, curiosity, and lost opportunities by Abigail Haddad

[Andy Wheeler @CrimeDecoder]

Abigail Haddad has a good blog post in response to the Glock piece on DOGE that I recommend folks read. Unlike private sector, it is hard to measure "how effective the gov is".

And she is pretty skeptical of Glock's claim that it is just as effective even after the exodus.