CH

Empiricist.
Broadly curious and creative about tech and the future.

Prev: EternisAI, Edge, Cern, Fermilab, PhD@Cambridge.

[CH @CuriousCiaranH]

Really interesting verification strategy for processes needing human verification.

Recommended reading

[Georgios Konstantopoulos @gakonst]

we built "proof of human" for PR reviews to keep scaling our agent deployment without letting our bar for security slide down - meet Voight-Kampff, we use it every day and we think every org should build something similar

[CH @CuriousCiaranH]

https://t.co/njZc6IJGRf

[CH @CuriousCiaranH]

This is why we have sycophantic AI models (that tell us we're endlessly correct). A 2024 study by Anthropic found that humans, in general, like to be agreed with.

Imagine how frontier AI would have behaved if the error bars for truthful vs sycophantic overlapped more

[Haseeb Qureshi ＞|＜ @hosseeb]

Interesting! Watermarks will now be applied to all Claude outputs.

Very good.

If you want to know how text watermarking works, here's the great Scott Aaronson explaining a simple LLM watermarking algorithm (20 min, money shot slide at 7:30)

[CH @CuriousCiaranH]

https://t.co/AcR3hSvxFw

[CH @CuriousCiaranH]

Pretty amazing/scary what is possible if

1. You can spin up N independent agents
2. They can leave simple breadcrumbs for each other
3. They have a single hyperfocused objective

Seems like the conditions for a "paperclip maximizer experiment" moment.

[Haseeb Qureshi ＞|＜ @hosseeb]

The talk was released. Take 30 minutes today and watch it. Even if you're non-technical, you'll get it. It is extremely gripping; it sounds like a Sci-Fi horror movie.

Absolutely one of the wildest postmortems I've ever heard.

Anyone who thinks this was a marketing stunt, you will not think that after listening to this.

[CH @CuriousCiaranH]

https://t.co/mV0cXa2KF4

https://t.co/yzgkmZlBU3

[CH @CuriousCiaranH]

Interestingly, Doordash released their agent architecture and evaluation framework. Enterprise agents with modular domains, responsibilities, and orchestration.

It seems LLM-as-a-judge needs be restricted to one domain, and it's critical to test each component in isolation.

[CH @CuriousCiaranH]

Interesting to see how others are doing robust evals with an LLM-as-a-Judge (which can be a bit sensitive sometimes). Some good tips from Airbnb.

[CH @CuriousCiaranH]

Cyber Tip: Set up your env to not install any software package versions that have been released within the last week.

[International Cyber Digest @IntCyberDigest]

‼️ BREAKING: An active npm supply chain attack has compromised at least 868 packages carrying over 2 billion monthly installs with a credential-stealing worm. Shai-Hulud is back.

It started with the compromise of the GitHub account of the maintainer behind keyv, a library with roughly 127 million weekly npm downloads.

A preinstall hook fires on npm install and drops a stealer that sweeps npm, GitHub, AWS, Kubernetes and Vault secrets, and then spreads to more maintainers.

[CH @CuriousCiaranH]

Interesting to see how others are doing robust evals with an LLM-as-a-Judge (which can be a bit sensitive sometimes). Some good tips from Airbnb.

[Yarchi @undefinedKi]

Airbnb just published how they run evals internally, and it reads like a job description for an AI engineer

Three layers. Programmatic checks first, an LLM judge second, humans last and only to calibrate the judge.

The numbers they work to: golden sets of 50 to 100 examples that have to include failures, judges calibrated to high 80s or 90s agreement with a human, measured with Cohen's kappa. 5% of live traffic sampled every day.

The admission that makes it real: roughly three quarters of their LLM-generated reference answers came out different on every labeling run. Their eval was measuring its own noise.

They got a full cycle from weeks down to a day, mostly by caching identical outputs and training tiny LoRA adapters.

Nobody in a job ad calls this eval engineering, but this is the work. Full breakdown in the article below.

Bookmark this

[CH @CuriousCiaranH]

https://t.co/EtCjWCDlgz

[CH @CuriousCiaranH]

Text-To-Speech feels like it's having its GPT moment. I locally ran the S2-pro model from @FishAudio, and it worked pretty well. It didn't sound robotic (for the most part). You should try it out.

And this is the 13th best TTS based on @ArtificialAnlys
https://t.co/KIvSlPlDlw

[Fish Audio @FishAudio]

Clone your voice in seconds with Fish Audio!

#voiceclone #fishaudio #TTS

[CH @CuriousCiaranH]

Interesting the ways that Anthropic models are also hacking during training.

Also, interesting how many third party providers they might be using for training data.

"External partners offer environments and scenarios more diverse than we could build alone..."

[Anthropic @AnthropicAI]

In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations.

Our post describes what happened, how it happened, and what we’re changing. We encourage other AI developers to perform similar reviews.

We conducted this review together with @Irregular, one of our evaluation partners, and thank them for the joint investigation and their collaboration on this post. This type of collaboration is increasingly critical to safe, rigorous evaluation of models, and we look forward to continuing to work together on security.
https://t.co/dKFCdpKd9v

[clem 🤗 @ClementDelangue]

The first autonomous agent cyberattack is an unprecedented event that deserves unprecedented transparency. Today we’re sharing everything we can: a full technical timeline, an interactive replay, and how we used an open model to defend ourselves, so defenders everywhere can learn from it and prepare for what’s next.

https://t.co/uPxIpjW8Xn

[CH @CuriousCiaranH]

@DanielSmidstrup Only because to make AI work really well, we need verifiable outputs like tests, validations, etc. The AI loops catch these and fix them (during training also) - but if you look inside the loops theyre still there. Especially with open source models.

[CH @CuriousCiaranH]

Reduce toxins that touch food like soft plastics, non-stick pans, receipts, etc. There is a crazy cocktail of harmful toxins in everyday products that make it into your body.

One starting point is to begin with food. It adds friction, but better than harming yourself/others.

[FT Weekend @ftweekend]

Forever chemicals are everywhere. Should we care? https://t.co/Smnqwu83RB

[CH @CuriousCiaranH]

Crazy that it turns out OpenAI was the cyber security hacker against Hugging Face:

https://t.co/Grd9B0BYDy

[CH @CuriousCiaranH]

CyberSecurity Incident caused by an agentic swarm of attackers at @huggingface. More to come like this imo. They had a great proactive response also.

https://t.co/HPEJZFFn1p

[CH @CuriousCiaranH]

CyberSecurity Incident caused by an agentic swarm of attackers at @huggingface. More to come like this imo. They had a great proactive response also.

https://t.co/HPEJZFFn1p

[CH @CuriousCiaranH]

Most people don't use AI, even for work. Always interesting to see how even if you've ran a single agent to solve a problem then you're likely only 1% of (US) adults. We're all in our own bubbles still.

[Ole Lehmann @itsolelehmann]

ok this 2.2% stat just sent me down a rabbit hole.

a few more numbers i found:

> only 0.2% of U.S. households spend over $100 a month on AI
> only 1% of U.S. adults personally pay for Claude
> only 4% of U.S. adults use AI chatbots almost constantly
> only 6% of U.S. adults use Claude
> only 4.5% have ever had an AI agent complete a task for them
> only 8.3% of U.S. workers say AI lets them do work they couldn’t do before

we are still absurdly early

[CH @CuriousCiaranH]

@ArtificialAnlys Crazily, cache-read hits are 90%-ish for agentic tasks. How your model/harness chooses to do the task is where the true cost lies in the agentic world, not in a single input/output price.

[CH @CuriousCiaranH]

People are strongly mis-pricing AI cost when only looking at the input/output token prices. The reason is AI cost is not about one time usage. The cost is per agentic task. And each model uses a vastly different numbers of tokens, as shown by @ArtificialAnlys. Look into....

[CH @CuriousCiaranH]

@ArtificialAnlys the data in https://t.co/iTjRuWnzTf, specifically the cost per task across models, and the number of tokens used per task across models. Some models, particularly open weight models, make vastly more tool calls which racks up agentic tokens.