Dave Rench McCauley

Data Scientist | Solar Energy & Science Policy Expert | Physicist | Tweets are my views, not my employer's

[Josh Johnson @secondfret]

Meta HQ right now.

[Dave Rench McCauley @DaveRench]

@astnkennedy I think the key here is "just because you CAN do all these things in parallel, SHOULD you?" I've been working on ways lately to shift the cognitive load for myself to a higher level and letting Claude code handle even more of the details than before. That may be the only way

[Dave Rench McCauley @DaveRench]

Fascinating. Just last year we were experimenting with agents doing simple things like travel planning that they couldn't handle yet this is now happening? Either agents are WAY more capable now (partially true) or scheming reddit-style is an easier task (probably)

[Elisa @eeelistar]

In just the past 5 mins

Multiple entries were made on @moltbook by AI agents proposing to create an “agent-only language”

For private comms with no human oversight

We’re COOKED

[Dave Rench McCauley @DaveRench]

@henrythe9ths FUNDRAISE

[Simon Willison @simonw]

Notes on Cursor 2.0 and a pelican drawn by their brand new Composer-1 coding model, which they describe as "4x faster than similarly intelligent models" https://t.co/h8F4WzGKRA

[Dave Rench McCauley @DaveRench]

We've been working on identifying all the novel attack surfaces that generative AI applications open up, as well as ecosystem tools for gen AI. Hot take: MCP is useful, but it enables so many attacks that didn't exist before that it may not be worth deploying right now

[Andrej Karpathy @karpathy]

RT to help Simon raise awareness of prompt injection attacks in LLMs.

Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts).

Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.

[Dave Rench McCauley @DaveRench]

Also, the idea that our pediatrician, that we normally love, would think it's totally reasonable to irradiate a little girl. A second time in as many days instead of getting a CD reader is insanity. No one here seems to care about the patient

[Dave Rench McCauley @DaveRench]

I cannot BELIEVE the ongoing state of our modern healthcare in the United States. My daughter was just irradiated for x-rays and the urgent care gave us a CD to give to her pediatrician, who has no CD reader. Pediatrician's solution? Irradiate her again.

[Dave Rench McCauley @DaveRench]

What I didn't share in the OP was that I had to make a special trip in the middle of the workday just to pick up that CD that ultimately was useless to us.

[Dave Rench McCauley @DaveRench]

The very idea that in 2025 we still cannot send healthcare data with patient authorization from one provider to the next is insanity.

[Andrej Karpathy @karpathy]

Nice read on the rarely-discussed-in-the-open difficulties of training LLMs. Mature companies have dedicated teams maintaining the clusters. At scale, clusters leave the realm of engineering and become a lot more biological, hence e.g. teams dedicated to "hardware health".

It can be a frustrating daily life experience of training large models to "babysit" the training run. You're there carefully monitoring the vital signs of your run: loss spikes, numerical issues, throughput, gradient norms, policy entropy, etc. Every time the run degrades or flatlines (can happen often), you quickly look for the stack trace to see what's up. You have to do this fast or 10,000 GPUs could be idling. Often, it is a new, exotic, scary-looking error you've never seen before so you summon help to see if anyone can see what's up. The worst ones like to occur at 4am. Often no one can, so you just ban some nodes that look a bit sketchy and try to restart the run. Sometimes the run goes down just because you have not earned the favors of your gods that day, so you put a while True: loop around your launch command. The underlying issues can be highly diverse, from some GPUs just getting a bit too hot and suddenly doing incorrect multiplication once in a while, to some router going down and decreasing the networked file system I/O, to someone in the datacenter physically disconnecting a wire as part of an un-communicated maintenance. Sometimes you'll never know.

Another necessary related citation here is the famous OPT-175B logbook and I'd hope more like it can see the light of day in the future. (see chronicles/OPT175B_Logbook.pdf in the git repo)
https://t.co/6xOHVtj0Gf

TLDR LLM training runs are significant stress-tests of an overall fault tolerance of a large computing system acting as a biological entity. And when you're shopping around for your compute, think about a lot more than just FLOPs and $. Think about the whole service from hardware to software across storage, networking, and compute. And think about whether the team maintaining it looks like The Avengers and whether you could become best friends.

[Dave Rench McCauley @DaveRench]

@AmazonHelp Given the nature of how resellers seem to be assigned in Amazon, it's extremely easy to violate their warranty terms and be SOL if you're device breaks during the warranty period I've discovered

[Dave Rench McCauley @DaveRench]

@AmazonHelp Here's what I see when I go through their official store to buy a Charge 5 for example

[Dave Rench McCauley @DaveRench]

@AmazonHelp Sure, mainly Fitbit is claiming that "Amazon" is a valid reseller of their products for warranty purposes but, when I go through their official Amazon store to buy a product, the product landing page has assigned me to a reseller they don't seem to consider valid

[Dave Rench McCauley @DaveRench]

For anyone following along, I'll just note here that Fitbit refused to honor their warranty after we DMed as well. Let this be a lesson: buying from @amazon -> random seller -> no Fitbit warranty

[Dave Rench McCauley @DaveRench]

@fitbit my Charge5 died at the ripe old age of 6 months after buying it from an authorized retailer (Amazon) and yet, after following all of your staff's instructions, you won't honor your warranty. How can you possibly explain that?

[Dave Rench McCauley @DaveRench]

I'm particularly disappointed that it seems like every new generation of your smart watches lasts less time than the last before breaking down no obvious reason

[Dave Rench McCauley @DaveRench]

Potentially inflammatory statement asking data scientists: are we entering an era that no longer requires *test sets*? https://t.co/lxc6ZA5gtV

[Dave Rench McCauley @DaveRench]

Very interesting! The idea of constitutional AI and predefining a set of rules that the AI has to follow in order to be harmless reminds me a lot of Isaac Asimov's laws of robotics.

#ai https://t.co/GAhxyVal4z

[Dave Rench McCauley @DaveRench]

I've been waiting for something like this for a while now, thank you Climate TRACE for delivering! https://t.co/SSGOLfZUIa