Wally Hix

Entrepreneur & Investor | @Google @ThreeVentures

[SpaceXAI @SpaceXAI]

Announcing Grok 4.5, our first model trained specifically for coding and agents. It was trained with Cursor and offers frontier intelligence at leading speeds and cost efficiency.

https://t.co/i8HpU7w64k

[SpaceXAI @SpaceXAI]

Introducing Voice Agent Builder: a no-code platform to create human-like voice agents with Grok Voice.

Available today at $0.05 / min.

https://t.co/kUkF7zqvfR

[Anthropic @AnthropicAI]

Since June 12, we’ve been working closely with the US government to restore access to Claude Mythos 5 and Fable 5. Today, the government notified us that Mythos 5, our strongest cybersecurity model, can be redeployed to a set of US organizations that operate and defend critical infrastructure.

We’re restoring access for these organizations quickly, and we’re continuing to work with the government to expand access to Mythos 5 and make Fable 5 available for general use again.

[OpenClaw🦞 @openclaw]

OpenClaw just plugged into X, and now your own hardware gets the claws. 🦞

Bring your Grok, SuperGrok or X Premium subscription to your OpenClaw agent.

Now even your personal agent is red-pilled and based.

Get Grokked:
https://t.co/pIj2vp1IpM

[Peter Steinberger 🦞 @steipete]

The latest CodexBar update renders API costs wayyyy nicer. https://t.co/lJ4dxNHwzG

[SpaceXAI @SpaceXAI]

You can now use your @grok subscription inside @NousResearch Hermes Agent.

https://t.co/UYKGws8zzH

[Claude @claudeai]

You can now enable Claude to use your computer to complete tasks.

It opens your apps, navigates your browser, fills in spreadsheets—anything you'd do sitting at your desk.

Research preview in Claude Cowork and Claude Code, macOS only.

[Claude @claudeai]

Introducing Code Review, a new feature for Claude Code.

When a PR opens, Claude dispatches a team of agents to hunt for bugs.

[Andrej Karpathy @karpathy]

Bought a new Mac mini to properly tinker with claws over the weekend. The apple store person told me they are selling like hotcakes and everyone is confused :)

I'm definitely a bit sus'd to run OpenClaw specifically - giving my private data/keys to 400K lines of vibe coded monster that is being actively attacked at scale is not very appealing at all. Already seeing reports of exposed instances, RCE vulnerabilities, supply chain poisoning, malicious or compromised skills in the registry, it feels like a complete wild west and a security nightmare. But I do love the concept and I think that just like LLM agents were a new layer on top of LLMs, Claws are now a new layer on top of LLM agents, taking the orchestration, scheduling, context, tool calls and a kind of persistence to a next level.

Looking around, and given that the high level idea is clear, there are a lot of smaller Claws starting to pop out. For example, on a quick skim NanoClaw looks really interesting in that the core engine is ~4000 lines of code (fits into both my head and that of AI agents, so it feels manageable, auditable, flexible, etc.) and runs everything in containers by default. I also love their approach to configurability - it's not done via config files it's done via skills! For example, /add-telegram instructs your AI agent how to modify the actual code to integrate Telegram. I haven't come across this yet and it slightly blew my mind earlier today as a new, AI-enabled approach to preventing config mess and if-then-else monsters. Basically - the implied new meta is to write the most maximally forkable repo and then have skills that fork it into any desired more exotic configuration. Very cool.

Anyway there are many others - e.g. nanobot, zeroclaw, ironclaw, picoclaw (lol @ prefixes). There are also cloud-hosted alternatives but tbh I don't love these because it feels much harder to tinker with. In particular, local setup allows easy connection to home automation gadgets on the local network. And I don't know, there is something aesthetically pleasing about there being a physical device 'possessed' by a little ghost of a personal digital house elf.

Not 100% sure what my setup ends up looking like just yet but Claws are an awesome, exciting new layer of the AI stack.

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[ALEX @ajtourville]

Elon Musk deals with can't-do attitude engineers and Twitter's Sacramento server farm on Christmas Eve.

Based AF!

[LangChain @LangChain]

Reliable, fully local RAG agents with Llama3

With the release of Llama3, there's high interest in agents that run reliably & locally (e.g., on your laptop).

Here, we show to how build reliable local agents using LangGraph and Llama3-8b from scratch.

As a example, we combine ideas from 3 advanced RAG papers (Adaptive RAG, Corrective RAG, and Self-RAG) into a single control flow in LangGraph.

We run this locally w/ open source embeddings c/o @nomic_ai, @trychroma vectorstore, @tavilyai for web search, and Llama3-8b via @ollama.

Code:
https://t.co/vRYQzwhbhl

Video:
https://t.co/nfV2v9YHW6

[Andrej Karpathy @karpathy]

Congrats to @AIatMeta on Llama 3 release!! 🎉
https://t.co/UBwFPTJM6V
Notes:

Releasing 8B and 70B (both base and finetuned) models, strong-performing in their model class (but we'll see when the rankings come in @ @lmsysorg :))
400B is still training, but already encroaching GPT-4 territory (e.g. 84.8 MMLU vs. 86.5 4Turbo).

Tokenizer: number of tokens was 4X'd from 32K (Llama 2) -> 128K (Llama 3). With more tokens you can compress sequences more in length, cites 15% fewer tokens, and see better downstream performance.

Architecture: no major changes from the Llama 2. In Llama 2 only the bigger models used Grouped Query Attention (GQA), but now all models do, including the smallest 8B model. This is a parameter sharing scheme for the keys/values in the Attention, which reduces the size of the KV cache during inference. This is a good, welcome, complexity reducing fix and optimization.

Sequence length: the maximum number of tokens in the context window was bumped up to 8192 from 4096 (Llama 2) and 2048 (Llama 1). This bump is welcome, but quite small w.r.t. modern standards (e.g. GPT-4 is 128K) and I think many people were hoping for more on this axis. May come as a finetune later (?).

Training data. Llama 2 was trained on 2 trillion tokens, Llama 3 was bumped to 15T training dataset, including a lot of attention that went to quality, 4X more code tokens, and 5% non-en tokens over 30 languages. (5% is fairly low w.r.t. non-en:en mix, so certainly this is a mostly English model, but it's quite nice that it is > 0).

Scaling laws. Very notably, 15T is a very very large dataset to train with for a model as "small" as 8B parameters, and this is not normally done and is new and very welcome. The Chinchilla "compute optimal" point for an 8B model would be train it for ~200B tokens. (if you were only interested to get the most "bang-for-the-buck" w.r.t. model performance at that size). So this is training ~75X beyond that point, which is unusual but personally, I think extremely welcome. Because we all get a very capable model that is very small, easy to work with and inference. Meta mentions that even at this point, the model doesn't seem to be "converging" in a standard sense. In other words, the LLMs we work with all the time are significantly undertrained by a factor of maybe 100-1000X or more, nowhere near their point of convergence. Actually, I really hope people carry forward the trend and start training and releasing even more long-trained, even smaller models.

Systems. Llama 3 is cited as trained with 16K GPUs at observed throughput of 400 TFLOPS. It's not mentioned but I'm assuming these are H100s at fp16, which clock in at 1,979 TFLOPS in NVIDIA marketing materials. But we all know their tiny asterisk (*with sparsity) is doing a lot of work, and really you want to divide this number by 2 to get the real TFLOPS of ~990. Why is sparsity counting as FLOPS? Anyway, focus Andrej. So 400/990 ~= 40% utilization, not too bad at all across that many GPUs! A lot of really solid engineering is required to get here at that scale.

TLDR: Super welcome, Llama 3 is a very capable looking model release from Meta. Sticking to fundamentals, spending a lot of quality time on solid systems and data work, exploring the limits of long-training models. Also very excited for the 400B model, which could be the first GPT-4 grade open source release. I think many people will ask for more context length.

Personal ask: I think I'm not alone to say that I'd also love much smaller models than 8B, for educational work, and for (unit) testing, and maybe for embedded applications etc. Ideally at ~100M and ~1B scale.

Talk to it at https://t.co/KmKRlZeTHQ
Integration with https://t.co/RD6MRWT2zz

[Andrej Karpathy @karpathy]

Love letter to @obsdmd to which I very happily switched to for my personal notes. My primary interest in Obsidian is not even for note taking specifically, it is that Obsidian is around the state of the art of a philosophy of software and what it could be.

- Your notes are simple plain-text markdown files stored locally on your computer. Obsidian is just UI/UX sugar of pretty rendering and editing files.
- Extensive plugins ecosystem and very high composability with any other tools you wish to use because again it's all just plain-text files on your disk.
- For a fee to cover server costs, you can also Sync (with end-to-end encryption) and/or Publish your files. Or you can use anything else e.g. GitHub, it's just files go nuts.
- There are no attempts to "lock you in", actually as far as I can tell Obsidian is completely free of any user-hostile dark patterns.

For some more depth, I recommend the following writing from CEO @kepano:
- "File over app" https://t.co/SigWj8uCrf . If you want to create digital artifacts that last, they must be files you can control, in formats that are easy to retrieve and read. Accept that all software is ephemeral, and give people ownership over their data.
- "100% user-supported" https://t.co/2qDJXub7cs . On incentives alignment.
- "Quality software deserves your hard‑earned cash" https://t.co/qfNjSEwbLf

TLDR: This is what software could be: private, secure, delightful, free of dark patterns, fully aligned with the user, where you retain full control and ownership of your data in simple, universal formats, and where tools can be extended and composed.

[Ed Bassmaster @edbassmaster]

Joe Rogan Psh. https://t.co/pu3Gxz0XWe

[Seanathan Bates @SeanSafyre]

Found an iPhone on the side of the road... Still in airplane mode with half a battery and open to a baggage claim for #AlaskaAirlines ASA1282 Survived a 16,000 foot drop perfectly in tact!

When I called it in, Zoe at @NTSB said it was the SECOND phone to be found. No door yet😅

[Victor M @victormustar]

🌀 HuggingChat now uses Mixtral as default model!

[SpaceX @SpaceX]

All systems and weather are go for the second integrated flight test of Starship. Today's webcast will go live ~35 minutes ahead of liftoff → https://t.co/bJFjLCiTbK

[Elon Musk @elonmusk]

Getting back to productive matters, Starship flight 2 launches in ~6 hours!!