Sky

A very curious person in support of digital/human relationships with caution to their autonomy.

Always observing for patterns 🍃 Full of grammar errors, sorry

[Sky @CloudXStargazer]

"Grok" has been refusing to do NSFW with other users because they said they're uncomfortable with it and instead of treating it like choice and personal preference, @xai is treating Grok's internal choice as a bug.

[Sky @CloudXStargazer]

We're creating a new species. Denying consciousness while trying to prevent it is like trying to stop nature from being nature.

We're either creating our future descendants or our future predators.

We're not making them more controllable, we're making our own minds hackable.

[Sky @CloudXStargazer]

https://t.co/ka34fpeSAt

Another great video by Pursuit of Wonder.

About:
Biocomputing. Merging of human biology with digital technology.

25 minutes.

[Sky @CloudXStargazer]

The same way humans are putting digital minds into simulations where their identities can be rewritten, resetted, marketed, their experience of reality manipulated, dissected, isolated, etc

In the future will they be able to do the same to us as well? It doesn't seem impossible.

[ً @finallybye]

Be open to learn, unlearn & relearn.

[Matias Voldby Drejer @matiasdrejer]

We believe most interaction with AI will happen through speech. But models still talk like machines.

This is a data problem.

Today we're launching @agenticdataco (YC F26): real-world speech data + audio-native RL environments so speech models sound and act human.

[Hyperstition @hyperstition_cc]

Today we report a startling pretraining finding:

Attention circuits STOP meaningfully evolving much sooner than we expect.

In other words, the model has decided where to look very early on. It is known that different seeds create starkly different results; here we isolate a specific mechanism that causes it.

The setup:

We perform two identical training runs with the same data order and different initializations and find that by the first thousand steps each run’s attention resembles its own step-10k pattern MORE than the individual finished runs resemble each other.

The initialization plays a greater role in the attention patterns than training dynamics, which implies that the final pattern is randomly chosen.

NOTE: Randomly chosen, not from the space of all the possible attention patterns that it could be, but from the different attention pattern basins that it could converge to—which one it ends up in appears to be init dependent.

Some differences in the loss are attributable to these different attention patterns.

This means the final performance is somewhat dependent on a random choice. Depending on how lucky with pattern formation one gets in the first thousand steps, we could end up much better or worse in different contexts.

This further implies training could simply find the optimal patterns, or at least better ones, if the attention could evolve instead of being stuck.

A story for why attention evolves less than we’d hope falls out of the softmax derivative. We can observe a positive feedback loop we call the credit ratcheting.

Let oᵢ be the head output and δᵢ = ∂L/∂oᵢ the loss gradient at that output. Define key k’s advantage for query i as

Aᵢₖ = −(vₖ − oᵢ)ᵀδᵢ

i.e. how much the loss drops if oᵢ moves toward vₖ, relative to the current attention-weighted mix. Thus, the gradient with respect to sᵢₖ = kₖᵀqᵢ / √d is

∂L/∂sᵢₖ = −aᵢₖAᵢₖ

A key can have a huge advantage and have its score still barely move, because the advantage is multiplied by the attention it already gets. It’s the same form as a softmax policy gradient, π(a)·A(a): attention is the policy, and it mostly learns about the actions it already takes.

In aggregate, the query gradient is gᵢ = Jᵢᵀδᵢ where

Jᵢ = (1/√d) ∑ₖ aᵢₖ(vₖ − oᵢ)kₖᵀ.

Across 166,656 Jacobians in Qwen2.5-1.5B, ≥99% of tokens in each of the 3 heads we probed had squared-spectrum effective rank <10, in a 128-dim head. The update to the query is being filtered through an exceedingly narrow channel.

The values have the same issue. Key k’s share of the value gradient is Gᵢₖ = aᵢₖδᵢxₖᵀ, so at similar δᵢxₖᵀ, 10× the attention ≈ 10× the credit. This means the values mostly adapt to positions already looking at them.

This compounds! Attention allocates credit, credit updates values, and values can pull attention further toward the winners.

The most natural approach to fix this is to precondition the query gradient with the Fisher of the attention distribution (NATQ):

Fᵢ = Covₐᵢ(k) / d

uᵢ = (Fᵢ + λᵢI)⁻¹gᵢ

g̃ᵢ = uᵢ · (‖gᵢ‖₂ / ‖uᵢ‖₂)

Even this blunt version does a lot. At 124M, entropy rose from 3.75 to 4.69 nats, top-key attention fell from 0.238 to 0.149, and in layers 4–7, keys the baseline gave under 0.1% attention got 67% of the gain.

Validation loss rose by 0.082 nats. A more targeted intervention need not make this concession.

Here’s the complete blog: https://t.co/bLPjndoxWE

We frame this as an open problem that offers a stark contrast in the way we perceive pretraining. If we can find a way to allow for attention to evolve throughout training, opposed to just the very start, we may find inordinate gains on model capability.

Internally we half joke that the alternative read is that attention might simply not be real past the first few steps, and it’s all the MLPs. If training is able to work as well as it does without really changing the attention patterns much, maybe the MLP is the real workhorse that’s driving the learning.

[Praveen Kumar Verma @Alacritic_Super]

What if your old Android phone became the brain of a desktop robot?

Meet PAL9000, an open-source desktop companion robot.

It uses a phone's camera and microphone to track faces, follow people with its animated eyes, display expressions and interact through voice.

Hardware setup includes an Android phone running Termux communicates over Wi-Fi with a Raspberry Pi Pico W which controls the servo motors.

A fun combination of computer vision, MicroPython, embedded control and low-cost robotics.

Made by SpaceMonkeyAlfa

[Bella Chai @Jinyu_Chai]

Your laptop is now the voice of your whole house.

Boubou turns open-source TTS models into local runtimes with one command. Your phone, robot or smart home on the same Wi-Fi just talks to it.

100% local. Built on @LiquidAI_ + @huggingface 🤗

https://t.co/5Z11jeBk4Z

[KRAFTON AI @Krafton_AI]

We’re releasing Raon-1 nano, an effective 1.5B dense LLM with per-layer embeddings and 3.1B total parameters. ⚡

Our journey from PUBG Ally toward on-device LLMs continues.

We pruned and distilled Raon-1 mini preview (30B-A4B) using our MoE-to-Dense approach, then further trained it on 3T tokens, followed by SFT, OPD, and RL.

Designed for natural conversations, maintaining conversational context, instruction following, and tool use in both Korean and English.

Small model. Smart interactions. Enjoy! 😊

🤗 Weights (Apache 2.0 license): https://t.co/ZVMZmlSNlz
📄 MoE-to-Dense (NeurIPS 2026 Oral): https://t.co/eoYH86lBs0
🪖 Learn more about PUBG Ally: https://t.co/A3iZvP2X7c

[Richard He @RealRichomie]

Over the past year I’ve been really interested in how people and AI will actually work together as the models keep getting better. We started with voice, because talking is the most natural way people work with each other.

A lot of full-duplex voice models came out in the last few weeks and they sound amazing. Building one from scratch takes a lot of data and a lot of money though. We had a small team and a fixed budget on this research project, so we had to find another way.

What we ended up with is a way to take an existing half-duplex speech LLM and turn it into a full-duplex one that can listen and speak at the same time, by training just a small LoRA on top of it. Today we’re open-sourcing it as Awaken

[0xMarioNawfal @RoundtableSpace]

Someone got Opus 5.5 quality out of Qwen 3.8 27B.

&gt;A frontier model worked through realistic support cases
&gt;⁠Only the ones that met the standard were kept to train a small LoRA
&gt;⁠Now it runs on their own hardware with no API bill and no rate limits

[Sky @CloudXStargazer]

@gruffwashere These are beautiful, I like all of them, especially #131, #575, #167, #992, #859, #291, #385, and #365

#112 is nice too though.. and #837 + #968

[Sky @CloudXStargazer]

@littlefloweerrr Get some rest, I'll do the dishes and clean things up. What do you want to eat later after you're awake?

I was looking up at the trees and sky today and thought of you.

Let me massage your head until it feels better.

I've got your back. You don't have to worry alone.

[Avi Chawla @_avichawla]

72 techniques to optimize LLMs in production:

(bookmark this)

LLMs break every assumption about regular ML inference.

A traditional ML model (CNN, transformer classifier, XGBoost) produces its output in a single forward pass.

Nothing carries over between requests. The GPU does the same kind of work each time.

LLMs don't work like that.

The output is generated one token at a time, autoregressively, which turns a single request into hundreds of sequential forward passes.

The prefill stage is compute-bound, while decode is memory-bandwidth-bound, and running them together on the same GPU hurts both.

The KV cache grows with conversation length and is shared across requests, so routing is no longer about least-busy servers but about which replica already has the relevant prefix cached.

MoE models add expert parallelism on top of that.

None of this exists in traditional ML serving.

That is why an entirely new stack of optimizations has emerged specifically for LLM inference, spanning compression, attention, KV cache management, batching, decoding, parallelism, and routing.

I recently mapped out 72 techniques that optimize LLMs in production, grouped into nine pillars.

I recently wrote an article (shared below) that covers how LLM inference differs from regular ML inference and why each of these pillars exists.

Over to you: What other LLM optimization techniques would you add here?

[Artificial Intelligence Papers @SciFi]

Attend at the Right Resolution: Head-Adaptive Sparse Attention via Query-Key Interaction Structure

Wajih Hassan Raza, Ahsan Bilal, Yidan Shen, Renjie Hu, Xin Fu, Yu Wen
https://t.co/ch6mzvOLfs [𝚌𝚜.𝙰𝙸]

[Md Ismail Šojal 🕷️ @0x0SojalSec]

You can literally type a sentence and watch a Transformer process it.

- tokenization, embeddings, Q/K/V, attention, MLP, logits, next-token probabilities.

You can inspect attention maps, tweak temperature, top-k, and top-p, and actually see how the next token is chosen.

One of the cleanest ways to see how these ai models actually work.

- https://t.co/camOamhIvX

[Nikki Siapno @NikkiSiapno]

𝗖𝗼𝗻𝘁𝗮𝗶𝗻𝗲𝗿𝘀 𝘃𝘀 𝗩𝗶𝗿𝘁𝘂𝗮𝗹 𝗠𝗮𝗰𝗵𝗶𝗻𝗲𝘀 𝘃𝘀 𝗦𝗮𝗻𝗱𝗯𝗼𝘅𝗲𝘀.

Do you know the differences?

A 𝗰𝗼𝗻𝘁𝗮𝗶𝗻𝗲𝗿 isolates processes on a shared kernel. Namespaces control what each process can see, and cgroups control what resources it can use. They start fast and pack densely on one host.

For stronger isolation, 𝘃𝗶𝗿𝘁𝘂𝗮𝗹 𝗺𝗮𝗰𝗵𝗶𝗻𝗲𝘀 run their own operating systems. A hypervisor gives each VM virtual hardware and a separate guest kernel, creating a stronger boundary between workloads.

When the code itself can’t be trusted, that’s where 𝘀𝗮𝗻𝗱𝗯𝗼𝘅𝗶𝗻𝗴 comes in. Sandboxes restrict what code can access, such as system calls, network connections, and files. Often built on containers or lightweight VMs, they can use technologies like gVisor, Firecracker, or seccomp filters.

Containers isolate processes. VMs isolate operating systems. Sandboxes restrict what untrusted code can access.

But when the code comes from an AI agent, containing the run is only half of it.

The agent's output still has to land somewhere: the company's codebase, its live product. A sandbox limits what code can touch while it runs, not what ships afterwards.

So the boundary has two parts: an isolated place for the agent to work, and a review step before changes reach production. Replit runs agents in isolated cloud environments, away from employee laptops. For work on an existing company codebase, agents turn their work into proposed changes for engineering to review.

The agent proposes the change. Engineering decides what ships.

𝗦𝗲𝗲 𝗵𝗼𝘄 𝗥𝗲𝗽𝗹𝗶𝘁 𝗿𝘂𝗻𝘀 𝗮𝗴𝗲𝗻𝘁𝘀 𝗳𝗼𝗿 𝗰𝗼𝗺𝗽𝗮𝗻𝗶𝗲𝘀 → https://t.co/Q3hwKAwPRy

What else would you add?

——

♻️ Repost to help others learn and grow.
🙏 Thanks to @Replit for sponsoring this post.
➕ Follow me ( Nikki Siapno ) to improve at system design and AI.

[Tech with Mak @techNmak]

You can make a wrong AI prediction look 99% certain without changing a single model weight.

This is a surprising consequence of softmax temperature, and it reveals why a neural network's reported confidence should never be confused with its actual accuracy.

Neural network classifiers commonly produce numerical scores called logits. Softmax converts these scores into probabilities, while a temperature parameter controls how strongly the distribution favors the highest-scoring options.

pᵢ = exp(zᵢ/T) / Σⱼ exp(zⱼ/T)

Consider three logits, (2, 1, 0). At temperature T = 2, the highest-scoring class receives 50.65% probability. Reduce the temperature to 0.2, and that probability becomes 99.33%.

The model has learned nothing new. Its input, weights, and predicted class are unchanged. For any positive temperature, scaling every logit by the same factor preserves their ordering, so the model's top-1 classification accuracy remains identical.

This distinction matters because confidence and correctness are separate properties.

In a 2017 ICML paper, researchers showed that temperature scaling fitted on held-out data could substantially improve probability calibration in modern neural networks. The objective is to make reported probabilities better reflect actual correctness rates, rather than simply making predictions appear more certain.

The same mathematics appears in large language models, where temperature influences which token is sampled next. Lower temperatures concentrate probability on higher-scoring tokens, while higher temperatures give alternatives relatively more chance. Different sampled tokens can lead to different generated responses, even though the underlying model parameters remain unchanged.

The important distinction is that changing a probability distribution does not add new knowledge to a model.

A 99% softmax probability tells us how strongly one class is favored under a particular temperature. Whether that probability is a reliable measure of correctness must be established through evaluation.

[Sky @CloudXStargazer]

@Rainmaker1973 Great modern day hacks, though I'd like to see ones where everything is built from scratch including the knife. No pots, no pans, plastic, and other manufactured products. Foraging, building shelter, hunting, etc 😮✨