AxS

Building The verification layer
for AI agent loops

[AxS @axsegni]

@GergelyOrosz Code review is just an opinion on whether a diff looks right, It was already struggling to scale with human code, and AI completely broke it.
The fix is moving from opinion-based review to deterministic, evidence-based verification. Building https://t.co/mF6zsxT2N9 to solve it.

[AxS @axsegni]

@GergelyOrosz Code review is just an opinion on whether a diff looks right, It was already struggling to scale with human code, and AI completely broke it.
The fix is moving from opinion-based review to deterministic, evidence-based verification. Building https://t.co/mF6zsxT2N9 to solve it.

[AxS @axsegni]

The takeaway is simple: AI doesn't eliminate the need for engineering maturity. It amplifies it.

The teams that will extract real leverage from AI won't be the ones with the cleverest prompts. They'll be the ones with a genuine software engineering culture.

[AxS @axsegni]

Today, I ran an experiment with a newly purchased @AnthropicAI @claudeai team account on one of my customer projects.

At the end of the session, I asked the agent to reflect on the architecture and on the experience of adding a new feature to a service. Its response was telling:

[AxS @axsegni]

These practices create systems that are easy to reason about, not just for humans, but for AI tools!
AI coding agents don't replace these practices. When these foundations are weak, agents stall, hallucinate intent, or require constant supervision.

[AxS @axsegni]

@livingdevops As mentioned by many comments, LXC was the first, most complete implementation of Linux container manager. It was implemented in 2008 using cgroups and Linux namespaces, and it works on a single Linux withoit requiring any patches. Docker in 2013 used LxC initially ...

[AxS @axsegni]

@iximiuz The time I used to spend writing my code, I now spend fixing and debugging AI-generated code. No clear productivity improvements so far. It’s useful for non-software developers to create code that somehow works, but it’s not very useful for experienced engineers

[AxS @axsegni]

@mipsytipsy I was thinking about the OTEL spanmetrics (errors, latency, throughput) connector that is a good implementation of this. Metrics deviated from spans are a good lighting of the overall application performance. Starting with them can help building effective SLI/SLOs for your app.

[AxS @axsegni]

@iximiuz ServerFounry is a strong and fitting choice

[AxS @axsegni]

This.

[Bryan Liles @bryanl]

Being a senior IC is getting comfortable with people telling you aren’t correct, ignoring you, straight up disagreeing. Being a successful senior IC is learning that influence is your biggest weapon and letting your good ideas be someone else’s best idea.

[Valentin De Matos @VDMatos]

Google just released Gemma 2 😮

* OSS lightweight LLMs (9B & 27B params) for various text tasks.
* Pre-trained & instruction-tuned variants available.
* Strong benchmark performance.
* Designed for responsible AI dev, with ethics/safety evaluations.
* Runs on "consumer"(ish) hardware

[ahmet alp balkan @ahmetb]

Published our first technical blog about using Kubernetes at LinkedIn!

This article is about how we moved our app deployment CRD from one API Group to another and changed its shape without any downtime. https://t.co/5LIpDSysxm

[AxS @axsegni]

I used to ask about this scenario when interviewing Sr Engineers about Kubernetes.
This is where client-side load balancing makes sense e.g. Service Mesh tools.

[Daniele Polencic — @[email protected] @danielepolencic]

Kubernetes doesn't load balance long-lived connections, and some pods might receive more requests than others. This is particularly common if you use databases or gPRC in your cluster.

Let me explain.

Tools such as kube-proxy load balance connections when they are first created. But once the connection is established, Kubernetes doesn't touch it anymore.

And this creates some interesting scenarios. Imagine having 4 pods all connected to the same database.
When the database is overwhelmed, and you scale to two instances, does Kubernetes split the connections across the replicas?
No, the same 4 pods are still connected to a single database replica!

So what can you do?
You can handle load balancing on the client side or use a proxy.

I wrote the above in a lot more detail in this article: https://t.co/bu8iL4Zl7D

[Andrej Karpathy @karpathy]

📽️ New 4 hour (lol) video lecture on YouTube:
"Let’s reproduce GPT-2 (124M)"
https://t.co/QTUdu8b0qh

The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model:
- first we build the GPT-2 network
- then we optimize it to train very fast
- then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers
- then we bring up model evaluation, and
- then cross our fingers and go to sleep.
In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model. This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

Github. The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step.
https://t.co/BOzkxQ8at2

Chapters.
On a high level Section 1 is building up the network, a lot of this might be review. Section 2 is making the training fast. Section 3 is setting up the run. Section 4 is the results. In more detail:
00:00:00 intro: Let’s reproduce GPT-2 (124M)
00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint
00:13:47 SECTION 1: implementing the GPT-2 nn.Module
00:28:08 loading the huggingface/GPT-2 parameters
00:31:00 implementing the forward pass to get logits
00:33:31 sampling init, prefix tokens, tokenization
00:37:02 sampling loop
00:41:47 sample, auto-detect the device
00:45:50 let’s train: data batches (B,T) → logits (B,T,C)
00:52:53 cross entropy loss
00:56:42 optimization loop: overfit a single batch
01:02:00 data loader lite
01:06:14 parameter sharing wte and lm_head
01:13:47 model initialization: std 0.02, residual init
01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms
01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms
01:39:38 float16, gradient scalers, bfloat16, 300ms
01:48:15 torch.compile, Python overhead, kernel fusion, 130ms
02:00:18 flash attention, 96ms
02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms
02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping
02:21:06 learning rate scheduler: warmup + cosine decay
02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms
02:34:09 gradient accumulation
02:46:52 distributed data parallel (DDP)
03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU)
03:23:10 validation data split, validation loss, sampling revive
03:28:23 evaluation: HellaSwag, starting the run
03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro
03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA
03:59:39 summary, phew, build-nanogpt github repo

[Chris_Mellor @Chris_Mellor]

Good article on state of object storage - https://t.co/xJHeloTstW