Reincarnate Ape

[Reincarnate Ape @ReincarnateApe]

@rankdim This can basically be extended to: if [xyz] can be verified semi-cheaply (in compute & time), then LLMs will probably get really good at it.

Once you figure out how to limit reward hacking, you can get LLM generated kernels that are better than the best human ones

[Reincarnate Ape @ReincarnateApe]

@marvoddle digital is 2000s video game

[Reincarnate Ape @ReincarnateApe]

@ethanparmet @jm0ney666 Yeah idk dude I think it’s fucked up also. I think most clothing doesn’t fit people with shorter torsos. Body proportions really matter.

[Reincarnate Ape @ReincarnateApe]

@adibvafa The memory footprint would be so large.

[Reincarnate Ape @ReincarnateApe]

@FleischmanMena @fleetingbytes I’ve added extra prose such as “minimize the cyclomatic complexity of your design” or “the implementation should focus on building the mvp”. Usually these agents tend to over-engineer, and the “minimally” viable product ends up being more than minimally viable.

[Reincarnate Ape @ReincarnateApe]

This is true. Nvidia will become an “ai factory” company. Want hardware to run models -> NV; want open source intelligence-> NV; want applied ai -> NV.

[Delia Lazarescu @tech__unicorn]

my theory is Jensen is playing the Elon game

acquisitions have started going into that leather jacket like candy

- $12.9B for Hugging Face
- $30B stake in Perplexity
- $6B licensing deal + stake in Poolside
- backing OpenAI, xAI, Safe Superintelligence

sell the chips, fund the buyers, own the shelf the models sit on

give it 2 years and Nvidia ia not a chip company anymore, it's a whole vertical stack

[Reincarnate Ape @ReincarnateApe]

option 2 is gonna prevail.

[Vítor Balocco @vitorbal]

Opinions on agentic engineering appear to be converging into two camps:

"You should put a lot of planning up front, all the way down to program design, then let the agents cook", aka the @humanlayer_dev way.

vs

"Do a shit ton of investment in architecture/linters/LLM checks/automatic verification upfront, and then let agents rip. Garbage collect periodically", aka the @poteto way.

I currently land somewhere in-between:
- A bunch of research and planning up front, but skip program design unless I identify that a net-new abstraction will be needed, or I know I'm working on high-stakes/core code.
- For "every day" stuff, I lean heavily on guardrails.
- Frequent sampling of PRs that have been landing on main to spot problematic code that should've been caught by a guardrail. Add more guardrails to catch it, burn down the offenders.
- Frequent sampling of hot spots in the codebase, to detect when human intervention is needed, e.g. an abstraction has grown out of control. Maybe I oughta build a heatmap visualization for this?
- I also love @poteto 's take on the tier of guardrails (opinionated architecture > static analysis > tests > LLM checks)

[Reincarnate Ape @ReincarnateApe]

None of these are frontier anymore. Maybe in 2024. Real LLM Inference Engineering scope is now focused on much harder, systems problems

[Amit Shekhar @amitiitbhu]

LLM Inference Engineering - Problem and Solution

Problem: LLMs are slow
Solution: KV Cache - Avoid recomputing previous tokens.

Problem: KV Cache consumes huge memory
Solution: PagedAttention - Manage KV memory efficiently.

Problem: GPU is underutilized
Solution: Continuous Batching - Dynamically add/remove requests.

Problem: Token generation is sequential
Solution: Speculative Decoding - Generate and verify multiple tokens together.

Problem: Attention requires large memory usage and expensive memory movement
Solution: FlashAttention - Reduce memory usage and memory movement during attention.

Problem: KV Cache is still large
Solution: MQA / GQA - Share K/V across multiple query heads.

Problem: Many optimizations need to work together
Solution: vLLM - Build an efficient inference engine.

Problem: Applications repeatedly process the same prefixes
Solution: RadixAttention / Prefix Caching - Reuse previously computed work.

Problem: Models are too large
Solution: Quantization - Reduce model memory and computation.

Problem: Large models are expensive
Solution: Distillation / SLMs / MoE - Make inference itself cheaper.

Keep Learning, Keep Sharing, and Keep Growing.

[Reincarnate Ape @ReincarnateApe]

@arpansplaining Nostalgia-bait. Web3 was one of the cringiest and most annoying eras of tech.

[Reincarnate Ape @ReincarnateApe]

@arpansplaining I don’t disagree that hard-tech people have inflated egos, but cmon most of these vertical ai startups boil down to “clod write md file for future clod”

[Reincarnate Ape @ReincarnateApe]

@lemon_logger3 @sethbannon Fs bro turn of that VPN and let’s see

[Reincarnate Ape @ReincarnateApe]

grass is green and water is blue

[wispem-wantex @wispem_wantex]

Code is bad. It has negative marginal value. All good software engineers know this-- if you don't, you're a junior. Adding more code makes a project worse.

Code only justifies itself by improving the product. If it doesn't, the extra code is just harmful.

[Reincarnate Ape @ReincarnateApe]

I wish, but complexity of tasks will just scale so that we still end up having to wait 5mins.

[shako @shakoistsLog]

work is gonna change a lot again when tokens/second regularly hits 1000. no more waiting around.

[Reincarnate Ape @ReincarnateApe]

@garrytan @jkbdco @AFergusonFTC Garry, respectfully, the people you are siding with see you as a chink who is next on their chopping block. They sided with Indians in 2024 to get the Indian vote for Trump. Now, the Indians are getting chopped. You’re next, and if you think you’re “different”. I got news for you

[Reincarnate Ape @ReincarnateApe]

this is already the case at frontier companies / labs. One issue worth tackling is: better agent observability. It’s frustrating to audit a long-horizon agent. My current soln is to point another agent at it and have it answer questions. Anyone know a better way?

[zhyncs @zhyncs42]

Over the next 6 months, agentic workloads will increasingly evolve toward long-horizon tasks — agents running for hours, days, or even weeks.

This will fundamentally change inference workloads as well. The optimization targets for LLM inference will shift beyond today’s low latency and cost metrics, and we’ll likely see a new class of inference optimizations emerge around long-running agent workloads.

[Reincarnate Ape @ReincarnateApe]

@allgarbled Agreed. I hate agentic engineering flow. Maybe I’m just bad, but I have no token limit access to SOTA models and they still can’t one shot items I give it. They generate too much code to process in one sitting and it’s just shitty.

[Reincarnate Ape @ReincarnateApe]

@47fucb4r8c69323 What do you do? I’ve been following you for a bit and I’m assuming it’s finance / trading related, but would u be able to share more specifically?

[Reincarnate Ape @ReincarnateApe]

+1. I would just listen during lecture, and then afterwards try to write a “technical” blog style post about the content covered. This method led to me identifying gaps in my knowledge much faster and led to better questions.

[hunter @HunterALanier]

Ive come to the conclusion that writing things down during lectures is a waste of time that is much better spent listening attentively

[Reincarnate Ape @ReincarnateApe]

Great take. Something that is increasingly evident: a strong link between “taste” and scarcity. You see this across travel destinations, art they enjoy, and other aspects of their lives. Fashion is the best example: it’s cool until the fugly dorks start wearing it.

[Jasper Lu @lu__jasper]

As someone who's been training models for design for the past two years...

Whether you can or can't rl taste is a silly binary.

The fact is that taste is highly individual and there's no golden ideal here. And because any foundation model's outputs will proliferate quickly, attempting to imbue specific taste at this layer is a dead end.

What you CAN do, however, is teach a model the general principles and rules of a creative domain -- the equivalent of sending a student to design school. Most open and closed frontier models already passed this bar for design early 2026.

As this work shows, you can also, with much effort, imbue a single person's or organization's tastes into a model.

Take Claude as an example. We rag on its design tics today as slop. But if the same designs had come from a boutique design firm at low volume, we would likely call them tasteful. By nature, once a design aesthetic is everywhere, it stops being tasteful.

The future of ai for "taste" is many n-of-1 models.