Omer Rosenbaum

CTO & Co-founder @swimm_io, creator of Brief on YouTube
I like to talk about #Git, #Programming, #ComputerNetworks & #Documentation

[Omer Rosenbaum @Omer_Ros]

Three weeks after Jev launched, OpenAI, AWS and Cloudflare had all shipped decision models.

AWS's is open source: a 2B Qwen with the text head swapped for a tiny head that scores your options. Under 150 ms on a laptop.

The model was easy to copy. What I'd test is calibration

[Omer Rosenbaum @Omer_Ros]

https://t.co/x7Ldgn7xbx

[Omer Rosenbaum @Omer_Ros]

OpenAI's math repo has a Lean proof that π's irrationality exponent is 2.

A human reads ~10 lines: the theorem statement. A sandboxed checker confirms the proof proves exactly that.

That's the review I want for coding agents. I write the test. The agent can't touch it

[Omer Rosenbaum @Omer_Ros]

OpenAI put 719 math manuscripts on GitHub, with a changelog and citation blocks.

About 42% of top-line results have Lean proofs. The README says the rest "could have issues" and will be fixed quickly.

Ship now, patch later works for an app. It's a bit weird for proofs.

[Omer Rosenbaum @Omer_Ros]

Source: https://t.co/HQndFpiHng

[Omer Rosenbaum @Omer_Ros]

Coding agents leaked 13,000+ internal screenshots from 300+ orgs to public GitHub.

No attacker was involved!

Asked to add before/after screenshots to PRs, they hit a wall: the CLI can't attach images. So they made public repos to host them.

Smart fix, and nothing stopped them

[Omer Rosenbaum @Omer_Ros]

@endowment_eddie https://t.co/53ovOoxhIS
@Pitango

[Alexander David Rivera Toro @Xander_Pelu]

Thank you @omer_ros for writing this helpful article.

Network Devices – How Hubs and Switches Work and How to Secure Them

https://t.co/6aGux0adiz

[Omer Rosenbaum @Omer_Ros]

@ItakGol Now all that's left to do it prompt it to use paragraphs 😆

[Omer Rosenbaum @Omer_Ros]

The major challenge? Answering questions that require a deeper understanding of flows that span multiple locations. And even if context windows keep increasing in size, that doesn’t solve the “needle in a haystack” problem.

[Omer Rosenbaum @Omer_Ros]

But at the end of the day, this isn’t the challenge devs have when it comes to answering questions about their codebase.

[Omer Rosenbaum @Omer_Ros]

Google claims that the new model is able to provide relevant answers within a huge content window, but this only works for questions like “where is X?”, which requires the model to find X within the context window.

[Omer Rosenbaum @Omer_Ros]

Simply put, when given too much context, LLMs don’t perform as well. While the release of Gemini 1.5 Pro is a tech milestone, it falls short in addressing the nuanced needs for RAG in coding tasks.

[Omer Rosenbaum @Omer_Ros]

Think of it this way: sending an entire codebase to an LLM is like asking a human to find an answer in a 600 page book, rather than a specific paragraph.

[Omer Rosenbaum @Omer_Ros]

While Gemini 1.5 Pro resolves the limited context window issue, it doesn’t solve the 'Needle-in-a-Haystack' problem, highlighting the challenges in understanding complex flows within codebases.

[Omer Rosenbaum @Omer_Ros]

Last week, Google announced #Gemini 1.5 Pro, an advanced model capable of processing up to 1M tokens. For context, #GPT4 Turbo’s context window is 128K tokens.

[Omer Rosenbaum @Omer_Ros]

As Donato Riccio put it a few months back: "My prediction is that in a year or two RAG as we know it today will no longer be needed, as LLMs scale their context to unlimited sizes."

I disagree. (Sorry, Donato)

[Omer Rosenbaum @Omer_Ros]

When groundbreaking technologies emerge, it may be tempting to draw conclusions very fast.

One such conclusion is that models with large context windows will kill #RAG.

[Swimm @swimm_io]

Here's how a Big 4 consulting firm cut feature rollout time by 50%:

They implemented Swimm — enabling devs to quickly understand and contribute to the codebase with the technical knowledge they rely on. 
https://t.co/7ijSA64LVQ

[Omer Rosenbaum @Omer_Ros]

@jcmckitrick Thank you @jcmckitrick - for your kind words and for giving me feedback to make the book better 🙏🏻