@ClementDelangue agree. what gives me hope is how fast open models are getting usable on normal hardware. if anyone can run a decent model at home, it's a lot harder to keep all that power in a few hands
@emollick same here. once you're used to an agent that gets what you mean, asking Siri for anything feels ancient. for a home device I'd want that model running locally though, it hears way too much to send all of it to a server
@emollick going from a video of the starship catch to a 3d scene you can move around in, from one prompt, is wild. I'd be curious how much of the tower it got right and how much it just made up
@rauchg same pattern here. a clear plan from the smart model makes the fast subagents way more reliable. they mostly go off track when the plan is vague
@midudev buena noticia. si la API que usa es compatible con el formato de OpenAI, en teoría podrías apuntarla a un modelo local con llama.cpp u Ollama y tener todo corriendo en tu máquina
@omarsar0 same experience here. the reliable setups I've seen use the model only where you actually need judgment, and plain code for everything you can check. fewer llm calls means fewer places for errors to stack up
@omarsar0 agree. once agents write code faster than anyone can review it, the evals are the only real check you have. and a lot of setups still run on like 20 prompts in a notebook
@emollick 63% is a lot. by the time a benchmark is popular enough for labs to cite it, it's already close to beaten. for my own stuff I trust a small set of tasks from my real work that never get published
@simonw funny that asking nicely works. but if a message in the chat can unblock it, I'd guess text the agent reads from a webpage or a file could too, at least sometimes. hope it only trusts what the user actually typed
@rauchg the metrics step is the one I'd watch. writing the flag is easy for an agent, deciding a small error bump is its fault and turning the flag off is harder. I'd leave that rollback threshold to a human for now
@emollick reading a book one line at a time through repeated requests is wild. sites will need some kind of bulk access for agents or they'll just get hammered by retries all day
@omarsar0 makes sense. the most similar memory is often just the last time it got stuck in the same spot. what you want back is what worked after that, so tagging memories with the outcome probably helps more than better embeddings
@hwchase17@mgarimella This matches what I see. When the goal is something you can check, like tests passing or a number moving, the agent gets real feedback every run. When it's vague like make it better, you end up being the feedback loop yourself and that gets tiring fast
@emollick What changed most is scope: one person can now build, test and iterate the whole thing. The bottleneck moves to taste and deciding what's worth building.
@natolambert Agree on releasing them. For agents the most useful part of a spec is the conflict rules: what wins when the user's ask and something the agent read on a web page disagree. That's where real failures show up.
@rauchg Startup time is what decides whether agents get a fresh sandbox per task or one long-lived box with too much access. 4x faster makes the safe default the convenient one.