I think it's better at instruction following, and I like it being cheaper. Can you make the guardrails for the cybersecurity better? Now even I am working on open source projects with no cybersecurity features I got flagged frequently and I don't even know what's the words in my prompt or in its thinking tokens triggered the cybersecurity flag, but man it's annoying.
It seems like the dominant pattern in autonomous research is recombining primitive ideas from humans. Despite it sounds trivial, it just works.
LLMs and humans are just such different forms of intelligence. LLMs have incredibly broad knowledge coverage, so sometimes they can simply combine two bodies of literature that no single human happens to know at the same time. Their speed of execution is also insanely fast. At the same time, coming up with fundamentally new primitive ideas still seems to be a uniquely human capability, at least for now.
I am re-learning playing legato, feel Chinese teachers are really stricter than the western ones, and I think the reason is Chinese teachers need to handle thoudsands of students (or already handled) given the population and market, thus they are impatient to answer same questions again and again. The curriculum after iteration and iteration seems better though, although sometimes I feel the method/solution is not personalized enough for each individual and I guess it can solve 80% porblem for all kinds but the rest 20% of people who are structurally different might need more tailored solutions and need to be more proactive to ask questions and get more guidances.
The low hanging fruit on harness engineering is insane.
Ontop of that, there's an interesting area of science/engineering on how to best study the effect of harnesses, from post training through eval/inference. Will be a fairly impactful area (in cost savings per performance)