Junseon

AI co-scientist for researchers. Starting from search that understands a paper's position, not just its topic. Founder, Pluto Labs (Scinapse).

[Junseon @junseon_89]

@AhmedMa34437965 @JAldrichPL Agree, but using an LLM doesn't mean the author delivers their findings poorly. As you pointed out, the delivery is the author's job. So I think we should not judge whether they used an LLM. We should judge how well the work is delivered.

[Junseon @junseon_89]

@plazined @chnli347218 @NeurIPSConf @konstantina_pll From what you describe this seems too harsh. In a D&B track the main contribution is the dataset, not the citation format.

[Junseon @junseon_89]

@VladimirGurevi2 @_reachsumit Agree. And k1 and b, the only parts anyone actually tunes, are hard-coded as probabilities in the derivation.

[Junseon @junseon_89]

@carlespedret Worth naming who exactly. Reviewers work free and editors are not paid per paper. The only revenue that moves with volume sits with the party that also decides what gets accepted. And the fee is the same whether the paper is good or bad.

[Junseon @junseon_89]

@kseniam0s @amilabs they build world model

[Junseon @junseon_89]

I think the trust part is downstream. What people imagine AI is already doing in science is ahead of what it actually does.

Almost everything I have seen work is in fields where the whole loop already runs on a machine. Math with a proof assistant. Computational physics. Computational chemistry. Elsewhere the help is real but narrow. Parameter sweeps and repeated runs. The slow part, not the hard part.

I ran bench experiments before this. What I remember as research was using a protocol in a way it was not written for, and building the measurement setup from instruments that were never meant to go together. I have not seen a model do that yet.

But that is about where the loop is closed, not about what machines are. It moves when someone builds a way for a model to run a real measurement and read the result back. That is a build problem, not a trust problem.

[Simon Maechling @simonmaechling]

We're living through a strange moment in history.

People distrust scientists because they're human.
Then trust AI because it's not.

One spent decades learning a subject.
The other draws pretty things in 3D.

Science didn't fail.
Our ability to recognize expertise did.

[Junseon @junseon_89]

OpenAI put out numbers on how much of their research agents are doing.

The chart people will quote is the token breakdown. The one worth reading is the intervention curve.

Success stays high as tasks get longer. Doing it unattended does not. The longer the task, the more often someone had to step in partway, and outright failure stays low throughout. So what grows with the horizon is not error. It is supervision.

I think research is hard in three ways.
- The equipment or the experiment is expensive.
- Nobody knows what right looks like yet.
- Or you know exactly what to do and it only gets done by sitting there long enough.

Agents took the third one. That is a large part of research and it is worth a lot. It is not the other two.

In software the third kind is pure chair time, so it can be handed over. At a bench the sitting is physical. You cannot do it from a chair, and nothing lets a model do it for you yet.

Until that changes, this is a software result.

https://t.co/zBcq6uzjfa

[Junseon @junseon_89]

An environment has to return a score. That is the scarce part, not the environment.

Code went first because tests were free. Math is going now that formalization keeps up. Binder design is halfway, since a plate scores a thousand at once but nothing scores which target to pick. Most of science has no readout at all.

[Junseon @junseon_89]

Agree with the diagnosis. The part I keep getting stuck on is what scores the new approach.

Simulating interventions inside a system model moves the verification problem up a level rather than removing it. The simulation still has to be checked against something, and now that check has to score a whole system instead of one binding event.

And the check we have is a trial with one pre-registered endpoint. A system level intervention can be right and still have no way to be counted. So the target paradigm may be downstream of what can be scored, not of how we think.

[Junseon @junseon_89]

The loop seems to close where the readout already exists. In the Anthropic binder work the scores ranked designs well inside each target but did not separate which targets would fail. And an author of a recent self-driving lab paper told me their assay was developed and optimized before the robot ever ran it, so it worked as a fixed program.

So the scarce thing may be a defined readout rather than throughput, and developing one has no score of its own.

[Junseon @junseon_89]

This lands in a conversation happening in a different field right now. Several groups building AI research agents this year have argued that failures and null results should be preserved and used as training data, because that is where the boundary conditions sit.

Your result is the part they skip over. The data is missing not because nobody wrote it down, but because the penalty is already there at the reading end.

[Junseon @junseon_89]

The detail I liked most is that the literature on Chowla sets was sparse, so the agent could not reach the solution by retrieving and interpolating. That is a good control to have built into the setup.

It makes me wonder if sparsity could work the other way too, as a way to choose which problems to point an agent at. Congratulations to the team on all three.

[Junseon @junseon_89]

The hard part is probably not willingness. The only durable record is written afterward, and by then you know which branch worked, so the failures stop looking like decisions and start looking like mistakes. Capturing them means writing while it is still uncertain, and nothing in the normal workflow does that.

[Junseon @junseon_89]

@julian_englert Congratulations. Orchestrating protocols across hundreds of instruments and still getting comparable numbers out is the part people underestimate from outside.

[Junseon @junseon_89]

The methodological divergence you report is the one I would want a number on. Papers and code have probably always drifted apart, but it is hard to measure, because you rarely have both artifacts from the same run and the ability to compare them.

Here you did. And it is the one failure that no authorship test catches. A detector reads the paper. The divergence only shows up when you diff the paper against what actually ran.

[Junseon @junseon_89]

Agree, and I think the timing is the part that hurts. Review is what separates a checked claim from an unchecked one, but it runs on a scale of months while a claim closes a problem the day it is posted. By the time anyone looks, the problem already reads as taken. That gap used to be covered by cost.

Producing something that looked like a solution was expensive, so a claim was decent evidence that a real result was behind it. That is the part that broke.

[Junseon @junseon_89]

@the_codewala Typescript

[Junseon @junseon_89]

That distinction is the one I would keep. Optimizing variants has a readout that tells you better or worse at every step. Developing the assay does not, because the thing that would score it is what you are still building.

In my case choosing infrared thermometry meant the substrate had to be silicon, and I committed to that before any data existed to justify it. Time and reagents were not what made it hard for me.

So the delegation idea is interesting. The unit doing assay development is working without the signal every other unit gets.

[Junseon @junseon_89]

The paper has a sharper version of this. The in silico scores ranked designs well inside each target, but across targets the ones that gave nothing scored about the same as the ones that worked. So for choosing which target to attempt, computation did not narrow anything down. You only found out by making them.