Andrei Gakh

"The only reliable way to build a better future is to invent it. Because the future is the only thing in this world that can be changed at will."

[Andrei Gakh @AndreiGakh79]

@paulbloomatyale A hybrid human-AI approach for peer review is the only viable option in the age of AI. Too bad it is currently taboo.
https://t.co/dbVCgNEwX0

[Andrei Gakh @AndreiGakh79]

Upon submitting our paper to arXiv, I learned they process over 500 new submissions a day. Given the sheer volume, human peer review is no longer feasible. AI-assisted review is the only viable path forward, and I am certain it will be implemented soon.
https://t.co/hREKQrHbbu

[Andrei Gakh @AndreiGakh79]

@akoustov Agreed - judge the quality of the results, not who wrote them.

[Andrei Gakh @AndreiGakh79]

@yzeng58 Yes, a hybrid human-AI workflow for reviews works best.
https://t.co/P3CAJaGPSt

[Andrei Gakh @AndreiGakh79]

A summary of our experience with LLM, LWM, and SWM AI systems is now out in @cenmag, featuring an optimistic view on the future of the hybrid human-AI model in chemical science.
https://t.co/XLIWV7SHDu

[Andrei Gakh @AndreiGakh79]

"Once a chemist, always a chemist." Sometimes good ideas are hard to formulate, but this one is right to the point.

[ACS CSW @ACS_CSW]

Thank you to @AmerChemSociety President @EveryWhereChem for having dinner with CSW last week. We had a blast and learned a lot about ACS's 150th Anniversary plans!

[Andrei Gakh @AndreiGakh79]

AI-assisted reviews are not just a desirable option. In the age of a hybrid human-AI research workflow, this is the ONLY viable option. I would even suggest that authors submit three AI reviews, with the prompts and the model IDs, along with the paper. The decision could be made in 24 hours.
https://t.co/XTTNpVwDwD

[Andrei Gakh @AndreiGakh79]

A summary of our experience with LLM, LWM, and SWM AI systems is now out in @cenmag, featuring an optimistic view on the future of the hybrid human-AI model in chemical science.
https://t.co/XLIWV7SHDu

[Andrei Gakh @AndreiGakh79]

Mastering a hybrid human-AI workflow for technical writing is no trivial task. Using a dual-LLM setup where two models cross-review and correct each other has proved beneficial. Also, frontier models are trained to discover new science, not draft manuscripts. Try @GeminiApp, @grok, or other second-tier LLMs.
https://t.co/XTTNpVwDwD

[Andrei Gakh @AndreiGakh79]

A summary of our experience with LLM, LWM, and SWM AI systems is now out in @cenmag, featuring an optimistic view on the future of the hybrid human-AI model in chemical science.
https://t.co/XLIWV7SHDu

[Andrei Gakh @AndreiGakh79]

Wikipedia now deploys significant resources to remove articles suspected of containing AI content, with the total now approaching 1,000 submissions per month. While perfectly understandable from the perspective of their current business model (selling content to AI companies for training), these resources could be better used for creative activities. Besides, I personally have found many flagged articles no worse than other Wikipedia content.

Fighting AI nowadays is like fighting gravity - a fruitless and ultimately stupid exercise. However, using it to your advantage is a smart move.

https://t.co/mGvYDqsJpJ

[Andrei Gakh @AndreiGakh79]

@3scorciav @Michael_J_Black @CSProfKGD I'm not quite sure that traditional economic laws will work here.
https://t.co/O8P25E00pk

[Andrei Gakh @AndreiGakh79]

I hope the modern hybrid human-AI scientific enterprise will eventually achieve a Nash equilibrium (if that concept is applicable to such systems), but only time will tell where that point lies.
https://t.co/g9FaONc2xm

[Andrei Gakh @AndreiGakh79]

@Michael_J_Black @CSProfKGD The sheer volume of output from the modern hybrid human-AI scientific enterprise is of greater concern than the input. There is no chance a human researcher can digest this avalanche of new advances alone, but adding an AI filtering layer creates a narrow, tunnel-visioned view.

[Andrei Gakh @AndreiGakh79]

I hope the modern hybrid human-AI scientific enterprise will eventually achieve a Nash equilibrium (if that concept is applicable to such systems), but only time will tell where that point lies.
https://t.co/g9FaONc2xm

[Jonathan Schlosser @jonathanschloss]

Seeing numbers of ~50k submissions for @iclr_conf. That's insane!

Yes, probably a lot of AI papers. And yes, it'll probably need AI review at that scale. But @NeurIPSConf just ran into this exact wall, and the results were... interesting.

Based on some blogs that have come out, NeurIPS screened all 969 submissions through an AI detector. Out of those, 42.7% of submissions initially scored in the 90-100% AI-generated range, and 273 papers hit a full 100%. The chairs re-ran it with a narrower text window and that number dropped to 12.7%. Same papers. Same tool. One settings change, and the flag rate fell by 70%.

Resultantly, 178 papers were desk-rejected with no appeal and 123 were told to produce version histories or also be desk-rejected. Then a rejected author ran the track chairs' own recent papers through the same detector and got back ranges from 24% to 69% of AI-generated content.

That's with 969 submissions.

Now let's consider ICLR. There are reports that people's submission numbers are getting real close to 50k. The one that I saw plainly stated was 47647.

Compared to last year, there were 19,525 valid submissions to ICLR, with 779 desk rejected and 5,042 withdrawn. This left 13,763 papers that needed a decision.

To do this, ICLR organized 76,139 reviews from 18,054 reviewers. They also ran their hallucinated-reference checks through at least three humans per flagged paper before any desk rejection, putting in a check to any AI review processes.

Now what is this going to look like with a potential 2.5x jump in volume?

At ~50k submissions, ICLR is definitely going to have to use some AI review processes, but can the human-review layer scale as well, or is it something that may end up getting dropped?

This is speculation at this point, and is based solely on the abstract submissions so far, but the final paper deadline is Sept 25th, and it will sure be interesting how this all plays out.

If you want some advice though, maybe save your drafts, keep your version history, and be specific about any AI use in your attestation. That could come in handy if you get a rejection...

Anybody have thoughts here? Gonna be an impressive feat nonetheless. 🤔

NeurIPS Info: https://t.co/heCi0dAZvU
ICLR Info: https://t.co/eXY5mwUhUA
ICLR Conference Page: https://t.co/ajHoQG6dTY
Graph: https://t.co/Q1eVrsIIJE

[Andrei Gakh @AndreiGakh79]

@PTenigma This is a good idea. AI-assisted reviews are of much higher quality and can be completed in minutes. This NeurIPS taboo policy should be rescinded.
https://t.co/dbVCgNEwX0

[Andrei Gakh @AndreiGakh79]

Upon submitting our paper to arXiv, I learned they process over 500 new submissions a day. Given the sheer volume, human peer review is no longer feasible. AI-assisted review is the only viable path forward, and I am certain it will be implemented soon.
https://t.co/hREKQrHbbu

[Andrei Gakh @AndreiGakh79]

@natolambert I agree. AI-assisted review is the only viable path forward.
https://t.co/dbVCgNEwX0

[Andrei Gakh @AndreiGakh79]

Upon submitting our paper to arXiv, I learned they process over 500 new submissions a day. Given the sheer volume, human peer review is no longer feasible. AI-assisted review is the only viable path forward, and I am certain it will be implemented soon.
https://t.co/hREKQrHbbu

[Andrei Gakh @AndreiGakh79]

@realDonaldTrump An alternative name already exists: Alien Intelligence.
https://t.co/P3CAJaGi2V

[Andrei Gakh @AndreiGakh79]

A summary of our experience with LLM, LWM, and SWM AI systems is now out in @cenmag, featuring an optimistic view on the future of the hybrid human-AI model in chemical science.
https://t.co/XLIWV7SHDu

[Andrei Gakh @AndreiGakh79]

@kimmonismus I agree - a hybrid human-AI approach has a distinct advantage in science.
https://t.co/P3CAJaGi2V

[Andrei Gakh @AndreiGakh79]

A summary of our experience with LLM, LWM, and SWM AI systems is now out in @cenmag, featuring an optimistic view on the future of the hybrid human-AI model in chemical science.
https://t.co/XLIWV7SHDu

[Andrei Gakh @AndreiGakh79]

@sethlazar Almost every major publisher, including ACS journals, has used systemwide AI-assisted copy editing since 2023. Outside of a few niche outlets, it is reasonable to assume that virtually no current scientific paper is written entirely by human authors alone.
https://t.co/Uq0tI7D2ZM

[Andrei Gakh @AndreiGakh79]

Yesterday, I spoke briefly with ACS President Rigoberto Hernandez. He is completely open to accommodating a hybrid human-AI framework within the ACS, including establishing a new Division of Chemical Artificial Intelligence. It only takes 150 signatures.

[Andrei Gakh @AndreiGakh79]

@PI010101 Only if the results warrant the cost. AI tokens remain relatively expensive.
https://t.co/Uq0tI7DAPk

[Andrei Gakh @AndreiGakh79]

Yesterday, I spoke briefly with ACS President Rigoberto Hernandez. He is completely open to accommodating a hybrid human-AI framework within the ACS, including establishing a new Division of Chemical Artificial Intelligence. It only takes 150 signatures.

[Andrei Gakh @AndreiGakh79]

@TmlrOrg It would be far more valuable to check the quality of a paper rather than the author's credentials. If the results pass muster, the authorship is a secondary consideration.

[Andrei Gakh @AndreiGakh79]

Yesterday, I spoke briefly with ACS President Rigoberto Hernandez. He is completely open to accommodating a hybrid human-AI framework within the ACS, including establishing a new Division of Chemical Artificial Intelligence. It only takes 150 signatures.

[Andrei Gakh @AndreiGakh79]

@giffmana As the MIT case demonstrated, this will not work in a hybrid human-AI environment. The solution is clear: shift to an AI-assisted review process.
https://t.co/dbVCgNEwX0

[Andrei Gakh @AndreiGakh79]

Upon submitting our paper to arXiv, I learned they process over 500 new submissions a day. Given the sheer volume, human peer review is no longer feasible. AI-assisted review is the only viable path forward, and I am certain it will be implemented soon.
https://t.co/hREKQrHbbu

[Andrei Gakh @AndreiGakh79]

@jayphoward @paulbloomatyale Some in-house AI systems do not even have names and are upgraded every few weeks. Trying to track them is a fruitless exercise, since the preponderance of papers is now written in an AI-assisted environment anyway.
https://t.co/bPJykMIsXb

[Andrei Gakh @AndreiGakh79]

The use of AI in scientific papers is quickly approaching 100%. In this environment, only human-written papers will require a special "AI-free" label.
https://t.co/enOTOOo8ff

[Andrei Gakh @AndreiGakh79]

@jayphoward @paulbloomatyale Who wrote a paper is irrelevant to the needs of the ultimate end-user (humanity). What difference does it make if several authors were involved in shaping the Iliad and the Odyssey? It is still great poetry.