@tonylfeng Obviously any attempt to metric-ify taste/interestingness will be ill-conceived. But do you think the "utility" of a proof could be more amenable to a quantitative definition?
@littmath I generally agree. However, the arxiv is overburdened (for all disciplines) atm. We need an AI-arxiv, or at a minimum, a tag for papers for which the authors claim no responsiblity/understanding
@doomslide The idea of accessing the internet with a text interface is not inherently bad. The idea of a few powerful companies designing/owning these abstractions (and only practical means of using the internet) is scary however
@natolambert I wonder how many submissions can be filtered out via the simple criteria (1) contains glaring mistakes? and (2) is reproducible?
obviously reproducing arbitrary papers from scratch remains an unsolved problem
@XYHan_ Imo the only solution is for journals/big conferences to develop proprietary screening tools that are hidden from the public
In practice this means fine tuning an llm to do autoreviews; this strat has drawbacks but is still preferable to the status quo
@XYHan_ This is good but a few issues remain
(1) in theory traditional reviews produce richer/more objective comments
(2) zoom interviews provide authors more chances to finesse their way to an acceptance via charisma
(3) people will cheat
@littmath imo AI is kind of like handing a veteran pickax and shovel-wielding miner an excavator;
that is, I don’t blame anyone bristling at this transition; and I expect future generations to prosper
@RyanWeather imo big data is largely an illusion; yes we have lots of data in the us, but much of the rest of world remains unobserved.
plenty of opportunities for new phds to develop new physics, conduct field campaigns, etc
@nihilunbounded Uni/national lab supercomputers exist partially for this purpose; although I agree that comparing ai industry to experimental physics is silly
@littmath Fantastic essay! These ideas are highly relevant across academia.
As disruptive as these times are, there's something very romantic about returning to oral exams given in front of a blackboard. Intuitively this seems like a much better way to measure depth of understanding
@praveshkkothari Exceptionally well put! I'm most curious to see if proof-driven domains will either (a) restructure around ai-proof proxies for success (e.g., citation count) or (b) begin to emphasize subjective skills: taste, clarity of communication, etc