@plumsirawit Lol
1. You clearly do not know what you are talking about.
2. Your nonsensical comments can only impress your fellow LARPers; otherwise it only contributes to your own public embarrassment
@_chenson__@GoogleDeepMind@axiommathai Here, I am not talking about not taking literally the same proof approach, but about the definition of concepts. The value question is relevant I think when compared to just directly asking frontier models to check for mistakes in natural language.
@_chenson__@GoogleDeepMind@axiommathai I am curious to hear your thoughts about the following. I am sure you know well about misalignments between autoformalized statements and their corresponding base statements. If one cannot 100% guarantee the match of meanings, what is the value of formalization?
@scottnarmstrong@rperezmarco I would say the autoformalization is meaningful if and only if the human supervisor either hand-wrote comparator or check thoroughly whether the comparator statements are exactly what they mean
@scottnarmstrong@rperezmarco But if autoformalization is to the point where you don't even know whether the final statement is what you think it means (which involves whether the definitions are correctly defined in Lean), then it is just another liability (in some sense worse that natural language slop)
@scottnarmstrong@rperezmarco I am not saying all autoformalizations are bad. For FLT, it is ok, because the final statement is extremely straightforward for a human to state, and having an OK with the human-configured statement would be good enough. No understanding of intermediate stuffs required
@scottnarmstrong@rperezmarco The point is this is not at all a certificate. The so-called "autoformalization" without any human check has the same authority as a ChatGPT session asking to check for mistakes.
@rperezmarco@scottnarmstrong This is exactly my point. I see autoformalization, when not configured correctly, adds very little value on top of LLM capabilities, and is a massive waste of resources. It would be better to use those tokens to rather write another slop (yet new) AI math paper.
@scottnarmstrong@rperezmarco I agree that this is another step to reduce mistakes, but the point of formalization to me is to eliminate the mistake, and I would argue that there is no philosophical difference between 0.1% chance of mistake and 0.0001% chance of mistake.
@scottnarmstrong@rperezmarco I want to ask y'all what you think the meaning of autoformalization is if humans don't hand-check at the very least the main headline statement by hand. If the process is like I believe what my LLM told me, how is it so different from just asking LLM directly check for mistakes?
@deepmatmul The main immediate use case even for human math is to use it as a tour-de-force to succumb mathematicians claiming false things. This has been a headache in some areas of math (look at the abc conjecture and IUT business around it).