@willdepue@joshuawolk@LukeFarritor This is extremely cool. Is it limited to the US? I'm a EU astrophotographer lately playing around with Astra/Fable on exoplanets research but I'm constrained on budget
@SuzanMaseki@imjustnewatai I agree, but I'm a Stockfish tester. There is a specific way to get the new net tested on a distributed test environment. I can tell you, this is going to fail SPRT. I can take 100:1 bet from anyone here :)
@imjustnewatai Small Sample Size. Let it run until 0/100% LOS (Likelyhood of superiority). Should be relatively quick on a STC (something like 10s+0,1s)
@Bayesian0_0@adonis_singh BALROG NetHack is confirmed to be a harness issue, as I briefly discussed with the maintainer. Some details here:
https://t.co/82TVrTwDUn
Harness difference yeah. My harness was built over 2 years ago, written by hand by me using Llama 3.1 70B to test it, which basically never even descended to dlvl 2.
Also it seems that the GUI used has lots of important information that can help the LLM navigate.
I'll try to add the new one as an option to BALROG if I can catch a break :D There's still room to go, currently most likely it's AGI. If we get 5 runs with ascension in a row with random characters it's likely ASI 😅