james

develooper | building @thereabouts_app so you never lose a new place to try

[james @http_413]

@coryparrry @btsouth opencodex

https://t.co/CwG1jKuglZ

[james @http_413]

feels so dirty

i'm a dirty boy

[james @http_413]

@btsouth @SlumBreeze_ > astra
> fast mode

lol

[james @http_413]

@notkevinzhang jevin please

[james @http_413]

@TokenGremlin dario: just one more model release then we slow down. for real, guys.

[james @http_413]

@JakeKAllDay @Presidentlin 0.50 to 0.20 is 95% huh?

[james @http_413]

@CapitanoJacko @Srasgon @insane_analyst wdym my team actually does this every morning

[james @http_413]

@AlanMCole i didn't think it was possible for realtors to do even less work.

[james @http_413]

@stevelizcano @Angaisb_ were you the groom

[james @http_413]

@Ananth7e if your codex limits are draining way too quickly, disable ultra reasoning.

the difference is night and day.

thank me later.

[james @http_413]

@headzoo @TheGBreaker oh hey i'm most of the way there

[james @http_413]

@Shex005 @ibocodes it said there's not 🥺

[james @http_413]

@Ananth7e perhaps you're going insane

[Alex Cheema @alexocheema]

As much as I'd like this to work, 1-bit / ternary models like Bonsai are (today) a dud.

The sweet spot appears to be ~4 to 5 bits per parameter. That's what @UnslothAI find to be the "knee" of the pareto curve (thanks @realGeorgeHotz for pointing this out).

Interestingly a synapse in the human brain uses about 4.7 bits of information so perhaps this points to something more fundamental.

You're much better off using a smaller model quantized to 4-bit than a larger model quantized to 1-bit or ternary.

We also find these 1-bit / ternary models to be overly verbose - so they can actually run *slower* on end to end tasks than 4-bit, even though the decode TPS is faster.

[james @http_413]

@Oakmanbearcow @dhh what do you mean less slop?

[james @http_413]

@haydendevs guessing terra is on life support rn with constrained compute. it was so bad they likely aren't planning a gpt-6 terra.

[james @http_413]

feels so dirty

i'm a dirty boy

[james @http_413]

@ForwardEditor

[james @http_413]

@TheRohanVarma why not build a messaging system into codex? 👀

[james @http_413]

@Blackfrost_AI jev said that address is wrong

[james @http_413]

@DennisonBertram @sicdev_pilnup just delete or everything and start fresh.