Jeff Rose

Mehh ...

[Jeff Rose @xjeffrose]

@benjitheblender https://t.co/1XoGsWt05s

Buy something like this, cut the PEX, put in new pipe, attach with the couplers, sinch with the provided straps. Will take less than 30 min

[Jeff Rose @xjeffrose]

@HustleChillson malpractice

[Jeff Rose @xjeffrose]

@JacsonBevens how in the world do you get an "F-" ?!

[Jeff Rose @xjeffrose]

I wonder what memegen looks like this morning ...

[Jeff Rose @xjeffrose]

@dylan522p "but the key takeaway is those serving an unmodified model (FP16) required batch sizes of 64+ to turn a profit. We believe that Mistral, Together, and Fireworks are serving Mistral at breakeven to slight profit margins."

Im curious how they do this

[Jeff Rose @xjeffrose]

@EddyElfenbein CIA Bloomberg Terminal

[Jeff Rose @xjeffrose]

@HustleChillson The hawks moved to the AFC West?!

[Jeff Rose @xjeffrose]

@TylerjAlsin

[Jeff Rose @xjeffrose]

@HustleChillson Was it Jake?

[Jeff Rose @xjeffrose]

@mattufford FTR

[Jeff Rose @xjeffrose]

@mookiealexander Hausch$$ slander will not be tolerated in this house

[Jeff Rose @xjeffrose]

@johndavidfraley There is a reed sized hole fo sho

[Jeff Rose @xjeffrose]

@abacaj I haven't seen any finetuning harnesses do it correctly at least. If you look at the mistral implementation, the only lever you really have is the weights on the gate for routing. You then have to coordinate the updating of the appropriate expert in conjunction with the router.

[Jeff Rose @xjeffrose]

@10x_er ... this is just a tribute

[Jeff Rose @xjeffrose]

@maximelabonne is this using the Mixtral gating layer as is for the routing?

[Jeff Rose @xjeffrose]

@giffmana PaLI or PaLI-X?

[Jeff Rose @xjeffrose]

@alexfmckinney @cgarciae88 i feel like this might be good enough for a first pass? https://t.co/oxBCKcvcpG

[TartWrangler @TartWrangler]

@SwiftOnSecurity I bet to differ

[Jeff Rose @xjeffrose]

@teknium @ericmitchellai there absolutely could be, but I would look for additional indicators / symptoms in addition to spiky loss.

[Jeff Rose @xjeffrose]

@ericmitchellai @teknium This has been my experience as well. You are training for a preference objective that wasn't captured in your SFT training set. Ride the wave and perform evals every 5000 steps or so.