michael

PhD student at @BoulderNLP @lecslab. LLMs for rare languages, automata, synthetic data

[michael @_michaelginn]

@adamkbaranowski @gabriberton https://t.co/puaWwEA6bP

[michael @_michaelginn]

In low-resource languages, speculative decoding may actually be hurting performance!

1/n

[michael @_michaelginn]

So what should we do instead? It turns out the simple n-gram models might be a better choice, thanks to incredibly fast inference speeds.

Sometimes, simpler is better!
4/n

[michael @_michaelginn]

In low-resource languages, speculative decoding may actually be hurting performance!

1/n

[michael @_michaelginn]

Distillation is a common way to improve acceptance rates, but we find that distillation on one task (translation) tends to generalize poorly to another task (story generation) in the language

3/n

[will brown @willccbb]

the God Model is a useful theoretical construct akin to a Worst-Case Adversary or a Busy Beaver Program or an NP Oracle, less compelling as a target to seek than as a foil for designing minimax programs which can be tangibly realized

[michael @_michaelginn]

@valmianski @rplevy @MasterTimBlais Among other issues, unitary certainly does not equal Turing computable, they’re unrelated

[michael @_michaelginn]

@valmianski @rplevy @MasterTimBlais “You can pretty obviously show that all human processes can be represented computationally”

Okay do it

[michael @_michaelginn]

@joseph_h_garvin @buildwithparas In my experience they’re actually pretty bad at it—but I think that’s an effect of RLHF and style training; base models are way better

[michael @_michaelginn]

@edmundheaphy @renegadesilicon It’s still the mechanism though. Understanding *how* we predict the next token doesn’t change that.

[michael @_michaelginn]

@p_maverick_b That’s not what the post is saying

[michael @_michaelginn]

@xboxbodywash My favorite was a student who couldn’t remember how to break out of a loop to end the program, so instead they entered an infinite inner loop

[michael @_michaelginn]

@TMoldwin Yup

https://t.co/P6Hdi5AOed

[michael @_michaelginn]

@gabriberton (Results may vary with hardware though)

[michael @_michaelginn]

@gabriberton Yup, even if you have task-specific data for distillation. It turns out since the forward pass is OOM faster, you get a favorable speed up even though the acceptance rate is a lot lower

[michael @_michaelginn]

@gabriberton Apple does for some of the on-device tiny models

I also actually just finished a paper (arxiv soon) showing that ngram models consistently work better for rare languages

[michael @_michaelginn]

@Sun_of_AZ @ml_pacheco_ - guy who can’t spell “lose”

https://t.co/DRYnfwJ96g

[Z @Sun_of_AZ]

Most people against this are foreigners who are already here mad there about to loose their jobs to Americans. This will make it more appealing to hire American citizens and American college grads.

The Americans against it are cultural suicidal drags

[michael @_michaelginn]

@ironcrakka Wrong, they also watch anime

[michael @_michaelginn]

@Ewaakaa @curiouswavefn So you admitted it was written in a rush, which would completely support the claim that it’s poorly written. Good job!