Josh Robinson

Ph.D. student at @CSatUSC/@nlp_usc. Research: sequence modeling, #NLProc, #ML.

[Michael Saxon @m2saxon]

@sintelion @BenZhou96 @muhao_chen Awesome analysis of what KNN-LM says abt training:

Is the seeming "free lunch" of KNN-LM (replacing top LM layers with embedding store and KNN lookup) due to a weakness of the LM objctve? Seems no!

Training a replacement MLP on the KNN does better! 🤔

https://t.co/o7eeKmHvOf

[Ting-Rui Chiang @ctinray]

“On Retrieval Augmentation and the Limitations of Language Model Training” (https://t.co/HrcV51Z9em) has been accepted to NAACL 2024!

While it is well known that kNN retrieval can decrease LMs’ perplexity, the underlying reason is unclear. We study two hypotheses 👇

[Aran Komatsuzaki @arankomatsuzaki]

Leveraging Large Language Models for Multiple Choice Question Answering

Finds that code/text-davinci performs much better on MCQ if the candidate answers are characters like "A", "B", etc unlike the original GPT3.

https://t.co/6yhECwJUaI

[Ethan Perez @EthanJPerez]

@rajammanabrolu @douwekiela @kchonyc Here are two I found quite interesting:
1. https://t.co/xSEC2YKHSF from @timo_schick @HinrichSchuetze
2. https://t.co/q9XG8jIamH from @ChrisRytting et al.

[Taylor Sorensen @ma_tay_]

Most methods for aligning LMs to tasks require many labeled data (prompt eng) or access to the model (soft-prompts). In our work @ACL, we select high-perf. prompts 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 𝘥𝘪𝘳𝘦𝘤𝘵 𝘢𝘤𝘤𝘦𝘴𝘴 𝘵𝘰 𝘵𝘩𝘦 𝘮𝘰𝘥𝘦𝘭 and 𝘸𝘪𝘵𝘩𝘰𝘶𝘵 𝘭𝘢𝘣𝘦𝘭𝘦𝘥 𝘦𝘹𝘢𝘮𝘱𝘭𝘦𝘴👇

[BYU @BYU]

What started out as an opportunity for extra credit in one of his BYU classes led to a month-long, all-expenses-paid trip to China, a “pretty sweet trophy and a very good scholarship" for BYU senior Josh Robinson:
https://t.co/jCOyZ53dyt

[Josh Robinson @_josh_robinson]

@andrew_n_carr @OpenAI Congrats!

[Josh Robinson @_josh_robinson]

@andrew_n_carr Congrats!