Xingyu Li

[Zhiyun Qian @pkqzy888]

The first author of the paper @XingyuLi816
is looking for internships. Give him a shot if you are hiring😀

[Zhiyun Qian @pkqzy888]

We tried hard to understand many aspects of how the ecosystem works in the industry. Unfortunately the paper was not published in a top academic venue. But the practical relevance of the work is what motivated us -- bridging the gap between academia and industry is a lot of fun.

[Xingyu Li @XingyuLi816]

thanks for liking our paper. it is really pleasure to get good feedback from you.

[Brad Spengler @spendergrsec]

Tons of great info in "An Investigation of Patch Porting Practices of the Linux Kernel Ecosystem": https://t.co/v16o4XdVQt that, if you've been listening to me, shouldn't come as much of a surprise.
Will share a few snippets/comments in thread:

[zack @wenquai]

If you’re an AI engineer or builder that relies on GPT4 and are worried about the API collapsing, here’s a contingency plan for you to follow:

First things first, make sure you apply to the Azure OpenAI service (link in replies). You can use it as a drop-in replacement for GPT4, as well as benefit from decreased latency and access to GPT4-32K. Supposedly Azure will also support GPT4-V, GPT4 FT and multimodal soon but this is TBD given all the chaos. Yes, it’s a pain in the ass to use Microsoft APIs, but this is still probably the fastest way to derisk. Also if you apply through Microsoft for Startups (link below) you’ll get $150K of Azure credits 🤑

Second, if you’re looking to move away from OAI/MSFT entirely, you should consider using Anthropic’s Claude. I’ve found Claude 2 to actually be better than GPT4 at many use cases (conversation, content generation, summarization). Claude still isn’t great at understanding logic/code so be wary of that. It also requires a slightly different style of prompting for best results - might release a guide on this in the future

If neither of these options work for you, your *last* resort should be to use GPT4 to generate synthetic data so you can fine-tune an OSS LLM (Mistral/LLaMa/Phi). This route requires the most engineering lift and technical risk, so caveat emptor. Will include link to some guides/resources below!

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

The KV-cache for self-attention is (arguably) the most effective/simple technique for speeding up LLM inference. I'm going to post a more extensive explanation tomorrow, but this writeup I found is super clear and even provides an implementation!

https://t.co/3G5MijbhFg

[Brendan Dolan-Gavitt @moyix]

Holiday hacking: a script that attempts to summarize a Ghidra-decompiled function/program using GPT-3 by recursively summarizing its dependencies in the call graph! https://t.co/PQUXQ4CTBB

[Lucas Beyer (bl16) @giffmana]

How good of a BERT can one get in ONE DAY on ONE GPU?

With all the recent studies about scaling compute up, this paper takes a refreshing turn and does a deep dive into scaling down compute.

It's well written, stock full of insights. Here is my summary and my opinions.

🧶 1/N

[elvis @omarsar0]

2022: A Year in Review (ML Papers Edition)

In this thread, let's take a look at some of the top trending ML papers of 2022 ↓

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Self-supervised pre-training is incredibly beneficial in NLP for deep networks like LLMs or BERT. But, making one simple mistake can drastically deteriorate pre-training quality. 🧵 [1/7]

[Olesia Nikulina @ol_nikulina]

Academic writing is a superpower. It's a key to publishing, getting a new job or tenure, and becoming known in the field

These 10 AI-powered websites will help you with EVERY part of your manuscript and will save you 100+ hours:

#openscience #AcademicTwitter #phdvoice #phdchat