Mark Ager

Professional cat wrangler

[Andrej Karpathy @karpathy]

Oops haven't tweeted too much recently; I'm mostly watching with interest the open source LLM ecosystem experiencing early signs of a cambrian explosion. Roughly speaking the story as of now:

1. Pretraining LLM base models remains very expensive. Think: supercomputer + months.
2. But finetuning LLMs is turning out to be very cheap and effective due to recent PEFT (parameter efficient training) techniques that work surprisingly well, e.g. LoRA / LLaMA-Adapter, and other awesome work, e.g. low precision as in bitsandbytes library. Think: few GPUs + day, even for very large models.
3. Therefore, the cambrian explosion, which requires wide reach and a lot of experimentation, is quite tractable due to (2), but only conditioned on (1).
4. The de facto OG release of (1) was Facebook's sorry Meta's LLaMA release - a very well executed high quality series of models from 7B all the way to 65B, trained nice and long, correctly ignoring the "Chinchilla trap". But LLaMA weights are research-only, been locked down behind forms, but have also awkwardly leaked all over the place... it's a bit messy.
5. In absence of an available and permissive (1), (2) cannot fully proceed. So there are a number of efforts on (1), under the banner "LLaMA but actually open", with e.g. current models from @togethercompute, @MosaicML ~matching the performance of the smallest (7B) LLaMA model, and @AiEleuther , @StabilityAI nearby.

For now, things are moving along (e.g. see the 10 chat finetuned models released last ~week, and projects like llama.cpp and friends) but a bit awkwardly due to LLaMA weights being open but not really but still. And most interestingly, a lot of questions of intuition remain to be resolved, e.g. especially around how well finetuned model work in practice, even at smaller scales.

[thaidn @XorNinja]

I reversed the Vietnam's contact tracing app, and found that it can silently upload all contact history to a centralized server, despite claiming that it only stores data on the client.

Somebody then said I was wrong and challenged me to provide evidences.

[Jenny Newman @Jennernugen]

I haven't found the words. I don't know if I'll be able to. I lost a part of my heart and soul today.
He was so generous and kind, so endlessly sweet and so loved by his incredible friends. I feel so lucky to have known him, to have loved & been loved by him.
I love you, honey.

[Mark Ager @Ager]

Perhaps we shouldn’t be surprised at how crazy the world is when our media is as fraudulent as @thisisinsider

[Greg Brockman @gdb]

An OpenAI employee printed out this AI-written sample and posted it by the recycling bin: https://t.co/PT8CMSU2AR

[Mark Ager @Ager]

Someone really dropped the ball over #BlackMirrorBandersnatch and #AppleTV. @Netflix was obviously well prepared with a helpful video and follow up email so what happened, @AppleTV?

[Mark Ager @Ager]

Seeing @elonmusk flap around in frustration with the media suggests that he doesn’t have a fully rounded team of advisors; hopefully some good people are reaching out to him behind the scenes.

[Mark Ager @Ager]

@thread_puller Stop practicing! I've gained weight and so need all the advantages my fat hands can grab.

[Mark Ager @Ager]

@thread_puller Frank's always a great choice. 🎤🎶Two thousand and seventeen, it was a very good year...🎶

[Mark Ager @Ager]

@thread_puller "Tune in after the break to see this little dog that laughed, and the unfolding scandal of a dish that ran away with a spoon".

[Mark Ager @Ager]

@thread_puller Next step, deep fried turkey (and the associated increase in visits to hospital with 3rd degree burns).

[Jeff Potts @jeffpotts01]

You wanted commercial support for standalone #Activiti from #Alfresco. Now you've got it. http://t.co/2xTgcYFC1A

[Mark Ager @Ager]

Yes! This -> http://t.co/HM6RvYUgTv

[Mark Ager @Ager]

@cdrcboston They don't appear to be on twitter. It's a shame since you're saying nice things about them!

[Mark Ager @Ager]

@cdrcboston I don't want to take credit for someone else's work, did you mention the right person?

[Mark Ager @Ager]

http://t.co/DaNkhIAUcN - bringing structure into large agile teams. This is probably something worth looking into. /via @joemckendrick

[Mark Ager @Ager]

.@kimjoar's excellent step-by-step for using backbone.js https://t.co/RoRzgKWL /via @snipeyhead

[Mark Ager @Ager]

@KaseyRamone I have a new song? Great! Is it any good?

[Kuali Feeds @kuali_feeds]

Congrats to Kuali Coeus on their release of #KC 5.0! Check out what's in this release: http://t.co/fOBGwUEF #software #release

[Daniel Kraft, MD @daniel_kraft]

Epic Challenge: What The Emergence of an EMR Giant Means For the Future of Healthcare Innovation - Forbes http://t.co/5wg2n01N #futuremed