icarus4

[Jim Fan @DrJimFan]

You'll soon see lots of "Llama just dethroned ChatGPT" or "OpenAI is so done" posts on Twitter. Before your timeline gets flooded, I'll share my notes:

▸ Llama-2 likely costs $20M+ to train. Meta has done an incredible service to the community by releasing the model with a commercially-friendly license. AI researchers from big companies were wary of Llama-1 due to licensing issues, but now I think many of them will jump on the ship and contribute their firepower.

▸ Meta's team did a human study on 4K prompts to evaluate Llama-2's helpfulness. They use "win rate" as a metric to compare models, in similar spirit as the Vicuna benchmark. 70B model roughly ties with GPT-3.5-0301, and performs noticeably stronger than Falcon, MPT, and Vicuna.
I trust these real human ratings more than academic benchmarks, because they typically capture the "in-the-wild vibe" better.

▸ Llama-2 is NOT yet at GPT-3.5 level, mainly because of its weak coding abilities. On "HumanEval" (standard coding benchmark), it isn't nearly as good as StarCoder or many other models specifically designed for coding. That being said, I have little doubt that Llama-2 will improve significantly thanks to its open weights.

▸ Meta's team goes above and beyond on AI safety issues. In fact, almost half of the paper is talking about safety guardrails, red-teaming, and evaluations. A round of applause for such responsible efforts!

In prior works, there's a thorny tradeoff between helpfulness and safety. Meta mitigates this by training 2 separate reward models. They aren't open-source yet, but would be extremely valuable to the community.

▸ I think Llama-2 will dramatically boost multimodal AI and robotics research. These fields need more than just blackbox access to an API.
So far, we have to convert the complex sensory signals (video, audio, 3D perception) to text description and then feed to an LLM, which is awkward and leads to huge information loss. It'd be much more effective to graft sensory modules directly on a strong LLM backbone.

▸ The whitepaper itself is a masterpiece. Unlike GPT-4's paper that shared very little info, Llama-2 spelled out the entire recipe, including model details, training stages, hardware, data pipeline, and annotation process. For example, there's a systematic analysis on the effect of RLHF with nice visualizations.

Quote sec 5.1: "We posit that the superior writing abilities of LLMs, as manifested in surpassing human annotators in certain tasks, are fundamentally driven by RLHF."

Congrats to the team again 🥂! Today is another delightful day in OSS AI.

[Jim Fan @DrJimFan]

Transformers are here to stay for a while. Not because it’s the absolute best architecture, but because the staggering amount of resources lock us to the existing weights.

Starting another model evolution tree will literally burn forests to ground (CO2). You only train once.

In a sense, Transformers won the pretraining lottery. Big companies (OpenAI) have little economic incentive to deviate from their backbone model, and indie players can’t afford to train from scratch. Llama/OPT makes the commitment even stickier.

[AI at Meta @AIatMeta]

Today we're releasing the Segment Anything Model (SAM) — a step toward the first foundation model for image segmentation.

SAM is capable of one-click segmentation of any object from any photo or video + zero-shot transfer to other segmentation tasks ➡️ https://t.co/qYUoePrWVi

[AI at Meta @AIatMeta]

In addition to the new model, we’re also releasing the SA-1B dataset, which is 400x larger than any existing segmentation dataset — we hope this work will help accelerate computer vision research and enable entirely new applications.
Get the dataset ⬇️
https://t.co/vizP1Oaqu2

[Jim Fan @DrJimFan]

Reading @MetaAI's Segment-Anything, and I believe today is one of the "GPT-3 moments" in computer vision. It has learned the *general* concept of what an "object" is, even for unknown objects, unfamiliar scenes (e.g. underwater & cell microscopy), and ambiguous cases.

I still can't believe both the model and data (11M images, 1B masks) are OPEN-sourced. Wow.😮

What's the secret sauce? Just follow the foundation model mindset:
1. A very simple but scalable architecture that takes multimodal prompts: text, key points, bounding boxes.
2. Intuitive human annotation pipeline that goes hand-in-hand with the model design.
3. A data flywheel that allows the model to bootstrap itself to tons of unlabeled images.

IMHO, Segment-Anything has done everything right.

[DV @DV2559106965076]

You might know that MSFT has released a 154-page paper (https://t.co/7ptdkaWjgb) on #OpenAI #GPT4 , but do you know they also commented out many parts from the original version?

🧵: A thread of hidden information from their latex source code

[1/n]

[icarus4 @icarus4chu]

2.8.0 加入了 prioritize flow，完全打中需求

對我這種只想跑一個 agent，但是不同 flow 的特性及 flow run 數量差距很大的情況下，事情變得超簡單。

[Prefect @PrefectIO]

Prefect 2.8.0 has been released! ✨

🏊 Prioritize flow runs with work pools
🌎 Connect flows to the world with generic webhooks
📊 Create flow runs in the UI with parameters from previous runs
💻 Reset concurrency limits via the CLI

...and more!
https://t.co/Sy2JHnfhfA

[Lauren @jenks]

Summing up my 11 years at Twitter:

-Twitter is private
-Jack is CEO
-Jack is not CEO
-Twitter is public
-Jack is CEO
-Jack is not CEO
-Twitter is private
-...Jack is CEO?

🤔

[tobi lutke @tobi]

OSX is really fun since M1 with @logseq, https://t.co/u4OhVD0K5m, and @raycastapp. Best productivity setup I’ve had in a decade. Bound common apps to the f1 keys like I had ages ago and wonder why I ever stopped.

[Ming Zhao @FabiusMercurius]

🧐How to Read 10Ks Like a Hedge Fund🧐

“Fundamentals don’t matter anymore!” I’ve heard this a lot lately on Fintwit.🙄

But, for those who’ve diversify beyond $GME and $DOGE, here’s a primer on what metrics fundamental buy-side PMs look at and why:
(real examples outlined)
👇

[Patrick Collison @patrickc]

Stripe Capital: https://t.co/mf4i02oUB4

[The Long View @HayekAndKeynes]

Macro data providers

Outside of Bloomberg and the Excel API, what’s your favorite and why? I’ve tried Macrobond, Haver, Datastream and even the FRED API and found each to have strengths but not enough to make me use them over Bloomberg.

[Sam Altman @sama]

How things get done in the world:

Relentless focus
Self-confidence
Personal connections
Effective leadership (communication, management, vision, and evangelism)

[Cabel Sasser @cabel]

Pro MacBook Pro Tip: have a Touch Bar with Touch ID? If you edit /etc/pam.d/sudo and add the following line to the top…

auth sufficient pam_tid.so

…you can now use your fingerprint to sudo!

[icarus4 @icarus4chu]

#rubyconftw
The slides of talk: "Building Efficient and Reliable Crawler System With Sidekiq Enterprise"

https://t.co/XpmsOU08Oz

[icarus4 @icarus4chu]

@qgray1984 123

[BBC News (World) @BBCWorld]

Hong Kong police use tear gas to disperse protesters - see footage of clashes http://t.co/RMKPOoNEiC

[icarus4 @icarus4chu]

@evadne
Thank you for your clear and helpful explanation.
Your backend architecture (and you) are really awesome :D

#railspacific

[lulalala_it @lulalala_it]

#railspacific @a_matsuda recommends @tenderlove Aaron Patterson's commits https://t.co/L6vfmmRL81 #refactoring

[lulalala_it @lulalala_it]

#railspacific @hsbt recommends Code Climate Blog http://t.co/MWLVAyOekA #refactoring