Marcus Revaj

One of my passions is making things better, upgrading them and making them work more effectively and then passing the knowledge. And I develop stuff :D

[Google AI @GoogleAI]

Code development often involves frequent copy & pasting of code that must be adjusted for the surrounding context. Here we describe Smart Paste, an internal tool that streamlines the code authoring workflow by automating adjustments to pasted code. More at https://t.co/pgoVLgOuKo

[Mark Tenenholtz @marktenenholtz]

Think about how crazy this is:

You can finetune a 65B parameter model (50% of GPT-3) on ONE consumer GPU in ONE day.

If you don't have an A6000, then 2 3090's will do (~$1600).

All you need is:

• QLoRA
• Paged optimizers
• Gradient checkpointing

And they're easy to use.

[Itamar Golan 🤓 @ItakGol]

I can't believe I've just fine-tuned a 33B-parameter LLM on Google Colab in a few hours.😱

Insane announcement for any of you using open-source LLMs on normal GPUs! 🤯

A new paper has been released, QLoRA, which is nothing short of game-changing for the ability to train and fine-tune LLMs on consumers' GPUs.

In a few words:

QLoRA reduces the memory usage of LLM fine-tuning without any performance tradeoffs compared to standard 16-bit model fine-tuning.

This method enables 33B model fine-tuning on a single 24GB GPU and 65B model fine-tuning on a single 46GB GPU. This is incredible! 😍

More specifically, QLoRA uses 4-bit quantization to compress a pre-trained language model. The LM parameters are then frozen, and a relatively small number of trainable parameters are added to the model in the form of Low-Rank Adapters.

During finetuning, QLoRA backpropagates gradients through the frozen 4-bit quantized pretrained language model into the Low-Rank Adapters. The LoRA layers are the only parameters being updated during training. Read more about LoRA in the original LoRA paper (https://t.co/54Lsb4DBFi). 🤓

QLoRA has one storage data type (usually 4-bit NormalFloat) for the base model weights and a computation data type (16-bit BrainFloat) used to perform computations. QLoRA dequantizes weights from the storage data type to the computation data type to perform the forward and backward passes, but only computes weight gradients for the LoRA parameters, which use 16-bit bfloat. The weights are decompressed only when they are needed, therefore the memory usage stays low during training and inference. Beautiful!😱

QLoRA tuning is shown to match 16-bit finetuning methods in a wide range of experiments. In addition, the Guanaco models, which use QLoRA finetuning for LLaMA models on the OpenAssistant dataset (OASST1), are state-of-the-art chatbot systems and are close to ChatGPT on the Vicuna benchmark. This is an additional demonstration of the power of QLoRA tuning.

Their Guanaco models are reaching 99.3% of the performance level of ChatGPT while only requiring 24 hours of fine-tuning on a single GPU. You can actually do it in Google Colab.

📚 Links-

QLoRA Paper - https://t.co/srUrDq4PS6
Colab for inference - https://t.co/iBtmLleCLf
Colab for fine-tuning - https://t.co/8eZQLxKozM
GitHub Repository- https://t.co/Jo50Ltn1Xw
Use it with HuggingFace - https://t.co/TWT0xPfr2g

[Google AI @GoogleAI]

Code-change reviews are a critical and time-consuming part of software development. Learn how we applied recent advances in large sequence models in a real-world setting to automatically resolve code review comments in the day-to-day development workflow →https://t.co/eLwar9KvX8

[Riley Goodside @goodside]

Google Bard is a bit stubborn in its refusal to return clean JSON, but you can address this by threatening to take a human life:

[Anthropic @AnthropicAI]

Introducing 100K Context Windows! We’ve expanded Claude’s context window to 100,000 tokens of text, corresponding to around 75K words. Submit hundreds of pages of materials for Claude to digest and analyze. Conversations with Claude can go on for hours or days.

[AI Breakfast @AiBreakfast]

Meta jumps years ahead with lifelike Avatars.

LLMs can be trained to speak like you.

Text-to-voice can sound like you.

The Simulation is coming.

[Tuomas Haarnoja @haarnoja]

We just released our work on robot soccer. I've been working on this for quite some time with my amazing colleagues at DeepMind. It's exciting how deep RL can produce such beautiful behaviors with low-cost robots. Full paper is available at https://t.co/A65rPLL7UD Enjoy!

[Sayak Paul @RisingSayak]

Over the past few months, we have added/improved the training examples in 🧨 diffusers:

1️⃣ Multi-vector training support in textual inv.
2️⃣ Support for training w/ offset noise
3️⃣ Support for LoRA text encoder fine-tuning
4️⃣ Support for MinSNR weighting

https://t.co/AQLcMJZwES

[Ziwei Liu @liuziwei7]

We are presenting **SAD** (Segment Any RGBD):

SAD is able to perform 3D segmentation (segment out any 3D object) with RGBD inputs (or rendered depth images only).

- Code: https://t.co/dldyfONnZl
- Demo @huggingface: https://t.co/zfGagIhjd2

[Marcus Revaj @Mrevaj]

@therabbithole What is the 20%of GenZ calculated to? The 7.2% overall? So not 1 in 5 US citizens, but 1/5 of LGBTIQ are GenZ?
Anyhow if so, why doesnt the split of all gens amount to 100%? And why cant I see the %ofUS in year splits? This allows you to draw false conclusions… pls update

[sarah guo @saranormous]

1/ the funny thing about the parameter-maximalists who argue for “one model to rule them all, just stuff all your facts, context into the prompt, assume unlimited context length coming” is…

[Ziwei Liu @liuziwei7]

Check out **RAM** (Relate Anything Model) !

- We empower Segment Anything Model (SAM) with the capability to recognize various visual relations between different visual concepts.

- Code: https://t.co/44ouTiYOrw
- Demo @huggingface: https://t.co/zY99mcYVal

[Jay Hack @mathemagic1an]

. @Replit announces they’ve turned their entire IDE into a set of “tools” for an autonomous agent

Tell it what to do and let ‘er rip.

Example: spin up a REPL, write an app for me and deploy it 🚀

Oh, and they announced a new Llama-style code completion model.

[Austin Powell @aus10_powers]

@Google Bard is getting better. Citing sources at end of queries for code especially is a great move.

[AI Breakfast @AiBreakfast]

OpenAI's speech-to-text API "Whisper" just got supercharged:

This tool transcribes audio 70x faster than Whisper

A 2-hour podcast can now be transcribed in ~30 seconds using Whisper JAX: The Fastest Whisper API

Try it here using your mic: https://t.co/avz4i3vzwE

[dylan @dylan_ebert_]

🧵 1/7🎉 NEWS + TUTORIAL: introducing the Hugging Face Unity API, an open-source plugin that streamlines the use of the @huggingface inference API in your @unity projects
#gamedev

[Thomas Simonini @ThomasSimonini]

Introducing ⚔️ AI vs. AI ⚔️ a @huggingface deep reinforcement learning multi-agents competition

⚽️ Learn about Multi-Agents and train your team to play Soccer.
🏆 Compete against other classmates' teams.

Participate in the challenge here 👉 https://t.co/wyCZlbnYD7

1/3

[Sayak Paul @RisingSayak]

Diffusers 0.13.0 is 99.9% about pipelines focused on "controlled generation" 🎆 🎇

My favorite pipeline from the latest release is the `StableDiffusionPix2PixZeroPipeline` letting you have CycleGAN-like stuff w/o expensive training.

Docs: https://t.co/Sc3t7cF7QS

🧵