Frederic Zhang

Scientist @amazon, photographer, substandard musician but a linguistic genius

[Frederic Zhang @fredzzhang]

Links:
Hugging face: https://t.co/WvFQzbzrEJ
Github: https://t.co/96DZu9PFrL

[Frederic Zhang @fredzzhang]

Is the low-rank formulation a necessity in parameter-efficient fine-tuning? In our #ICLR2025 paper, we proposed a full-rank PEFT method that outperforms LoRA across vision and language tasks, while maintaining parameter and memory efficiency.

[Paul Albert @PaulAlbert31]

New paper at ICLR 25 !
https://t.co/kkHO2yLmvQ
https://t.co/HadQ8ayMaU

We evidence the low-rank bottleneck of LoRA, showing that scaling trainable parameters isn't always enough.
Plus, RandLoRA optimizes gradient computation to reduce memory use over LoRA !
#peft #ICLR2025

[Frederic Zhang @fredzzhang]

Links:
Hugging face: https://t.co/WvFQzbzrEJ
Github: https://t.co/96DZu9PFrL

[Frederic Zhang @fredzzhang]

Excited to announce that the codebase for our #neurips2024 paper Knowledge Composition using Task Vectors with Learned Anisotropic Scaling (https://t.co/yCekofQ6xq) is now available. In this paper, we showed how to combine or transfer the knowledge across different models. [1/6]

[Frederic Zhang @fredzzhang]

(5) aTLAS is a strong PEFT method, especially when data is limited. In addition, exisiting PEFT methods such as LoRAs can be seamless integrated into aTLAS for better memory efficiency. [6/6]

[Frederic Zhang @fredzzhang]

(3) aTLAS is complementary to previous few-shot methods, in that one third of the examples it improves upon are unique; (4) aTLAS is robust to domain shift and achieves better generalisations on out-of-domain datasets; [5/6]

[Frederic Zhang @fredzzhang]

We show that (1) aTLAS implicitly exploits the low intrinsic dimensionality of pre-trained models, leading to few learnable parameters; (2) It is particularly effective in low-data regimes, and improves CLIP by 6.5 points in accuracy across 22 datasets with unlabelled data; [4/6]

[Frederic Zhang @fredzzhang]

We learn an independent scaling coefficient for each parameter block of the task vector, e.g., weight matrices or its partitions, resulting in anisotropic scaling at the task vector level. We test our method across five different applications and reveal interesting insights [3/6]

[Frederic Zhang @fredzzhang]

We propose aTLAS, an algorithm that simplifies otherwise complex problems to learning a linear combination of task vectors. The task vectors, which are weight deltas between a pre-trained model and its fine-tuned variants, have been shown to encode task-specific knowledge. [2/6]

[Australian Institute for Machine Learning @TheAIML]

In our first Researcher Spotlight, Dr. Frederic Zhang, AIML Postdoctoral Fellow, shares insights into his work on 'task vectors'—a method for improving multitasking in machine learning. Learn more and watch his interview: https://t.co/p7wMIgL27e #AI #machinelearning

[Min Choi @minchoi]

This is wild 🤯

AI imagines stars with young self together

[Grant Sanderson @3blue1brown]

I just posted the first of what will be several chapters about transformers, which in the last year has been the most requested topic for the channel.

https://t.co/j1LMXSor6c

[Frederic Zhang @fredzzhang]

I’ll be presenting our paper in person for the first time! If you are interested in human-object interaction detection, come check out our poster (#147) in Room Nord this morning from 10:30 to 12:30. #ICCV2023 @dylanjcampbell_ @sgould_au

[Kosta Derpanis (back home, Toronto ❤️) @CSProfKGD]

#ICCV2023 Awards 🏆

[Frederic Zhang @fredzzhang]

It was quite a rollercoaster ride with so many decisive games before the tie breaker. Most enjoyable WCC I’ve ever watched!

[Chess.com @chesscom]

Ding Liren wins the 2023 FIDE World Championship 🏆

Congratulations to Ding on becoming the new FIDE World Champion, and cementing his place in chess history after a thrilling match! 👏

[Kevin Fischer @kevinafischer]

My mind is blown 🤯🤯🤯

go to any github repo but use https://t.co/gVeKJkKfGG instead of https://t.co/PHQ162gDwH

it's now VS code IN BROWSER

[Andrej Karpathy @karpathy]

This is a baby GPT with two tokens 0/1 and context length of 3, viewing it as a finite state markov chain. It was trained on the sequence "111101111011110" for 50 iterations. The parameters and the architecture of the Transformer modifies the probabilities on the arrows.

E.g. we can see that:
- state 101 deterministically transitions to 011 in the training data, so the probability of that transition becomes higher (79%). Not near 100% because we only did 50 steps of optimization.
- state 111 goes to 111 and 110 with 50% probability each, which the model almost learns (45%, 55%).
- states like 000 are never encountered during training, but have relatively sharp transition probabilities, e.g. 73% of going to 001. This is a consequence of inductive biases in the Transformer. One might imagine wanting this to be 50%, except in a real deployment almost every input sequence is unique, not present in the training data verbatim.

Not really sure where I was going with this :D, I think it's interesting to train/study tiny GPTs because it becomes tractable to visualize and get an intuitive sense of the entire dynamical system. Play with here: https://t.co/8jdceMLpqy

[OpenAI @OpenAI]

Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment: https://t.co/TwLFssyALF

[Australian Academy of Science @Science_Academy]

What if you actually could build Rome in a day? Prof Richard Hartley is one of the founders of Multiview Geometry, which establishes the construction of 3D models from sets of images or videos. He receives the Hannan Medal for his pioneering work.

[Adam Kortylewski @AdamKortylewski]

Thrilled that our workshop on "Generative Models for Computer Vision" got accepted to #CVPR2023!

We have a fantastic speaker lineup with whom we will discuss how image synthesis can benefit image analysis.

Stay tuned! :-)

[Stephen Gould @sgould_au]

I'm recruiting for a 1-year postdoc to work at the intersection of object tracking and NeRF models. @ANUComputing

https://t.co/XpbBbsFH0u

Applications close 23 October 2022. Ideal candidate will preferably be able to commence by January 2023. Contact me if you're interested.