Benjamin Bossan

[Benjamin Bossan @BenjaminBossan]

- PSOFT by https://t.co/Cxr2ktPmHb
- Lily and PEANut by @tmux_1
- AdaMSS by https://t.co/lyNvV68iX2

Find a short description of these PEFT methods and the full release notes here: https://t.co/Afn9lXCs8n

[Benjamin Bossan @BenjaminBossan]

Today, we released PEFT v0.19.0 and it's a big one. Not only did we add 9 new PEFT methods, the release also contains a bunch of improvements to make PEFT more useful. Check the thread for details:

[Benjamin Bossan @BenjaminBossan]

We also added 9 new PEFT methods. There is not enough room here to describe them in detail, but here is a list:

- GraLoRA by https://t.co/hx9vq0Bv7F
- BD-LoRA by https://t.co/GnH5wm7bfC
- Cartridges and TinyLoRA by @krasul
- PVeRA by @leofillioux

[LeRobot @LeRobotHF]

We just merged support for Parameter Efficient Fine-Tuning (🤗PEFT) in LeRobot.

This means that you can now use methods like low-rank adapters (LoRA) for training your pre-trained policies like Pi0 and SmolVLA.

[Benjamin Bossan @BenjaminBossan]

Apart from these new PEFT methods, there have been many improvements and other additions, as well as bug fixes and other changes. The full release notes can be be found here: https://t.co/VwnOFy1t99

To give it a try, install the latest version with:
python -m pip install -U peft

[Benjamin Bossan @BenjaminBossan]

PEFT has reached 20k GitHub stars ⭐⭐⭐⭐⭐

Right on time for the new PEFT release, v0.18.0. And it's not a small one at that, with lots of new PEFT methods and other improvements. Details in the 🧵

[Benjamin Bossan @BenjaminBossan]

https://t.co/Q4aQAoR3Wi added Orthogonal Fine-Tuning (OSF). It works by freezing the high-rank subspace of the targeted weight matrices and projecting gradient updates to a low-rank subspace. It is worth checking out on tasks where forgetting previously learned tasks is a concern

[Benjamin Bossan @BenjaminBossan]

If you're interested in using this or collaborating, hit me up. The repo can be found here:
https://t.co/aS7qKfhChi

[Benjamin Bossan @BenjaminBossan]

I was tinkering with a tool that lets AI coding agents query Python code bases with SQL. The idea is that using cat, grep, sed etc. is inefficient. LLMs are proficient with SQL, so they should pick it up easily. Here is an query to find all uses of `tqdm` in the PEFT code base.

[Benjamin Bossan @BenjaminBossan]

As I don't really have the time or knowledge to evaluate this on coding benchmarks involving tool use, I can't say if it really helps or not. In my anecdotal testing, OpenAI Codex can use it with the right instructions, but not consistently.

[Sayak Paul @RisingSayak]

We cover a relatively untouched topic seen in conferences -- "Testing in ML Libraries".

@BenjaminBossan & I poured our learnings from maintaining two widely used libraries (PEFT & Diffusers) while working on the presentation.

Below is the overview:

* Revisiting the existing topic of tests 🤷
* A bit about ML libraries and their kinds
* Approaching tests for the OSS libraries at 🤗
* Practical concerns and how we address them (with concrete examples)

Benjamin recently had a chance to present this in person at @pydataamsterdam.

I will leave links to the full video and slides in the comments ⬇️

[Benjamin Bossan @BenjaminBossan]

For me, this is a prime use case for coding with LLMs: Building small but specialized tools that you need in the situation. Without AI, I just wouldn't have bothered with writing such an app, especially when it comes to the UI, and the refactor would have been more difficult.

[Benjamin Bossan @BenjaminBossan]

I wanted to reduce the amount of code duplication in 🤗 PEFT. As I didn't find a tool to let me interactively explore code duplication hotspots, I vibe coded my own. End result: a net reduction of 3000 lines of code in PEFT.

[Benjamin Bossan @BenjaminBossan]

Besides duplication, this tool also determines type annotation coverage, finds TODO comments, calculates cyclomatic complexity etc. It is a bit rough around the edges and only supports Python, but can still be very useful. You can find it here:

https://t.co/aS7qKfhChi