@0xSero +1
That’s why I started distillforge, an LLM proxy to automatize that transparently
https://t.co/iVnxoPioQF
Work in progress, feedback welcomed
@julien_c qu'est que tu penses d'un service de proxy LLM qui distille automatiquement un model frugal pour chaque tache a partir des traces initiales sur le lllm frontier ?
Seamless integration & host HF https://t.co/6nHuncyaLL
@Dorialexander I started that poc, distillforge
https://t.co/iVnxoPioQF
Add a transparent http proxy to use your expensive LLM endpoints.
It will collect trace, generate synthetic results, train a local small model, and switch on it if performance are good enough.
It’s must be fully auto
@babgi Vendredi soir, j’ai effectué le portable d’un logiciel scientifique de modélisation agricole de 25 ans avec 2 millions de lignes de code et la je termine une plateforme de montage vidéo automatique.
Nous ne sommes pas prêt
La version française https://t.co/5kOUZgHBUq
@lemire I can confirm that, as far back as 15 years ago, I recompiled the Debian package from source code for the BLAS and LAPACK libraries, so that I could also use the grid search optimiser on L1/2/3 levels.
And when you deploy your application on a 2,000-node cluster, it makes a gap
@morganlinton@Brooooook_lyn It’s never enough fast :)
Async tasks like a claw agent (with limited context 8k) etc yes
Software dev not really
Try but keep in mind that almost of your RAM is used