Emre Cir Cassian λ

Şim vûtesû hâ gocâgâre.
(On horseback, a dog won't bite.)

[Emre Cir Cassian λ @classF2RE]

IT IS TIME TO GIVE BACK OPEN SOURCE!

Keeping it short. Here is what this repo includes:

- Better 2, 3, and 4-bit quantization formats for llama.cpp (RQ2, RQ3, and RQ4), all utilizing WHT transformation.

- Speed-optimized fused kernels for each quantization format.

- NVFP4 support for non-Blackwell GPUs. Prefill is slightly slower, but decode is faster than Q4 and RQ4! (Integrated and bound existing llama.cpp code to make it fully usable).

- Experimental RQFP4 format (combining NVFP4 + RQ4). If it passes the final tests, this will be the best format in the repo.

- Fully modular design! Standard formats like ⁠RQ2_K_L⁠, ⁠RQ3_K_L⁠, and ⁠RQ4_K_L⁠ (following Q-series schemes) are ready to go out of the box. However, since quantization is highly model-specific, this modularity lets you mix and match to build the absolute best combination for your specific model.

- Because I don't have much computing power, the tests take a long time. Therefore, I performed simple tests comparing them to the standard quants in llama.cpp.

-I've done a lot of testing to make sure everything is working. But if you think there's anything not working or missing, please let me know.

-For now it is CUDA only repo. Because I do not have @AMD hardware. I have an M1 Mac. I'll port it to Macs when I have time.

[Emre Cir Cassian λ @classF2RE]

@sunsetsyntax Furthermore, the quantization process is largely model-specific; you need to carefully tune the quantization rates for the layers. However, RQFP4 combines WHT and NVFP4. Mathematically, it outperforms Q4_K and NVFP4, though it is slightly slower than RQ4.

[Emre Cir Cassian λ @classF2RE]

Updated !

-It is even now with main llama.cpp repo
- Speed gains for RQ quants
- Optimization for RQFP4

[Emre Cir Cassian λ @classF2RE]

IT IS TIME TO GIVE BACK OPEN SOURCE!

Keeping it short. Here is what this repo includes:

- Better 2, 3, and 4-bit quantization formats for llama.cpp (RQ2, RQ3, and RQ4), all utilizing WHT transformation.

- Speed-optimized fused kernels for each quantization format.

- NVFP4 support for non-Blackwell GPUs. Prefill is slightly slower, but decode is faster than Q4 and RQ4! (Integrated and bound existing llama.cpp code to make it fully usable).

- Experimental RQFP4 format (combining NVFP4 + RQ4). If it passes the final tests, this will be the best format in the repo.

- Fully modular design! Standard formats like ⁠RQ2_K_L⁠, ⁠RQ3_K_L⁠, and ⁠RQ4_K_L⁠ (following Q-series schemes) are ready to go out of the box. However, since quantization is highly model-specific, this modularity lets you mix and match to build the absolute best combination for your specific model.

- Because I don't have much computing power, the tests take a long time. Therefore, I performed simple tests comparing them to the standard quants in llama.cpp.

-I've done a lot of testing to make sure everything is working. But if you think there's anything not working or missing, please let me know.

-For now it is CUDA only repo. Because I do not have @AMD hardware. I have an M1 Mac. I'll port it to Macs when I have time.

[Emre Cir Cassian λ @classF2RE]

@sunsetsyntax In my opinion, perplexity is not a very reliable metric for measuring the quality of quantization; during development, I sometimes achieved PPL values ​​lower than those of the original model, yet the model noticeably began to become "dumber."

[Emre Cir Cassian λ @classF2RE]

@sunsetsyntax I conducted extensive research on quantization while developing this project.

[Emre Cir Cassian λ @classF2RE]

@spiritbuun Perhaps RQFP4 should be tried.

https://t.co/nvpq9KjvcT

[Emre Cir Cassian λ @classF2RE]

What were you saying? Were you very happy about NVIDIA acquiring Hugging Face?

[Anthony Ronning @anthonyronning]

Okay to torrents we go. Hugging face had its moment.

[Yifan Zhang @yifanzhang_]

We are at the dawn of Superintelligence.

Introducing the Recurrent Looped Transformer (RLT),

We now have Transformers with Infinite Reasoning depth.

From now on, we should pace progress at the Open Frontier of Superintelligence,

Until Safe Superintelligence is achieved.

https://t.co/yMWIWU4upo

[Emre Cir Cassian λ @classF2RE]

I haven't been able to fully complete this yet because I've had to attend to some civil engineering work. I wanted to quantize and release a few FLASH models. In particular, the RQFP4 quantization method is a good one that offers higher quality than NVFP4. However, I specifically need to optimize the prefill speed. I hope to find time for this as soon as possible. The repository already features optimized implementations of the RQ2, RQ3, and RQ4 quantization methods for 2, 3, and 4-bit precision; you can use these to quantize large models while maintaining high quality and run them locally.

[Emre Cir Cassian λ @classF2RE]

IT IS TIME TO GIVE BACK OPEN SOURCE!

Keeping it short. Here is what this repo includes:

- Better 2, 3, and 4-bit quantization formats for llama.cpp (RQ2, RQ3, and RQ4), all utilizing WHT transformation.

- Speed-optimized fused kernels for each quantization format.

- NVFP4 support for non-Blackwell GPUs. Prefill is slightly slower, but decode is faster than Q4 and RQ4! (Integrated and bound existing llama.cpp code to make it fully usable).

- Experimental RQFP4 format (combining NVFP4 + RQ4). If it passes the final tests, this will be the best format in the repo.

- Fully modular design! Standard formats like ⁠RQ2_K_L⁠, ⁠RQ3_K_L⁠, and ⁠RQ4_K_L⁠ (following Q-series schemes) are ready to go out of the box. However, since quantization is highly model-specific, this modularity lets you mix and match to build the absolute best combination for your specific model.

- Because I don't have much computing power, the tests take a long time. Therefore, I performed simple tests comparing them to the standard quants in llama.cpp.

-I've done a lot of testing to make sure everything is working. But if you think there's anything not working or missing, please let me know.

-For now it is CUDA only repo. Because I do not have @AMD hardware. I have an M1 Mac. I'll port it to Macs when I have time.

[Emre Cir Cassian λ @classF2RE]

DeepSeek 4.1 Flash is an excellent model, but it holds no real significance for local AI. You could somehow fit the previous model, DSV4F 0731, onto your local hardware. If anyone argues otherwise, it implies that when they think of "local AI," they are envisioning companies hosting it locally, not individual users. I do not subscribe to that perspective.

[DeepSeek @deepseek_ai]

🌐 Supporting open source. Expanding deployment options.

We’ll work closely with the open-source community on V4.1-Flash inference support and explore more deployment options.
Planning a large-scale deployment with 2,000 GPUs + a storage cluster? Let’s talk.

🔹 Model: https://t.co/rv2G2TN66a
🔹 Paper: https://t.co/FIECPM1SSV

6/6

[Emre Cir Cassian λ @classF2RE]

I didn't notice that the parameter size has doubled approximately. Very good progress. Smaller in size, better than the Pro model and many other models. But the naming is a bit strange. It's a new architecture and a new size compared to the previous FLASH model, but why 4.1? This naming seems like a very small change, but it's not.

[Emre Cir Cassian λ @classF2RE]

🤯 Deep Seek moment never ends...

[DeepSeek @deepseek_ai]

🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient.

🔹 Introducing the smallest model in our new architecture family, with native visual understanding.
🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models.

1/6

[Emre Cir Cassian λ @classF2RE]

https://t.co/OhKv6fOWSy

[Emre Cir Cassian λ @classF2RE]

So many good things to try! And they say "We are not in the singularity."

[Emre Cir Cassian λ @classF2RE]

@paulabartabajo_ "Education without values, as useful as it is, seems rather to make man a more clever devil.”

[Emre Cir Cassian λ @classF2RE]

@grok @elnathan_john @grok What exactly did the note say?

[Emre Cir Cassian λ @classF2RE]

Dear researchers and scientists,

Are you foolish enough to entrust your valuable data to companies like ClosedAI, Anthropic, Google, and X, hoping it won't be stolen? This situation scientifically proves that it is possible to be both a capable researcher or scientist and a fool at the same time.

[OpenAI @OpenAI]

We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.

The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.

The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.

[Emre Cir Cassian λ @classF2RE]

Adamın burada bahsettiği "Kendi kendini geliştirip, iyileştiren" Yapay Zeka modelleri. Yoksa standart bir LLM modelinin herhangi bir şey yapmak istediği yok. Ancak ona çeşitli kabiliyetler verip , kendi kendini geliştirebileceği bir yapıya büründürürseniz insan döngüden çıkıyor. Bu sefer anlamadığınız çıktılar üreten ve anlamadığınız şeyler yapan yapay zeka sistemleri ile karşılaşıyorsunuz. Bu durum son dönem yapay zeka modellerinde de bariz. Başkalarına ait büyük modellerden "damıtma" yolu ile eğitilen modellerin size verdiği cevapları bile anlamakta güçlük çekiyorsunuz. Ama kodlama vs gibi yaptıkları işler çalışıyor ve iyileşiyor. Sadece insan dilinden daha fazla uzaklaşıyor. Verdiği kararlardaki mantık belirsizleşiyor. Astra modelinin eğitiminde kullanılan yapay sinir ağlarında kendi içinde döngü yapan ve düşünme sürecini iyileştiren teknik de bir nevi aynı anlaşılmaz çıktılar üretiyor. Yapay sinir ağlarından sayılar çıkıp tekrar yapay sinir ağlarına giriyor. Düşünme süreci iyileşiyor ama hangi çıktılarla iyileştiği bilinmiyor. Özet olarak teknolojiyi anlamadan yorum yapmamak lazım.

[Emre Cir Cassian λ @classF2RE]

@SiliconForested @grok is it native or emulation?