zkstx

[zkstx @zkstx]

@eisokant Consider tagging the updated version slightly differently (e.g. S 2.2 or adding a date tag similar to mistral and deepseek) Right now if I go to any third party API there is essentially no way to tell if I am being served the updated and fixed version or the old one.

[zkstx @zkstx]

@ItsmeAjayKV @rewin123 @poolsideai what batch size do you use?

[zkstx @zkstx]

@ilyakochik @victorstorchan Did you run any of the evals on the quantized ggufs you provide? I saw multiple posts of people reporting disappointing performance. Maybe it just degrades quickly under quantization? Could be a template issue? Unfortunately I don't have enough disk space to check myself atm.

[zkstx @zkstx]

@maxbittker @just_be_dev is the listed deepseek v4 the flash or the pro model?

[zkstx @zkstx]

@LLMJunky @l0gix5 The displayed text is correct, the model you are showing in the video is just the 1.7B model which is why it is so fast. They did release the 27B very recently too but that one will be very slow on a phone.

[zkstx @zkstx]

@matthew_d_green okay, not sure what prompting strategies you tried. maybe filling the context with some related code (that you would approve of) might help. in many cases you can just ask the models to first look up specific information you deem relevant to reduce the risk of hallucination

[zkstx @zkstx]

@mithilyaganti @0xSero *on the domains used to determine what to reap and what to keep. Anything out of domain will take a considerable hit

[zkstx @zkstx]

@ItsmeAjayKV @mr_r0b0t dspark will probably improve speeds a bit, but will take a little while until it's supported in llama.cpp, dflash support was merged just a few days ago

[zkstx @zkstx]

@usr_bin_roygbiv @NosytLabs omp seems like a joke. it's like 150MB for a tui? for what? how can people work with a harness where editing earlier messages is essentially unsupported? If I prompt for something and see the model going off track I want to stop and rephrase the last message

[zkstx @zkstx]

@unilightwf higgs audio v3 might be better in some aspects

[zkstx @zkstx]

@mweinbach @BenGoihman @theo @serenaa_ge https://t.co/HCHIllLJYz

[Junyang Lin @JustinLin610]

A clarification of Qwen3.5 Plus and 397B:

1. for opensource, we follow the tradition to make parameters apparent so we use the name with the number of total parameters and active params.

2. Qwen3-Plus is a hosted API version of 397B. As the model natively supports 256K tokens, Qwen3.5-Plus supports 1M token context length. Additionally it supports search and code interpreter, which you can use on Qwen Chat with Auto mode.

3. To help the opensource community, we also put the original 397B model for testing, which has thinking and fast (non-thinking) modes.

[zkstx @zkstx]

@PengmingWang @teortaxesTex I know it's not really something you can put into the report but I would love to see QAT versions for the GPU poor https://t.co/UEWz4QoY94

[zkstx @zkstx]

@JamesLee1033176 @0xSero And step3.5 flash and mimo V2 flash (probably also v2.5 once it is released) and hy3

[zkstx @zkstx]

@scale_AI I would love to see some more open weight models such as Step 3.5, Qwen 3.5, DS V3.2, Mistral Large 3, Longcat Flash.

[zkstx @zkstx]

@yunus_eee @miniapeur

[zkstx @zkstx]

@cherry_cc12 Thanks for your work! Please consider releasing a version with quantization aware training (QAT), the community doesn't have access to the original training distribution so mismatched replications would generally lead to some performance degradation

[zkstx @zkstx]

@0xSero @softpoo @grok Last time I checked they were asking for a phone number even though I wanted to bring my own models.. Really don't feel like giving out my number when OpenCode, Zed, Crush, Aider, Kilo and alike are a thing and don't ask for one. Did they change this in the meantime?

[zkstx @zkstx]

@x86machine @0xSero What alternative do you suggest?

[zkstx @zkstx]

@0xSero @JohnDoer6ii I also think there would be advantages to adapting the "lightweight teacher-correction fine-tuning phase" / distillation outlined in the Nvidia Minitron paper to REAP. Doing everything (Pruning + Quantization + Distillation) end-to-end would probably be optimal

[zkstx @zkstx]

@0xSero @JohnDoer6ii So for example if you have a ~120B MoE is it better to REAP 50% of experts with a 4 bpw quant (≈30GB parameter weight) or should you rather go for 25% experts REAPed @ 2.67 bpw to get to 30GB of weights?