Abdul Mateen

Startup Advocate, Java, Python, Self-Improvement Expert.

[Abdul Mateen @axmateen]

@WescheNex1q Prefill of 52tok/s is quite slow no?

[Abdul Mateen @axmateen]

@loktar00 You will not regret it, but CUDA 12.8 is quite old, the only thing it lacks is the support from Nvidia.

[Abdul Mateen @axmateen]

@0xSero Do you have any benchmarks on B60 Arc 24/24?

[Abdul Mateen @axmateen]

@VadimStrizheus Anthropic is for enterprises with huge token budgets not individual users...

[Abdul Mateen @axmateen]

@0xSero I felt the same GLM 5.3 Flash is still better than DSv4.1 which I think released in bit of hurry

[Abdul Mateen @axmateen]

@spiritbuun What is prefill or prompt processing?

[Abdul Mateen @axmateen]

@bountyAIhunter DDR5 is around $1033USD alone for servers so it is still a lot of capax

[Abdul Mateen @axmateen]

@ARNOfficiall Is 16% CAGR is real?

[Abdul Mateen @axmateen]

@blackwellboy @UnslothAI V100 32 GB Qwen3.6 35bA3B MoE

[Abdul Mateen @axmateen]

@JawadFaroo94817 What reason behind AGP?

[Abdul Mateen @axmateen]

@MrAhmadAwais Although legacy, I am running V100s on Dell R730 serving with llama.cpp, Since they are like old data centers with DDR4, configured very deep numa configurations so that CPU-Memory-GPU should not cross QPI path. It has been working perfectly for smaller Q4 models like Qwen 3.6.

[Abdul Mateen @axmateen]

@0xSero Have you tried Qwen 3.6?

[Abdul Mateen @axmateen]

@0xSero Which hardware?

[Abdul Mateen @axmateen]

@Alibaba_Qwen Qwen3.6 35B MoE is much better in speed and accuracy than this model I tried both today.

[Abdul Mateen @axmateen]

@TheGeorgePu Try Alibaba?

[Abdul Mateen @axmateen]

@WuMinghao_nlp I would be great if there are like MoE models in 50b range.

[Abdul Mateen @axmateen]

@iotcoi vLLM is not designed to give single user latency rather it is designed for better concurrency. Ampere is not old that would be suprising if the sams results cane with Volta ( v100s ).

[Abdul Mateen @axmateen]

@dealignai Is this only Mac Users OR dedicated GPU users as well?

[Abdul Mateen @axmateen]

@SenSanders This did'nt answer regarding the first question i.e Job impact and impact on political process?

[Abdul Mateen @axmateen]

@Amank1412 How can this be possible?