Tor Tor

[Tor Tor @tor1457174]

@mitchellhynes What the f are you talking about -- Gemini Flash 2.5 caught up to Sonnet 3.5 two years ago🤣

[Tor Tor @tor1457174]

@marcthiessen 你以为她真傻啊 🤣

[Tor Tor @tor1457174]

@feifei_qiu @Google What is wrong with simply enjoying your life?
You only live once, and the world will keep spinning long after you are gone.

[Tor Tor @tor1457174]

@hypergameme To run a crappy local Chinese model
Like qwen 3.8 flash next at mxfp4
Which surpasses 95% human coders
You don't need 100k
6k is enough in today's price
-- buy 2 amd r9700, 64g ddr5, 512g nvme, done.

[Tor Tor @tor1457174]

@mweinbach tcclaviger/Qwen3.8-Flash-Next-MXFP4-FP8-GPTQ · Hugging Face https://t.co/Yo47c09jmn

This one is the cheapest hw setup you can get:
> 100 tps decode for batch 1,

[Tor Tor @tor1457174]

@docmilanfar Wild cat survived the wild world

[Tor Tor @tor1457174]

@talphiel Man, I am from communism China.
I am not worshipping anything
You don't need to be religious to know
That human are limited, transient existence--that only make me be more appreciative of this short life.

[Tor Tor @tor1457174]

@satellitedown Gavin Newsom would require you to have a FBI background check to buy 5090 soon.

[Tor Tor @tor1457174]

@TechMDAI Are you going to start your own cloud company?

[Tor Tor @tor1457174]

@DaveShapi What do you mean 'solve' aging?
That sounds arrogant.
We are mortal creatures,
Aging, death is a designed feature,
Not a bug.

[Tor Tor @tor1457174]

@nikitabier Just name one SW 'product' comes from this paradigm -- I'd say 'NONE'

[Tor Tor @tor1457174]

@yetone 变成我儿子的游戏主机

[Tor Tor @tor1457174]

@anah_sahh Why not?
This way I can always just share the data I want to Claude

[Tor Tor @tor1457174]

Why not AMD?
-- a single RTX 6000 (96GB) can get you about 8 AMD Pro R9700 (32 x 8 = 256GB), with aggregated VRAM bandwidth 640x8 = 5TB/s, vs 1.8 TB/s, 766 x 8 = 6 PFLOPS at FP8, vs 2PFLOPS at FP4.
-- physics is physics, man, people get https://t.co/ONutA0t4u3 to run on just 2 R9700, with > 100 tps batch = 1 decoding, > 3000 tps prefill
--

[Tor Tor @tor1457174]

Decode:

podman exec -e OPENAI_API_KEY="$VLLM_API_KEY" qwen38_vllm \ vllm bench serve --backend openai --base-url http://127.0.0.1:8090 \ --endpoint /v1/completions --model Qwen3.8-Flash-Next --tokenizer /app/models \ --dataset-name random --random-input-len 131072 --random-output-len 512 \ --num-prompts 2 --max-concurrency 2 --num-warmups 1 \ --ignore-eos --temperature 0 --seed "$RANDOM" \ --percentile-metrics ttft,tpot,itl

# ============ Serving Benchmark Result ============ # Successful requests: 2 # Failed requests: 0 # Maximum request concurrency: 2 # Benchmark duration (s): 68.45 # Total input tokens: 262144 # Total generated tokens: 645 # Request throughput (req/s): 0.03 # Output token throughput (tok/s): 9.42 # Peak output token throughput (tok/s): 66.00 # Peak concurrent requests: 2.00 # Total token throughput (tok/s): 3839.15 # ---------------Time to First Token---------------- # Mean TTFT (ms): 45169.63 # Median TTFT (ms): 45169.63 # P99 TTFT (ms): 60350.37 # -----Time per Output Token (excl. 1st token)------ # Mean TPOT (ms): 15.27 # Median TPOT (ms): 15.27 # P99 TPOT (ms): 15.29 # ---------------Inter-token Latency---------------- # Mean ITL (ms): 15.37 # Median ITL (ms): 15.17 # P99 ITL (ms): 19.09 # ==================================================

[Tor Tor @tor1457174]

Got this Qwen 3.8 Flash Next running on my dual AMD R9700 machine
peak 21,000 😨😨😨😨tps prefill speed
stead 52 tps decode speed (no MTP, because I only have 32GB memory 😂)
-- beat OPUS 5 on a long context knowledge retrieving task (Opus gave Qwen 89, Qwen gave Opus 64 😂)

-- buddy, if you have 5K, buy this instead of paying $200 a month!

[Tor Tor @tor1457174]

prefill performance from standard vLLM perf test:

for LEN in 4096 16384 65536 131072 200000; do echo "=== input_len=$LEN" podman exec -e OPENAI_API_KEY="$VLLM_API_KEY" qwen38_vllm \ vllm bench serve --backend openai --base-url http://127.0.0.1:8090 \ --endpoint /v1/completions --model Qwen3.8-Flash-Next --tokenizer /app/models \ --dataset-name random --random-input-len "$LEN" --random-output-len 4 \ --num-prompts 2 --max-concurrency 1 --num-warmups 1 \ --ignore-eos --temperature 0 --seed "$RANDOM" \ --percentile-metrics ttft,e2el 2>/dev/null | grep -E "Median TTFT|Total input" done

# === input_len=4096 # Total input tokens: 8192 # Median TTFT (ms): 890.31 # === input_len=16384 # Total input tokens: 32768 # Median TTFT (ms): 2038.54 # === input_len=65536 # Total input tokens: 131072 # Median TTFT (ms): 7408.44 # === input_len=131072 # Total input tokens: 262144 # Median TTFT (ms): 15432.40 # === input_len=200000 # Total input tokens: 400000 # Median TTFT (ms): 24045.92

[Tor Tor @tor1457174]

@Fei2411 刚刚和Grok聊了一个中文帖子中的技术论题
我倒，全是中文那些互联网公司的黑话
我说你不会讲正常的中文吗？还是这就是现在的正常的中文？😨😨😨
-- 只好让它讲英文，冰冷的机器感立马回来了，这下感觉舒服多了。🤣

[Tor Tor @tor1457174]

Added 32GB DDR to my machine ($480, 😰😰😰😫) ， now the machine has 64GB DDR5, running at 5600 MT per s

Enabled MTP, vision, 4 concurrent request at full 256k context.

23.5k tps peak prefill speed
92 tps decode speed.

Now the whole machine need about $6000 for a full new build, I think 🤔

-- but I almost have my own data center grade server now 😁🤭🤗

[Tor Tor @tor1457174]

@vineerpasam Free/open is the only way non-frontier can gain customers -- also all AI company need real world client data.

[Tor Tor @tor1457174]

@0xSero $5000 setup (dual amd r9700)
Qwen 3.8 flash next at mxfp4
Got prefill peak at 21k tps, decode 52 tps at 128k context
Beat opus 5 on my private benchmark
-- if only I can afford another 64gb ram, I think the decode speed can easily be x3, but that's another $1200 ? 😂