Toni S

Challenge yourself to explore unfamiliar subjects and concepts regularly

[Toni S @VoltableCom]

Using Qwen 27B probably will be hundrens of tokens.
With Laguna S.21 just fewer. Open source ai models need to win.

[Toni S @VoltableCom]

@Tono_Ken3 Or buy one M2 drive Samsung 9100 Pro 1-4TB 14GB

[Toni S @VoltableCom]

@sflorimm remove the users 😜

[Toni S @VoltableCom]

Only if drops everything and do a fully rewrite from zero.

[Toni S @VoltableCom]

@PenWrangler You want a better packages !? PNY sure -> Pro 6000 + better package = $15000

[Toni S @VoltableCom]

@0xSero Why you choose Asus over Asrock ?

[Toni S @VoltableCom]

I'm flexing that I do not have a Kimi plan yet

[Mia @MiaAI_lab]

People flexing they have a Kimi plan is next level 😁

[Toni S @VoltableCom]

My AI local box:

One RTX Pro 6000 Blackwel. It was under 10K :)
Asrock WRX90 WS Evo
AMD Threadripper 9965WX
224 DDR5 ECC 5600 (2 x 64GB + 1 x 96GB)
HDD 20TB - To keep local models
1 x Samsung 990 2TB
1 x Samsung 9100 4TB

For now I run Nvidia Qwen 3.6 27B NVFP4. Around 130 tps decode usign vllm.

[Toni S @VoltableCom]

Now chat conversations are saved into a Obsidian vault. On the server, if browser doesn't support File Access Api or local if it support it.

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

I integrated web search using SearXNG.
It's working, but I need some tweeks.

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

Good to here we have more options.

Kimi 2.x T, Qwen 2.x T – open source for big/mid-sized corporations. Less accessible for individuals, especially if you can’t afford it.

[Qwen @Alibaba_Qwen]

Qwen3.8 is launching and going open-weight soon!🌐

With a massive 2.4T parameters, this model is continuously evolving. We believe it’s one of the most powerful model available today, compatible to leading frontier AI models , second only to Fable 5.

You don't have to wait to test it. Just now, the Qwen3.8-Max-Preview made its debut on Alibaba’s Token Plan, Qoder, and QoderWork. Be among the very first to try it out.

Can't wait to hear what you build. Stay tuned! 🚀 

Token Plan
international：https://t.co/YRvcGdB9Bv
China：https://t.co/PKMUNwUuRp

[Toni S @VoltableCom]

Next future: save all prompts to some kind of database. Maybe a obsidian vault. Right now promtps and conversations are save in localStorage of my browser.

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

Next step is to integrate in workflow SearXNG to help local ai to have a better answer for me. Work in progress

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

This will help me learning more about local llm

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

I taken the web ui part from llama.cpp and make a version to solve my need of using local ai models.

For now I added 'local ai servers' section.

[Toni S @VoltableCom]

etc/docker# cat daemon.json
{
"log-driver": "local",
"log-opts": {
"max-size": "10m",
"max-file": "3"
}
}

Thanks me later.

[Toni S @VoltableCom]

@thdxr Those from casino get unlimited 'tokens' too ?

[Toni S @VoltableCom]

@LLMJunky I will wait and buy at half the price the rtx 3090 card, if rtx 5080 super came alive

[Toni S @VoltableCom]

@sudoingX 224 GB DDR5 ECC, 96 GB DDR5, 64 GB DDR5, 16 + 16 DDR5, 8 + 8 DDR4 + 8 VRAM + 96 VRAM, hope it will be more

[Toni S @VoltableCom]

Buy a minipc and do all harnesses on minipc and leave LLM stuff on your main computer. So you have all VRAM available for LLM

[Hikari∣LocalLLM⚡ @Hikari_07_jp]

To maximize the VRAM capacity of the RTX RPO 6000, I will be adding an RX 9070XT next month. I plan to have the 9070XT handle all processing except LLM, allowing as much of the expert functions as possible to be loaded into VRAM.