not tango uniform

veteran and polyglot

[not tango uniform @tangouninformed]

@LLMJunky This is good info to level set & at least it's qualified w/ real price. Most local LLM posters on X never talk price. They say "if you **just** have" ... quad RTX 6000's, or quad/dual DGX Sparks, a 512/256GB Mac, etc. When most real people are looking for sub $50/mo solutions.

[not tango uniform @tangouninformed]

@mattpocockuk Isn't calling this the "system prompt" a misnomer? The system prompts are published https://t.co/6JCpdaD8Jg and this is not changing them. I think the idea of turning off unused features that add to context is good. But this is in user-land prompt, not system.

[not tango uniform @tangouninformed]

@choephix @mattpocockuk Anthropic's documented system prompts are generally around 20k characters in total. They publish them: https://t.co/6JCpdaD8Jg

[not tango uniform @tangouninformed]

@gtupak @0xSero Wow, everyone talks about how wonderful Qwen3.6-35B-A3B performs at Q8 so that's a new data point. It's difficult to compare information on X b/c user1 is writing their English paper & user2 is working a 100k LoC TypeScript repo. The local LLM space is very tough to guage.

[not tango uniform @tangouninformed]

@gtupak @0xSero I have the same issues with Q4_K_XL. I was told that a better quant was needed for “high context” (I was using 115k) w/ tool calling b/c the accuracy at Q4 was not high enough for reliability. But I have 4080 16gb so Q6 or Q8 is dicey. Not sure what to test next.

[not tango uniform @tangouninformed]

@MiaAI_lab @sachindetrax @UnslothAI @NVIDIAAI Do you have a recommendation for a small model for Linux, 4080 Super 16GB VRAM, specifically for Python agentic coding w/ tool calling? I’ve tried the usual Qwen 35b MoE Q4 & the accuracy isn’t there. Someone said move to smaller but Q6 or Q8. Do you have a recommendation?

[ThePrimeagen @ThePrimeagen]

i still think about this tweet a lot

the rent is so free at this point

[Kyle Hessling @KyleHessling1]

Good morning y'all!

Qwopus-3.6-35B-A3B-MTP-Coder is live! All GGUF's will be populating over the next few hours!

It's a lightning-fast MOE with the coder curriculum recipe. Similar to the 27B coder, it shines with thinking disabled, offering significantly faster wall time for similar, and in some cases superior results to same-sized thinking alternatives! With thinking disabled, it goes toe-to-toe with the new Ornith 35B MoE across a huge eval suite (performed by @no_stp_on_snek), edging it on the coding trajectories and decisively on speed and cost, even though Ornith was run with thinking enabled.

See the model card for the full test results, and shoutout to Tom, @no_stp_on_snek, for thoroughly evaluating the model for us before launch!

With MTP and thinking disabled, along with the MOE speed, it runs so quickly in harnesses like @opencode that it almost feels instant @ 253 tps on my 5090.

No 8k tokens of thinking before a coherent output is actioned. This is especially useful in long contexts, where the base models will progressively start thinking for tens of thousands of tokens before replying.

Compared to the base models with thinking off, the coder curriculum really advances the no-think frontier. Especially in terms of how creative it can be. Run temp hot as usual, 0.85-1, and make sure your harness isn't overriding the temp setting of your server at runtime.

If you want to use it to its full ability, I would recommend giving it very thorough prompts. I have been using it in opencode, and I have been blown away by the results it generates autonomously with chunky prompts. Please see links to the demo's Aether Dominion (RTS Game), and a slide deck presentation the model made about itself that turned out beautifully, links in comments below!

I am getting results on this incredibly fast local model (with thinking disabled) that I couldn't get in some thinking frontier models over a year ago.

Open source is accelerating fast, and in light of recent events, there's never been a better time to get your local AI workflows tightened up. This MOE would be a great one to play with, and it's also a great one if you don't have much VRAM because it can run fast offloaded partially to system memory!

All of that said, please give it a run with thinking off and build something you'd like to see. We'd love to see your results and any feedback on specific use cases in the comments below!

Also, thanks so much for 5k followers, you all make up such an enjoyable and knowledgeable open source community, and I am so blessed to be able to collaborate and discuss this research with all of you. I can't express how grateful I am for every comment. As always, I will try to reply to them all!

If we ever get monetized on X, I will put every penny into buying more hardware for our lab!

Have a blessed day, my friends, looking forward to your thoughts!

https://t.co/0WkjglsaWS

[herdr @herdrdev]

herdr -- remote user@host

one command, nothing else.

https://t.co/eX6o7abq3n

[not tango uniform @tangouninformed]

@DanielMiessler I'm a mixture of both. I think the *business* of AI is a bubble created by human greed & scummy finance practices. However, I think the tech has value, & learning its good & bad is important. Decision making requires knowledge & experience. Improvise, Adapt & Overcome!

[not tango uniform @tangouninformed]

@UnslothAI I don't have one. I have 16G VRAM & want agentic with tool calling but they fail. @UnslothAI qwen3.6-35B-A3B Q4_K_XL is working at OK speed with 125k context but it fails on real work. Guessing I need a better quant with a smaller model?? Suggestions welcome.

[John @nullc0py]

@MTSlive Rules for thee, but not for me

[not tango uniform @tangouninformed]

@johndeanl Are folks glossing over “Once you do all of the under the hood engineering work to make this "just work" (e.g. across tools, integrations, compute environments, memory, security, etc.)”? IF you already have done 99% of the engineering, is a huge IF.

[Ayaan 🐧 @twtayaan]

Apple just made Docker Desktop optional on Mac.

And it is completely free.

This is apple/container. 26.5k stars no Github.

You can now run Linux containers natively on your Mac without installing Docker Desktop, without a background daemon hogging your RAM, and without paying $21 a month per developer for a commercial license.

Here is what it does:

→ Runs Linux containers as lightweight VMs directly on Apple Silicon using macOS 26 virtualization
→ Fully OCI compatible. Pull any image from Docker Hub, GitHub Container Registry or anywhere else
→ Written in Swift and optimised specifically for Apple Silicon. Faster and lighter than anything Docker Desktop does on Mac
→ Standard container CLI syntax. If you know Docker commands you already know how to use this
→ Push images you build to any standard container registry and run them anywhere

Docker Desktop charges $21 per developer per month for commercial use. Apple's version costs nothing and ships as open source under Apache-2.0.

Microsoft made Docker Desktop optional on Windows with WSL Containers last month.

Apple just did the same on Mac.

Docker is not going anywhere. But the era of paying for a GUI wrapper around containers on your own machine is quietly ending.

Repo here: https://t.co/uFJ867sul6

[mRr3b00t @UK_Daniel_Card]

the most important thing I think in this game is to be able to say, sorry i was wrong.

the number of people who seem unable to do this is mad.

i get things wrong, i change my mind, I sometimes fuck up.... we all do.

[BlackRoomSec @blackroomsec]

So, it should worry you, but allow me to ease your mind, some. I have never worked for the NSA but I have worked for 10 major corporations, one of which was Honeywell and GE, both of whom did and do work with the federal government.

I have been cybersecurity Director for 9 DNA forensic labs which did work with the DoD.

I am an expert in cybersecurity compliance (NIST, CJIS, ISO, to name just a few and there are hundreds) and as far as my hacking ability goes my disclipines (read: strengths, we call them disciplines) are:

Stack smashing/Memory exploitation
Reverse engineering hardware/software
Privilege Escalation
Malware creation/modification/reversing
Social Engineering (physical and programmatically)
to name just a few.

I have over three decades experience and have done every job role in IT and Cybersecurity up to CISO (my last role) I now advise one of the most prominent DA's here in New York at the county level as their lead cybersecurity expert.

With that said, no system is 100% secure or foolproof. So we do what is called defense-in-depth to ensure that we protect as much as we can, the best that we can, with what we have at our disposal.

People controls are part of that.

The NSA has very strict guidelines, protocols, policies and procedures about who can access their systems and how, internally.

None of their core network systems are exposed to the Internet.

Most of them are not connected to each other.

Lateral movement is when an attacker breaks into one system, escalates their privileges and then moves to other systems within the same network, in order to gain access to them and collect more loot (files, information, etc).

An air gapped system is NOT accessible because it is disconnected. There's nowhere to move laterally TO.

An attacker cannot just waltz into the NSA, sit down at any of their systems and start running commands or dropping payloads.

If they did, not one but ALL defensive measures have failed and this is just not going to happen.

The staff of the NSA have to use multiple factors to access these systems and they work there.

They are monitored continuously by thousands of staff.

Cameras everywhere.

Armed guards, the whole nine.

This was one closed test by people with the appropriate access in order to conduct it.

That does NOT mean this particular test is possible from a real-world perspective and that is why we are calling it out. And, we will continue to.

If the NSA was *this* insecure, any garden variety hacker would have found the misconfiguration and exploited it years ago.

It isn't.

So, please don't worry about the NSA. What you should do instead is read their excellent hardening guides and cybersecurity guidance on how to protect YOUR systems.

They aren't idiots.

Most of us in this industry know people who work there. They are trained the exact same way we are.

They know what they're doing.

And whomever wrote this article did not do their research, published this BS irresponsibly and should have to publish a mea-culpa retraction.

You can tell them I said that. I'll be happy to repeat it to them and educate them SOUNDLY.

Havce a nice day. Go touch grass. :)

[dax @thdxr]

i have low conviction on model routers - very open to changing my mind but this is a snapshot of my current thoughts

- i don't think it's good to not be aware of what model you're using. coding with LLMs is a skill you develop and getting a feel for models is part of that

- people (at scale) don't have this skill right now which is why a lot of companies are complaining that people are using expensive models for dumb things. a model router promises to solve this without the user having to do anything but i think the issue is missing feedback loops to the user. id rather we figure out how to help users get smarter

- i dont even know how much you can model route when factoring in things like prompt cache. only so much you can do

- their effectiveness is a bit exaggerated by the same dynamic that's impacting everything AI. so many companies desperately searching for opportunities and trying anything. model routing is the one thing models labs cannot do so everyone is jumping on it

[not tango uniform @tangouninformed]

@sudoingX Is a a NVIDIA 3090 better than an Intel Arc Pro B70 at 32Gb VRAM?

[not tango uniform @tangouninformed]

@mvanhorn Please make Part 3 the same kind of detail and reference based article on the setting up the harness with “evaluators” that people are using for real SWE work.