leo

local AI field notes @ heavy industries - AI lab @ inference @ HP (opinions my own)

[leo @leonardoeloy]

I KNOW MY PROFILE PIC LOOKS LINKEDINISH. KTHXBIE.

[leo @leonardoeloy]

@yacineMTB "While drunk, your boyfriend trying calling his ex. We're not determining malicious cause or ill intent."

[leo @leonardoeloy]

@OwnerEmeritus @cohere Idk why, that has been demonstrated publicly already. We’re in for a rough ride

[Cohere @cohere]

While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.

Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.

Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.

But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?

We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.

Read the full report: https://t.co/TZSugXi2ZE

[leo @leonardoeloy]

@Duderichy That was actually express service tbh

[leo @leonardoeloy]

@imdariotoo @KyleHessling1 Bad news: it doesn’t. At 32 VRAM, even though you apply all the model card says, it will reason for 30 minutes before an answer. I ran a parallel test with Qwen3.8-27B, and it finishes quick and provides the correct answer. Using a Nanbeige4.3-33B-Q4_K_M equivalent.

[leo @leonardoeloy]

@FlashVMM Very good point. I’ve been away from cybersecurity and lost touch with the MO of most exploits.

Maybe I need to open source that thing, too.

[leo @leonardoeloy]

Advantages of harness autism: I have an agent that scrapes *places* (we can't publicly say) for breached dependencies.

I have another agent that runs a honeypot MicroVM for every single dependency install before whitelisting it.

[leo @leonardoeloy]

You can do a lot just by checking if certain files were accessed, bogus DNS resolve calls, or regular TCP/UDP sockets being used in weird ways.

[leo @leonardoeloy]

Is this their AI moment?

[Scotch McClure @scotchmcclure]

We are developing a drug that opens up the capillaries and fills the body’s tissues with a compound that - according to our partner military labs - destroys all known pathogenic bacteria, all known pathogenic fungi, biofilms, influenza virus as well as viruses of the herpes family.

These are facts.

The FDA recently designed this product a Qualified Infectious Disease Product (QIDP) for all cause pneumonia, giving us FDA Fast Track, and Priority Review.

Good news for the day. ❤️‍🩹🫡

[leo @leonardoeloy]

@KyleHessling1 Update: at Q4 on 32GB VRAM and unlimited thinking budget, it will keep going on and on; so, highly unusable. This model requires 64GB.

[leo @leonardoeloy]

@KyleHessling1 Testing now. So far, Qwen still wins. Both at Q4.

[leo @leonardoeloy]

@ScalaWilliam @bnjmn_marie The problem with Q8_K/Q4_V with all that unlimited thinking the model card suggests is that it just takes A LONG TIME to spit out the answer. Oddly enough, it often does so only in the reasoning stream, but keeps iterating over it.

My verdict is 64GB land for now.

[leo @leonardoeloy]

“But, Leo, if your SaaS is not fully automated, blablabla”

Bro, you have no clue the levels of hand holding at enterprise software that runs the world.

[leo @leonardoeloy]

Support is still the moat 👇

If your SaaS is frantic about CX, you’ll win, bro.

[Fred @hustle_fred]

4 months ago I cloned Airtable for a client in 5 days because they refused to pay Airtable $500/mo

months later, I’m still maintaining it

every two weeks it’s another edge case, a UI tweak, or a "quick 10-minute request." They’ve easily spent 10x more in headaches than the $2,000 they would’ve paid Airtable

just proving that SaaS isn't dead. Cloning isn't enough, if you want it to be same quality as original it becomes a full time job

and at this point isn't it easier to just pay?

[leo @leonardoeloy]

Let me correct myself: the moat in software is still support. I still think the old saying “nobody was fired for buying IBM” holds true for any >= 9-digit operation.

[leo @leonardoeloy]

@RoliumGens Watching bro Cook 🧑‍🍳

[leo @leonardoeloy]

in software. “There’s no moat” in software.

Try getting a SKU to a consumer or enterprise. At scale. Globally.

[jack @jack]

there is no moat

[leo @leonardoeloy]

I’d appreciate if @nanbeige can release some inference recipes, I’m not able to get it up and running as the comp spec sheet claims. Running at Q8 and Q4

[leo @leonardoeloy]

@KyleHessling1 Preliminary results: Q4 bad for keys, bumping up to Q8. Reasoning budget is a must and top P at a .95 minimum. Progressing here. Without reasoning this thing is a beast with tool usage.

[leo @leonardoeloy]

@0xSero I’m testing it now, requires a lot of effort to keep it under control at Q4. Trying Q8 K and Q4 V to see where it goes. The main key is a reasoning budget and top P at .95 at least.