Patrick Devine

Software Guy @ Ollama

[Patrick Devine @pdev110]

For your pandemic Friday viewing enjoyment. 'kubectl run -it --rm --image=https://t.co/y1ra99OfKZ tif' or for the docker inclined 'docker run -it --rm https://t.co/y1ra99OfKZ'

[Patrick Devine @pdev110]

@martin_casado

[Patrick Devine @pdev110]

@rexsalisbury Keep the train yard but put a lid over it and then put homes/stores over the top of that and use the proceeds to pay for the tunnel to the transbay terminal, amiright?

[ollama @ollama]

.@GoogleDeepMind EmbeddingGemma 2 is now available on Ollama.

Made for consumer devices and is multimodal.

ollama pull embeddinggemma-2

[Patrick Devine @pdev110]

@ronaldmannak Ich brauche die vegane Version davon in meinem Leben.

[Patrick Devine @pdev110]

@Meffysto69 @mchiang0610 We defaulted to the 8bit models, so Nimble is about 9.5GB and Tev1 is between 4.5GB and 800MB. There is a 4bit quantization for Nimble that is about half the size though, but it loses some quality.

[ollama @ollama]

Ollama now supports Jev-like decision models all locally.

Use decision models like Nimble for tasks like ticket triaging, model routing, and content moderation.

ollama pull nimble

Here’s Nimble playing Ollama racer through the new local /v1/systemone API by making decisions in real-time. 🏎️

[Patrick Devine @pdev110]

@mustafajamil My wife told me about someone the other day going off on oxford commas. It's just wild out there.

[Patrick Devine @pdev110]

@martin_casado I mean, someone's gotta hack the Gibson, right?

[Patrick Devine @pdev110]

@farquaadguy @ddalcu Yes, I saw the post, but I also ran some benchmarks on Qwen3.8. It's much closer than what this graph shows, even at each of those prompt sizes.

[Patrick Devine @pdev110]

It's very fast, but it's not comfortably 2x faster. The posted benchmarks for Ollama used the llama.cpp backend and not the MLX backend, and it had spec-decoding turned off. In reality I think it's closer to around 10% on prefill and a bit faster than that on decode.

[Patrick Devine @pdev110]

@KluepfelMark @ollama The price stayed the same, but off-peak hours got a big discount.

[Patrick Devine @pdev110]

@DJSnM Who wouldn't want to do flight seeing by airship? Also, I just think they're neat.

[Patrick Devine @pdev110]

@martin_casado OK, that's just wild.

[David Sacks @DavidSacks]

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.

You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.

I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.

But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.

Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.

Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.

So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.

If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

[Patrick Devine @pdev110]

@tlbtlbtlb @martin_casado Personally I find it really fun, but unless you're willing to commit large amounts of time it gets pretty dangerous. There's the whole "spewing lead particulate matter" all over everyone thing as well.

[Patrick Devine @pdev110]

@tlbtlbtlb @martin_casado I think this one is true, but with caveats. Small aircraft *can* be a massive time saver, but there are a lot of regulatory hurdles (licensing and certification) which drives the cost up enormously. Couple that with the perceived danger factor and last mile connectivity problems.

[Patrick Devine @pdev110]

@sxzz Boundary Bay Airport (CZBB) is the GA airport for Vancouver and has customs clearance (i.e. it's a Port of Entry). I've flown in there w/ an SR-20 from the US before. ATC was probably confused why you weren't flying there instead.

[Patrick Devine @pdev110]

@electroheadfx @ollama It should be working. Here's a session with kimi.

[ollama @ollama]

Ollama v0.33 is here!

You can now easily configure Claude Desktop to seamlessly work with Ollama as a third-party gateway provider.

One toggle. Cloud & local models just work👇

[Patrick Devine @pdev110]

@redp314 @ollama This is a good tip. We're looking at being able to swap the kvcache out to disk as well, that way we can free up some more memory, but that's a little ways off.