jon3k

healthcare cio, rhce and 少し日本語

Disclaimer: views are my own and not necessarily representative of my employer.

[jon3k @jon3k]

"How's AI going?"

Me:

[jon3k @jon3k]

@KhunPat27 It's because humans have to lift your bag on and off the plane. They have to set a limit somewhere.

[jon3k @jon3k]

@perrymetzger I feel the exact opposite. I can show you dozens and dozens of chats with me correcting AI and it admitting it is wrong. Happens daily. But it's still very useful. I think both can be true.

[jon3k @jon3k]

@analogalok 😲 I'll check it out! Thanks!

[jon3k @jon3k]

@NeoAIForecast What's your llama.cpp config look like?

[jon3k @jon3k]

@DylanBurnette @cjzafir Kimi K2.6 the Deep seek v4 pro are pretty comfortably the top two for coding.

[jon3k @jon3k]

@DylanBurnette @cjzafir Coding, agents, something else?

[jon3k @jon3k]

@PawelJLisowski @vxunderground I can't think of a worse set of data to have than the unhinged ramblings of a couple billion Facebook users. Can you imagine how braindead an LLM trained on that slop would be?

[jon3k @jon3k]

@DylanBurnette @cjzafir What kind of hardware do you plan to run it on? Some of these are massive.

[jon3k @jon3k]

@Pallavi_345 I use it on Android Studio and it's great for the simple utilities I've whipped up.

[jon3k @jon3k]

@LTGraphicEssays @sudoingX I'm using unsloth and their mmproj: mmproj-BF16.ggu

vision working via slack

[jon3k @jon3k]

@ivanfioravanti Is anyone producing data on this that is publicly available?

[jon3k @jon3k]

@UlrichRozier @FoolaFroos I'm running it on a 7800XT (16GB) with a 200k context window. Using unsloth q5. This is the best model for agents I've ever been able to run locally on that card. It's fantastic.

[jon3k @jon3k]

@0x45o https://t.co/is4qalMl27

[jon3k @jon3k]

@theo What's the source that the limit is $60 of inference?

[jon3k @jon3k]

@housecor Plot twist: they hired 8x more engineers

[jon3k @jon3k]

@_sirajuddeen_ @notnullptr Let me know how it goes! Curious what kind of context window size you can get on 8GB. My use case is agents with a huge context on 16GB.

[jon3k @jon3k]

@AlexFinn > and give you a couple use cases that will blow your mind

still waiting

[jon3k @jon3k]

@Ominousind gemma 4 12b is a weird savant model. I don't know how it's this good at this size.

[jon3k @jon3k]

@BillQueens @notnullptr Nope. This model is crazy good if you know how/where to use it.