Amartya

SWE @Walmart | GPU / CUDA / Inference Engineering | Developing a taste for Design & UX 🧐

[Amartya @evilseyee]

last night i made the classic engineer mistake.

"how hard can it be to replace part of my claude code workflow with a local setup?"

famous last words.

i started around midnight thinking i'd spend an hour installing ollama, hook it up to cline, generate a few files and call it a day.

instead, i spent the entire night staring at terminals, "nvidia-smi", "ollama ps", changing configs, reading docs, questioning my life choices and slowly realizing i wasn't debugging a model... i was debugging an entire inference stack.

my setup was pretty simple:

• rtx 4060 laptop (8gb)
• ollama
• qwen3:8b
• cline

the first task? literally generate a "hello.html" file.

it worked... but painfully slowly.

naturally, my first instinct was, "guess i need a better model."

turns out that was probably the wrong conclusion.

here's what i learned after spending way too many hours chasing the wrong problem:

• "ollama run" and cline don't spin up separate models. they're just different clients talking to the same ollama server.

• chatting with a local model and running a coding agent are two completely different workloads. by the time cline sends your "simple" prompt, it's already bundled in system prompts, tool definitions, workspace context and conversation history.

• i finally understood why my model looked like ~5.6gb in one moment and ~9.6gb the next. KV cache went from being a buzzword i kept hearing on twitter to something i could literally watch eating my VRAM in real time.

• seeing "100% GPU" in the terminal but "64% GPU / 36% CPU" inside cline wasn't some mysterious bug. it was a reminder that context, memory pressure and runtime configuration matter just as much as the model itself.

the funniest part?

i spent the first half of the night convinced i needed to download a "better" model.

i spent the second half realizing i probably needed a better understanding instead 😭

safe to say the experiment did exactly what i wanted it to do. i wasn't trying to find the perfect local model. i wanted to understand what actually happens between hitting "enter" and an ai agent writing code.

turns out the answer is: a lot more than i gave it credit for.

for everyone who's been down this rabbit hole already...

what's the one thing that completely changed the way you think about local inference once it finally clicked?

[Amartya @evilseyee]

@gregpr07 what is it about ?

[Amartya @evilseyee]

@BosonJoe local inferencing all the way

[Amartya @evilseyee]

@tokenbender that time of the year

[Amartya @evilseyee]

told ya, this Brazil vs Japan might turn into a major upset 👀

[Amartya @evilseyee]

can we expect the first upset in this world cup today ?

Brazil 🇧🇷 vs Japan 🇯🇵 Round of 32, probably the 2nd most interesting draft after Morocco 🇲🇦 vs Netherlands 🇳🇱

[Amartya @evilseyee]

@prateek_0041 into the same journey for a couple of months, interesting stuff & too many opportunities to create something :P

[Amartya @evilseyee]

can we expect the first upset in this world cup today ?

Brazil 🇧🇷 vs Japan 🇯🇵 Round of 32, probably the 2nd most interesting draft after Morocco 🇲🇦 vs Netherlands 🇳🇱

[Amartya @evilseyee]

https://t.co/t5sUtM0kxA

[Amartya @evilseyee]

one of the best skill / plugin that i have been using lately gotta be Ponytail

crazy how simple instructions change the way your agent codes

[Amartya @evilseyee]

@Hikari_07_jp what is your local inference stack ?

[Amartya @evilseyee]

@HeyGen This is what we lacked! Finally 🙌

[Amartya @evilseyee]

@leerob I want to dive deeper into Evals and Benchmarking, any good resources ?

[Amartya @evilseyee]

@cszhu Haha , best of luck for that

[Amartya @evilseyee]

@levelsio bro flexed hard

[Amartya @evilseyee]

@alex_heritier @sudoingX They are the real saviour

[Amartya @evilseyee]

@dprophecyguy @dejavucoder this shit is below average ai slop :P

[Amartya @evilseyee]

@sudoingX @0xSero isn't the background distracting :P

[Amartya @evilseyee]

should i buy a dgx spark or cloud gpu vm for local inference?

[Amartya @evilseyee]

last night i made the classic engineer mistake.

"how hard can it be to replace part of my claude code workflow with a local setup?"

famous last words.

i started around midnight thinking i'd spend an hour installing ollama, hook it up to cline, generate a few files and call it a day.

instead, i spent the entire night staring at terminals, "nvidia-smi", "ollama ps", changing configs, reading docs, questioning my life choices and slowly realizing i wasn't debugging a model... i was debugging an entire inference stack.

my setup was pretty simple:

• rtx 4060 laptop (8gb)
• ollama
• qwen3:8b
• cline

the first task? literally generate a "hello.html" file.

it worked... but painfully slowly.

naturally, my first instinct was, "guess i need a better model."

turns out that was probably the wrong conclusion.

here's what i learned after spending way too many hours chasing the wrong problem:

• "ollama run" and cline don't spin up separate models. they're just different clients talking to the same ollama server.

• chatting with a local model and running a coding agent are two completely different workloads. by the time cline sends your "simple" prompt, it's already bundled in system prompts, tool definitions, workspace context and conversation history.

• i finally understood why my model looked like ~5.6gb in one moment and ~9.6gb the next. KV cache went from being a buzzword i kept hearing on twitter to something i could literally watch eating my VRAM in real time.

• seeing "100% GPU" in the terminal but "64% GPU / 36% CPU" inside cline wasn't some mysterious bug. it was a reminder that context, memory pressure and runtime configuration matter just as much as the model itself.

the funniest part?

i spent the first half of the night convinced i needed to download a "better" model.

i spent the second half realizing i probably needed a better understanding instead 😭

safe to say the experiment did exactly what i wanted it to do. i wasn't trying to find the perfect local model. i wanted to understand what actually happens between hitting "enter" and an ai agent writing code.

turns out the answer is: a lot more than i gave it credit for.

for everyone who's been down this rabbit hole already...

what's the one thing that completely changed the way you think about local inference once it finally clicked?

[Amartya @evilseyee]

@ronaldwichhart @levelsio It does run qwen3:8b but the problem arises when I combine it with vscode / roo code or cline to have a cursor like experience.
Reason being agent frameworks like cline send a huge context alongwith each prompt that makes inference run 64% on GPU and spills 36% on CPU

[Amartya @evilseyee]

with every claude release, they build a state of the art launch video , probably using framer motion.
whereas when I make them, they're just another slop.

Does anyone know the secret sauce to make SOTA videos using framer motion or any other library? any particular skills ?