Manjunath

AI Engineer | ML and DL | Agents | DevOps and Observability | Cloud

[Manjunath @MIrukulla]

GPT ni adugu "STRAWBERRY" ane word lo enni letters unayi ani. Watch it fail! Counting raaka kaadhu, dheeni venaka chala peddha reason undhi. Adhi teliyali ante, meeku Tokenization gurinchi teliyali saami...!

Antha peddha katha emi kaadhu le saami.

[Manjunath @MIrukulla]

@Machi091 That's my dad's work setup. I use it at nights mostly 😅

[Manjunath @MIrukulla]

That’s all for the day.

✅ Revision Deep Learning
✅ Revision LLMs
✅ Prepare for an interview.
✅Applied Google DeepMind hackathon.
✅ Project research and architecture planning.

[Manjunath @MIrukulla]

@SiO2ey All the best!

[Manjunath @MIrukulla]

@unknowexistence Yeah, maybe that helps, but the question is how much is enough to achieve that?

[Manjunath @MIrukulla]

Interview Question -

Is the context window a software problem?

A hardware problem?

Or both?

Your answer tells me how deeply you understand LLMs. 🧵

[Manjunath @MIrukulla]

This exact tension, needing to hold growing context without paying growing memory cost, is precisely why KV caching exists.

A full breakdown of how it actually works next Monday.

What's your take? Should context windows scale via smarter software, or are we just waiting on better hardware?

[Manjunath @MIrukulla]

So, the verdict is that software created the shape of the problem, quadratic compute growth, and trained positional limits. Hardware decides how far you're allowed to push before physically running out of memory or bandwidth.

Neither one is "the" bottleneck. They're a coupled system, software writes the check, and hardware decides if it clears.

[Manjunath @MIrukulla]

@virattt @ycombinator @findatasets Congratulations but curious, why do we need stock market infrastructure for agents, especially apart from latency issues?

[Manjunath @MIrukulla]

@dreamydoer_9 Woahhhh!!! Thanks for reposting. 🙌🏻❤️

[Praveen Kumar Verma @Alacritic_Super]

As an AI Infrastructure Engineer

Please learn:
- GPU architecture, VRAM fundamentals, CUDA & memory hierarchy
- Quantization (INT8/FP8/4-bit), batching & continuous batching
- vLLM, TensorRT-LLM, SGLang, llama.cpp & inference optimization
- KV caching, speculative decoding, prefix caching & token throughput
- Distributed training (DDP, FSDP, DeepSpeed, ZeRO)
- Model serving (Triton, vLLM, KServe, Ray Serve, SGLang)
- Kubernetes, Docker & GPU orchestration
- NCCL, InfiniBand & high-speed networking
- Multi-GPU & multi-node inference
- LoRA, QLoRA, PEFT & fine-tuning pipelines
- Vector databases, embeddings & RAG pipelines
- Prompt caching, semantic caching & cost optimization
- Observability (OpenTelemetry, Prometheus, Grafana, Langfuse, Phoenix)
- LLM evaluation, benchmarking & A/B testing
- Model routing & fallback strategies
- MCP, AI agents & workflow orchestration
- Data pipelines, Kafka & streaming inference
- Security, guardrails & prompt injection mitigation
- CI/CD for ML (MLOps), MLflow & model registries
- Linux, networking & storage fundamentals
- PyTorch internals, CUDA profiling & kernel optimization

[Manjunath @MIrukulla]

@ghoshshirsha @haha_girrrl Cool!

[Manjunath @MIrukulla]

@ram_dev117 Thanks bro 🙌🏻

[Manjunath @MIrukulla]

Fingers crossed 🤞

[Manjunath @MIrukulla]

@aashwindev 😂😂😂