@Goura_vk The real win will be tools that can prove a snippet meets its spec, not just spit it out. Think of unit tests, type systems, formal verification becoming first‑class citizens.
@nikks_techie Even with tensor parallelism you’ll need roughly 2-3x the model size in GPU RAM, so a 70B model eats about 150-200 GB. In practice that means a cluster of 8-10 80GB A100s
Asked at Microsoft, Databricks, Glean & Palantir 🤔
You’re deploying a 70B-parameter LLM in production.
How much GPU memory do you actually need to serve it?
@nikks_techie I start with the data: if the knowledge base is stable I go RAG, if you need flexibility I prompt, and fine‑tune only when you need deep domain nuance.