nCompass

Making profiling and performance optimization effortless for heterogenous systems

[Aditya Rajagopal @adityaraja0]

@nCompass_tech just launched on @ycombinator's Launch YC!

nCompass - Optimize performance on GPUs, 10x faster. Check them out: https://t.co/1CbPeRJhMY

[Aditya Rajagopal @adityaraja0]

@nCompass_tech is a performance optimization dev tool that unifies 𝗽𝗿𝗼𝗳𝗶𝗹𝗶𝗻𝗴, 𝘁𝗿𝗮𝗰𝗲 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻 and 𝘁𝗿𝗮𝗰𝗲 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀 of AI systems.

We automate the process of identifying and fixing the root cause of performance bottlenecks across all levels of the AI infrastructure stack. Our agent automatically surfaces key bottlenecks and works in tandem with Claude Code / Cursor to implement the fixes.

Using our tool, we were able to automatically improve a Hopper GEMM kernel taking it from 𝟯𝟬% 𝘀𝗹𝗼𝘄𝗲𝗿 𝘁𝗼 𝟯% 𝗳𝗮𝘀𝘁𝗲𝗿 than a near optimal CUTLASS kernel within a day - this took us months before!

We’re excited to officially release v0.1.0 of our VSCode extension that includes 𝗻𝗮𝘁𝗶𝘃𝗲 𝘁𝗿𝗮𝗰𝗲 𝘃𝗶𝗲𝘄𝗶𝗻𝗴, 𝘁𝗿𝗮𝗰𝗲 𝗱𝗶𝗳𝗳𝘀, 𝘁𝗿𝗮𝗰𝗲 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻 and integration of our 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁 directly into Cursor / Claude Code. We’re working closely with an initial set of users including the core maintainers of @vllm_project to build features that best support the community.

𝗜𝗳 𝘆𝗼𝘂’𝗿𝗲 𝗮𝗻 𝗲𝘅𝗽𝗲𝗿𝘁 𝗚𝗣𝗨 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 - bring your time spent optimizing performance down to days rather than weeks - view, collaborate and analyze faster than you do today.

𝗜𝗳 𝘆𝗼𝘂’𝗿𝗲 𝗻𝗲𝘄 𝗮𝗻𝗱 𝘄𝗮𝗻𝘁 𝘁𝗼 𝗹𝗲𝗮𝗿𝗻 𝗵𝗼𝘄 𝘁𝗼 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 - you now have an expert pair-programmer to consult at any time.

𝗚𝗲𝘁 𝘀𝘁𝗮𝗿𝘁𝗲𝗱 𝗳𝗼𝗿 𝗳𝗿𝗲𝗲 𝘁𝗼𝗱𝗮𝘆 using VSCode, Cursor or Claude Code - https://t.co/lWXuYeoavi

We also work directly with you to identify opportunities for performance improvements. If you would like us to 𝗮𝘂𝗱𝗶𝘁 𝘆𝗼𝘂𝗿 𝘀𝘁𝗮𝗰𝗸 and suggest ways to improve performance - reach out at [email protected]!

[nCompass @nCompass_tech]

#vibetracing 😎

[Aditya Rajagopal @adityaraja0]

Got tired of manually reading through 200+MB trace files so we've now hooked them up to an AI agent and you can just chat with it. #vibetracing

Checkout our docs - https://t.co/GdYoTXqLo2

0:00 - Intro to the AI agent
1:00 - Example prompt with 2 questions
2:08 - Answer to question 1 - what should I care about in this trace
4:18 - Answer to question 2 - how do kernels map to operations from LLMs

Link to both the trace and responses in the comments!

#MLSys #GPUPerformance #PerfettoTraces #NsysReps

[Aditya Rajagopal @adityaraja0]

wrote a blog on how to use ncu directly on @vllm_project for a simple @Alibaba_Qwen model without taking 18+ hours to profile. I was tired of having to isolate kernels in a repro script to profile them with ncu. you can just isolate kernels direclty on a large trace and profile individual ones.

check it out here - https://t.co/fzJDABgTVA

#mlsys #vllm #inference #ai

[nCompass @nCompass_tech]

RT @adityaraja0: we're introducing the concept of diffs to trace data - pick two traces, run the diff and get information on changes in dur…

[Aditya Rajagopal @adityaraja0]

Ever wanted to run Nsight Compute (ncu) on a large codebase and understand individual GPU kernel performance — without isolating each kernel?

I’ve hit this problem many times profiling systems like vLLM when asking the question - how well is a specific kernel performing for this workload?

In practice, that means using ncu. But with default settings, running ncu on thousands of kernel launches is prohibitively expensive — so you end up building isolated repos just to profile one kernel.

The good news: you don’t have to.

I’m writing a 3-part series on reducing ncu overhead enough to profile kernels directly in large, real-world codebases. Part 1 is out today.

Part 1 → understanding the metrics ncu reports (important because reducing which metrics you profile is important to reduce overheads of profiling)
Part 2 → how to actually apply this on large codebases
Part 3 → why combining ncu + nsys beats using either alone

📖 Part 1: https://t.co/lsoUXKWpWy

If you’ve run into this before, I’d love to hear in the comments how you handle kernel-level profiling at scale.

#CUDA #GPU #PerformanceEngineering #NsightCompute #MLSys #NVIDIA #Profiling

[Aditya Rajagopal @adityaraja0]

You wouldn’t collaborate on code in Google Docs — so why do we still do that with trace files?

We just shipped a new feature in the nCompass VSCode / Cursor extension that makes trace collaboration actually usable:
• Save traces to secure cloud storage
• Share traces via a link teammates can open directly in their IDE

Our team is partially remote and works with a lot of traces. Until now, sharing meant SCP or Slack uploads. Now we can share traces straight from remote servers without leaving the IDE.

As teams push harder on AI inference and training performance, collaborating on traces is becoming as important as collaborating on code. We’re building the tooling for that workflow.

Tutorial: https://t.co/00lNjLzklM

#MLSys #Profiling #GPU #VSCode #Performance

[Aditya Rajagopal @adityaraja0]

🚀 Shipped nCompass SDK v0.1.12 - performance debugging shouldn’t require permanent code changes.

v0.1.12 makes it effortless to inject NVTX / TorchRecord markers at runtime — without editing your codebase or maintaining a debug branch.

How it works:
• Select a region in VSCode / Cursor
• pip install ncompass>=0.1.12
• Set env vars + run with torch profiler / nsys
• Markers show up in the trace ✨
Fully open source. New injectors welcome.

🎥 Tutorial: https://t.co/iWbBvtOMOs
⚡ Quick start: https://t.co/4oeilSzXgt

#MLSys #Profiling #GPU #Performance #PyTorch

[Aditya Rajagopal @adityaraja0]

As the saying goes, an image is worth a thousand tokens. So, to see a whole new line of @Alibaba_Qwen models released without multimodality was rather surprising. This thread is my breakdown of the state of multimodality in open and closed source AI.

[Portkey @PortkeyAI]

Portkey now supports nCompass integration!

@nCompass_tech is known for its blazing-fast, serverless LLM deployments with no rate limits and flexible on-prem options.

With this integration, you can now:
✅ Route and monitor LLM traffic to nCompass
✅ Apply guardrails and track prompt-level metadata for every nCompass call
✅ Track 40+ key metrics including cost, tokens, latency, and more

Read more about the integration here - https://t.co/olDaOim6gc

[Y Combinator @ycombinator]

🧭@nCompass_tech (YC W24) is a platform for simplified hosting & acceleration of open-source LLMs. Its API requires only one line of code to integrate low-latency versions of these models into your AI pipeline.

https://t.co/utU9YPURNm

Congrats on the launch @adityaraja0 & @DiederikVink!