Alex Alexapolsky

Python developer/contractor

[Alex Alexapolsky @TheWake]

Built an open-source tool for debugging LLM agents:

- Record runs with full traces
- Replay w/o API calls
- Diff to see what changed

"Agent worked yesterday, broke today" - now you can see exactly why.
Works with PydanticAI, LangGraph, CrewAI, etc.

https://t.co/7myMi5JxWZ

[Alex Alexapolsky @TheWake]

@vboykis Greek code review: "Are you aware that Socrates’ questioning was basically the first code review in history?"

[Alex Alexapolsky @TheWake]

@gvanrossum Agents are caterpillars. Look like pipelines with survival instincts. We'll soon see into what they evolve.

[Alex Alexapolsky @TheWake]

Finding a RAG engineer is itself a RAG problem.
Right resume. Wrong chunk.
https://t.co/hNJqSzhcSL

[Alex Alexapolsky @TheWake]

Built a chunker that knows where documents actually break: articles, sections, tables stay intact instead of getting sliced at character 600. Now I can stuff good ingredients into LangChain!

pip install chunkweaver

https://t.co/cWKw35omZq

[Alex Alexapolsky @TheWake]

https://t.co/2egq5rSz6g
https://t.co/vkRTtXIMUi

[Alex Alexapolsky @TheWake]

Built ragprobe — pre-deployment domain difficulty diagnostic for RAG.

One specificity score tells you whether your benchmark will transfer to your actual domain. No embeddings. No API keys. Just: is this domain easy or hard to retrieve?

pip install ragprobe

[Alex Alexapolsky @TheWake]

@svpino Skill issue" is unfalsifiable without tooling. When a web server breaks you get a stack trace. When an agent breaks you get vibes. Record the run, replay it, diff it against the one that worked. That's what work-ledger does. https://t.co/7myMi5JxWZ

[Alex Alexapolsky @TheWake]

@ysu_ChatData @ihteshamali Needle coverage: annotate queries with exact answer spans, check if they appear in top-k. Recall@K can be 1.0 while needle coverage is ~30%. Measure retrieval first—more steps on weak retrieval mostly add cost. https://t.co/mJhhxlkmcy

[Alex Alexapolsky @TheWake]

"RAG is dead"? I tested PageIndex on GDPR. 44%. Same as vector RAG. Maybe RAG is undead, like Raggie. See why: https://t.co/UEFx1D602M

[Alex Alexapolsky @TheWake]

@idzikbartosz or if too low (out-of-domain query). Basically tells you whether it's a chunking problem, an embedding problem, or a prompt problem.

[Alex Alexapolsky @TheWake]

I built "EXPLAIN ANALYZE" for RAG retrieval.

After debugging LLM pipelines blind for months, I made a CLI to see what's actually happening.

Works with pgvector, Qdrant, Weaviate, Pinecone — real infrastructure, not toy demos.
https://t.co/mJhhxlkU26

[Alex Alexapolsky @TheWake]

@idzikbartosz Yes -- that's the core of "ragtune explain". It shows each chunk with its similarity score, source doc, and text. Plus it runs score diagnostics automatically: distribution shape, spread, top-gap between #1 and #2, and warns you if scores are too tight (chunks indistinguishable)

[Alex Alexapolsky @TheWake]

@akshay_pachaar It's just not correct. PageIndex is good thing, but not a full replacement. Let's be thoughtful and correct, like old time engineers. We still need to employ our brain and not jump to the conclusions.

[Alex Alexapolsky @TheWake]

Benchmarked structural vs naive chunking for RAG on GDPR + RFCs.
Both find the right doc. Only one gives the LLM enough context to cite the right section.
Recall@K is incomplete — we need context-quality metrics.
Full experiment: [https://t.co/N94glJxI6d]

[Alex Alexapolsky @TheWake]

@goyalshaliniuk Great overview. One thing that cuts across all these RAG types is evaluation.
As systems become agentic, reproducibility and retrieval matter more than new architectures.
That’s exactly the gap we’re exploring with https://t.co/J36RkcbWJm — query-level, LLM-free RAG benchmarking.

[Alex Alexapolsky @TheWake]

@martinfowler "There are no unit tests for context engineering" - love this framing.

But there could be: record a run, replay deterministically, diff against the next run.

That's how you test probabilistic systems.

[Alex Alexapolsky @TheWake]

An LLM call is superposition - infinite possible outputs exist simultaneously.
Recording the call and it's result - is the observation that collapses it to one.

[Alex Alexapolsky @TheWake]

@agrover112 Exactly. Retrieval is fundamental - it's just specializing by domain. Code - agentic search. Structured - tree based. Semantic-embeddings. "RAG is dead" really means "naive RAG is dead." The retrieval problem didn't go away.

[Alex Alexapolsky @TheWake]

@dr_cintas "Vector DBs disrupted" is a fun headline, but facts are: PageIndex uses document trees, great for structured docs (finance, legal). But embeddings still dominate for: Unstructured text, Semantic similarity, Cross-modal retrieval. The right answer is: benchmark both on YOUR data.

[Alex Alexapolsky @TheWake]

@rajshetgar @techwith_ram @iPullRank Totally agree: low temperature + highly relevant retrieval is the combo that actually works in production. Curious, what re-ranker do you use?

[Alex Alexapolsky @TheWake]

@rajshetgar @techwith_ram @iPullRank This is the underrated problem. You can't fix what you can't measure. We built https://t.co/J36RkcbWJm to debug exactly this - run your queries, see which ones retrieve confidently, which ones are borderline, and why. The "in-between" come from chunking, embeddings etc.