Prajjwal Yadav

Developer Advocate @weaviate_io | UX Approver @KnativeProject

[Prajjwal Yadav @PrajjwalYd]

Just told Claude to import a PDF into Weaviate.

It figured out the schema, vectorised every page, batched the whole thing.

I typed one sentence

(works with CSV and JSON/JSONL too, but that feels less impressive to say)

This is the part where I'm supposed to feel useful

Try the Weaviate agent skills: https://t.co/HoMKlIN3DM

[Kamil Tyborowski @kamtybor]

Your customers can notice slow queries instantly.

But can you figure out what makes those queries slow?

Was the time spent traversing HNSW, evaluating a filter, scoring BM25 results, or reading objects from disk?

𝗤𝘂𝗲𝗿𝘆 𝗽𝗿𝗼𝗳𝗶𝗹𝗶𝗻𝗴 𝗶𝗻 𝗪𝗲𝗮𝘃𝗶𝗮𝘁𝗲 answers that for the exact query you’re debugging.

Enable `query_profile` in the request metadata and the response includes a per-stage timing breakdown, organised by shard and node. The coordinating node collects profiles from every participating shard, so you get the whole cluster view without having to stitch logs together.

For example, imagine a shard takes 48.2ms:

• vector_search took: 8.4ms
• filters_build_allow_list_took took: 2.1ms
• objects_took: 36.8ms

Object hydration takes the longest, so investigate the page cache, storage, result limit, or payload size.

If filter evaluation dominates and matches millions of IDs, the filter is too broad. Make it more selective before adding hardware.

Previously, diagnosing this meant enabling the slow query log, waiting for the query to cross a threshold, then reconstructing timings across node logs. A restart could also leave you measuring cold page caches rather than the original conditions.

Now a single opt-in flag returns the cross-cluster profile directly in the query response.

Learn more on our blog: https://t.co/X6NREvQ01I

[Prajjwal Yadav @PrajjwalYd]

Kimi K3 made GPT-2 look like a pixel.

I plotted frontier open-weight models from GPT-2 to Kimi K3 on the same scale.

1.5B parameters in 2019.
2.8T in 2026.

This is the growth in open-weight model parameter capacity.

One caveat: Kimi K3 is a sparse MoE model, with 16 of 896 experts active per token. So total parameters and per-token compute are very different numbers.

[Prajjwal Yadav @PrajjwalYd]

And The Name Arrives After the Pain

(10/10)

[Prajjwal Yadav @PrajjwalYd]

I kept pretending I understood all the latest agent terms.

Then I actually tried. Here are the notes:

(1/10) ➡️

[Prajjwal Yadav @PrajjwalYd]

Every New Term Is An Old Problem

(9/10)

[Prajjwal Yadav @PrajjwalYd]

@kamtybor So it technically built a maglav

[Prajjwal Yadav @PrajjwalYd]

Yesterday's truth is today's hallucination.

A memory can be completely accurate when stored and completely wrong when retrieved days later.

Preferences evolve. Facts become stale. Agents learn better ways to use tools. If every version remains equally retrievable while the world keeps changing, the model has to reconcile conflicting versions of reality on every request.

Engram moves that reconciliation to ingestion:

• 𝗘𝘅𝘁𝗿𝗮𝗰𝘁 a candidate fact matching a configured topic
• 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗲 related existing memories from @weaviate_io during the context-aware transform
• 𝗧𝗿𝗮𝗻𝘀𝗳𝗼𝗿𝗺 the old and new information by rewriting, merging, deduplicating, keeping, or deleting as needed
• 𝗖𝗼𝗺𝗺𝗶𝘁 the finished changes so they become queryable

The important part is that this is configurable infrastructure.

Topics control what deserves to become memory. Scopes control who can influence it. Project-wide memories can improve an agent for a trusted team, while user-scoped memories are isolated through Weaviate multi-tenancy on both writes and queries.

The pipelines run asynchronously, with durable, in-order processing per scope. Applications can fire-and-forget new data, then inspect the run and its changes later.

Not every message should become memory.

But every memory that does should remain coherent as the world changes.

Start building with Engram: https://t.co/4UjT5HE2EJ

[Kamil Tyborowski @kamtybor]

https://t.co/ozuQZXn4Xt

[Prajjwal Yadav @PrajjwalYd]

@kamtybor team knowledge the best knowledge

[Prajjwal Yadav @PrajjwalYd]

HTTP 200 is a terrible place to stop checking an import.

A batch request can succeed while individual objects fail inside it. Rate limits, vectorizer errors, and schema mismatches do not always take the whole request down with them.

That is one of several things that barely matter in a small prototype, then become very expensive when the dataset gets real.

A safer import pipeline needs a few boring properties:

• stream rows instead of loading the dataset into memory
• let the server apply backpressure instead of guessing one fixed batch size
• generate deterministic UUIDs from a stable source key
• persist failed_objects and retry only that subset
• checkpoint progress and dead-letter repeat failures

The deterministic IDs are especially useful because they make retries idempotent: the same source row maps to the same @weaviate_io object, rather than creating duplicates on every rerun.

And server-side batching still needs error handling. It handles pacing, not correctness.

The illustration shows one failure mode. The full blog post goes further into data types, media ingestion, blobHash, multimodal vectorisation, etc.

Link: https://t.co/b81H202cJd

[Prajjwal Yadav @PrajjwalYd]

what if j-space is just gstack’s premium tier

[Prajjwal Yadav @PrajjwalYd]

@MatinMnM @victorialslocum @moroz_i_holod Nice, which model is used in this?