When building Sift, I had to choose: pure chunk storage, or compile into WikiPages first?
Chunks solve "can I find it." WikiPages solve "is what I found actually usable."
When a chunk is cut from its source, context breaks. You retrieve a passage but lose whether it's an argument, an example, or a conclusion. High vector similarity doesn't fix that.
Inspired by Karpathy's LLM Wiki: compile raw captures into structured knowledge before any query.
Source relationships preserved. Concept graph intact. Agents get something they can reason with, not just pattern-match against.
@hasantoxr The 3-apps-to-1 compression is real — most "workflow automation" just moves friction around, rarely eliminates it.
What does the output actually look like? Curious if the handwritten style holds up on complex diagrams.
100k markdown files is impressive — and also a little terrifying to maintain 😄
Curious: at that scale, how do you handle knowledge decay? Old files that are outdated but still surface in queries?
I've been building Sift around a different bet — instead of files, everything compiles into Source + WikiPage with merge history, so the knowledge stays fresh rather than just growing.
Would love to compare notes.
That last line is exactly why I built Sift.
Same insight: raw capture → AI compiles into structured knowledge → queryable, reusable context.
The difference: no scripts, no raw/ directories, no manual Obsidian setup. Just drop anything in — links, screenshots, text — and the pipeline runs in the background.
Also added Agent/MCP endpoints so external tools can pull your knowledge as long-term context.
Still early, but this is the product I wanted to exist. Happy to share if you want to try it.
Most people's workflow: see something useful → "I'll save it properly later" → never happens.
That habit made sense before AI. You had to organize manually or you'd never find it again.
Now AI can read untitled text, extract meaning from screenshots, and connect dots across 10 unrelated articles.
The job of "organizing" moves to the background.
Your only job: catch it before it disappears.
Capture first. Let AI do the rest.