Francesco Pensabene

Digital Art Director, Expressionengine lover, Photographer, Fly Fisher, Dreamer, Italian. Latest work uhu.it - bostik.it - arexons.it - svitol.it

[Andrej Karpathy @karpathy]

LLM Knowledge Bases

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Data ingest:
I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them.

IDE:
I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides).

Q&A:
Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale.

Output:
Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base.

Linting:
I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into.

Extra tools:
I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries.

Further explorations:
As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows.

TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

[Andrej Karpathy @karpathy]

This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc.

More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage:

1) raw text (hard/effortful to read)
2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default
3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default
...4,5,6,...
n) interactive neural videos/simulations

Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral https://t.co/z21CP5iQfu

There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen.

TLDR The input/output mind meld between humans and AIs is ongoing and there is a lot of work to do and significant progress to be made, way before jumping all the way into neuralink-esque BCIs and all that. For what's worth exploring at the current stage, hot tip try ask for HTML.

[Zain Shah @zan2434]

Imagine every pixel on your screen, streamed live directly from a model. No HTML, no layout engine, no code. Just exactly what you want to see.

@eddiejiao_obj, @drewocarr and I built a prototype to see how this could actually work, and set out to make it real. We're calling it Flipbook. (1/5)

[Francesco Pensabene @CottonHen]

This is the "legendary" quality of K-Swiss after just 20 days. Asked @tennispointde for help: NO RESPONSE. Asked @KSWISS, and you can read for yourself the kind of "help" they offer. These are going straight in the bin. Getting myself a pair of @ASICSItalia @ASICS_JP instead.

[Francesco Pensabene @CottonHen]

@rolandgarros Dajeeeeeeeeeeeeeeee

[Francesco Pensabene @CottonHen]

Hi @Freepik, I subscribed to use Google Nano Banana Pro instead of Google AI Pro. But I always see 'high demand' warnings with 6+ min wait times, while my Google subscription generates instantly. What's going on? #NanoBananaPro

[Francesco Pensabene @CottonHen]

@floraai pdf file of Catherine Chung Case study is corrupted, I'd like to read it. Can you update it on you website home page?

[Jan 🏴󠁧󠁢󠁳󠁣󠁴󠁿 @RosaryMum]

Every December 8 on the Feast of the Immaculate Conception, firefighters in Rome honor the Virgin Mary by placing a wreath of flowers on her statue.

[Francesco Pensabene @CottonHen]

@invideoOfficial STARBOY

[Francesco Pensabene @CottonHen]

@mikefutia Banana

[Francesco Pensabene @CottonHen]

@recap_david Fit

[۟ @jnksnrs]

like holy fuck?

[Francesco Pensabene @CottonHen]

@KyloRxns Rolex requires it…

[Francesco Pensabene @CottonHen]

@CincyTennis Too Hot, Too Long, The Wrong Winner. @janniksin @JannikSinner_Up #CincinnatiOpen #CincinnatiOpen2025

[Francesco Pensabene @CottonHen]

@ceracoat @atptour @carlosalcaraz @CincyTennis What an asshole

[Francesco Pensabene @CottonHen]

@lewisgreenall @atptour @carlosalcaraz @CincyTennis Maybe the same you just had…

[Francesco Pensabene @CottonHen]

@TheTennisLetter This is not tennis, this is HELLFIRE. Too hot! So sorry for @janniksin

[Blackmagic Design @Blackmagic_News]

The Beginner’s Guide to DaVinci Resolve 19 now available! This newly updated training guide covers how to use the traditional edit page, color correction, Fusion VFX, Fairlight audio tools and more! Download now from https://t.co/oYfG558orC

[Frenkie_Woody @Frenkie_Woody]

Sarà che ormai non si regge più in piedi e la sofferenza ne ha modellato il carattere ma questo discorso di Papa Francesco di qualche giorno fa è semplicemente STREPITOSO!

"Puoi avere difetti, essere ansioso e perfino essere arrabbiato, ma non dimenticare che la tua vita è la più grande impresa del mondo. Solo tu puoi impedirne il fallimento. Molti ti apprezzano, ti ammirano e ti amano. Ricorda che essere felici non è avere un cielo senza tempesta, una strada senza incidenti, un lavoro senza fatica, relazioni senza delusioni.

"Essere felici è smettere di sentirsi una vittima e diventare autore del proprio destino. È attraversare i deserti, ma essere in grado di trovare un'oasi nel profondo dell'anima. È ringraziare Dio ogni mattina per il miracolo della vita.È baciare i tuoi figli, coccolare i tuoi genitori, vivere momenti poetici con gli amici, anche quando ci feriscono.

"Essere felici è lasciare vivere la creatura che vive in ognuno di noi, libera, gioiosa e semplice. È avere la maturità per poter dire: "Ho fatto degli errori". È avere il coraggio di dire "Mi dispiace". È avere la sensibilità di dire "Ho bisogno di te". È avere la capacità di dire "Ti amo". Possa la tua vita diventare un giardino di opportunità per la felicità ... che in primavera possa essere un amante della gioia ed in inverno un amante della saggezza.

"E quando commetti un errore, ricomincia da capo. Perché solo allora sarai innamorato della vita. Scoprirai che essere felice non è avere una vita perfetta. Ma usa le lacrime per irrigare la tolleranza. Usa le tue sconfitte per addestrare la pazienza.

"Usa i tuoi errori con la serenità dello scultore. Usa il dolore per intonare il piacere. Usa gli ostacoli per aprire le finestre dell'intelligenza. Non mollare mai ... Soprattutto non mollare mai le persone che ti amano. Non rinunciare mai alla felicità, perché la vita è uno spettacolo incredibile.".

PAPA FRANCESCO

[Swish 🍒 Tennis @Zwxsh]

Congratulations to you. Yes, you reading this tweet.

You won as many sets at the Australian Open as Nick Kyrgios this year.