One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.
@karpathy This is very true and I find over time the rambles get better. I think we’re so accustomed to proper speech with other people that we forget that LLMs don’t need perfect speech. This is also why I find it funny when people spend so much time correcting the typos in their prompt.
@thsottiaux More persona and life context. Proactive notifications would be nice too. Would love it if built in it could give me updates on recent work being done on other machines. Suggest new actions.
Anouncing Sonitura reinvented. This one is open source and costs nothing if you have a ChatGPT subscription. Leveraging your existing subscription this tool allows you to click (fn + ctrl) and ask an AI agent to help you with whatever you’re looking at on your screen. Keeps context, has long term memory and is something I’ve been using for the last couple of days with huge success. It was fun to build too! Since it requires a lot of permissions from Mac to work well I think this kind of tool needs to be open source.
It can even type for you when talking to your agents, if you’re ever talking to a coding agent and need help responding, Sonitura will type a response directly in context.
I’m on an open sourcing roll right now. Excited to share all the other agent tooling I’ve built over the last year with my 6000+ commits.
https://t.co/sMiSFr9obc
@thsottiaux Also built https://t.co/dh5lyW4e1G to meter your AI agent spend and put caps. Open source. Detects how much subscriptions would cost if charged at API rates. It’s been a busy week of building.
@travisk@DavidSacks@elonmusk For people saying this is not real, I was able to reproduce it as well. If they haven't patched it you might be able to do the same.
https://t.co/KUPR20rkOZ
@thsottiaux Built this A voice assistant that sees your screen, hold a key, ask, and it answers out loud on the ChatGPT plan you already pay for. Local speech-to-text + neural TTS, on-screen agent, private local memory. Open source, macOS.
https://t.co/wuD3aXc7xe