Pack

We plan, you pack

[Pack @trypackai]

https://t.co/upzDezs9ey

[Pack @trypackai]

We want to be your travel agent for free - "Knicks game then offsite then week at beach w/ my wife"

Travel is personal and complex

A bad suggestion is awkward, an uncatchable flight is awful

We want to solve 300 question onboarding and having to double check every detail

[andrew gao @itsandrewgao]

i had to prompt inject the @united airlines bot because it kept refusing to connect me with a human

🧵 what led up to this breaking point

[Pack @trypackai]

@karpathy Haiku is fantastic if you have good frameworks and prompts

[Pack @trypackai]

@signulll AI is finally far enough along this is possible, working on one that doesn't suck all the other launches I've seen haven't actually solved my problems

[Pack @trypackai]

@Suhail Travel agentic AI that doesn't have a 40% failure rate

[Pack @trypackai]

Something about putting in the effort for KMS IAM and Minimal Access patterns is so relaxing

[JT @jiratickets]

ever since I was a little boy I dreamed of configuring IAM polices for an enterprise AWS environment

[Pack @trypackai]

Getting close to finally having my problem actually solved

[Pack @trypackai]

@jxnlco 1000% and I think this is only something you can solve with enough time spent working on LLMs/AI to understand ranking/preference weighting in production.

Working on solving this for travel :)

[Sahil Lavingia @shl]

Automation makes jobs harder, not easier

[Pack @trypackai]

Wouldn't have it any other way, all I want to do is solve my own problems

[Calvin Chen @calvinchen]

you gotta be a special type of delusional to build consumer

[Pack @trypackai]

@hthieblot 👋 working on Personal Assistant AI that isn't probabilistic

[Pack @trypackai]

@OfficialLoganK Generalized AI that has 100% accurate evals you say

[Pack @trypackai]

@kyliebytes Working on it

[xjdr @_xjdr]

you should legally be required to disclose what quantization level you are serving your current model at like it was a nutrition label. you should also be banned from dynamically adjusting quantization based on demand without notification. (you know who you are ...)

[Pack @trypackai]

@jdjohnson @tnm ;) working on it

[Pack @trypackai]

;)

[Andrej Karpathy @karpathy]

We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning?

Pretraining is for knowledge.
Finetuning (SL/RL) is for habitual behavior.

Both of these involve a change in parameters but a lot of human learning feels more like a change in system prompt. You encounter a problem, figure something out, then "remember" something in fairly explicit terms for the next time. E.g. "It seems when I encounter this and that kind of a problem, I should try this and that kind of an approach/solution". It feels more like taking notes for yourself, i.e. something like the "Memory" feature but not to store per-user random facts, but general/global problem solving knowledge and strategies. LLMs are quite literally like the guy in Memento, except we haven't given them their scratchpad yet. Note that this paradigm is also significantly more powerful and data efficient because a knowledge-guided "review" stage is a significantly higher dimensional feedback channel than a reward scaler.

I was prompted to jot down this shower of thoughts after reading through Claude's system prompt, which currently seems to be around 17,000 words, specifying not just basic behavior style/preferences (e.g. refuse various requests related to song lyrics) but also a large amount of general problem solving strategies, e.g.:

"If Claude is asked to count words, letters, and characters, it thinks step by step before answering the person. It explicitly counts the words, letters, or characters by assigning a number to each. It only answers the person once it has performed this explicit counting step."

This is to help Claude solve 'r' in strawberry etc. Imo this is not the kind of problem solving knowledge that should be baked into weights via Reinforcement Learning, or least not immediately/exclusively. And it certainly shouldn't come from human engineers writing system prompts by hand. It should come from System Prompt learning, which resembles RL in the setup, with the exception of the learning algorithm (edits vs gradient descent). A large section of the LLM system prompt could be written via system prompt learning, it would look a bit like the LLM writing a book for itself on how to solve problems. If this works it would be a new/powerful learning paradigm. With a lot of details left to figure out (how do the edits work? can/should you learn the edit system? how do you gradually move knowledge from the explicit system text to habitual weights, as humans seem to do? etc.).