from prompt to context to harness engineering.
three terms keep coming up in AI engineering, and they get conflated all the time. here is the cleanest way to understand what each one is and how they fit together.
๐ฝ๐ฟ๐ผ๐บ๐ฝ๐ ๐ฒ๐ป๐ด๐ถ๐ป๐ฒ๐ฒ๐ฟ๐ถ๐ป๐ด ๐ถ๐ ๐๐ต๐ฒ ๐บ๐ฒ๐๐๐ฎ๐ด๐ฒ.
the model has no memory of anything before this single call, so the prompt has to carry the full universe of what it needs to know. that means a role, some background, the instructions, a few examples, and a format.
these get assembled into one input and sent to the model. when the output falls short, the skill is figuring out which ingredient is actually letting you down, not rewriting the instructions every time.
the unit of work is one input.
๐ฐ๐ผ๐ป๐๐ฒ๐ ๐ ๐ฒ๐ป๐ด๐ถ๐ป๐ฒ๐ฒ๐ฟ๐ถ๐ป๐ด ๐ถ๐ ๐๐ต๐ฒ ๐บ๐ฒ๐บ๐ผ๐ฟ๐.
across multiple steps, the window is finite and the information available is not, which forces a curation step. without it, important details get buried under stale tool outputs and old turns, and the model's attention degrades on the things that actually matter.
a curator selects what stays, compresses what is useful but bulky, and drops the rest. each step's output then feeds into the next step, where good curation is more about knowing what to throw away than packing more in.
the unit of work is what stays in the window, step by step.
๐ต๐ฎ๐ฟ๐ป๐ฒ๐๐ ๐ฒ๐ป๐ด๐ถ๐ป๐ฒ๐ฒ๐ฟ๐ถ๐ป๐ด ๐ถ๐ ๐๐ต๐ฒ ๐บ๐ฎ๐ฐ๐ต๐ถ๐ป๐ฒ.
on its own, a model just generates text. the harness is what turns it into something that can take actions, check its own work, and recover when a step goes wrong.
the full loop has three phases:
- ๐ด๐ฎ๐๐ต๐ฒ๐ฟ pulls together everything the model needs
- ๐ฎ๐ฐ๐ runs the model and calls tools or sub-agents
- and ๐๐ฒ๐ฟ๐ถ๐ณ๐ checks the output with tests or a judge
on failure, the whole loop retries with updated context, which is the entire difference between calling an API and running an agent.
the unit of work is the machine itself.
here is the part that ties it together.
prompt engineering and context engineering both live inside ๐ด๐ฎ๐๐ต๐ฒ๐ฟ. the harness is the outer container, context is what it curates, and the prompt is what it finally hands to the model.
zoom out and the unit of work gets bigger. zoom in and you are back at the prompt.
i also published this deep dive (article) on agent harness engineering, covering the orchestration loop, tools, memory, context management, and everything else that transforms a stateless LLM into a capable agent.
the article is quoted below.
The Elder Scrolls Online AI Skills set
Link: https://t.co/LAKHMq7Fpe
Aurbis is a collection of skills that turn your AI into a dedicated personal assistant for The Elder Scrolls Online.
You're stuck.
You've been slamming your head against a target for days and you can't get anywhere.
You've plateaued in a target.
Here's how to break through:
Prelude: if you've been hacking on this target for less than 8 hours then you haven't plateaued yet, you've just barely started. Ignore this list and keep going.
If you have hacked for at least 8 hours:
1. Write down what you know about the target. (Paper works best for me)
Write down interesting things you've seen, parts of the app unexplored, gadgets you've found, ect. Collect your thoughts.
2. Take a breather
If you're already tilted, you're likely not forcing an unwarranted mental context switch. Go touch grass. Go lift weights. Go for a run. Meditate. Pray.
3. Consider why you may be tilting
Are you not finding bugs? Is there a lot of friction to testing on this target (short sessions, annoying hacker deterrents, etc) that you could automate away? Do you not find the app interesting?
4. Solve your tilt
If you're not finding bugs, switch up your technique. Are you not going deep enough? Go deeper. Are you going to deep and needlessly rabbit hole-ing? Resolve to test 30 endpoints in the next hour.
Is this part of the scope just not interesting? Go back to recon and find a new app to hack.
Is there too much friction? Automate it away, move on, or resolve to push through.
5. Give the above a shot for a couple of hours
If you don't break through, call it a day and try again tomorrow (1-3x)
If you don't break through still and you hate your life, switch targets. It's hard to stay engaged on a target like this and the best way to find bugs is to be in your flow.
Glhf, find bugs ;)
Here are my slides from my @0x41con presentation on Appleโs Lockdown Mode:
https://t.co/GQP0uIXbBC Enjoy ๐
โ ๏ธ Ironically the slides won't load when you have Lockdown Mode enabled ๐
See README for more info here - https://t.co/CIucgwnIss
๐บNew on the Apple Security Research blog: we pit our hardened kalloc_type XNU allocator against SockPuppet, a powerful vulnerability from the past: https://t.co/UyTkz1slu3
โThe only person that is paid to actually understand the system from top to bottom is the attacker. Everybody else is usually paid to understand their partโ (@halvarflake)
Itโs a bit old(2020) but I swear this is one of the best talk Iโve ever seen:
https://t.co/46oGLJGt0g
you see, `syscall` actually calls `do_syscall`. which in turn calls `_do_syscall`. got it? and that turns around and calls `____syscall`. which is basically a wrapper for `__syscall`, that's where the magic happens. it does `arg->ops->syscall` which was set in a diff code path.