Frances Liu

Co-Founder of Promptless

[Frances Liu @robobobots]

Something that no one is talking about is how all of the emerging knowledge stored in Skills or AGENTS.md or Obsidian is just different forms of documentation.

[Frances Liu @robobobots]

Genuine question, can someone explain to me what PG @paulg is talking about when he says do things that don’t scale? At what point does it cross into romanticizing busywork? For example - if I hand-wrap 300 gifts for our users. Is that “do things that don’t scale” or is that dumb

[Frances Liu @robobobots]

docs drift

#documentation #docs #Developers

[Frances Liu @robobobots]

@ay_ushr @mintlify hey you should try @promptlessai

[Frances Liu @robobobots]

For @promptlessai, our doc agents have a wide range of tools, including navigating codebases, retrieving context, browsing the web, etc. There is rarely a single gold answer for “the correct” documentation update, but there are almost always better and worse trajectories.

[Frances Liu @robobobots]

Counter intuitively, acting (tool calls) and thinking are not automatically aligned in agentic RL. In fact, they hurt each other with conventional LLM training.

[Frances Liu @robobobots]

Pairwise comparisons are more generalizable than scoring. The reward model becomes a critic, not a number generator. That makes it useful for best-of-N selection, self-reflection loops, and iterative agent improvement, etc.

[Frances Liu @robobobots]

You don’t need ground truth to build a good tool-call critic. Instead of requiring human experts to label the “correct” tool call in every context, ToolRM uses pairwise preferences.

[Frances Liu @robobobots]

Training Qwen3-4B/8B on their constructed dataset, the reward models achieved up to 17.94% higher accuracy, substantially outperforming frontier LLMs and existing reward models that are magnitudes larger.

A few non-obvious takeaways:

[Frances Liu @robobobots]

The method is very practical.
1. Collect agent trajectories, break them into context–tool call pairs
2. Sample alternative tool call choices from LLMs, and construct pairwise comparisons (“Option A vs Option B”)
3. Train reward model (RM) using GRPO.

[Frances Liu @robobobots]

when you jointly train reasoning and tool use, they can interfere. Gradients that improve reasoning can degrade tool behavior, and vice versa.

ToolRM proposes a lightweight reward modeling approach that is specifically designed to evaluate tool-use behavior, replacing generic RM

[Frances Liu @robobobots]

Last Friday at @promptlessai paper club, we read TOOLRM: Towards Agentic Tool-Use Reward Modeling: https://t.co/GJQsqA5asU.

This paper clicked for me because it complements another paper: Reasoning and Tool-use Compete in Agentic RL from the ByteDance team.

[Frances Liu @robobobots]

You can tag "Promptless" anywhere in a public github repo and get docs drafted for you. Pretty magical. @promptlessai

[Frances Liu @robobobots]

@promptlessai Related:
[2404.11018] ICL (In-Context Learning) https://t.co/HbiViXvxiB

[2507.19457] GEPA (Genetic-Pareto) (Beats GRPO by 6%) https://t.co/YpNrH1tAJN

[2504.07952] DC (Dynamic Cheatsheet) https://t.co/FOaJ9jgSi5

[Frances Liu @robobobots]

Last Friday, we read "Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models” (https://t.co/pkeYfoAWHU) as part of the @promptlessai paper club. Here's some tidbits that we found useful, so want to share it with other AI agent builders out there

[Frances Liu @robobobots]

@promptlessai (3/3) You don’t necessarily need ground truth labels for learning from the trajectories. Structured feedback from execution (success/failure signals) is enough. That’s really key for us as a small startup.

[Frances Liu @robobobots]

@promptlessai (2/3) Over-compression of context reduces performance. Instead of periodically compressing knowledge into short snippets. it’s better to build and update a synthesized context incrementally (A bulleted list playbook in this particular paper).

[Frances Liu @robobobots]

@promptlessai Takeaways that are not super obvious
(1/3) Don’t be afraid to feed models long, rich context. You can maintain long context without quadratic latency/cost blow-ups thanks to the many inference + attention optimizations.

[Frances Liu @robobobots]

@promptlessai It proposes a process that treats context as a first-class object that evolves over time with experience. It proposes a three-role loop — Generator (tries tasks), Reflector (analyzes outcomes), and Curator (incorporates insights) — to iteratively refine an agent’s playbook.