everyone is building agents and testing them by using them twice and calling it a day.
then a user asks something slightly off script and the thing books two flights instead of one.
simulated users is the one piece nobody wants to build in house. i have tried. you end up writing fake users who are too polite and too logical. real ones are neither.
and evals that check actual tool calls instead of the final message, that is the part that saves you. agents are very good at saying done.
great work @ilan_kadar and team :)