Prateek Agarwal

FDE field notes

[Pranav Pillai @PranavAPillai]

Pretty cool seeing @ModusAudit on Anthropic's first startup newsletter!

@Arushj01 @vinaykasat_ this counts as ROI on our token spend, right?

[Prateek Agarwal @prateeka6]

Change management is central to the Modus (and overall AI) thesis. Approaching this problem from a purely engineering mindset just doesn’t work. Developing a vision for the ideal way to run an audit firm is just the first step. The harder part is finding the fastest path to that goal state.

Part of finding that path is developing a deep intuition for the customer. What level of incremental change can they tolerate? What feels obvious to them that doesn’t feel obvious to me? How do I balance product changes they request vs process changes they need to make? How do I build observability in day 1 to link their behavior to progress towards my goal state? How do I change the rules, bit by bit, until it’s hard for them not to win?

[Prateek Agarwal @prateeka6]

Before building these kinds of feedback loops for customers, we first built these internally. We built a simple set of skills to “compound” the value of our customer calls. Each call is already recorded and transcribed with Fathom. The skill:

1. pulls in a transcript
2. finds the product discovery topic(s) it relates to
3. pastes the raw transcript (for quotes) and updates a summary for that topic (for high level learnings)

Couple benefits already:
•new engineers onboard incredibly fast (we’re hiring!)
•client calls are much higher signal. we never ask the same questions twice
•Another skill analyzes our performance as the interviewers and gives feedback to improve
•when developing software solutions, direct product feedback is available to the coding agent itself. This has proven invaluable to get right. The first step towards a self-healing solution

[Satya Nadella @satyanadella]

https://t.co/vLmiBKTtX3

[Prateek Agarwal @prateeka6]

Finding the wedge to bring users over to the platform can also just mean building for parity. We’re building a step change past 20yr old software that has barely been updated to suit our auditors needs. Change is hard though, so pushing a full fledged platform for adoption right off the bat is intimidating. Instead, delivering a re-skinned version of one of their core tools is much more palatable. The wedge here is “same thing but faster” rather than “delegate entire workflows to AI immediately”. Building that gentle adoption curve up front will (hopefully) pay dividends for more complex releases down the road

[Prateek Agarwal @prateeka6]

Every adoption problem is a trust problem first. Our auditors are very skeptical of new products. They've also been trained to review for decades - they'll catch every assumption and point out every edge case. This forces us to find the simplest possible thing to ship first to get them onto our platform so we can make incremental changes from there. It's not trivial to find that wedge but now that we have it we can expand

[Prateek Agarwal @prateeka6]

Engineers obsess over features. Users abandon tools after one bad experience. During a third of our onboarding sessions, something failed on the first try. Managers forgave us. Staffers immediately lost interest. Reliability isn't a nice-to-have. It's the whole game.

this week has been a lot learning how to put myself in the user's mind and anticipate their feedback. Interview time is precious and we can't waste it on known issues

[Prateek Agarwal @prateeka6]

The right beta tester may be the wrong champion.

I've been building a "glass pane" across data from a bunch of ancient account applications. Working with one manager in a tight loop was valuable to stay focused and fast. Every session got me closer to something that clearly proved value, even if for just one person to start. Once I had that product, I was itching to see it in production.

But he wasn't in a rush. To him, I'm just another tool in his stack. So instead of waiting, I called every senior and staff below him, got their buy in, and they're all onboarding tomorrow. They gave me more feedback and more confidence that we're in the right direction. Bottom up in an afternoon. Now his own team is pulling him into the adoption curve.

[Prateek Agarwal @prateeka6]

"Build it and they will come" is the easiest way to hide behind software. More often than not the user doesn't want more features, they want to be white gloved onto the platform. Do that well, and you have your first user. Do that really well, and you have a champion who will spread your product for you