Gaurav Koley

Poet, Design enthusiast, Doctoral student @BU_CDS, Creator of Gratia and Pdfvuer. Vue and RoR fan. Web Science researcher, @IIITB_official, @microsoft alum

[Gaurav Koley @arkokoley]

@thsottiaux Down to start. Where do we build the Foundation? I'll bring my book on Psychohistory.

[Gaurav Koley @arkokoley]

@avataram Glad to have been in the circle for a long time now!

[Gaurav Koley @arkokoley]

Yes. Clicking is the demo. Session, scope, and verifying resulting state are the product.

Same design constraint while building Narada: granted tabs, approvals, activity, and whether the page actually changed have to live in the browser, not in whichever model is on the other end of the wire.

https://t.co/ViEoEMCviT

[Gaurav Koley @arkokoley]

This is the exact failure mode that made me put take-over / pause in the browser instead of hoping the model is polite.

If the agent can grab focus on password managers and credential UI, "computer use works" is not the same as "this is safe for a person sharing a machine."

Browser has to own scope + sensitive-action stops. Person has to be able to yank the leash without killing the whole session.

[Gaurav Koley @arkokoley]

Yes. Clicking is the demo. Session, scope, and verifying resulting state are the product.

Same design constraint while building Narada: granted tabs, approvals, activity, and whether the page actually changed have to live in the browser, not in whichever model is on the other end of the wire.

https://t.co/ViEoEMD38r

[Gaurav Koley @arkokoley]

This matches what broke for me too.

Screenshot-first agents feel general, then fall over on multi-step work: stale coordinates, duplicate labels, page looks ready while nav is still running.

Accessibility / structured tree is a huge win when it exists. I still keep visual as a fallback for canvas/PDF/opaque UI, but not as the only language.

Wrote that split while building Narada:
https://t.co/H9cyCjnktn

[Gaurav Koley @arkokoley]

Hot take from building an agent browser this week:

Computer use is an incredible fallback. It is a terrible only language for multi-step work.

The browser already knows the tab, the URL, loading, and interactive controls. Making the model rediscover all of that from pixels every step is leaving free information on the floor.

I wrote up the distinction (visual when seeing adds info; structured state when the browser already knows):
https://t.co/H9cyCjmMDP

Building this as Narada → https://t.co/5ES3tBQ6cl

[Gaurav Koley @arkokoley]

If an agent can act inside a browser, who should remember what it is allowed to do?

The model could. The app that connected it could. A chat panel could keep a list somewhere. All three work until the model changes, another client connects, or the person takes the tab back.

The browser is the only part that sees the whole situation: tabs, the page, the connection asking for an action, whether that action is sensitive, and whether the person is using the page again.

I kept running into this while building Narada. The model could finish a turn, but the browser still had to decide scope, approval, and take-back. So the browser has to own those rules.

That is the product model: not a browser with an agent bolted on, but a browser as the control plane for agents acting on a person's behalf.

In Narada the browser owns that interface. External agents can connect. The first-party Agent Host uses the same path inside the app. Models and clients can change; the browser's rules do not have to move with them.

Scope starts with granted tabs, not silent access to everything open. Actions show up in browser activity. Sensitive actions stop for approval. If you start interacting with the tab, that is a change in authority, not noise for the agent to ignore. Take over pauses the agent so you have the page again.

"I clicked it" is not enough. The browser should report what actually changed so the agent can continue, recover, or stop.

The objection is that an agent-aware browser gives the agent more control over the person. It should mean the opposite. Follow, take over, cancel, and approval stay browser decisions because they belong to the person, not to whichever model is running.

Models will keep changing. Identity, scope, approvals, activity, and user control need a stable home in the browser.

Blog: https://t.co/ViEoEMD38r
Narada: https://t.co/5ES3tBQE1T

[Gaurav Koley @arkokoley]

6/ Agent-aware software should not take control away from its user. Follow, take over, cancel, and approval remain browser decisions because they belong to the person, not the current model.

Blog: https://t.co/RHB7XhGl8A
Narada: https://t.co/gVpxaWpFFK

[Gaurav Koley @arkokoley]

2/ I kept running into this while building Narada. The model could finish a turn, but the browser still had to decide scope, approval, and whether the person had taken the tab back. I think the browser has to own those rules.

[Gaurav Koley @arkokoley]

@thsottiaux 's resets have been very very useful in building :D

[Gaurav Koley @arkokoley]

Computer use is a remarkably general browser interface. It is also strange to make it the only one. The browser already knows the active tab, URL, loading state, and interactive controls. Why make the model rediscover all of that from pixels?

[Gaurav Koley @arkokoley]

Next: if an agent is more than a remote hand on the user's mouse, what should the browser itself own?

Blog: https://t.co/cye7XV1ORv

[Gaurav Koley @arkokoley]

Been my experience every time I have tried to use Fable

[Spencer Shulem @Spshulem]

When Fable thinks for 25 mins and hits your session limit before giving your response

[Rahul Raj @x_rahulraj]

For most of our lives, we are measured by ordinary questions.

How much do you earn? How much have you saved? What car do you drive? What is your company's valuation? Where did you spend your vacation? Which restaurants you eat in?

But every once in a while, you come across people who have quietly stopped playing that game.

They choose to wager their youth, their careers, their savings, and often their peace of mind on an idea that appears absurd to everyone else. They are drawn not by certainty, but by possibility. Their ambitions are so disproportionate to the odds that failure is almost the expected outcome.

Engineers who attempt moonshots belong to this rare tribe.

Today's Skyroot launch is one of those moments.

[sphinx @protosphinx]

PRIVATE DEDICATED ORBITAL LAUNCHERS: FIRST ORBIT

🇮🇳 Skyroot Vikram-1 (350 kg) ············· 18-Jul 26 ··· Attempt 1
🇺🇸 SpaceX Falcon 1 (420 kg) ·············· 28-Sep 08 ··· Attempt 4
🇺🇸 Firefly Alpha (1,030 kg) ·············· 01-Oct 22 ··· Attempt 2
🇺🇸 Astra Rocket 3.3 (~150 kg) ············ 19-Nov 21 ··· Attempt 4
🇺🇸 Virgin Orbit LauncherOne (500 kg) ····· 17-Jan 21 ··· Attempt 2
🇺🇸🇳🇿 Rocket Lab Electron (300 kg) ·········· 21-Jan 18 ··· Attempt 2
🇨🇳 i-Space Hyperbola-1 (~300 kg) ········· 25-Jul 19 ··· Attempt 1
🇨🇳 Galactic Energy Ceres-1 (~400 kg) ····· 07-Nov 20 ··· Attempt 1
🇷🇺 Comparable private orbital launcher: none

[Gaurav Koley @arkokoley]

@adelwu_ 👋

[Gaurav Koley @arkokoley]

Burnt through the last reset on Codex. Debating whether I should use a reset or wait for Tibo to give us another one!