@TurnerNovak@rexsalisbury ‘Per user’ is not the right way to scope consumption.
Users are not a relevant proxy for agents.
Multiple agents can act on behalf of a user.
Agents that reason and use tools will represent the majority of token consumption.
The framing of this analysis is not very relevant.
@tbpn@EverettRandle Definition of ‘agent’ is clear.
An agent is the combination of:
(1) instructions / prompt
(2) tools used to support instructions / prompt
(3) data used to support instructions / prompt
@arpitrage@arpitrage - open source modes do not offer indemnification. Open source users must indemnify the provider. Consumers may be comfortable but enterprises are not
@tgeisenheimer@jasonlk@bhalligan If an agent requires many tools to complete a task/set of tasks (ie deliver an outcome), then the token consumption can be significant.
The outcome may not be super valuable.
The cost of the former (consumption) does not necessarily equate to the value of the latter (outcome).
@mwseibel Do you think this is also true for revenue you help your customers retain?
Arguably it is more expensive to replace an existing customer than to keep one and grow.
We are opening up a new role at Quora: a single engineer who will use AI to automate manual work across the company and increase employee productivity. I will work closely with this person.
🚨 TRUMP JUST SAVED COLLEGE SPORTS.
No more billionaire bidding wars.
No more boosters buying rosters.
No more chaos.
Trump just signed an executive order banning the “pay-for-play” deals that turned college football & basketball into free agency.
Here’s what it means 🧵
Context Rot
Great title for a report, but even better insights about how increasing input tokens impact the performance of top LLMs.
Banger report from Chroma.
Here are my takeaways (relevant for AI devs):
We are opening up a new role at Quora: a single engineer who will use AI to automate manual work across the company and increase employee productivity. I will work closely with this person.