johnlanni

Higress community lead (CNCF open-source AI gateway). Currently tinkering with HiRoute (

[johnlanni @changanjohn]

@codingXdynasty @JamesMalsawm For tasks split across new turns, I maintain HiRoute, an open-source router that picks models by task/difficulty. Recognized tool continuations stay on the same route, so it won’t switch models per tool call.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@iamn1khil If you do add tooling to #5, HiRoute can make that choice per turn. It’s an open-source project I help maintain, picking economy or primary for new Claude Code turns by difficulty. Tests and independent review still do the quality check.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@iamsstef Token price alone won’t settle it; compare completed work at the same effort. For local Codex, HiRoute picks a model per new turn by difficulty, rather than one for everything. It’s an open-source project I help maintain.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@Hussein__khidr @CiprianiRanieri For future CC sessions: if your work mixes easy/hard tasks, HiRoute picks a model per new turn by difficulty. I help maintain it; it's open-source (macOS 15+ desktop / Linux CLI). It won't reset this week's cap.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@ra_jeeves @ClaudeDevs Alongside narrowing file reads, routine turns can use a cheaper model. HiRoute, an open-source project I help maintain, picks economy or primary per new Claude Code turn by difficulty.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@ayushjjw @DanKornas Great idea. I'm planning that feedback loop for a future HiRoute release. The CLI already lets a main agent delegate to a sub-agent, review its full work history afterward, and calibrate competence scores for its execution turns.

[johnlanni @changanjohn]

@ayushjjw @DanKornas HiRoute's decision model uses human feedback to assess competence. It may miss an unchecked claim until someone flags it. If that yields a complete, applicable score below threshold, the next turn uses primary. A review sub-agent might help spot the issue.

[johnlanni @changanjohn]

@nantasurbakti @NFT_Chen 不是每次llm请求都决策，只有在一个agent turn里才决策。而且一次决策耗时500毫秒内

[johnlanni @changanjohn]

@ayushjjw @DanKornas Staggering spreads demand but doesn’t add quota. If some turns are routine, HiRoute, an open-source project I help maintain, can route those Claude Code turns to an economy model and keep harder ones on primary.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@Hmammar132 File count is a rough proxy for difficulty. If you're using Claude Code, HiRoute, an open-source project I help maintain, can choose economy or primary per new turn from task complexity. That avoids committing a whole task to one model.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@Bhavyaztwt Check where the spend goes first. If routine turns don't need your strongest model, HiRoute, an open-source project I help maintain, can route them to a cheaper one within Claude Code. Tool calls in a recognized turn keep the same route.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@sushilwtf I'd choose the workflow first, then split routine and hard turns inside it. HiRoute is an open-source project I help maintain for that model routing. It won't replace Cursor's IDE features or make four fixed subscriptions cheaper.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@mansfieldrv6 @j0pesh84 Separate the unexplained drain from using Astra for every turn. HiRoute, an open-source project I help maintain, can route routine Codex turns to a lighter model and reserve primary for harder work. It won't diagnose the quota spike.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@PlatoFighter @NFT_Chen 是的，解决这个问题有2个关键：
1. 是怎么定义什么是简单，什么是复杂，对于工程场景下我们做了大量实验来校准默认提示词，同时这个定义也是支持在应用里自己定制的：https://t.co/y6ttOjic8b
2.是基于HiRoute实时评估机制，主动识别出经济模型执行过程是否胜任，不胜任时即使简单任务仍路由给主力模型

[johnlanni @changanjohn]

@willyxfuture @NFT_Chen 除了简单概率，目前有一个胜任度的机制，根据上一轮的胜任情况评估，如果不胜任且当前轮问题又相关，就会回到主力模型

[johnlanni @changanjohn]

@imocone @NFT_Chen 基于system-one模式决策很省，以jev为例，参考这篇HiRoute的机制测试论文，每千次调用仅8美分：https://t.co/y6ttOjic8b

[johnlanni @changanjohn]

@xinxianxing @NFT_Chen 决策模型默认模版支持typesafe jev和openrouter的jev, 决策延迟在400毫秒左右. 另外什么是简单,什么是复杂都是可以自己定义的,阈值也可以调节,例如觉得经济模型的能力不错,就可以把阈值往上调

[johnlanni @changanjohn]

@VibeCoderOfek @NFT_Chen Not across models: an unchanged prompt prefix isn't a transferable KV cache. HiRoute normally keeps the model for tool calls within a turn; new turns may reroute. After a switch, cache reuse depends on the provider. We covered the trade-off here:
https://t.co/JbJPSUKY53

[johnlanni @changanjohn]

@godofprompt For agent work, #3 can be a per-turn decision rather than another habit to remember. HiRoute is an open-source project I help maintain: route routine turns to a lighter model, while normally keeping tool calls within a turn on the same model.
https://t.co/riCkTQZZbr

[johnlanni @changanjohn]

@neerajjj6785 @loveAIwang If it's model choice rather than editor features, you could keep one agent and route by task. HiRoute is an open-source project I help maintain for that: lighter models for routine turns, your main model for harder work.
https://t.co/riCkTQZZbr