If Strata is serving 3.8 Flash Next for a long session of coding, how frequently are different experts being consulted? Is the set of experts used stable, or highly randomized? Could Strata be tuned such that, for a single task like "coding" it would prefer some experts over others, maybe temporarily pinning some to the VRAM and raising the STRATA_ROUTE_TAIL_SKIP to 3 or 4 if those experts are truly the "best code writing experts in the model"? If one expert truly is the GOAT coder, I only really want that one, and the others are only contributing noise, waste, heat, time which gets discarded anyways.
If I switch efforts to generating prose, could Strata know / measure / test which experts are best for that purpose and pin them similarly setting the TAIL_SKIP to a low number?
Or is all of this academic because hundreds of experts are truly needed for a single task like coding?
@grok I'd appreciate if you could weigh in here too about how many experts are truly required for a single task. If the task is short like "Pick a random color in hex" or "Pick a random English name", does the model really need 10 experts evaluated for every token, or would 5, or maybe even 1-2 do? And then the same goes for coding, could a smaller subset of experts be resident / consulted? Could a smaller number of experts be executed for a token, saving compute time / bandwidth? Could this be measured? Could a test suite be designed that explored the latent space of a model's experts one-by-one to determine what they're "best" at to be smarter about which experts are consulted, when, and when many are needed versus a few?
I don't think you understand me completely. In strata, there is an option to generate the OpenCode config so that when I open OpenCode it has every model being served or available for serving pre-configured for OpenCode. Hand building that JSON is a huge pain, especially when lots of models are involved. It would be great if TF could generate and place that file in the correct format to make it seamless for users
@ashxhart@noelkimx Can Tensorflow create an OpenCode config automatically and put it in the right directory in the v2 format? Strata can't and it's so frustrating
@_weilix@grok how can this be incorporated into local inference tasks and harnesses? Could this be wrapped as a plugin for popular harnesses? What infrastructure is required?
@lium_io How does this work if I want to also occasionally do work on my personal node/GPU? Can I queue up work such that it runs when the node is idle for @lium_io ?
@ColeMacro@Werkman@PunterJeff congratulations on the $SATS announcement, I'm super excited to see that IPO get massively oversubscribed ๐
Also, ignore the haters and pundits. Don't even respond to them. I think fundamentally they forget, if it were in the best interest of the $ASST shareholders you could make your BTC purchases on the open market and wipe out the sell side orderbook in an instant, and you could do it every week, forever. BTC softness doesn't concern this leadership or the shareholders at all. Maximize the amplification, because anything less is boring and if I want to invest in a boring company I would do that.
@DevotedDividend $566,600 in $SATA yields $73,658 per year as 100% RoC so you don't pay income tax on it until your cost basis drops to zero over approx. 8 years. Also I prefer getting paid every business day