Just launched Systune (a macOS app for disk space management and performance tuning) on Better Launch.
Please help me out by upvoting on Better Launch!
See the comments.
@tdevane especially given the history of abuses some of these companies have 💀hard to balance innovation and regulation esp as the open weight nonUS models are miles ahead of anything homegrown
Some people are interpreting this as a way for OSS models to bypass encryption of closed model APIs. If that is what you are taking away from this, please understand that the title is a massive overstatement in this context.
The point of encryption here is to keep the inference protocol stateless; I assume labs are fully aware of some side channel implications and do not care about preventing replay (why would they?) Encrypted traces offer a stateless inference protocol so inference backend can be distributed across hundreds or thousands of data centers without the need for coordinating sessions.
If your side channel is to ask the LLM "true or false" questions for the universe of words that might be contained in a trace, you'd go broke decoding a single trace. So you'd have to assume there's some super special magic trace that lived within some encrypted blob that's worth attacking in this way.
Overall definitely an interesting limitation of the encrypted vs stored traces, but you aren't going to be able to steal traces this way to train a competing model. That would cost more than building the model with the standard training arsenal.
a little saturday morning lite expo writing
New Post:
Of Assholes and Obsessives
on the distorted intensities of ego and conviction
https://t.co/1bF1cpH9hp
I’m excited to finally reveal Near Horizon to the world!
In early 2023, @mfk , @mjacobstein and I were frustrated at how great solo founders were told to “go find a co-founder” by traditional VC firms ...
...so set out to create a new kind of early stage venture firm!
1/
I’m excited to finally reveal Near Horizon to the world!
In early 2023, @mfk , @mjacobstein and I were frustrated at how great solo founders were told to “go find a co-founder” by traditional VC firms ...
...so set out to create a new kind of early stage venture firm!
1/
OpenAI API compatibility shipped for 100+ models on @togethercompute API.
Replace GPT calls with Mixtral or Llama-70B, get faster responses and for less $$
🚀🚀🚀