Sagi Bashari

Founder, Varve | system of record for object storage | ex-CTO, MyHeritage

[Sagi Bashari @sagiba]

@thsottiaux do you get charged long context rates (2x) when enabling this? so basically you pay both for the longer context + the rate itself is higher? also with subscription?

[Sagi Bashari @sagiba]

@blakeflayton השיער שלך נראה מצויין

[Sagi Bashari @sagiba]

Apparently they released an option to bump this recently, see the model_context_window param docs at https://t.co/4VgcPhTmuf

Downside is that once you pass 272K you're charged Long Context rates, which are 2x more expensive (on par with Fable pricing).

[Sagi Bashari @sagiba]

@rezoundous do you know if this is because of session transcript storage, or because of artifacts the agents are creating for various outputs? I happen to work a couple of OSS projects, each tackling a part of this, curious to know what's the problem you're seeing :)

[Sagi Bashari @sagiba]

Listing a billion-object S3 bucket with no inventory takes weeks with the AWS CLI. Swath does it in parallel: guesses how to split the keyspace, fixes the guesses as real keys arrive, steals work as it learns the bucket's shape. Out today: https://t.co/75QWOUd159

[Sagi Bashari @sagiba]

@awscloud Permissions are the easy darkness.

The scarier one is the 1 PB bucket nobody knows what’s inside anymore.

[Sagi Bashari @sagiba]

https://t.co/pzM15plIMR

[Sagi Bashari @sagiba]

Wrote up some thoughts on S3 Intelligent-Tiering. It’s about as smart as it can be, but only because it doesn’t know much. It sees access patterns. Sometimes you actually know what the data is and what should happen to it, and S3 doesn’t really give you a place to say that.

[Sagi Bashari @sagiba]

@QuinnyPig, your Scribd episode with R. Tyler Croy stuck with me. The 5PB a data scientist abandoned in 2012 is a symptom: S3 has no first-class place for the metadata that would tell you what a dataset is, who owns it, whether it's still alive.

[Sagi Bashari @sagiba]

Amazon S3 has buckets and objects, but no first-class concept of a dataset. A lot of cost, governance, and operational pain in object storage starts with that gap.

https://t.co/HyISfaVQ6o

[Sagi Bashari @sagiba]

Amazon S3 has buckets and objects, but no first-class concept of a dataset. A lot of cost, governance, and operational pain in object storage starts with that gap.

https://t.co/HyISfaVQ6o