DataSpreads

@DataSpreads! is an ultra-fast edge platform for data that you trust, own and share.

[DataSpreads @DataSpreads]

@yaamehn DataSpreads is on pause, what is published is a proof of concept. Spreads went to a wrong direction (with generics) and is (very slowly) being redesigned. DataSpreads project depends on Spreads, no ETA...

[DataSpreads @DataSpreads]

A lot was done in sibling & private repos: fast IPC/network series sync, fast compressed storage, BBG-quality charts. PM if interested.

[DataSpreads @DataSpreads]

@ClumsyPilot @github It's embedded real-time TSDB engine

[DataSpreads @DataSpreads]

Поехали! Open source ultrafast embedded real-time cross-process time series database is on @github. #dotnetcore #LMDB #SQLite #CosmonauticsDay https://t.co/Utc23USmfO

[Victor Baybekov @buybackoff]

Released zero (alpha) version of @DataSpreads. No docs yet, but there is a sample that sends and receives messages using #csharp 8.0 async streams at 14 million messages per sec accross processes. Not stable yet, but it's a matter of time to stabilize it.

[DataSpreads @DataSpreads]

Working on documentation and releases of Spreads library 1.0 and @DataSpreads alpha. Doc site template is work-in-progress, has only generated API docs now. But please subscribe to early access announcements. No spam promised, only important milestones. https://t.co/5L4kzDwd6n

[Victor Baybekov @buybackoff]

It's 4 AM... During the day improved initial throughput (for 512 writers) by 70% + moved 10msec latency from 99.9% to 99.999%. This includes generic serialization, works across processes, almost ready to use as a library and has awesome absolute performance! #dotnet @DataSpreads

[Victor Baybekov @buybackoff]

Utf8Json is fastest #json, all credits go to @neuecc. But with some #dotnet discipline it could be even faster. It also could read from native memory without copying and reuse buffers from pools without cleaning them for primitives. @DataSpreads now is faster. Code will be later.

[Victor Baybekov @buybackoff]

@McCrews @bruinbrown93 @DataSpreads I will be pushing it to 1.0 in the following months with actual production use case for trading (real time, low latency, no GC reqs) and backtesting (high throughput req). Please leave issues/comments/wishes and uncovered use cases on GitHub.

[Anthony Brown @bruinbrown93]

@McCrews I'm not sure whether you've checked out Spreads yet (https://t.co/XBj1xxt0sM) but it could be something to look at

[DataSpreads @DataSpreads]

New interfaces towards 1.0 [WIP]
https://t.co/xxBD8HMgrT

[DataSpreads @DataSpreads]

Updated to .NET Standard 2.0. Small change. Updating VS to 15.3.3 took longer. The library was already 1.6. Support for .NET 4.5.1 remains.

[DataSpreads @DataSpreads]

To make it beta and then stable, community feedback, evaluation and contribution is much needed & appreciated! https://t.co/Zeu4w4rQsG

[Victor Baybekov @buybackoff]

Released @DataSpreads as 1.0-alpha1. Many things have been reworked & optimized recently. #dotnet #csharp #fsharp https://t.co/TAr4TTpTW4

[Victor Baybekov @buybackoff]

70% faster dictionary, 2x faster comparers with @DataSpreads Unsafe lib. First step to Spreads 1.0. #dotnet #csharp https://t.co/sS0dZ5y9dV

[DataSpreads @DataSpreads]

With the previous perf boost from April 14th it is now 14x times faster than 1.5 months ago... Not bad :) But sadly no more room left.

[DataSpreads @DataSpreads]

Now the internals design looks right from nano-optimization perspective, time to add other useful features! https://t.co/bH7hT4JSJS

[Victor Baybekov @buybackoff]

Performance is a feature! New Zip implementation in @DataSpreads is 7x faster due to deed type specialization. #dotnet #csharp #fsharp

[Victor Baybekov @buybackoff]

Doubled performance of one of the main @DataSpreads features - ZipN. Mostly due to avoiding IComparer interface calls and better inlining.

[DataSpreads @DataSpreads]

Published #NuGet packages targeting Netstandard1.6 framework & tested on #Linux, including native #Blosc serialization with #LZ4 and #Zstd.

[DataSpreads @DataSpreads]

Complete .NET Core package is on its way https://t.co/bYR4tzDfCy

[Victor Baybekov @buybackoff]

Yay! First green Travis build for @DataSpreads. The Core project with a native library works on #Linux + #dotnetcore

[DataSpreads @DataSpreads]

50x faster than naive implementation for window size = 200 and no allocations! https://t.co/3nkg6JHAQy

[Victor Baybekov @buybackoff]

Moving median is hard! Ported online algo from @LIGO to #csharp. Excited to have such a great origin! @DataSpreads https://t.co/9pb4Oqr8Ab