ironArray

A multi-dimensional data container and computational engine optimized for big-data applications.

[ironArray @ironArray]

#Caterva2 cannot only be used for sharing your compressed datasets in the internet, but also to efficiently perform operations on datasets exceeding available memory.

Look at our new blog explaining how this works 👉 https://t.co/vEdgGYXvmG

Make compression better 😀

[ironArray @ironArray]

Honored to put our team and knowledge in this work!

[Blosc @Blosc2]

📢 Learn how using Blosc2 and Btune is improving the compression ratio of data (both lossless and lossy) coming from photon sciences. We were able also to reach compression speeds exceeding 23 GB/s.

Report 👉https://t.co/VC8DlWnYZe

Make compression better 😀

[ironArray @ironArray]

ironArray joined Twitter 3 years ago already? Really? Proud to see how much progress we did during these intense years! 🎉🎉 🚀🚀

[ironArray @ironArray]

See why #ironArray can make your #data simulation code to fly 🪁🪁:
https://t.co/Kcta2u6Esj

[ironArray @ironArray]

Reductions are the bread and butter of everyday data analysis. ironArray allows doing this for on-disk data faster and consuming much less memory (up to 20x) than e.g. Dask+Zarr.
https://t.co/vWBCBcBzbM
Do more with less.

[ironArray @ironArray]

User Defined Funtions are a data mini-language (DSL), where code is intertwined with compressed data at a fundamental level to achieve best performance.

See how to create libraries of UDFs for operating with your data at ⚡️ speed: 👇
https://t.co/tOmzv8Pd2Z

Do more, with less.

[ironArray @ironArray]

🎉🎉New blog on estimating 𝜋 by throwing darts and using ironArray:
https://t.co/tOmzv8xCbr

In this installment: high quality random number generators, User Defined Functions and reductions. See how ironArray can consume much less time and memory, compared with #NumPy/#Numba 👇

[ironArray @ironArray]

ironArray comes with a nice assortment of blazing fast random generators.
 Check our tutorial here: https://t.co/7hp0M6KP6h

API: https://t.co/qCw1zYoxQm

But as there is more to life than speed, our tests also guarantee that distributions pass the Kolmogorov-Smirnov test 👇

[ironArray @ironArray]

ironArray implements a matrix multiply that consumes (much) less time & memory than other parallel solutions. Even on Intel machines, ironArray can go at a speed that is just 2x slower than NumPy+MKL.
Read more 👉 https://t.co/DOm7Y3xEkx
Save time & energy with ironArray ⚡️🌲

[ironArray @ironArray]

Persistent storage has been gaining performance by leaps and bounds since solid state disks introduction. Check how out-of-core computation can lead to huge savings in memory compared with other solutions, while also performing decently fast: https://t.co/DvjUiCIRkX

[ironArray @ironArray]

Did you know that ironArray comes with AI-driven compression algorithms that lets it easily adapt to your preferences?
SPEED: favor speed
CRATIO: favor compression ratio
BALANCE: balance among the two above
No more time wasted deciding the codec to use! 👉https://t.co/UtnhfgdPZ5

[ironArray @ironArray]

Did you know that our array constructors work in parallel and can go more than 10x faster than NumPy?
And that by leveraging compression you can host way more data using the same memory resources?
More info 👉https://t.co/7RZWoeWAlN
Enjoy speed and compactness with ironArray!

[ironArray @ironArray]

Did you know that our Enterprise license comes with up to 100 hours of support (negotiable)?
We want to *collaborate* with you to make your interaction with ironArray (both the library but *also* the company) as fluid and interactive as possible.
Your success is our fulfilment!

[Blosc @Blosc2]

🎉We are happy to announce that we have moved most of our documentation into https://t.co/J1lkRCdiPm (see the new navigation bar on top).
The doc pages in https://t.co/cjxMuql7jq have been removed for now. Sorry for the inconvenience.
Thanks to Marc Garcia, for all the help!

[ironArray @ironArray]

Did you know that ironArray can access #Zarr remote arrays in a transparent way via the Zarr proxy array?
See our tutorial here: 👇 https://t.co/r8fJvi9FhL

[ironArray @ironArray]

Often times you need to think about a problem from the inside-out. This is how we implemented User Defined Functions to re-define computing for a more cost effective usage of your computing facilities:
https://t.co/Tp5jboxUyw

[ironArray @ironArray]

Surpassing the memory wall requires a wise and well balanced combination of computation and compression, at the right time *and* at the right place.

Here is how we are achieving this:
https://t.co/kNzCOdRGUM

[ironArray @ironArray]

If you need constructors tailored to your needs, you can easily code them via our powerful User Defined Functions (UDFs).
E.g. the UDF below can build a triangular matrix at 3x the speed of NumPy and up to 5x faster than Numba (fully multi-threaded), while using 100x less memory.

[ironArray @ironArray]

In latest 2022.2 release we introduced *parallel constructors* to build arrays using multithreading.
This, combined with fine-tuned random generators, allows for amazing speed-ups and memory savings. Check it out in our latest tutorial:
https://t.co/jcmmWk58Mp

[ironArray @ironArray]

And the random Poisson (integers) constructor is up to 40x faster, while consuming 10x less memory (with lossless compression):