@NOAA's NWS is adopting Icechunk for CIRRUS, the next-gen system replacing AWIPS.
๐ง ACID transactions + cloud-native I/O at PB scale
๐ 100x faster than existing NetCDF workflows
๐๏ธ Serving FAA, FEMA, NASA, USDA, DOE + every state
https://t.co/0Tc8kvDgJI
Very proud of the work we are doing. Free cloud datasets for science, SLAs for businesses, best consistency and performance we know of. See also our recent ERA5 dataset https://t.co/hwLyIESpO0
๐ฐ๏ธ Old file format? No problem! We made the full GOES-16 satellite archive queryable as a single Zarr store โ without copying a single byte ๐ง Find out how: https://t.co/tQ37HkMApA
๐ฐ๏ธ Old file format? No problem! We made the full GOES-16 satellite archive queryable as a single Zarr store โ without copying a single byte ๐ง Find out how: https://t.co/tQ37HkMApA
Ship fast, break nothing. ๐
We built a test that runs Icechunk v1 AND v2 in a single Python process, randomly interleaves their calls, and auto-shrinks failures โ using a tool called third-wheel ๐
Result: zero v1/v2 compatibility issues reported. โ
Full write up link๐
๐ Arraylake now speaks MCP.
Connect Claude, ChatGPT, or Cursor to your array data to browse repos, query Flux, render maps, all in natural language. Agents empowerment, no notebook required. โก
Full announcement & links to the docs๐
๐ง Icechunk V2: made more reliable for the scale V1 users wanted with help from great OSS tools.
๐ค๏ธ shuttle (@awscloud) systematically explores async task orderings to find race conditions
โข๏ธ toxiproxy (@Shopify) poisons network connections to harden retry logic
Links ๐
From GeoTIFF chaos โ cloud-native climate risk: Eoliann built AIRIS to predict floods, fires & droughts with precision across time horizons to 2050.
Powered by Earthmover, they turned TBs of satellite data into fast, decision-ready insights: https://t.co/FwOvZpETYZ
Icechunk V2 is coming soon- so to check compatibility with V1 pre-release, @xarray_dev Ian Hunt-Isaak built third-wheel, a tool to ๐ข๐ง๐ฌ๐ญ๐๐ฅ๐ฅ ๐ฆ๐ฎ๐ฅ๐ญ๐ข๐ฉ๐ฅ๐ ๐ฏ๐๐ซ๐ฌ๐ข๐จ๐ง๐ฌ ๐จ๐ ๐ญ๐ก๐ ๐ฌ๐๐ฆ๐ ๐ฉ๐๐๐ค๐๐ ๐ ๐ฌ๐ข๐๐ ๐๐ฒ ๐ฌ๐ข๐๐: witness the magic.
Announcing the Earthmover Data Marketplace: cloud native access to high-velocity weather, climate, & E/O datasets. Get access to ARCO ready data cubes from trusted providers, including forecasts from @ECMWF@NOAA- and exciting new AI-based forecasts!
๐ง Icechunk 2 is now available in alpha: Move/rename nodes without rewriting, shift chunks without consuming I/O, write metadata at the repo level, host repos on any HTTP server, and so many performance improvements. Read @rabernat and @paraseba's post for the full story! ๐
In case you missed it:ย Tomorrow @cherian_deepak will give a live demo that will take you from Zarr/Icechunk to Mapbox/MapLibre with a click of a button!ย Build faster, smoother maps with no pre-processing using our latest feature: Tiles. https://t.co/57nBKgIiJx
๐ง Icechunk 1.0 is production-ready!
Deployed by weather & climate data teams for rapid operational access to data on cloud
โก๏ธ 100x faster reads vs. NetCDF
๐ petabyte scale
Big day for #WeatherData teams!
๐ https://t.co/dAFQYOYSoV
#climateTech#OpenSource#meteorology
Lindsey Nield & @cherian_deepak from @Earthmover's #CNG2025 talk is live on YouTube. Learn about Zarr, Icechunk & Xarray for cloud-native data cube analysis.
Watch now: https://t.co/PVxkqGpAAi
Seriously blown away by this talk - likely the best I've ever seen on open data. @cherian_deepak gives a captivating recount of the historic context, clearly shows where we are falling short, and crucially, what we can do about it.
Don't miss it! https://t.co/60dOHlZpte
I had a boatload of fun preparing and delivering this talk! Watch the recording here: https://t.co/Xa64nipAkD Please reach out if you have comments or other hot tips on this topic!
1/ ๐ Solving #NASA โs cloud data dilemma: Icechunk unlocks 100x faster access to archival data formats
We're thrilled to publish results from our pilot project with @NASA + @developmentseed to enable high-performance cloud-native access for NASAโs 100s of PBs of EO data.
Join us tomorrow Oct 23 at 4 PM EDT for a Pangeo showcase on Icechunk: An Open-Source Transactional Storage Engine for Zarr by @rabernat! https://t.co/lsfzcXYaqn