Gerald Woo

Interested in AI/ML.

[Ameet Talwalkar @atalwalkar]

We’ve released a technical report for Toto 2.0 detailing the data, architecture, training recipe, μP/u-μP hyperparameter transfer pipeline, and benchmark results behind our 5-model open-weight release. Report linked below.

[Gerald Woo @woo_gerald]

@BitsOfChris This is so cool Chris!!

[Chris Lettieri @BitsOfChris]

I bought a $200 weather sensor and fed the data into the time series foundation model my research team at Datadog just released.

Call it personal observability for the weather.

It works surprisingly well.

My team at Datadog AI Research just released Toto 2.0. It’s a family of five open-weight time series foundation models from 4m to 2.5B parameters. We achieved the top spot on GIFT-Eval, BOOM, and TIME forecasting benchmarks.

Toto is a transformer, just like most LLMs. Except Toto predicts the next patch of numbers instead of the next word. It was trained on observability and synthetic data, never weather. That it works on weather at all is the whole point of foundation models.

Now in my backyard the weather app has been horribly wrong. My wife and I would complain about the temperature forecast being 5-10 degrees below what actually seems to happen in the afternoon.

The setup is simple. I use the Ecowitt WS-90 sensor and the GW3000 gateway. This streams live data from my backyard to my Hugging Face Space. The data is then fed to Toto-2.0-22M and forecasts the next 48 hours. I then track the accuracy against the National Weather Service’s forecasts.

Five days in, Toto is winning the 1-hour forecast on both temperature (1.9°F MAE vs NWS 2.5°F) and humidity (4.6% vs 7.0%). NWS pulls ahead at 3h and 12h.

Toto reacts fast to fresh sensor data, which dominates the near term. NWS has radar, satellites, and neighbor stations feeding the regional context that dominates longer horizons.

I just love that Toto works here. Trained on server metrics, never on weather, finds enough signal in noisy backyard readings to beat a national forecasting agency at the near term.

Live on HF: https://t.co/sOWwD0fyVZ

[clem 🤗 @ClementDelangue]

Are scaling laws finally working for time series foundation models?

Today, @datadoghq is releasing Toto 2.0 weights in Apache 2.0 on @huggingface. It's a family of open-weights TSFMs from 4M to 2.5B parameters, where every size beats the last from a single hyperparameter config. First across the leading benchmarks: BOOM, GIFT-Eval, and TIME.

Most TSFM families ship multiple sizes that all perform roughly the same. This one doesn't.

Why it matters: scaling laws gave language and vision a predictable relationship between compute, data, parameters, and downstream performance. Time series hasn't had that curve until now. Once you have it, you can scale data and compute with confidence, and start asking which new capabilities emerge at the next order of magnitude.

2.5B open-source weights: https://t.co/prpcGoCw0U
4M open-source weights: https://t.co/5d6rw5NYL2

Blogpost: https://t.co/xKazgTMh1I

[Ameet Talwalkar @atalwalkar]

Today we’re releasing Toto 2.0: a family of open-weights time series foundation models spanning 4M to 2.5B parameters.

The question we set out to answer was simple (yet previously open): Do time series foundation models get reliably better as they scale?

Our answer: yes! 🧵

[Chris Lettieri @BitsOfChris]

Never bench press to Ed Sheeran.

I was surprised to learn you’ll lift at his tempo.

Here’s another surprise: you can train a time series foundation model with zero public data and still top every public benchmark.

That model is Toto 2.

My team at Datadog AI Research is releasing today: a family of 5 open-weight models from 4m to 2.5B parameters.

A few things worth flagging:

→ Scaling works. Every size beats the one below it, with no saturation at 2.5B.
→ Trained only on Datadog observability + synthetic data. No public forecasting data in pretraining, yet it leads on general-purpose benchmarks.
→ Top of every benchmark we tested — BOOM (observability), GIFT-Eval (general-purpose), and TIME (a new contamination-resistant zero-shot benchmark).
→ 7× more parameter-efficient than Toto 1 at matching quality, and a lot faster at inference.

Read the blog: https://t.co/qQBCZOZuue

Try it out on HF: https://t.co/VA4qlyUY7Y

[Othmane @ThisIsOthmane]

Scaling finally works for Time Series Foundation Models.

Introducing Toto 2.0: open-weights TSFMs from 4M to 2.5B params, where every size beats the last from a single hyperparameter config. #1 on leading benchmarks: BOOM, GIFT-Eval, and TIME.

Most TSFM families ship multiple sizes that all perform roughly the same. This one doesn't.

[Datadog, Inc. @datadoghq]

Toto 2.0 is here: Datadog AI's 5 open-weights forecasting models (4m-2.5B params) finally make scaling work for time series forecasting! #1 on BOOM, GIFT-Eval, and TIME. Weights/code Apache 2.0.
🔗 Read the blog post for more details: https://t.co/CEQSfUhvOA

[Stephan Xie @stephofx]

How well do AI systems (LLMs, VLMs, time series FMs) answer questions about time series data📈?

On ARFBench, the best models achieve ~63% accuracy on real incident data. But models and human experts fail in different areas: combining them achieves 87% accuracy.

🧵1/

[Ameet Talwalkar @atalwalkar]

Heading to @iclr_conf next week? Datadog AI Research is hosting an event along with the ICLR SPOT workshop organizers. Join us to meet our team and learn about our work on observability world models + agentic post-training!

[Gerald Woo @woo_gerald]

@cloneofsimo We’re all in a very Chinese time of our lives

[Gerald Woo @woo_gerald]

@bryancsk Somewhere out there, there’s a cow called Junwei

[Ben Cohen @bthecohen]

I will be at @NeurIPSConf in San Diego this week with colleagues from @datadoghq presenting our work on Time Series Foundation Models for observability! If you want to chat about AI/FMs for o11y, agentic RL, or scaling RL environments and RL infra, DM me! We’re hiring for RS/RE.

[Ameet Talwalkar @atalwalkar]

I'll be at  @NeurIPSConf with several of my Datadog colleagues. We're actively hiring Research Scientists & Research Engineers, and would love to chat at our booth in the expo hall, during the poster session where we're presenting Toto, or at my talk at the TSFM workshop!

[Gerald Woo @woo_gerald]

@mvaloatto @ClementDelangue @huggingface @jonatasgrosman @Lin_Chen_98 @Emily_Alsentzer @patrickjohncyh

[Horace He @cHHillee]

For too long, users have lived under the software lottery tyranny of fused attention implementations.

No longer.

Introducing FlexAttention, a new PyTorch API allowing for many attention variants to enjoy fused kernels in a few lines of PyTorch.
https://t.co/IXeUS6AkrY
1/10

[Caiming Xiong @CaimingXiong]

👋Hi everyone👋,

we are releasing our updated time-series foundation model Moirai-1.1-R and our time-series code repo UNI2TS 1.1.0!

The new Moirai model achieved significant improvements (~20%) for low-frequency cases like Yearly and Quarterly data in Normalised Mean Absolute Error (NMAE) for 40 datasets on the Monash repository.

The new code repo adds hpparams context manager to support hyperparameter adjustment; and includes more example notebooks for forecasting to help you onboard your use cases faster!

Code repo: https://t.co/kPWaRyItb4
Model: https://t.co/K7DX1zvyiY

@woo_gerald @JunchengLiu3 @ChenghaoLiu15 @silviocinguetta @CaimingXiong @doyensahoo @SFResearch

[Salesforce AI Research @SFResearch]

Our team has 4 papers accepted to ICML this year! Congratulations to all of our paper authors on your accepted work.

[Gerald Woo @woo_gerald]

Happy to share that Moirai has been accepted to ICML! See you Vienna!

https://t.co/BDtKZAu6pv

[Caiming Xiong @CaimingXiong]

🚀 Exciting Update: Moirai is now open source! 🚀

Foundation models for time series forecasting are gaining momentum! If you’ve been eagerly awaiting the release of Moirai since our paper (Unified Training of Universal Time Series Forecasting Transformers https://t.co/HkjA5PvPSr) dropped, here’s some great news!

Today, we’re excited to share that the code, data, and model weights for our paper have been officially released. Let’s fuel the advancement of foundation models for time series forecasting together!

Code: https://t.co/PzCZFqPkVd
LOTSA data: https://t.co/Doa3lzbtbW (of course via @huggingface)
Paper: https://t.co/fUoN6EcSV3
Blog post: https://t.co/NWFvcuGm2G
Model: https://t.co/sj5RnOUnEx

@woo_gerald @ChenghaoLiu15 @silviocinguetta @CaimingXiong @doyensahoo

[Lucas Beyer (bl16) @giffmana]

@TheZachMueller Why not combine? Gym with @ykilcher in the background.
Maybe we can ask him to, in future videos, say “switch” every 3 minutes for the gym listeners.