Ann Huang

I like food, and some other things too. Making ML magic 🪄 at @huggingface.

[David Berenstein @davidberenstei]

Observers: A Lightweight SDK for AI Observability

TLDR;
- Track and record interactions with AI models
- Store observations in multiple backends @huggingface, @argilla_io @duckdb
- Query and analyse your AI interactions with ease

GitHub:
https://t.co/F2F6aoAgRe

[Ann Huang @AnnInTweetD]

We're planning roll out our new storage backend on the Hub in early 2025. Read more about CDC and our learnings! https://t.co/lyI2LaWERP

[Ann Huang @AnnInTweetD]

We're turning @huggingface Hub's files into content-defined chunks to speed up your workflows!⚡️

This means:
- 🧠We store your file as deduplicated chunks
- ⏩ You only upload changed chunks when iterating!
- 🚀 Pulling changes? Only download changed chunks!

[Ann Huang @AnnInTweetD]

@huggingface In our benchmarks, we found that using CDC to store iterative model and dataset version led to transfer speedups of ~2x - we'd love to learn about more real world examples to see how we perform!

[Cyril Zakka, MD @cyrilzakka]

Super excited to introduce Halo: A beginner's guide to DIY health tracking with wearables! 🤗✨

Using an $11 smart ring, I'll show you how to build your own private health monitoring app. From basic metrics to advanced features like:
- Activity tracking
- Heart rate monitoring
- Sleep analysis
- and more!

[Vaibhav (VB) Srivastav @reach_vb]

If you were to fix one thing on the Hugging Face Hub, what would it be?

What's your biggest gripe with the hub - let's fix it; the more, the merrier.

[Caleb @calebfahlgren]

How can the Hub be better? Let us know!

[Caleb @calebfahlgren]

The @huggingface SQL Console now has Embeds!

🔗 Nice URL to Share / Save your Query Results
🖼️ Embed Results into Web Pages via IFrame

In this example, I use handy DuckDB regex functions to find the Code Feedback conversations with the most markdown code blocks

[Daniel van Strien @vanstriendaniel]

How do you release an impactful dataset on the @huggingface Hub?

We're enhancing how we track dataset downloads on the Hub, so I wanted to share some common themes I've noticed for datasets with high downloads. 🧵

[Yacine Jernite @YJernite]

Glad to see the @OpenSourceOrg release their OSAI definition process after an extensive collaborative process, and especially happy to see the role of training data enshrined!

Head over to the OSI HF org page if you want to discuss the definition on @huggingface 🤗

1/2🧵

[Ann Huang @AnnInTweetD]

@ExceptionAtAll @lvwerra @EugeneVinitsky We're updating the Hugging Face backend right now to address Git LFS limits!

If you have patterns you'd like to see addressed, reach out or start a discussion on our team page:
📈 https://t.co/myfPmebFXW

[Gradio @Gradio]

🆕 𝚜𝚊𝚏𝚎𝚑𝚝𝚝𝚙𝚡: a new open-source library from the Gradio team 🆕

This library is a product of our collaboration with @TrailOfBits and allows you to make asynchronous GET requests while avoiding Server Side Request Forgery.

A 🧵 on why this is important!

[Cyril Zakka, MD @cyrilzakka]

The source code for HFChat macOS🤗is now fully open source and accepting PRs! Looking forward to see what folks will build.
You'll also find some hidden features that never made it to the release: https://t.co/804uBWU1Nm

[Martin Görner @martin_gorner]

Did you know that you can load the newest checkpoints (like Llama 3.2) into Keras directly from the original HuggingFace release (safetensors)?

I tried - and lived to tell the tale: https://t.co/o3yvGYpXjq

[Vaibhav (VB) Srivastav @reach_vb]

What a great day for Open Science! @AIatMeta released models, datasets, and code for many of its research artefacts! 🔥

> Meta Segment Anything Model 2.1: An updated checkpoint with improved results on visually similar objects, small objects and occlusion handling. A new developer suite will be added to make it easier for developers to build with SAM 2.

Model checkpoints: https://t.co/FWx9qs4nwR

> Layer Skip: Inference code and fine-tuned checkpoints demonstrating a new method for enhancing LLM performance.

Model checkpoints: https://t.co/x1k7gcZxgX

> SALSA: New code enables researchers to benchmark AI-based attacks to validate security for post-quantum cryptography.

Repo: https://t.co/AKqUjCVZrT

> Meta Lingua: A lightweight and self-contained codebase designed to train language models at scale.

Repo: https://t.co/BIw2zMeaw4

> Meta Open Materials: New open source models and the largest dataset to accelerate AI-driven discovery of new inorganic materials.

Model checkpoints: https://t.co/KS3eVdRmTn

> MEXMA: A new research paper and code for our novel pre-trained cross-lingual sentence encoder covering 80 languages.

Model checkpoint: https://t.co/3IFeupCXJM

> Self-Taught Evaluator: a new method for generating synthetic preference data to train reward models without relying on human annotations.

Model checkpoint: https://t.co/j09g0COe7r

> Meta Spirit LM: An open-source language model for seamless speech and text integration.

Repo: https://t.co/hoERsWT7FD

[clem 🤗 @ClementDelangue]

👀👀👀

[Ann Huang @AnnInTweetD]

@huggingface Views like this help us understand real-world access patterns so we can architect a more efficient, geo-distributed system for the Hub's storage backend. What else should we be looking at?

[Ann Huang @AnnInTweetD]

Sweet visualization of S3 PUT requests to @HuggingFace Hub over a 24-hour period, showing upload density across the globe. 🌏

[Ann Huang @AnnInTweetD]

@severo_dev @davidgasquez @vanstriendaniel Definitely worth an investigation… will report back!

[Ann Huang @AnnInTweetD]

Did you know that @Huggingface Hub holds over 29 PB of Git LFS files across datasets, models, and spaces? 📈

That's the equivalent of 64 @CommonCrawl downloads - and it's growing every day. So what's inside? 🧵