Eric Hofesmann

Machine Learning + Computer Vision engineer @voxel51
CSE M.S. @UMich
Love me some cooking 🍲, games 🎲, and dogs 🐶

[Eric Hofesmann @ehofesmann]

If you're training object detection models, #Detectron2 integrated with #FiftyOne allows you to iterate over both your model and dataset to quickly achieve higher performance!

Check out my recent blog covering this!

https://t.co/FYLu6nR0xA

#ai #ml #opensource #computervision

[Jimmy at Voxel51 @jimmy_voxel51]

1 in 3 teams pay to annotate data that never makes it to production. That's not a labeling problem. That's a strategy problem - https://t.co/6XK2HG8fQb

FiftyOne Annotation treats annotation as a data flywheel — not a one-time chore.

* Label the right data first. Smart Data Selection surfaces the rare, high-signal samples that actually move model performance — teams ship with 60–80% less annotated data.

* Agentic labeling. VLMs handle the first pass, so humans review and correct instead of drawing every box from scratch.

* 2D, 3D & video in one place. Bounding boxes, SAM2 click-to-segment, lidar cuboids, temporal events — all native.

* Intelligent Review. Embeddings + model comparison catch systematic label errors before they poison training.

* Close the loop. Model eval failures feed straight back into the next annotation round. Every iteration compounds.

The magic isn't any single feature — it's that curation, annotation, and evaluation finally live in ONE platform. No handoffs. No lost context. Just faster models, less waste. Stop labeling everything. Start labeling what matters.

#computervision #ai #artificialintelligence #machinevision
#machinelearning #datascience #physicalai #mcp #agents

[Jason Corso @_JasonCorso_]

Builders gonna build! Eric Hofesmann from the @Voxel51 rolled out a new #FiftyOne plugin last week for interactive visualization of the hyperspectral imagery including the ability to vary the transfer function dynamically.

Check out the plugin here with instructions on how to install it: https://t.co/1FDAs1RKL5

[harpreet @DataScienceHarp]

PaliGemma2-Mix is now integrated into FiftyOne! You can use this model for:

• Image captioning (multiple detail levels)

• Object detection

• Semantic segmentation (Not perfect, but good for initial exploration)

• Optical character recognition (OCR)

• Visual question answering

• Zero-shot classification

All with just a few lines of code!

Check out the example notebook here: https://t.co/NwpOH7eP7t

[Voxel51 @Voxel51]

If you’re deploying models into the wild, this new open source technique for visualizing model certainty on unlabeled data at scale could change the game for you. ⬇️

In our latest post, we dive into:

✅ A simple method for turning model outputs into actionable certainty scores

✅ How we used this technique across 190 ResNet18 models, each trained on a unique variation of the CIFAR100 training set

✅ Why typical embeddings fall short—and what works better

This is a great way to answer the question AI developers really care about: will our deployed model fail or not, and if so, on which data?

Read the full breakdown: https://t.co/Ioe4fCb3gn

#ComputerVision #AI #VisualAI #MachineLearning #ModelEvaluation #VisualAI #OpenSource #FiftyOne

[Jason Corso @_JasonCorso_]

Selective Transparency and The Battle for Open Source

What a great way to start a new week other than a new contributed article in VentureBeat!!! Focus: the battle for open source AI through the serious risk that selective transparency poses.

https://t.co/VC6XQBQDup

[harpreet @DataScienceHarp]

DeepSeek AI is on fire! Just yesterday they shipped Janus-Pro, a Unified Multimodal Model for multimodal understanding and text-to-image generation.

The model comes in two model sizes:

• Janus-Pro-1B, which is a little over 1.5 billion parameters
• Janus-Pro-7B with 7 billion parameters.

According to the technical report, the model can perform several vision-language tasks, such as :

• Image descriptions

• Landmark recognition

• OCR

Interestingly, the authors claim that this model excels in meme perception.

Oh, wait a minute...I read that wrong in the report...that doesn't say MEME perception.

It says MME perception.

Well, anyway, I'm gonna see just how well this model does at meme perception, specifically for:

• extracting captions from a meme

• understanding a meme

• generating captions for a meme

And in the spirit of competition, I'll pit it head to head against another newly released model: moondream2

🐋 vs 🌕 - Let's go 🚀!

Code and all you need 👇🏼

#deeplearning #artificialintelligence #computervision #genai

[Voxel51 @Voxel51]

Did you hear? DeepSeek just dropped Janus-Pro 7B, an open source game-changer for multimodal AI. And here's where it gets even better...

Thanks to our own @DataScienceHarp, we're excited to introduce our Janus-Pro VQA Plugin for FiftyOne. 🔥

Tap into Janus-Pro’s visual question understanding capabilities directly within FiftyOne.

🚀 Dive in here: https://t.co/Fy8Mvp4mrW

And here: https://t.co/XhDyiRa9Xi

#DeepSeekAI #DeepSeek #JanusPro #OpenSource

[Jason Corso @_JasonCorso_]

🏈 Hey, NFL: Meet Visual AI, Your Old Friend 🏈

1/n

The NFL is a game of inches, split-second decisions, and an ever-evolving playbook of strategies. While the league has embraced modern technology in areas like player safety and fan engagement, the disastrous 4th down call that may have kept my Bills from advancing to the Super Bowl is just one example of the untapped potential of using Visual AI to revolutionize the game. And most surprisingly, this is the one league that has classically decided to use Visual AI for fan engagement only (e.g., the televised first down marker) rather than advertising.

Also, @BuffaloBills I'd be happy to chat!

[Voxel51 @Voxel51]

💎 Introducing ZCore: Find hidden gems in massive datasets without labels.

This open source solution cuts datasets by 95%—perfect for efficient model training, fine-tuning, or eval.

See how it works: https://t.co/737G2x5y0g

#AI #ML #ComputerVision

[Dan Gural @dan_gural]

How to make the best Self-Driving Dataset 👇 🚗

I shared all my thoughts on how to bring to life the best driving dataset possible here in 2025, covering topics like:

💡 Multimodal datatsets, using LIDAR, RADAR, and more!

💡 Embeddings for natural language search and outlier detection

💡 Gaussian Splats, diffusion models, and more in the next frontier!

Read or watch more here: https://t.co/gTmp7uyE2I

#machinelearning #ai #ml #fiftyone #datasets

[Voxel51 @Voxel51]

🔥 New blog post! Dive into the intricacies of data bias and discover how embeddings provide insightful perspectives on the differences between real and synthetic data.

Key findings:

💡 Some synthetic images that look realistic to humans show unexpected patterns in feature space

💡 If you're not careful, distributions of synthetic data can diverge significantly from natural data distributions

💡 Tools like FiftyOne can help detect these hidden biases before they affect your models

Whether you're working with synthetic data generation or trying to understand dataset bias, this analysis offers practical insights for improving your visual AI pipelines.

Read the full article here:
https://t.co/9mnPB8kE5q

#MachineLearning #ML #DataScience #ComputerVision #VisualAI #AI #SyntheticData #FiftyOne

[Jason Corso @_JasonCorso_]

⚠️ 📈 ⚠️ Annotation mistakes got you down? ⚠️ 📈 ⚠️

There's been a lot of hooplah about data quality recently. Erroneous labels, or mislabels, put a glass ceiling on your model performance; they are hard to find and waste a huge amount of expert MLE time; and importantly, waste you money.

With the class-wise autoencoders method I posted about last week, we also provide a concrete, simple-to-compute, and state of the art method for automatically detecting likely label mistakes. And, even when they are not label mistakes, the ones our method finds represent exceptionally different and difficult examples for their class.

How well does it work? As the figure attached here shows, our method achieves state of the art mislabel detection for common noise types, especially at small fractions of noise, which is in line with the industry standard (i.e., guaranteeing 95% annotation accuracy).

Try it on your data!

👉 Paper Link: https://t.co/Y6ZlRs8TG3
👉 GitHub Repo: https://t.co/6OuG8ZEAtA

@QuantumMarks @Voxel51

[Jason Corso @_JasonCorso_]

🎥🖐🎥🖐 New Video GenAI with Better Rendering of Hands --> Instructional Video Generation 🎥🖐🎥🖐

New Paper Alert . Instructional Video Generation – we are releasing a new method for Video Generation that explicitly focuses on fine-grained, subtle hand motions. Given a single image frame as context and a text prompt for an action, our new method generates high quality videos with careful attention to hand rendering. We use the instructional video domain as driver here given the rich set of videos and challenges in instructional videos both for humans and robots.

Try it out yourself.  Links to the paper, project page and code are below; and a demo page on HuggingFace is in the works so you can more easily try it on your own.

Our new method generates instructional videos tailored to *your room, your tools, and your perspective*. Whether it’s threading a needle or rolling dough, the video shows *exactly how you would do it*, preserving your environment while guiding you frame-by-frame. The key breakthrough is in mastering **accurate subtle fingertip actions**—the exact fine details that matter most in action completion. By designing automatic Region of Motion (RoM) generation and a hand structure loss for fine-grained fingertip movements, our diffusion-based im model outperforms six state-of-the-art video generation methods, bringing unparalleled clarity to Video GenAI.

👉 Project Page: https://t.co/LlwIZALQfh
👉 Paper Link: https://t.co/aARZsRcsuI
👉 GitHub Repo: https://t.co/mOnEq0CxKu

This paper is coauthored with my students @YayuanLi and Zhi Cao at @UMRobotics, @michigan_AI, @UMichECE, @UMichCSE, @Voxel51

[harpreet @DataScienceHarp]

In this paper from NeurIPS 2024, researchers challenge the growing trend of using AI-generated images to train vision models.

They compared two approaches: synthetic images from Stable Diffusion versus carefully selected real photos from the same dataset used to train it.

The results?

Real images won – consistently and decisively.

Across five major benchmarks, from ImageNet to aircraft recognition, retrieved real photos outperformed synthetic ones, often requiring far less data to achieve better results.

This finding disrupts the current synthetic data hype, suggesting that AI-generated images might be an unnecessary middleman.

Instead of generating new data, the key lies in intelligently mining existing real-world datasets.

Want to dive deeper into this line of research?

Read the full breakdown in my blog, where I dive deep into the paper and share what I've learned from it.

Sometimes, reality truly is better than fiction.

#NeurIPS2024

[Voxel51 @Voxel51]

🎉 It’s Giving Tuesday, and we’re doubling down on giving back! Here's how:

Today, we’re donating to three amazing charities on behalf of our AI Meetup community: @ai4allorg, @Heart_to_Heart, and @oceana. 🌎

And because it’s Giving Tuesday, we have the chance to 2x our annual contribution to make an even bigger impact!

See details on the organizations we’re proud to support below.

#GivingTuesday #NonProfits #Donations #FiftyOne

[Dan Gural @dan_gural]

Only in Munich, the Eisbachwelle! 🌊🏄

A famous man-made standing wave on the Eisbach River, located in the English Garden (Englischer Garten), it’s one of the most iconic urban surfing spots in the world! Even in freezing temperatures that doesnt stop these surfers!

Fun fact, though people started experimenting surfing the wave in the 70s, it was outlawed for many years and only became officially legal in 2010!🤯

[Jason Corso @_JasonCorso_]

The potential revocation of President Biden’s AI Executive Order, as I discussed in Friday's VentureBeat article by @miyadavid, raises serious concerns for the future of AI regulation and innovation.

While voluntary responsible AI practices are vital, I shared my thoughts about the long-term impact of losing federal resources that drive groundbreaking innovation. As the regulatory landscape shifts, it’s more important than ever for the industry to take proactive steps toward transparency and accountability.

Article Link: https://t.co/43rdZUWRgY

#AI #Policy #ResponsibleAI #Innovation

[Jason Corso @_JasonCorso_]

I recently learned that some companies are adding "Must have published papers in XYZ conference" where XYZ is something like CVPR, ICLR, NeurIPS, etc.

PLEASE STOP.

Not only is published XYZ work mostly irrelevant to the actual work they'll be doing at your company, it's not helping our research communities.

[Voxel51 @Voxel51]

Amidst the rush to build bigger models, the real game-changer for AI success is quality data. 🔍

In our latest blog, we unpack:

👉 Real-world examples of model failures due to bad data
👉 The secret to building high-performing models
👉 And more!

https://t.co/N6Z7kqKrIz

#ML #AI

[harpreet @DataScienceHarp]

Paper: https://t.co/T6Gdi4JmRE

Project Page: https://t.co/kKBohX0MbF