OpenTrain AI

The Data Labeling Marketplace | Where AI Builders and AI Trainers Connect to Build the Future | Find, hire, & securely pay data labelers for any annotation tool

[OpenTrain AI @OpenTrainAI]

@AlicBa4 Hi @AlicBa4, just responded to your live chat :)

[OpenTrain AI @OpenTrainAI]

@natolambert They could hire & pay the same talent from these "data foundries" directly on https://t.co/w9eGGWOMGE. Savings are often 50%+

[OpenTrain AI @OpenTrainAI]

@Suhail Here to help! https://t.co/qRIxZvPw56

Post your data labeling project and connect with freelancers and/or data labeling agencies.

We also offer a fully-managed solution, where we build your own annotation team and manage the whole process.

DMs are also open.

[martin_casado @martin_casado]

At this point I feel like we understand pretty well what's going on with LLMs:

- Outputs are roughly equivalent to kernel smoothing over positional embeddings (https://t.co/oo07Wu3VWi)
- The learned computation model is *probably* bounded by RASP-L (https://t.co/TW8RDyxw3F)
- LLMs learn structure primarily from human generated content (text, images) which is far more structured and predictable than the universe.
- LLaMa3 shows us that the higher quality the annotations on the human generated content, the better the LLMs do (10million messages is a lot!)
- Multi-turn labeling is currently very expensive and so likely driving the costs of the models.
- Right now we're likely bottlenecked not on CPU, or size of data, but number and quality of annotations.

So tl;dr. Great at predicting what a human would do or say by averaging in distribution data in the corpus. No emergent generality. Currently bottlenecked by high quality annotated data.

@hausdorff_space did I miss anything?

[François Chollet @fchollet]

The perfect quote to describe LLMs can be found in a 1946 Jean Cocteau movie -- "Réfléchissez pour moi, je réfléchirai pour vous" (think for me, I will reflect for you).

What you get from the model is always a reflection of the training data you put in -- itself a by-product of human thinking.

[OpenTrain AI @OpenTrainAI]

Learn more about OpenTrain's AI Interviewer here: https://t.co/wu1cDchiyp

[OpenTrain AI @OpenTrainAI]

NEW FEATURE:

AI Live Chat Interviews: Using GPT-4 to interview, score, and screen job applicants from data labelers/AI trainers.

One-on-one interviews, at scale!

How It Works:

#GPT4

[OpenTrain AI @OpenTrainAI]

With the power of LLMs, we can now do all of this for you behind-the-scenes and then present an AI Qualification Score.

From there, you can get a clear view of who are the most qualified candidates.

[OpenTrain AI @OpenTrainAI]

Traditionally, selecting who to hire required significant time for:

-Viewing applicants' resume
-Reading cover letters
-Messaging follow up questions
-THEN judge if they are a good fit for your specific job

[OpenTrain AI @OpenTrainAI]

@ylecun We're working on this. Hire & pay data labelers/contributors directly, with only a small transaction fee. But if the resulting dataset will be open sourced, ZERO FEES.

https://t.co/w9eGGWOMGE

[OpenTrain AI @OpenTrainAI]

Agreed! But some datasets are just too large/require too much time. This is when many turn to crowdsourcing. The issue with crowdsourcing is it's largely a "black box". You don't get any feedback on issues with the dataset or anomalies. This is becoming even more important as people start building more advanced models. Advanced models require labelers with domain-expertise that is not available through crowdsourcing.

Here's the data labeling process we'd recommend:

1) Data scientist/engineer do the initial labeling (10-20 hours recommended)
2) Source & hire INDIVIDUAL data labelers with expertise in your data's domain (on https://t.co/BNQ8RP724X of course 😉)
3) Work closely with the data labelers (daily sync up, open communication channels, etc)

This allows for deep understanding of training data, while still being scalable for large datasets.

[OpenTrain AI @OpenTrainAI]

A big reason for this is the cost of hiring data talent. The big data labeling platforms are prohibitively expenses and openly have a 70% margin (and pay their data labelers poorly). We're working on making it easier & more cost-effective for the open source community to get custom data by allowing people to post a dataset need > Get proposals from freelancers (many of whom also work for the large data providers) > Pay them directly with only a small transaction fee for completed milestones. https://t.co/BNQ8RP724X ...We also wave the transaction fee if the resulting data set will be open sourced

[OpenTrain AI @OpenTrainAI]

We're working on a solution for this. There are a lot of great tools, like Label Studio, but finding specialized talent to do the actual annotation work can be difficult.

With OpenTrain, you post your job and select your labeling software > workers submit proposals > you hire and simply add the workers to your labeling software account.

You then pay the talent directly in escrow milestones. Also, if the specialized talent isn't on our platform yet, we recruit them and bring them onboard for your job.

We only charge a small % on successful milestone payouts...vs the big data labeling companies who charge a massive markup and long term contracts.

We support 20+ data labeling tools and hiring in 110+ countries.

[OpenTrain AI @OpenTrainAI]

@kristoph coming soon ⌛️

[OpenTrain AI @OpenTrainAI]

Grok-1 by @xai utilized "AI Tutors", or human subject matter experts to create custom training data & provide RLHF.

We're making this easy for anybody to do this with https://t.co/BNQ8RP724X: The data labeling marketplace to find, hire, & pay training data experts for ANY data labeling software.

Simply setup the data workflows on your data labeling tool of choice, & use OpenTrain to find the subject matter experts to work from that tool.

Like UpWork, but for "AI Tutoring"

Post jobs & collect applicants 100% free!

#rlhf #llm #datalabeling #trainingdata

[Arena.ai @arena]

🔥Excited to introduce LMSYS-Chat-1M, a large-scale dataset of 1M real-world conversations with 25 cutting-edge LLMs!

This dataset, collected from https://t.co/4LVJjx4pZi, offers insights into user interactions with LLMs and intriguing use cases.

Link: https://t.co/koniYAR4MD

[OpenTrain AI @OpenTrainAI]

An in-depth look at RLHF by @natolambert from @huggingface. The need for high-quality, task-specific data in RLHF is crucial. With OpenTrainAI, you can find, hire, & pay the human experts essential for responsible and effective RLHF. Post your job today! #RLHF #MachineLearning

[Muratcan Koylan @muratcan]

Reinforcement Learning from Human Feedback (RLHF) is gaining traction. This field aims to make AI more responsible by including human values and preferences.

In this video, @natolambert, a research scientist and RLHF team lead at @huggingface explores its inner workings, applications and industry impact.

RLHF has gained the spotlight in recent years. The growth of language models like Anthropic’s Claude and OpenAI's ChatGPT have increased interest in human-feedback integration.

"There are some rumors that Open AI had two teams; one was doing RLHF and the other instruction fine-tuning. And the RLHF team kept getting more and more performance."

Understanding RLHF
The RLHF process has three main steps:
Pre-training: Much like with GPT models, the journey starts with pre-training on a large corpus of data. This can range from text data, web scrapes, to specialized datasets.
Reward Modeling: This is the RLHF counterpart of supervised fine-tuning in large language models. This stage involves creating a reward model that resonates with human values and preferences.
RL Optimization: This stage parallels reward modeling and reinforcement learning in traditional AI models. The AI system fine-tunes itself based on the reward model, employing reinforcement learning algorithms for that extra layer of optimization.

The Data Challenge
Data collection and curation in RLHF closely resemble the challenges you'd encounter in large language model training.

Datasets from organizations like OpenAI can serve as a useful foundation. However, the need for high-quality, task-specific data cannot be overstated.

Implementing RLHF: A Practical Guide
If you’re someone who loves getting hands-on with AI libraries like Hugging Face, implementing RLHF is right way to do.

It’s essential to understand its limitations. Think about model stability, over-optimization, and exploration strategies, much like you would when prompt engineering.

Ongoing Research and Next Steps
While he suggests that some basics figured out, there are layers of complexity that still need to be unraveled:

1. New Benchmarks: How do we measure the effectiveness of RLHF?
2. Preference Modeling: How can the model be made to understand human preferences better?
3. Interpreting RLHF: Much like explainability in traditional models, how do we make RLHF more interpretable?
4. System-Wide Evaluation: Going beyond individual performance, how does RLHF affect an entire system?

The Transformative Power of RLHF
Whether you're an AI developer, a business analyst, or a marketer, RLHF promises to revolutionize your domain.

Imagine customer service chatbots that understand human emotions better, or content generators that align more closely with human values.

RLHF is an emerging field that focuses on enhancing machine learning models through human feedback.

While it tackles important issues like bias and ethics, its broader goal is to improve system performance across various applications.

Whether you're deeply invested in the ethics of AI or simply curious about advancements in machine learning, RLHF offers valuable insights.

If you're interested in the next wave of AI development, this area is definitely one to watch.