Comparing new LLM releases via benchmarks is now a bit like using IQ tests for hiring decisions.
Sure, benchmarks are useful baselines and so on, but beyond that, I don't know how indicative they are of our individual tasks and needs. (Yes, also looking at that release yesterday)
Introducing Trelis/Duplex-0
https://t.co/vojuDi1Eeq
An Apache 2-licensed speech-to-speech prototype model and training scripts.
Duplex-0 is a ~200M parameter duplex model, trained on ~750 hrs of audio.
It is a research checkpoint that is useful for ablating data mixes and model architectures... its speech is about half as intelligible as real speech in our measurements.
Duplex-0 comes in three architectural flavours:
1. two-track audio
2. two-track audio + 1-track model text
3. 2-track audio + 1-track model text + 1 track caller text (ASR, 320 ms delay)
These three training options were compared on the same data, seed and settings.
All three produce speech of similar quality: 60 to 68 confirmed words per minute after the conversational stage, against 132.8 for real speech.
Here are some further notes on its design and training.
A Speech to Speech Pre-training Dataset
https://t.co/UoLcKULyX0
This is the first in a series of datasets and models we'll be releasing around speech-to-speech.
It's a small filtered and tokenized pre-training dataset for training dual-track speech-to-speech models.
It holds long spans of English speech with one or two speakers, chosen to hold little or no music.
It's drawn from YODAS3 (English), which is a Creative Commons Youtube dataset, and referenced by shard and member.
The dataset ships audio tokens (Mimi here) in place of raw audio, so it's lightweight.
The dataset includes rich machine annotations:
- speaker-attributed transcripts with word timings, including overlapping speech;
- vocal bursts such as laughter;
- per-utterance emotion labels;
- where two speakers are present, each speaker on their own token track.
Notably, where there are two speakers, we also split the audio into separate tracks! On 856 of the 3630 spans two people speak, and each speaker has their own audio token track alongside the mixed audio, with that speaker's word timings.
YODAS3 - and therefore this dataset - is mostly single speaker, but stayed tuned for dual-speaker data very soon.
Introducing Claude Fable 5.2 and Claude Mythos 5.2, designed to tackle the most daunting technical challenges.
They’re capable collaborators on ambitious projects in many fields, and show improvements in areas like coding, reasoning, and vision.
Get ready for full duplex audio and video for conversations with state of the art performance from @boson_ai with our 45B MoE model.
Big Bench Audio: 98.9, highest in the comparison, vs. 96.5 for Thinking Machines' 276B model with its background agent. Beats Google's Live models on 6 of 7 benchmarks and TML on 4 of 6.
Get a first look at Verbex's highly expressive Bangla TTS (Text-to-Speech) Engine.
Generate natural-sounding, expressive speech with fine-grained control over emotion, emphasis, and delivery using descriptive tags. Learn more about what we're building: https://t.co/KQvwxJEgAU