Kelby | kelby.eth

AI Engineer | Techie | Fly Fisherman | Crypto addict

[Kelby | kelby.eth @EnevoldJames]

I had entirely too much fun playing with ADK. Rather than just stuffing it in a private repo I opted to share my first test here: https://t.co/V7xPNFgbK3
Next step, allow for "personas" on each agent. I need help!
Debate Agent A Persona: ???
Debate Agent B Persona: ???
Topic: ???

[Jim Fan @DrJimFan]

Unmute. Sora now gets synthetic audio @elevenlabs. It's prompted by text, but the right conditioning should be on both text and video pixels. Learning an accurate video->audio mapping would also require modeling some *implicit* physics in the latent space.

Here's what an end2end transformer needs to figure out to simulate sound waves correctly:

1. Identify each object's category, materials, and spatial locations.
2. Identify the higher-order interactions between objects: is a stick hitting a wooden, metal, or drum surface? At what speed?
3. Identify the environment: restaurants? Space station? Yellowstone? Japanese shrine?
4. Retrieve the typical sound patterns of objects and surroundings from the model's internal memory.
5. Run "soft", learned physical rules to piece together and adjust parameters of the sound patterns, or even synthesize completely new ones on the fly. Kind of like "Procedural Audio" in game engines.
6. If the scene is busy, the model needs to overlay multiple sound tracks based on their spatial locations.

None of the above is an explicit module! All will be learned by gradient descent through massive amounts of (video, audio) pairs, which are naturally time-aligned in most internet videos. Attention layers will implement these algorithms in their weights to satisfy the diffusion objective.

We don't have such a high-quality AI audio engine yet, but here's a very inspiring paper called "The Sound of Pixels" from 5 years ago: https://t.co/i2XXosRKL4

[Kelby | kelby.eth @EnevoldJames]

V-JEPA and Sora seriously blow my mind.
https://t.co/IV3rtghH9S

https://t.co/h841xllKRb

What is our future going to be like? The idea that simulating digital worlds is the next step towards AGI is wild… I’m excited? Scared? Blown away?!

[Kelby | kelby.eth @EnevoldJames]

@jonbonso @TechWithLucy @nealkdavis @JulieAElkins @JulieAElkins I love seeing your name all over the place! This is awesome!

[Kelby | kelby.eth @EnevoldJames]

I have so much catching up to do on ETH and ENS updates/proposals.🙃

[Kelby | kelby.eth @EnevoldJames]

@xelfer I'm going! I'll be there Monday through Friday, at Mission Ignite and at the AWS afterparty.

[Kelby | kelby.eth @EnevoldJames]

I haven't been on twitter in AGES. Hellllloooooo everybody!

Are there any psychology/history/philosophy books or podcasts you would recommend?

I'm tired of listening to music while I work!

[Kelby | kelby.eth @EnevoldJames]

@anthonydjames It’s such a ponzi scheme. At least this spring will be great for crypto portfolios, haha.

[vitalik.eth @VitalikButerin]

Happy birthday beacon chain!

Here's an updated roadmap diagram for where Ethereum protocol development is at and what's coming in what order.

(I'm sure this is missing a lot, as all diagrams are, but it covers a lot of the important stuff!)

[Kelby | kelby.eth @EnevoldJames]

@fayecloudguru A WILD MOOSA! @mk_0418

[Mike G Chambers @mikegchambers]

#AWS #reInvent2021 Keynote - much faster...

[Kelby | kelby.eth @EnevoldJames]

@christophelimp this gave me a chuckle.

[Kelby | kelby.eth @EnevoldJames]

Great read about the FB outage:

https://t.co/65HOd5o3cK

[The account formerly known as @lfowler_LA @lfowler_LA]

This is a fantastic time to be part of the Azure world. Get over to @acloudguru and get your learn on. In five years you’ll really thank yourself.
#Azure #CloudAllTheThings #NeverStopLearning #KeepBeingAwesome

[AWS Partner Network @AWS_Partners]

Watch our on-demand @Twitch discussion with @acloudguru & explore exciting new features added to the platform in the last few months: https://t.co/zGMSfCl9h0

[Kelby | kelby.eth @EnevoldJames]

If you haven't already watched this, do it, it's amazing! I can't even remember lyrics to a real song...this is impressive.

[Forrest Brazeal @forrestbrazeal]

168 AWS services in 2 minutes. *inhales*

[Kelby | kelby.eth @EnevoldJames]

@TiaWilliams_LA He was an incredible human being and such an inspiration to so many people.

[Kelby | kelby.eth @EnevoldJames]

@anthonydjames 1. Pihole
2. TICK stack (telegraf, influxdb, chronograf, kapacitor, sometimes people prefer grafana over chronograf)
3. RetroPi to show my kid all the old games (or to make a mini-arcade)
4. Hopefully IoT garden this fall, MudPi or something
5. IoT Kegerator