Lezhi Li

[Lezhi Li @lezhili]

Our work at Apple on scalable video generation models!

[Zongyu (Johnson) Lin @zy27962986]

🚀🚀🚀Want to develop a cutting-edge video generation model towards Sora? Please dive into Apple’s latest recipe and studies for scalable video generation models🔥🔥🔥. In this work, we aim at providing a transparent and detailed recipe 📖 for model architecture, training strategy and data for scalable text-image conditioned video generation.
Arxiv: https://t.co/VDZ2o3X2bz
Hugging Face link: https://t.co/iL8QmdpRtA

[Saining Xie @sainingxie]

Representation matters.
Representation matters.
Representation matters, even for generative models.

We might've been training our diffusion models the wrong way this whole time. Meet REPA: Training Diffusion Transformers is easier than you think! https://t.co/lyWwiTYjEt(🧵1/n)

[Nathan Lambert @natolambert]

PSA: I'm open to guest posts on @interconnectsai covering areas I'm not an expert in, video gen, image gen, architectures, etc. Will be a high bar though.

[Bojan Tunguz @tunguz]

La Baie Area by @trbdrk

[Dogan Ural @doganuraldesign]

AI tools have no creative control; they're like slot machines.

Yeah buddy, sure.

This is ReshotAI. In the coming months, we will see many more tools like this. Future of AI is bright ✌️

[Joel Simon @_joelsimon]

Quick tests of CLIP directions with flux-schnell. The latent space is still entangled and jumpy, but I'm finding higher-level sliders like 'complexity' and 'playfulness' fun and useful for navigating. 🧭🎚️

[Damon Beres @dlberes]

People will be like, “generative AI has no practical use case,” but I did just use it to replace every app icon on my home screen with images of Kermit, soooo

[apolinario (poli) @multimodalart]

I'm genuinely impressed by Kolors IP Adapter! 🎨

Just put out a demo so you can play with image variations and reference 🖼️

▶️ https://t.co/mS8cfs5Fbp

[Hugues Bruyère @smallfly]

In the past few weeks, I deep dived into an exploration revolving around the use of physical interfaces to feed and interact with a real-time img2img diffusion pipeline using Stream Diffusion and SDXL Turbo. What really captivated me is to use my hands, objects, art supplies, tools, and light to create images and scenes.

𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗠𝗲𝗱𝗶𝗮 + 𝗧𝗼𝗼𝗹𝘀

I experimented with clay, manipulating different types and colors along with a selection of prompts. I used a magnifying glass, tracked in real-time, to focus the diffusion process on specific areas. Combining these tools created a dynamic and inspiring experience. Using magic clay to layer shapes and colors as a base for revealing landscapes and hidden worlds, and the magnifying glass to focus and reveal these details, was particularly effective.

𝗣𝗵𝘆𝘀𝗶𝗰𝗮𝗹 𝗟𝗶𝗴𝗵𝘁

I used light as my method of interaction with the img2img diffusion. This approach felt special right away. There was something magical about holding a physical light source and seeing it influence the generated visuals. I iterated on this technique with themes like Rococo architecture, flowers, Brutalist architecture, hidden worlds, and origami landscapes.

𝗜𝗻𝗸 + 𝗛𝘆𝗯𝗿𝗶𝗱 𝗙𝗼𝗿𝗺𝗮𝘁𝘀

I also used ink in milk as a means of physical interaction with the diffusion pipeline. As I drop ink into milk, shapes come alive instantly. By learning to manipulate the combination of physical and digital elements, I steered the generated output toward my areas of interest. These iterations extended beyond ink in milk to include the format in which these elements are contained: a circular plate or a triptych of small stainless steel trays. These formats provide a structured yet flexible framework to explore themes and narratives across multiple visual spaces. It's magical.

Some of that last iteration has been captured in this insightful article by Fast Company: https://t.co/XCPRJXHSfx

#stablediffusion #realtime #ai

[Johannes Stelzer @j_stelzer]

New way to navigate latent space. It preservers the underlying image structure and feels a bit like a powerful style-transfer that can be applied to anything. The trick is to...

[alex peysakhovich @alex_peys]

just released a paper with will berman on multimodal inputs for image generation

main idea: describing things just in text is often hard. can you train a model that uses interleaved text/image prompts for image generation? the answer is yes.
🧵

[Noah Tye @noahlt]

Image generation AI is a cognitive prosthetic for aphantasiacs. Opposing it is ableist.

[Dogan Ural @doganuraldesign]

The new 'Style References' is mind-blowing!

I tried to transfer the styles of some movies.

🧵 Here is how and the results:

[Soumith Chintala @soumithchintala]

If you have questions about why Meta open-sources its AI, here's a clear answer in Meta's earnings call today from @finkd

[Dogan Ural @doganuraldesign]

Google just announced a new image generator!

ImageFX (It's also available in Bard)

🧵 Comparisons and more in the thread:

[Lezhi Li @lezhili]

In a world where new things to learn never fall short, finding an effective learning path is essential. I've not found a better Diffusion models tutorial than https://t.co/uNSBttXwUN. It explains things better than a whole quarter of Stanford course I took.

[Lezhi Li @lezhili]

What a fruitful #kdd week! Learned so much and also of course witnessed how LLM is the hottest way to do recommendation systems / causal inferencing / outlier detection……
Plus, sharing the best experiment result I saw:

[Lezhi Li @lezhili]

I will be attending the #kdd2023 conference next week. Come say hi if you’re around!
You are also welcome to come visit the Apple booth and learn about our latest research publications and career opportunities in AI and ML. See you there!

[Lezhi Li @lezhili]

My latest side project on the topic of small LLMs! Thanks amazing collaborators for making this happen. You’re welcome to leave a comment on it on Kaggle if you find it helpful: https://t.co/mG7PXIpf2l

[Lezhi Li @lezhili]

Daily dose of life lessons from stats-teaching youtubers:
* [Markov chain]: "the future is not independent of the past, but the future is conditionally independent of the past given the present".
* [Ergodic therum]: "anything that can happen, will happen".