L 💭

Human after all

[L 💭 @lhenault]

#AI generated #Makeup 💄for lips with #GAN.

Because you're worth it. 👄

[Google AI @GoogleAI]

Today we introduce an AI co-scientist system, designed to go beyond deep research tools to aid scientists in generating novel hypotheses & research strategies. Learn more, including how to join the Trusted Tester Program, at https://t.co/1eqmTTZOLr

[Vaibhav (VB) Srivastav @reach_vb]

That’s a 1.5B model, multilingual, supports voice cloning, trained on 200K hours of audio - apache 2.0 licensed! 🤯

[Jiayi Pan @jiayi_pirate]

We reproduced DeepSeek R1-Zero in the CountDown game, and it just works

Through RL, the 3B base LM develops self-verification and search abilities all on its own

You can experience the Ahah moment yourself for < $30
Code: https://t.co/UcGKN2SVGj

Here's what we learned 🧵

[DeepSeek @deepseek_ai]

🔥 Bonus: Open-Source Distilled Models!

🔬 Distilled from DeepSeek-R1, 6 small models fully open-sourced
📏 32B &amp; 70B models on par with OpenAI-o1-mini
🤝 Empowering the open-source community

🌍 Pushing the boundaries of **open AI**!

🐋 2/n

[DeepSeek @deepseek_ai]

🚀 DeepSeek-R1 is here!

⚡ Performance on par with OpenAI-o1
📖 Fully open-source model &amp; technical report
🏆 MIT licensed: Distill &amp; commercialize freely!

🌐 Website &amp; API are live now! Try DeepThink at https://t.co/v1TFy7LHNy today!

🐋 1/n

[Ali Behrouz @behrouz_ali]

Attention has been the key component for most advances in LLMs, but it can’t scale to long context. Does this mean we need to find an alternative?

Presenting Titans: a new architecture with attention and a meta in-context memory that learns how to memorize at test time. Titans are more effective than Transformers and modern linear RNNs, and can effectively scale to larger than 2M context window, with better performance than ultra-large models (e.g., GPT4, Llama3-80B).

[MiniMax (official) @MiniMax_AI]

MiniMax-01 is Now Open-Source: Scaling Lightning Attention for the AI Agent Era

We are thrilled to introduce our latest open-source models: the foundational language model MiniMax-Text-01 and the visual multi-modal model MiniMax-VL-01.

💪Innovative Lightning Attention Architecture, with Top-tier Model Performance
This series of models (MiniMax-01) incorporates bold innovations, marking the first large-scale implementation of a novel Lightning Attention mechanism in the industry, offering a new alternative to the traditional Transformer architecture.

🚀4M Ultra-Long Context, Spearheading the AI Agent Era
MiniMax-01 efficiently processes up to 4M tokens - 20 to 32 times the capacity of other leading models. We believe MiniMax-01 is poised to support the anticipated surge in agent-related applications in the coming year, as agents increasingly require extended context handling capabilities and sustained memory.

💰Unbeatable Cost-Effectiveness for Continuous Innovation
With our our proprietary architectural innovations and infrastructure optimization, we are able to offer both model APIs at the industry's most competitive price points: USD $0.2 per million input tokens and USD $1.1 per million output tokens.

Try Now for FREE: https://t.co/WXPvhNWeJj

📑Paper：https://t.co/leZXTL3oMd
📖Read more: https://t.co/IswiryeO0z

[DeepSeek @deepseek_ai]

🚀 Introducing DeepSeek-V3!

Biggest leap forward yet:
⚡ 60 tokens/second (3x faster than V2!)
💪 Enhanced capabilities
🛠 API compatibility intact
🌍 Fully open-source models &amp; papers

🐋 1/n

[wh @nrehiew_]

How to train a 670B parameter model.

Let's talk about the DeepSeek v3 report + some comparisons with what Meta did with Llama 405B

[Miguel | AP @angrypenguinPNG]

Flux Concept Injection with Image2Image and IC LoRA

Pretty amazing

[merve @mervenoyann]

OmniVision-968M: a new local VLM for edge devices, fast &amp; small but performant 👏

it's based on SigLIP-so-400M and Qwen-2.5-0.5B
💨 9x less image tokens, super efficient
📖 aligned with SFT and DPO for reducing hallucinations
🔥 Apache 2.0 license

[Avthar @avthar]

VECTOR DATABASES ARE THE WRONG ABSTRACTION. Here’s a better way: introducing pgai Vectorizer, a new open-source PostgreSQL tool that automatically creates and syncs embeddings with source data, just like a database index.

❌ Why vector databases fail
Vector databases treat embeddings as independent data, divorced from the source data from which embeddings are created, rather than what they truly are: derived data.

This pitfall means that many AI projects that start out as simple vector search implementations inevitably evolve into a complex orchestra of monitoring, synchronization, and firefighting.

😓 Keeping embeddings in-sync is hard
In an attempt to avoid stale embeddings, engineering teams have to build and maintain a maze of ETL pipelines, juggle multiple databases (vector DB, metadata store, lexical search), and manage complex queuing systems for updates.

Add monitoring for data drift, alert systems for stale results, and validation checks across systems - and you have a brittle infrastructure that inevitably breaks down, leading to stale embeddings and wasted engineering hours.

What if you could just use Postgres instead?

✅ Pgai Vectorizer: Vector embeddings as database indexes
Pgai Vectorizer treats embeddings like database indexes. It automatically creates, updates, and maintains embeddings as your data changes. Just like an index, the database handles all the complexity: syncing, versioning, and cleanup happen automatically.

This means no manual tracking, zero maintenance burden, and the freedom to rapidly experiment with different embedding models and chunking strategies without building new pipelines.

🤔Why did we build pgai Vectorizer?
Our team at @timescaledb built pgai Vectorizer because many developers regard PostgreSQL as the “Swiss army knife” of databases, as it can handle everything from vectors and text data to JSON documents.

We think an “everything database” like PostgreSQL is the solution to eliminate the nightmare of managing multiple databases, making it the ideal home for vectorizers and the foundation for AI applications.

⚙️How does pgai Vectorizer work?
Check out the code snippet below –  it takes just 6 lines of SQL to put your embedding creation pipeline on autopilot with pgai Vectorizer!

Under the hood, pgai Vectorizer checks for modifications to the source table (inserts, updates, and deletes) and asynchronously creates and updates vector embeddings in an external worker.

🧑‍💻 Sounds exciting! How can I get started?
Pgai Vectorizer is open-source under the PostgreSQL license and available for free to use on any PostgreSQL database. You can find installation instructions on the pgai GitHub repository (see end of post). It’s also available as a managed service in Timescale’s PostgreSQL cloud platform.

📚Learn more
[1] Pgai github repo: https://t.co/hut1MxuwPZ
[1] Technical explainer post: https://t.co/A9hOz482Rg

Share this post with your followers to let them know about pgai Vectorizer and comment your reactions and questions.

[Sebastian Raschka @rasbt]

"What Matters In Transformers?" is an interesting paper (https://t.co/2O6TxZK5Mx) that finds you can actually remove half of the attention layers in LLMs like Llama without noticeably reducing modeling performance.

The concept is relatively simple. The authors delete attention layers, MLP layers, or entire transformer blocks:

- Removing entire transformer blocks leads to significant performance degradation.
- Removing MLP layers results in significant performance degradation.
- Removing attention layers causes almost no performance degradation!

In Llama 2 70B, even if half of the attention layers are deleted (which results in a 48% speed-up), there's only a 2.4% decrease in the model benchmarks. The author also recently added Llama 3 results to the paper, which are similar.

The attention layers were not removed randomly but based on a cosine-based similarity score: If the input and output are very similar, the layer is redundant and can be removed.

This is a super intriguing result and could potentially be combined with various model compression techniques (like pruning and quantization) for compounding effects.

Furthermore, the layers are removed in a one-shot fashion (versus iterative fashion), and no (re)training is required after the removal. However, retraining the model after the removal could potentially even recover some of the lost performance.

Overall, a very simple but very interesting study. It appears there might be lots of computational redundancy in larger architectures.

One big caveat of this study, though, is that the focus is mostly on academic benchmarks (HellaSwag, MMLU, etc.). It's unclear how well the models perform on benchmarks measuring conversational performance.

[Peter Kraft @petereliaskraft]

Scalability! But at what cost?

This paper is an absolute classic because it explores the underappreciated tradeoffs of distributing systems.

It asks about the COST of distributed systems--the Configuration that Outscales a Single Thread. The question is, how many cores does a big distributed system need to outperform some moderately-optimized single-threaded code running on your laptop?

As it turns out, scalability often comes with an extremely high COST. The authors examine several graph processing systems--including some big names like Spark--and find that they need dozens to hundreds of cores to outperform a single-threaded program.

Why is this the case? It's not because these distributed systems are badly designed, but because distributing computation is inherently inefficient for many problems.

Fundamentally, a distributed system cannot rely on all processors sharing state, at least not efficiently. This is a big issue! In graph algorithms, it means servers need to expensively exchange data and eliminates a wide swathe of algorithms and optimizations that rely on shared state. In distributed databases, it means expensive coordination is required to distribute transactions to ensure participating servers have consistent views of data.

Does this mean we shouldn't build scalable systems? Of course not! Many problems are well beyond the capability of a single server, no matter how optimized. But it does mean we should be mindful of the efficiency costs of scaling.

As an aside, I think this kind of thinking is why Postgres is so popular, despite not being distributed. A large Postgres server can handle a vast amount of traffic (especially with read replicas, which can be cheaply maintained). You need a huge company or incredibly heavy workload to outscale that single server, and when you do, the alternatives come with huge tradeoffs!

[Jacob Matson @matsonj]

Are you kidding me? Look at this:

[DeepLearning.AI @DeepLearningAI]

Introducing Serverless Agentic Workflows with Amazon Bedrock, a new course made in collaboration with @awscloud!

Build scalable agents, deploy them serverlessly, and implement guardrails for secure and responsible operations. Enroll for free 👇
https://t.co/nwt8RRJQRt

[Martin Nebelong @MartinNebelong]

Adobe.. has released a tool that combines Generative Gaussian Splats with a Diffusion layer and it's not all over the internet? WHAT IS GOING ON :D

I had to test this out ofc! Substance 3D viewer, the new 3d viewer just released by Adobe, not only supports viewing of a large, large list of 3d formats, but it also supports generating of 3d models as Gaussian Splats. Combine that with the built-in 3d to image functionality and you basically have at least a few of the steps I've been doing for the last year or so using multiple tools. Exciting stuff!

The new Photoshop Beta also supports the 3D viewer and you can import 3d files directly in photoshop as smart objects linked to the viewer. The elements you prompt in based on your models, can be easily exported to your clipboard without backgrounds making it very easy to use this in your normal image editing workflows.

I have a suspicion that we'll see much more of this in other Adobe tools and I'm very curious to see this used for things like this in more 3d tools than "just" Adobe Neo (which is fantastic btw).

#adobe #art #gaussiansplatting

[Sebastian Raschka @rasbt]

Just put together a short Jupyter notebook with tips and tricks for reducing memory usage when loading larger and larger models (like LLMs) in PyTorch: https://t.co/fEx2e8E7jS

(PS: This is an LLM example but the same concepts apply to any PyTorch model)

[AI at Meta @AIatMeta]

📣 New course now available on @DeepLearningAI: Introducing Multimodal Llama 3.2! The course covers both Llama 3.1 & Llama 3.2 and includes detailed rundowns on multimodal prompting, custom tool calling, Llama Stack + more.

Take the 1h course for free ⬇️
https://t.co/d1058HFVkU

[Skizi @SkyZiiHD]

Tu nous manqueras, Renault