Lama Hélène

Researcher in Security/Cryptography/AI
Université du Luxembourg- SnT/SEDAN
I tweet about Cybersecurity| AI | Politics | Random 🐾

[Guy Van den Broeck @guyvdb]

This might be a good time to mention my recent keynotes titled: "Agentic AI is Neurosymbolic AI"

[François Chollet @fchollet]

I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a "harness"), running at inference time, orchestrating thousands of calls to a neural network for any given task, is the exact definition of a "neurosymbolic architecture"

[Khairallah AL-Awady @eng_khairallah1]

🚨 Anthropic's own team just showed how to actually use Claude Code properly.

25 minutes. free. from the people who built it.

watch the workshop. bookmark it.

you've been using Claude for months without knowing the commands.

Then read the guide below.

[How To Prompt @HowToPrompt__]

Yann LeCun was right the entire time. And generative AI might be a dead end.

For the last three years, the entire industry has been obsessed with building bigger LLMs. Trillions of parameters. Billions in compute.

The theory was simple: if you make the model big enough, it will eventually understand how the world works.

Yann LeCun said that was stupid.

He argued that generative AI is fundamentally inefficient.

When an AI predicts the next word, or generates the next pixel, it wastes massive amounts of compute on surface-level details.

It memorizes patterns instead of learning the actual physics of reality.

He proposed a different path: JEPA (Joint-Embedding Predictive Architecture).

Instead of forcing the AI to paint the world pixel by pixel, JEPA forces it to predict abstract concepts. It predicts what happens next in a compressed "thought space."

But for years, JEPA had a fatal flaw.

It suffered from "representation collapse."

Because the AI was allowed to simplify reality, it would cheat. It would simplify everything so much that a dog, a car, and a human all looked identical.

It learned nothing.

To fix it, engineers had to use insanely complex hacks, frozen encoders, and massive compute overheads.

Until today.

Researchers just dropped a paper called "LeWorldModel" (LeWM).

They completely solved the collapse problem.

They replaced the complex engineering hacks with a single, elegant mathematical regularizer.

It forces the AI's internal "thoughts" into a perfect Gaussian distribution.

The AI can no longer cheat. It is forced to understand the physical structure of reality to make its predictions.

The results completely rewrite the economics of AI.

LeWM didn't need a massive, centralized supercomputer.

It has just 15 million parameters.

It trains on a single, standard GPU in a few hours.

Yet it plans 48x faster than massive foundation world models. It intrinsically understands physics. It instantly detects impossible events.

We spent billions trying to force massive server farms to memorize the internet.

Now, a tiny model running locally on a single graphics card is actually learning how the real world works.

[Allen Braden @allen_explains]

This 2-hour Stanford lecture breaks down how models like ChatGPT and Claude are actually built, clearer than what many people in top AI roles ever get exposed to.

Save this and set aside two hours today. It might end up being the most valuable thing you learn all week.

[Matthew Berman @MatthewBerman]

.@AnthropicAI just published a WILD new AI jailbreaking technique

Not only does it crack EVERY frontier model, but it's also super easy to do.

ThIS iZ aLL iT TakE$ 🔥

Here's everything you need to know: 🧵

[Wolf of X @WolfofX]

Ever thought about viewing the world differently..? This thread on cool maps is here to blow your mind

1. All roads lead to Rome

[Edward Snowden @Snowden]

As information comes in about the exploding beepers in Lebanon, it seems now more likely than not to be implanted explosives, not a hack. Why? Too many consistent, very serious injuries. If it were overheated batteries exploding, you'd expect many more small fires & misfires.

[UpliftUnwind @UpliftUnwind]

Today's Reminder

[Antonine University (UA) @uAntonine]

Join us at the Open Day for #ArtificialIntelligence organized by UA’s Faculty of Engineering (FOE), TICKET Lab, and @IEEEComSocLeb!

Check out the program and register now 🔗 https://t.co/FHjoTpMe26

#uAntonine #uA_Engineering #AI

[Bishal Nandi @LearnWithBishal]

People who use checklists finish their work 50% quicker.

However, 99% are not making their checklists correctly.

Here's the right way to do it:

[🔖Bookmark and Follow]* ↓

[Andrej Karpathy @karpathy]

# automating software engineering

In my mind, automating software engineering will look similar to automating driving. E.g. in self-driving the progression of increasing autonomy and higher abstraction looks something like:

1. first the human performs all driving actions manually
2. then the AI helps keep the lane
3. then it slows for the car ahead
4. then it also does lane changes and takes forks
5. then it also stops at signs/lights and takes turns
6. eventually you take a feature complete solution and grind on the quality until you achieve full self-driving.

There is a progression of the AI doing more and the human doing less, but still providing oversight. In Software engineering, the progression is shaping up similar:

1. first the human writes the code manually
2. then GitHub Copilot autocompletes a few lines
3. then ChatGPT writes chunks of code
4. then you move to larger and larger code diffs (e.g. Cursor copilot++ style, nice demo here https://t.co/u8ueY0mGxZ)
5....
Devin is an impressive demo of what perhaps follows next: coordinating a number of tools that a developer needs to string together to write code: a Terminal, a Browser, a Code editor, etc., and human oversight that moves to increasingly higher level of abstraction.

There is a lot of work not just on the AI part but also the UI/UX part. How does a human provide oversight? What are they looking at? How do they nudge the AI down a different path? How do they debug what went wrong? It is very likely that we will have to change up the code editor, substantially.

In any case, software engineering is on track to change substantially. And it will look a lot more like supervising the automation, while pitching in high-level commands, ideas or progression strategies, in English.

Good luck to the team!

[Rosa @Rosaserhane]

"لا تجعلوا من المآسي أعباء عليكم...

فوراء كل مأساة تختبئ عبرة...

ومن خلف كل ألم، تشرق شمس الوعي!".

"اللاوعي إن حكى" (ص26)
ج ب م المعلّم 💜

[Min Choi @minchoi]

The OpenAI Sora team dropped more wild AI videos.

100% AI 🤯

1. Cybernetic German Shepherd

[Lama Hélène @LamaSleem]

Would you be interested to join a live webinar that helps you create easily your chatbot or AI avatar for your company/just for fun ?

[Santiago @svpino]

LoRA is a genius idea.

To understand the fine-tuning of Large Language Models, you must understand how LoRA works.

By the end of this post, you'll know everything important about how it works.

Large Language Models are good generalists, but they have little specialization. We train them in many different tasks, so they know a bit about everything but not enough about anything.

Think of a kid who can play three different sports at a high level. While he can be proficient across the board, he won't get a scholarship unless he specializes. That's how the kid can reach his full potential.

We can do the same with these large models. We can train them to solve a particular task and nothing else.

We call this process "fine-tuning." We start with everything the model knows and adjust its knowledge to help it improve on the task we care about.

Fine-tuning is revolutionary, but it's not free.

Fine-tuning a large model takes time, care, and lots of money. Many companies can't afford the process. Some can't pay for the hardware. Some can't hire people who know how to do it. Most companies can't do either.

That's where LoRA comes in.

We realized we could approximate a large matrix of parameters using the product of two smaller matrices. There was a lot of wasted space within these large models. What would happen if we find a new, more optimal representation?

Did you ever buy a map at a gas station? Giant pages showing every small road, path, and lake around you. They were exhaustive but hard to navigate. These are like parameters in a large model.

LoRA turns a gas station map into a cartoon treasure map. Every useless parameter is gone. Only two roads, a palm tree, and a cross pointing at the treasure. We don't need to fine-tune the entire model anymore. We can only focus on the small treasure map that LoRA gives us.

It's a mind-blowing trick.

We can train the small approximation matrices from LoRA instead of fine-tuning the entire model. LoRA is cheaper, faster, and uses less memory and storage space.

You can also merge the approximation matrices with the model during deployment time. They work like simple adapters. You load up the one you need to solve a problem and use a different one for the next task.

Then, we have QLoRA, which makes the process much more efficient by adding 4-bit quantization. QLoRA deserves its own separate post.

The team at @monsterapis has created an efficient no-code LoRA/QLoRA-powered LLM fine-tuner.

What they do is pretty smart:

They automatically configure your GPU environment and fine-tuning pipeline for your specific model. For example, if you want to fine-tune Mixtral 8x7B on a smaller GPU, they will automatically use QLoRA to keep your costs down and prevent memory issues.

The @monsterapis platform specializes in no-code LoRA-powered fine-tuning. It's the fastest and most affordable offering for fine-tuning models in the market. They sponsored me and gave me 10,000 free credits for anyone who uses the code "SANTIAGO" in their dashboard:

https://t.co/DBsB9EXgYR

If you want to read their latest updates, get free credits and special offers, join their Discord server: https://t.co/oecoxEec14

TL;DR:

• Traditional fine-tuning trains the entire model. It requires a complex setup, higher memory, and expensive hardware.

• LoRA: Trains a small portion of the model. It's faster, requires much less memory, and affordable hardware.

• QLoRA: Much more efficient than LoRA, but it requires a more complex setup.

• No-code fine-tuning with LoRA/QLoRA: The best of both worlds. Low cost and easy setup.

[Lama Hélène @LamaSleem]

Nature is scary.. yet beautiful

[Lama Hélène @LamaSleem]

https://t.co/5bQDFnXl3Q
A good Visualization of LLMs
#LLMs #visualization #transformers

[Sayak Paul @RisingSayak]

The final half of @karpathy's talk on "Intro to Large Language Models" is actually a gem of an overview of LLM security! It's so good!

https://t.co/xF6MchJymQ