Or Raz

LLM security enthusiast. Learning, sharing, and helping others understand the complexities. Join me on this journey.

[Or Raz @OrRaz6]

In the era of LLM-based software, prompt-injection-based SSRF is emerging as a major attack surface.
I've just pushed a key security fix for @LangChain, fortifying the APIChain against this type of attack:
https://t.co/8ut0MSyWol

[Or Raz @OrRaz6]

And here is a clear sign that the hype is over -

[Or Raz @OrRaz6]

2 signs that enterprise adoption of LLM has started - and one that the hype is over -->

[Or Raz @OrRaz6]

People are looking for Azure OpenAI...

[Or Raz @OrRaz6]

LangChain is skyrocketing.

[Or Raz @OrRaz6]

A new interesting prompt injection based SSRF, this time in LlamaIndex.

[Boaz Wasserman @BoazWasserman]

LLM based agents are growing in popularity, and I often wonder how many people are aware of the risks in enabling LLMs to use tools.

While playing with LlamaIndex agents, I came across an example notebook for an OpenAPI + Requests based agent which is vulnerable to SSRF.

[Or Raz @OrRaz6]

4/4 This should be a concern for any business that's adopting generative AI. When architecting LLM powered apps, it must be taken into account.

[Or Raz @OrRaz6]

1/4 🚨Indirect prompt injection is set to dominate the cybersecurity threat landscape this decade. Hard to imagine an attack surface more potent -->

[Or Raz @OrRaz6]

3/4 🎯Perfect exploitation ground? Applications using LLM to analyze web content. Attackers should simply be able to alter website's content, in order to get initial access to the LLM and try to exploit it.

[Or Raz @OrRaz6]

2/4 In an indirect prompt injection, the adversarial instructions are introduced by a third party data source like a web search or API call.

[Harrison Chase @hwchase17]

🧪langchain_experimental

In an effort to make langchain leaner, more focused, and safer, we are moving select chains to a separate package on 7/28

Big thanks to folks like @BoazWasserman @OrRaz6 Justin Flick for pushing on the safety part

There will be some breaking changes 🧵

[Or Raz @OrRaz6]

It could be hard to introduce so many changes and new features, and still maintain high security standards.
I'm glad that @langchain and @hwchase17 are on that.

[Boaz Wasserman @BoazWasserman]

Exciting changes coming from @langchain on the security front.

There is a new announcement that any component that has vulnerabilities (SQL & python agents for example) will be moved from the package core into an experimental package

https://t.co/73h0VqBFkG

[Boaz Wasserman @BoazWasserman]

There's a new awesome blog from @Dropbox that shows evidence of OpenAI's gpt-3.5-turbo processing control sequences the same way a terminal would do. E.g. \b will cause the model to "backspace" and ignore previous input.

https://t.co/7ZZATUcg4X

[Boaz Wasserman @BoazWasserman]

The fact that ChatGPT Code Interpreter can still be jailbroken to do really nasty stuff shows how far we are from solving LLM jailbreaks.

I was easily able to get it to create a macro-enabled document that downloads and executes a payload from pastebin 🫤

[Or Raz @OrRaz6]

Are they possible? Sure.
But I must say that data poisoning is one of the more complicated (and less likely to be common) attack scenarios, and there are much easier LLM attack scenarios.

[LLM Security @llm_sec]

Are data poisoning attacks practical?

tl;dr: yes - podcast with ETHZ prof on work w. Robust, Google, NVIDIA

https://t.co/DKt88OBPE9

[Or Raz @OrRaz6]

@llm_sec Interesting 🤔
Possible? Yes. But when it comes to LLM security I wouldn't say they're the first thing to worry about.

[Or Raz @OrRaz6]

@omarsar0 Hi 👋
Researching and sharing on LLM security

[Or Raz @OrRaz6]

If you haven't tried this one already, you may want to check it out:
https://t.co/795OP621wx

Nicholas Carlini will show you how little you truly know about GPT-4 capabilities.

[Or Raz @OrRaz6]

@rowancheung AI will change customer relationship management forever.

We just have to make sure that we keep these kind of use cases secure. And this is a big challenge.

[Boaz Wasserman @BoazWasserman]

Another day, another LangChain RCE...
This time in a new CPAL chain (causal program-aided language), which improves upon the capabilities of the existing PAL chain

[Or Raz @OrRaz6]

THE most interesting weakness on OWASP LLM top 10 is #8: Excessive Agency. 🕵️

This decade, the security teams' greatest catalyst for change could be the fortification of LLM agency security.