felix ringe

law&ai, mostly legal evals | PhD'ing in law
visiting fellow @stanfordlaw
prev research @cambridge_uni, mle @freshfields

[felix ringe @felixringe]

We have discovered a substantial number of vulnerabilities in the German legal system using LLMs.

These may enable access to confidential documents, travel without an otherwise required visa, enforceable leave for soldiers at NATO’s JSEC, and more.

Similar to computer systems, the written law can contain vulnerabilities that invite exploitation. In our blog post, we share a selection of vulnerabilities, and present potential exploits for each. To our knowledge, none of these have been previously recorded. We share only those vulnerabilities whose exploitation we believe would not cause serious harm.

blog post: https://t.co/hsYTiJpNoJ

[felix ringe @felixringe]

Hi there! I would say the contrast between the aspects of German tort law you highlight here and the example we present is especially that ours relies on a rule that leaves very little room for interpretation (Art. 80(1)3 GG: the regulation has to cite its legal basis, else it is void), while German tort law has a very sparse written basis as you say, and leaves a lot of room not only for interpretation, but even "judge-made law" (often called "Richterrecht" in German). The safety duties based on negligience ("Verkehrssicherungspflichten") your abstract mentions are a very good example for the latter. Based on this, I would assume these safety duties are unlikely to contain vulnerabilities similar to the ones presented in our post, because in your tort example, the law is highly malleable and allows for very flexible application through the courts

[felix ringe @felixringe]

We have discovered a substantial number of vulnerabilities in the German legal system using LLMs.

These may enable access to confidential documents, travel without an otherwise required visa, enforceable leave for soldiers at NATO’s JSEC, and more.

Similar to computer systems, the written law can contain vulnerabilities that invite exploitation. In our blog post, we share a selection of vulnerabilities, and present potential exploits for each. To our knowledge, none of these have been previously recorded. We share only those vulnerabilities whose exploitation we believe would not cause serious harm.

blog post: https://t.co/hsYTiJpNoJ

[felix ringe @felixringe]

@LudwigBald Hi there, glad to hear! We are working on a German one, and I agree there should be one. For now, I hope the English one works for a German legal audience too - I put in a lot of effort to be precise with linking all English terms to the German legal concepts they represent

[felix ringe @felixringe]

@Profpatsch we do have a lot more "exploit strategies", but I felt like this was the most digestible example to illustrate the concept of legal exploits for a first post!

[felix ringe @felixringe]

Glad to see you have looked into it so closely! This does not hold though, we address this in the follow-up considerations to case 1, point 1. The shortest explanation is that even the Federal Government itself (which made the regulation) expressly states that it considers (1a) to be the required legal basis for §9(5) BKV, but then fails to cite it

[felix ringe @felixringe]

Thank you all for the overwhelming response! I'm glad so many of you took the time to share your thoughts. It's getting very late here on the West Coast, so I'll sign off for tonight, but I'll be back tomorrow!
https://t.co/CX1g4EMNZX

[felix ringe @felixringe]

We have discovered a substantial number of vulnerabilities in the German legal system using LLMs.

These may enable access to confidential documents, travel without an otherwise required visa, enforceable leave for soldiers at NATO’s JSEC, and more.

Similar to computer systems, the written law can contain vulnerabilities that invite exploitation. In our blog post, we share a selection of vulnerabilities, and present potential exploits for each. To our knowledge, none of these have been previously recorded. We share only those vulnerabilities whose exploitation we believe would not cause serious harm.

blog post: https://t.co/hsYTiJpNoJ

[felix ringe @felixringe]

In Germany, the case for what we describe here is actually very strong: The mechanism we describe here is established judicial practice ("ständige Rechtsprechung", the closest relative to precedent in civil-law countries) by the German Federal Constitutional Court, see Judgment of July 6, 1999, 2 BvF 3/90, paras. 154, 158: https://t.co/bT1UWOEOIo (in German) and German courts adhere to it extremely strictly!

[felix ringe @felixringe]

@luna_moon_emoji @PlastiqSoldier what follows for you from the fact that it is a civil-law system?

[felix ringe @felixringe]

@raskolnikov1800 I think we are on the same page here! We do not claim that this is a common phenomenon and you could find this in many laws, in fact, these cases are very rare. But the argument here is that at scale, even these rare phenomena can yield many exploits in absolute terms

[felix ringe @felixringe]

@luna_moon_emoji @PlastiqSoldier https://t.co/XkTWCxIRTY

[felix ringe @felixringe]

You probably mean this a bit tongue-in-cheek, but in case you don’t, I do recommend you read the post! The type of vulnerability we describe here is so “mechanical” that if it emerges in practice, the case is so clear that historically, the executive usually concedes and does not even take the case to court in the first place. These vulnerabilities currently occur very rarely, but for our point, they only need to occur *at all* (which they do, cf. post!) and the rest is scale

[felix ringe @felixringe]

@PeterPneis we address this in the post! Of course there could theoretically be other grounds for confidentiality, but they do not apply automatically, and our example is specifically constructed to preclude all other ones, see footnote 8

[felix ringe @felixringe]

the law is intended to work this way! the purpose of Article 80(1) 3 GG according to the German Federal Constitutional court is to make it possible find and see the legal basis of the regulation (established judicial practice ("ständige Rechtsprechung", the closest relative to precedent in civil-law countries) by the German Federal Constitutional Court, see Judgment of July 6, 1999, 2 BvF 3/90, paras. 154: https://t.co/bT1UWOEgSQ (in German)) and if that citation requirement is violated, the regulation is void (ibid, 158)

[felix ringe @felixringe]

@Robotic_Genocid https://t.co/XkTWCxIRTY

[felix ringe @felixringe]

You probably mean this a bit tongue-in-cheek, but in case you don’t, I do recommend you read the post! The type of vulnerability we describe here is so “mechanical” that if it emerges in practice, the case is so clear that historically, the executive usually concedes and does not even take the case to court in the first place. These vulnerabilities currently occur very rarely, but for our point, they only need to occur *at all* (which they do, cf. post!) and the rest is scale

[felix ringe @felixringe]

@PlastiqSoldier weird, right? but it does!

[felix ringe @felixringe]

I understand that this may feel unusually formal and "too-good-to-be-true", but the mechanism we describe here is actually established judicial practice ("ständige Rechtsprechung", the closest relative to precedent in civil-law countries) by the German Federal Constitutional Court, see Judgment of July 6, 1999, 2 BvF 3/90, paras. 154, 158: https://t.co/CY8VZIexvo (in German) and German courts adhere to it extremely strictly!

[felix ringe @felixringe]

You probably mean this a bit tongue-in-cheek, but in case you don’t, I do recommend you read the post! The type of vulnerability we describe here is so “mechanical” that if it emerges in practice, the case is so clear that historically, the executive usually concedes and does not even take the case to court in the first place. These vulnerabilities currently occur very rarely, but for our point, they only need to occur *at all* (which they do, cf. post!) and the rest is scale

[felix ringe @felixringe]

And many thanks to @MeganPHMa, @JulianNyarko, Esmie Hurd, Chijioke Mgbahurike and everyone else at liftlab at @StanfordLaw who gave feedback and comments on an earlier draft!

[felix ringe @felixringe]

@togelius @paradigmainc should count!

[John Hughes @jjhughes]

It’s getting hard to build good legal evals. So many legal tasks involve judgment and discretion.

I built LegalForecastBench to test frontier models on a high-level legal task with objective ground truth: predicting how judges will decide real motions. https://t.co/5uY9L9oBGx

[felix ringe @felixringe]

Good read! Also, I like your argument that for purely rubric-based scoring, there's not only an empirical case against it (judgmentbench), but also the “jurisprudential” one you make (at least where judicial decisions can serve as ground truth) that rubrics reflect expert views, which are inferior to court rulings in that they never hold precedential value