Paul Drm

Letsgo twitter

[Paul Drm @PaulDarm]

Where did I leave my satellite again? 🛰️🧐In our upcoming #ACL2023 industry track paper (w/ the great @AVMiceliBarone & Shay B. Cohen @EdinburghNLP, and my supervisor Annalisa Riccardi @icelab_uk) we develop a Q&A system for @esa operational engineers https://t.co/WQhnzIYqsl 1/3

[Paul Drm @PaulDarm]

@MartinB14053768 @SparesCeltic Hey - cannot DM you but would be interested

[Paul Drm @PaulDarm]

@darrenmcfadden9 @SparesCeltic @darrenmcfadden9 cannot DM you. You need to update your message settings.

[Paul Drm @PaulDarm]

@darrenmcfadden9 @SparesCeltic Take one if still available

[Paul Drm @PaulDarm]

@SparesCeltic looking for one today. Scan in if possible would be great.

[Paul Drm @PaulDarm]

Code & datasets: https://t.co/9LyIdANrco

DM me if you want to collaborate in representation engineering!

[Paul Drm @PaulDarm]

Representation engineering might be the key to controllable AI systems 🤖
Our latest research shows how adding specific vectors to LLM attention heads can steer model behavior in precise ways.
Thread 🧵

[Paul Drm @PaulDarm]

Key finding: Fine-grained interventions at select attention heads outperform:

Full layer interventions
Supervised fine-tuning
Other broad steering methods

Only a few example completions needed vs. traditional fine-tuning datasets.

[Paul Drm @PaulDarm]

@ChatGPTapp Cringe

[Paul Drm @PaulDarm]

@EhudReiter The main user would be humanitarian aid workers. We developed around conversations with domain experts. Personally, I think showing sources and snippets from sources is a good thing to increase trustwothiness. You can test yourself https://t.co/gVs9FPcPVS pw: floodbrain

[Paul Drm @PaulDarm]

@EhudReiter Very important topic! Albeit, I think at least the first three points Cost/Response Time/Unstable (LLMs beeing discontinued) will be solved with more and cheaper compute. E.g. fine-tuned LLaMa models that run locally on a M2 chip at adequate speed and scratch GPT-3.5 performance.

[Paul Drm @PaulDarm]

We overcame these using a specialized architecture utilizing out-of-domain training data and in-context learning of LLMs. We match GPT-4 results with significantly less compute, enabling us to run the assistant locally on consumer hardware while maintaining data security 3/3

[Paul Drm @PaulDarm]

Where did I leave my satellite again? 🛰️🧐In our upcoming #ACL2023 industry track paper (w/ the great @AVMiceliBarone & Shay B. Cohen @EdinburghNLP, and my supervisor Annalisa Riccardi @icelab_uk) we develop a Q&A system for @esa operational engineers https://t.co/WQhnzIYqsl 1/3

[Paul Drm @PaulDarm]

Engineers may struggle with complex database queries. Our approach helps them use everyday language for satellite management tasks like re-entry analyses or collision avoidance. Practical challenges were the sensitivity of the data as well having no training set available. 2/3

[Paul Drm @PaulDarm]

@RichardSocher @elonmusk @YouSearchEngine 2. They could also offer companies a way to promote their products. Imagine talking to a Mickey Mouse assistant who recommends the newest attraction in Disney Land. Challenges are probably enabling targeted products recommendations and ensuring output aligns with company policies

[Paul Drm @PaulDarm]

@RichardSocher @elonmusk I think personalized assistants are a massive market. They let you engage directly with your favorite character/idol from any franchise/time period, and @YouSearchEngine seems indeed capable of doing that, even adapting to unique speech patterns like Master Yoda's!

[Paul Drm @PaulDarm]

@aimatej @ykilcher Have a look at https://t.co/pjNQ7KhG2q