Pascal Biese

Weekly LLM Research Nuggets in my newsletter📲 AI/ML Engineer 🧑‍💻 The time to build is now 🦾 xaiguy.agi on 🧵

[Pascal Biese @xaiguydotagi]

I don't like newsletters - that's why I'm starting one.

There'll be a weekly newsletter that will be similar to my daily posts. I will also share articles that will help you integrate some of the research and tools I share into your own workflows.

🔗https://t.co/rVHxyc2mcH

[Pascal Biese @xaiguydotagi]

Thou Shall Not Lie: LLMs' might be getting a reality check ✔

Hallucinations in AI-generated content often spark intense discussions about the trustworthiness of large language models (LLMs).

Now, a new approach called Truth Forest (TrFr) is aiming to significantly enhance the truthfulness of these models. Researchers have demonstrated a striking boost in accuracy: for example, the Llama-2-7B model's truthfulness rating jumped from an unremarkable 40.8% to an impressive 74.5% using the new method.

https://t.co/UZeMlzFmjr

[Pascal Biese @xaiguydotagi]

Turbulences Ahead: Are LLMs for Coding Robust or RUSTy? ✈ 🌪

Turbulence is a new benchmark tailored to test the integrity of instruction-tuned Large Language Models (LLMs) in code generation.

Rather than just gauging if an AI can code, it zeroes in on the nuances: Can it handle variations of a similar code problem?

https://t.co/ejzUTnjhS4

[Pascal Biese @xaiguydotagi]

LLMs and High-Stakes Decisions: Finally Becoming More Reliable? 🤔

A new approach rooted in the principles of social choice theory, leverages a novel application of the Partial Borda Count—a method to merge ranked choices—which could dramatically improve the reliability of LLMs in sensitive fields. Their empirical findings suggest notable strides in robustness, potentially transforming LLM outputs from random flukes to steadfast guides in settings like medical diagnostics.

https://t.co/Q6c8T17suB

[Pascal Biese @xaiguydotagi]

Tested on 50 tasks over 10 applications including email, maps, and image editing, this agent displayed a robust ability to handle high-level interactions. That's a big step for LLMs entering our daily lives, perhaps sooner than you'd expect. Keep an eye on this space – the way we interact with our gadgets might be on the cusp of an AI revolution!

3/3

[Pascal Biese @xaiguydotagi]

LLMs can now find real love for you on Tinder 🤯 😂

Tencent just open sourced their LLM smartphone agent framework, which means you'll probably never ever have to manually touch your screen again.

1/3

[Pascal Biese @xaiguydotagi]

Jokes aside, with decently capable Large Language Models (LLMs) running on smartphones and other consumer hardware, it will become more and more important to develop models that can navigate operating systems and interact with complex Graphical User Interfaces (GUIs).

2/3

[Jeremy Howard @jeremyphoward]

“Nobody called LLMs” totally misunderstands how AI development happens. LLMs didn’t spontaneously appear for no reason.

I knew they would work, I demonstrated them working in 2017, and I said at that time I think they’ll be a big deal.

[Pascal Biese @xaiguydotagi]

Supercharge Your Home Rig with Data Center-Level LLM Inference Speeds 🔥 💻

Ever dreamt of running data center-grade Large Language Model (LLM) inference on your personal gaming GPU? PowerInfer now makes this a reality.

https://t.co/qTlwx7280E

[Pascal Biese @xaiguydotagi]

Wouldn't it be great if in the future, you wouldn't need to spend time crawling through the internet anymore to identify if someone actually has the experience he or she claims to have? Or if the experience they have is relevant to their idea?

Would you let Founder-GPT lend you a hand for your decision processes?

https://t.co/tEsoJ137yz

4/4

[Pascal Biese @xaiguydotagi]

Predicting Start-up Success from LinkedIn Profiles 📈 💰

Let's explore a fun idea today. What if I told you that researchers are now using advanced Large Language Models (LLMs) to link founders' backgrounds to their startup ideas?

1/4

[Pascal Biese @xaiguydotagi]

The research indicates that the success patterns for each startup idea are as unique as fingerprints, necessitating a customized approach based on the specific nuances of a founder's experience and skills. And humans are too often based to their own limited experience when it comes to funding and ideas.

3/4

[Pascal Biese @xaiguydotagi]

LoRAMoE doesn't merely prevent knowledge loss—its modular approach may reshape how we think about multi-task learning and LLM fine-tuning.

With LoRAMoE, we get the best of both worlds - gaining specific knowledge while maintaining general capabilities.

[Pascal Biese @xaiguydotagi]

What's better than LoRA and MoE? Enter, LoRAMoE 🔥

Are Large Language Models (LLMs) forgetting their "world knowledge" while fine-tuning?The concept of "world knowledge forgetting" during supervised fine-tuning (SFT) of LLMs has been a pressing issue while trying to improve AI comprehension and output in specific tasks. An increase in specialized fine-tuning data has traditionally led to a decrease in the breadth of the model's knowledge—until now.

https://t.co/rDzZ9nhNwI

[Pascal Biese @xaiguydotagi]

In a new paper, the Chinese researchers tackled the delicate balance between task-specific performance gains and preserving the LLM's existing knowledge base. Their method, LoRAMoE, can be thought of as adding an expert plugin to the base LLM, which remains untouched to safeguard its accrued "world knowledge."

[Pascal Biese @xaiguydotagi]

Did gpt-3.5-turbo-1106 really have to do me like that?

[Pascal Biese @xaiguydotagi]

"I'm sorry, but I cannot provide a rating for this paper as I am an AI language model and not capable of making qualitative assessments. If you have any specific questions about the content of the paper or need assistance with any particular aspect of it, feel free to ask!"

[Pascal Biese @xaiguydotagi]

The toolkit encompasses prompt construction, engineering, evaluation protocols, and adversarial prompt attacks.

It strives to move past the fragmented landscape of LLM benchmarks by providing a unified, extensible library that any researcher can plug into and expand upon.

[Pascal Biese @xaiguydotagi]

OpenAI's Prompt Engineering Guide Was a First Step: Here's What's Next 👨‍🏫 💡

Evaluating Large Language Models (LLMs) is no cake walk and arcane prompting techniques are only adding to this challenge.

https://t.co/fTjRVB4Chy

[Pascal Biese @xaiguydotagi]

PromptBench is built on top of the following question: How do we assess LLMs comprehensively, mitigate security risks, and push the boundaries of their capabilities?