Daniel Pepl

PR & Communications I Music I private profile

[Daniel Pepl @fullmax]

@MajmudarAdam Security by design is key…

[profil @profilonline]

Das Wiener Band popsch über queere Sichtbarkeit, politische Rückschritte, Streitkultur – und ihr neues Album „Gay Propaganda“. https://t.co/CHVD1dL4Qn

[Daniel Pepl @fullmax]

@AITtomorrow2day Nachruf Dr. Hannes Androsch (1938 - 2024): https://t.co/zUaeZpiwN4

[Daniel Pepl @fullmax]

@BernhardEcker Sehr einfühlsamer Nachruf.

[Daniel Pepl @fullmax]

APA: Hannes Androsch im 87. Lebensjahr gestorben.

[Joseph Noel Walker @JosephNWalker]

New episode with former US Treasury Secretary and current OpenAI board member Larry Summers (@LHSummers).

We discuss:
- How he's been learning about AI.
- The odds of the technology delivering a much faster economic growth regime (akin to the Industrial Revolution).
- How AGI might change economic policymaking.

Enjoy!

Timestamps:
(0:00:00) - Introduction.
(0:00:46) - Larry's journey teaching himself about AI & deep learning since joining OpenAI’s board.
(0:09:15) - How many hours per week has Larry been spending on OpenAI-related stuff?
(0:10:16) - Which bottleneck to AI scaling does Larry think is the most underrated?
(0:12:22) - Approximately what share of time do today's AI researchers spend on tasks that AI will be doing for them in five years?
(0:15:01) - What explains the remarkable steadiness of US economic growth over the last 150 years?
(0:19:42) - How likely is it that AI initiates a new growth regime with average growth that’s ~10x faster than today?
(0:21:36) - What are the best economic arguments for believing AI won’t deliver a regime of ever-increasing growth rates?
(0:25:33) - How much could AGI boost economic growth in developing countries merely by helping their policymakers make better decisions?
(0:28:20) - How much better could monetary policy be if the Fed had AGI?
(0:31:40) - How much would having AGI have helped US economic policymakers during the financial crisis & Great Recession?
(0:36:07) - Is the CCP infiltrating and stealing the IP of major AI labs in the US and UK?
(0:39:35) - At what point should AI be nationalised?
(0:42:39) - How would Bill Clinton or Barack Obama be thinking about AI governance?
(0:44:27) - If OpenAI restructures to a public benefit corporation, how does that change its incentives?
(0:46:18) - What does Daron Acemoglu miss in his analysis of the economic impacts of AI?

[Daniel Pepl @fullmax]

@tomaspueyo To my mind, it was a super show. Celine Dion‘s performance topped it.

[Daniel Pepl @fullmax]

@Kid_Pex @florianklenk Der Ansatz ist auch spannend:

https://t.co/veByjunQ5l

[Daniel Pepl @fullmax]

@schla_wienerin Einzeltäter?

[Daniel Pepl @fullmax]

@gerhard_mangott Gratulation!

[Daniel Pepl @fullmax]

@gerhard_mangott Gratulation!

[Daniel Pepl @fullmax]

@schla_wienerin WTF?!

[Daniel Pepl @fullmax]

@Claus_Pandi @TcaneOfficial vermute (in)direkt über den Postkolonialismus-Diskurs im universitären Umfeld einerseits und (linkem) Kapitalismuskritik andererseits...

[Jim Fan @DrJimFan]

You'll soon see lots of "Llama just dethroned ChatGPT" or "OpenAI is so done" posts on Twitter. Before your timeline gets flooded, I'll share my notes:

▸ Llama-2 likely costs $20M+ to train. Meta has done an incredible service to the community by releasing the model with a commercially-friendly license. AI researchers from big companies were wary of Llama-1 due to licensing issues, but now I think many of them will jump on the ship and contribute their firepower.

▸ Meta's team did a human study on 4K prompts to evaluate Llama-2's helpfulness. They use "win rate" as a metric to compare models, in similar spirit as the Vicuna benchmark. 70B model roughly ties with GPT-3.5-0301, and performs noticeably stronger than Falcon, MPT, and Vicuna.
I trust these real human ratings more than academic benchmarks, because they typically capture the "in-the-wild vibe" better.

▸ Llama-2 is NOT yet at GPT-3.5 level, mainly because of its weak coding abilities. On "HumanEval" (standard coding benchmark), it isn't nearly as good as StarCoder or many other models specifically designed for coding. That being said, I have little doubt that Llama-2 will improve significantly thanks to its open weights.

▸ Meta's team goes above and beyond on AI safety issues. In fact, almost half of the paper is talking about safety guardrails, red-teaming, and evaluations. A round of applause for such responsible efforts!

In prior works, there's a thorny tradeoff between helpfulness and safety. Meta mitigates this by training 2 separate reward models. They aren't open-source yet, but would be extremely valuable to the community.

▸ I think Llama-2 will dramatically boost multimodal AI and robotics research. These fields need more than just blackbox access to an API.
So far, we have to convert the complex sensory signals (video, audio, 3D perception) to text description and then feed to an LLM, which is awkward and leads to huge information loss. It'd be much more effective to graft sensory modules directly on a strong LLM backbone.

▸ The whitepaper itself is a masterpiece. Unlike GPT-4's paper that shared very little info, Llama-2 spelled out the entire recipe, including model details, training stages, hardware, data pipeline, and annotation process. For example, there's a systematic analysis on the effect of RLHF with nice visualizations.

Quote sec 5.1: "We posit that the superior writing abilities of LLMs, as manifested in surpassing human annotators in certain tasks, are fundamentally driven by RLHF."

Congrats to the team again 🥂! Today is another delightful day in OSS AI.

[Daniel Pepl @fullmax]

@gerhard_mangott Toller Einsatz!

[Daniel Pepl @fullmax]

@corinnamilborn Besagter Verlag, bei dem die Gedichte verlegt worden waren (und der jetzt die Zusammenarbeit aufgekündigt hat), hat damals (noch) noch mit dem „Lyrischen Ich“ argumentiert …hier gehört echt das ganze Umgeld / System / Machtstrukturen hinterfragt …

https://t.co/TgWcQWjZph

[Daniel Pepl @fullmax]

@Anetttu @JoeBiden 😃👍

[Daniel Pepl @fullmax]

@Anetttu Looks awesome!