Jacky Renno

Geek, Sportif , G.O., père de famille et le reste du temps... président et fondateur Kantic Analytics

[Punk（2898 🙌💎） @punk2898]

OpenAI 的天塌了，karpathy 是 OpenAI 的创始成员、AI 圈公认的“活菩萨”级人物、开源狂魔，现在去 Anthropic 了🤣

看看他的履历：

- OpenAI 创始成员之一

- Tesla 前 AI 总监， Elon 亲自挖过去，带队干 Autopilot 计算机视觉，直接推动 FSD

- 斯坦福 CS231n（深度学习视觉神课）主要讲师，教了全球几十万人

- 开源狂魔：nanoGPT、micrograd、char-rnn 等项目，直接手把手教你从零实现 GPT（各种 LLM Wiki 博主都是搬运的他的）

- 写了一堆神级博客 + 拍了大量 YouTube 视频，把最难的 backprop、Transformer、LLM 讲得异常清晰

但是对开发者来说可太棒了！

[Andrej Karpathy @karpathy]

This works really well btw, at the end of your query ask your LLM to "structure your response as HTML", then view the generated file in your browser. I've also had some success asking the LLM to present its output as slideshows, etc.

More generally, imo audio is the human-preferred input to AIs but vision (images/animations/video) is the preferred output from them. Around a ~third of our brains are a massively parallel processor dedicated to vision, it is the 10-lane superhighway of information into brain. As AI improves, I think we'll see a progression that takes advantage:

1) raw text (hard/effortful to read)
2) markdown (bold, italic, headings, tables, a bit easier on the eyes) <-- current default
3) HTML (still procedural with underlying code, but a lot more flexibility on the graphics, layout, even interactivity) <-- early but forming new good default
...4,5,6,...
n) interactive neural videos/simulations

Imo the extrapolation (though the technology doesn't exist just yet) ends in some kind of interactive videos generated directly by a diffusion neural net. Many open questions as to how exact/procedural "Software 1.0" artifacts (e.g. interactive simulations) may be woven together with neural artifacts (diffusion grids), but generally something in the direction of the recently viral https://t.co/z21CP5iQfu

There are also improvements necessary and pending at the input. Audio nor text nor video alone are not enough, e.g. I feel a need to point/gesture to things on the screen, similar to all the things you would do with a person physically next to you and your computer screen.

TLDR The input/output mind meld between humans and AIs is ongoing and there is a lot of work to do and significant progress to be made, way before jumping all the way into neuralink-esque BCIs and all that. For what's worth exploring at the current stage, hot tip try ask for HTML.

[Julien Simon @julsimon]

I used AI to reverse-engineer the Apollo 11 source code.
@NASAArtemis is about to fly, the first crewed mission beyond low Earth orbit since 1972. It felt like the right moment to look at the software that got us there the first time.

40,000 lines of 1960s assembly. A 15-bit computer with 4 KB of RAM. A dead architecture that almost no one alive can read. I pointed @claudeai at it anyway.

The result: 6,500 lines of technical analysis covering the job scheduler, the restart system that saved the landing, the bytecode VM that made the code fit in memory, and the guidance equations that flew Armstrong and Aldrin to the surface.

The biggest surprise? The team invented patterns we think are modern: crash-only design (decades before Erlang), cooperative multitasking (decades before goroutines), virtual DOM diffing (decades before React), and bytecode interpretation (decades before the JVM).

"Our codebase is too legacy for AI." I keep hearing this. If Claude can make sense of 1's-complement assembly for a computer with less RAM than a bank card chip, your Java monolith is going to be fine. No code is too old.

I shared the full walkthrough, including all prompts and what the AI got right and wrong, on @githubstatus

https://t.co/KSlT0TCB2p

[Clara Chappaz @ClaraChappaz]

900 millions d'euros levés par AMI, la start-up d'IA de Yann LeCun, basée à Paris à ambition mondiale.🏆
Après les LLMs, place aux world models ! La France est le lieu où l'on construit l'IA de demain.
Fière de ce que @ylecun et son équipe incarnent pour notre écosystème. Bravo !

[Powlisher @powl_d]

https://t.co/eQXtiHxlVe

[L'Équipe @lequipe]

🙌​🇫🇷 HISTORIQUE !

Christo Popov remporte les World Tour Finals après sa victoire face au n°1 mondial Shi Yu Qi, c'est une première pour un Français !

#lequipeBADMINTON

[Frederic CAVAZZA @fredcavazza]

Une équipe de chercheurs japonais dévoile une architecture multi-agents particulièrement performante qui combine plusieurs modèles entre eux. #GenAI #AIagent
https://t.co/Dx33GwcL6E

[Quentin '🐧' ADAM @waxzce]

Bon, pour ceux quui l'aurait loupé, il faut aller écouter ça https://t.co/NRW8b6qesy

[Yann LeCun @ylecun]

#Openalwayswins

[Infos Françaises @InfosFrancaises]

HANDBALL : La France🇫🇷 se qualifie pour la finale de l'Euro après sa victoire renversante contre la Suède🇸🇪 en prolongation (34-30). #FRASUE

[Andrew Ng @AndrewYNg]

After reading the @nytimes lawsuit against @OpenAI and @Microsoft, I find my sympathies more with OpenAI and Microsoft than with the NYT.

The suit:
(1) Claims, among other things, that OpenAI and Microsoft used millions of copyrighted NYT articles to train their models
(2) Gives examples in which OpenAI models regurgitated NYT articles almost verbatim

But the presentation muddies (1) and (2), and I saw a lot of commentary on social media that -- because of what I believe is a muddied presentation -- draws a link between them that I'm not sure is what people think it is.

On (1): I understand why media companies don't like people training on their documents, but believe that just as humans are allowed to read documents on the open internet, learn from them, and synthesize brand new ideas, AI should be allowed to do so too. I would like to see training on the public internet covered under fair use -- society will be better off this way -- though whether it actually is will ultimately be up to legislators and the courts.

On (2): I suspect a lot of the examples of ChatGPT regurgitating articles nearly verbatim were due to a RAG-like mechanism where the user prompt causes the system to browse the web, retrieve a specific article and then print it out. (If my statement here isn't accurate, I would love to see the @nytimes clarify this.) If this is the case, then (i) To OpenAI's credit, they seem to have already updated their software to make this much less likely, and (ii) This is also a much easier problem to fix than if an LLM were to regurgitate text using only the pre-trained weights, which AFAIK very rarely happens (and which, given its rarity, also raises the question of how much harm to NYT this has actually caused).

To be clear, I believe independent media is important for democracy and must be protected. I also sympathize with media businesses worried about Generative AI disrupting their businesses. But I'm not convinced the NYT lawsuit is the right way to do this.

Usual caveat: I am not a lawyer and am not giving legal advice or any other form of advice here.

You can also read more details of my take on this below. https://t.co/wkZSMHsvNA

[Bindu Reddy @bindureddy]

Why LLMs kickass at Code Generation and will they Replace Programmers?

It's probably not all that surprising that LLMs can understand machine language (python, java etc.) better the human language. Code generation has evolved to be the killer application of these large language models.

So why are they so good at this skill and will they replace human programmers?

Fundamentally, it's because code has repeatable patterns, we have a large amount of training data and LLMs are particularly good at understanding context.

Code has less nuance than human language, has specific design paradigms, and follows some structured rules. Code needs to be less ambiguous than human language. This makes it so that it is easier to generate code that is syntactically correct

Programming languages have limited vocabulary and we are not constantly inventing new words and slang that literally needs an urban dictionary.

While LLMs are pretty good at understanding context, code requires much less contextual understanding than text. For example, a sorting algorithm will not require a broader context, unlike a complex piece of text.

Code tends to be more logical, functional, and less creative than text, making it simpler to generate correct code. Another big advantage is that it's easy to test the validity of generated code by simply executing it and looking at the errors.

All this means that LLMs kickass at code generation. Does this mean they will soon replace programmers? The short answer is NO in the next 1-3 years and YES beyond 3-5 years.

LLMs don't have the overall context of the problem and are only really good at generating snippets of code. You still need a programmer to translate the business problem into a set of programs. Typically this involves knowledge of the overall system, creative thinking, and design skills all of which today's LLMs don't possess.

Today LLMs can be a fantastic AI assistant to boost programmer productivity from 10 to 50%. This could mean that instead of hiring 10 developers, you may need to hire just 5 to finish a particular project. Most companies, however, won't reduce the number of hires but instead will choose to get more projects done faster.

A decade ago, it would take months to develop a good website or app, now it can be as fast as a couple of days. So programmer productivity has gone up significantly over the last few years. Yet we have seen an increase, not a decrease, in the number of jobs as the overall pie has expanded and software has been eating the world.

Long term, however, LLMs will become smarter and we will be able to chain multiple AI bots together to tackle bigger and bigger tasks. Eventually, you may not need a programmer to translate your mock-ups and PRDs into functioning systems. Until then... keep building ;)

[Andrew Ng @AndrewYNg]

In addition to prompting LLMs, many developers are now also experimenting with fine-tuning. I describe in The Batch how to choose from the growing menu of options for building applications with LLMs: Prompting, few-shot, fine-tuning, pre-training. https://t.co/NgPg0snzNt

[Brian Roemmele @BrianRoemmele]

“DoctorGPT is an LLM that can pass the US Medical Licensing Exam. It works offline, it's cross-platform, &amp; your health data stays private”

Working on a new local model of this that will use telemetry from 16 human sensors to create a daily health check. https://t.co/NTJhMIyrZT

[Andrew Ng @AndrewYNg]

Do Large Language Models really "understand" the world, or just give the appearance of understanding? Evidence (e.g., Othello-GPT) shows LLMs build models of how the world works, which makes me comfortable saying they do understand. More in The Batch: https://t.co/e0JGU2wUbf

[Yann LeCun @ylecun]

This is huge: Llama-v2 is open source, with a license that authorizes commercial use!

This is going to change the landscape of the LLM market.
Llama-v2 is available on Microsoft Azure and will be available on AWS, Hugging Face and other providers

Pretrained and fine-tuned models are available with 7B, 13B and 70B parameters.

Llama-2 website: https://t.co/PKrrXgHdem
Llama-2 paper: https://t.co/aINNrXNhMb

A number of personalities from industry and academia have endorsed our open source approach: https://t.co/N7HwgW9Suh

[Yannick Jadot @yjadot]

L’embauche de madame Scott Morton, citoyenne américaine et lobbyiste des GAFAM, à la DG concurrence de la @EU_Commission est un scandale.
Je demande à @vestager et @vonderleyen d’annuler cette nomination, contraire à l’éthique et à notre souveraineté numérique.

[Gwennael Buchet @GwennaelBuchet]

C’est toujours un très beau projet qui se co-construit en équipe.

Si vous souhaitez en discuter et, pourquoi pas, m’accompagner n’hésitez pas à m’écrire.

[Gilles Babinet @babgi]

Comment les Français sont devenus les rois de l'IA... dans la Silicon Valley.
https://t.co/UlZBG9CEw4

[Gilles Babinet @babgi]

Gpt-4 a désormais des capacités de diagnostic qui lui permettent de passer haut la main l'examen de médecin aux USA. Il est meilleur à cet exercice que bien des médecins américains. C'est le Dr @dr_l_alexandre qui va être content.
https://t.co/MwYMMCdIdC