mkwld

[Polymarket Money @PolymarketMoney]

JUST IN: Andrej Karpathy, a top AI scientist at Anthropic, is reportedly barred from accessing the company’s most advanced AI model because he is not a U.S. citizen.

[mkwld @mkwld]

Can we claim that Mythos has achieved ASI in digital world? I'd say yes.

[mkwld @mkwld]

The new iOS font feels exaggerated. I prefer the plain and straight style on previous versions

[NIK @ns123abc]

BREAKING: OpenAI staff member claims that they have achieved AGI

LMAO

[Tsarathustra @tsarnick]

Former Google CEO Eric Schmidt says that the Russian invasion of Ukraine prompted him to become a licensed arms dealer in order to integrate AI and robotics into warfare

[mkwld @mkwld]

@_jasonwei This seams more commonly seen in deep learning experiments. I remember my initial dl model was such a “yolo” run and it hit sota. Possibly somehow connected to the nature of deep networks…

[LlamaIndex 🦙 @llama_index]

✨Retrieval-augmented generation over video✨
Check out this brand-new notebook in which @lancedb shows you
➡️ How to index video
➡️ Retrieve both text and images in response to your prompt
➡️ Answer questions about video

https://t.co/q7V9XtINsb

[AK @_akhaliq]

Google Deepmind presents Grandmaster-Level Chess Without Search

paper page: https://t.co/qwpbAb9DL7

largest model reaches a Lichess blitz Elo of 2895 against humans, and successfully solves a series of challenging chess puzzles, without any domain-specific tweaks or explicit search algorithms. We also show that our model outperforms AlphaZero's policy and value networks (without MCTS) and GPT-3.5-turbo-instruct. A systematic investigation of model and dataset size shows that strong chess performance only arises at sufficient scale. To validate our results, we perform an extensive series of ablations of design choices and hyperparameters.

[Paul C. Jeffries @PaulJeffries]

General-purpose AI is a commodity.

It’s hard to see how it can be the basis of a large, sustainable business in and of itself. It costs a lot to make, but not much to replicate, and probably doesn’t have a sufficient lifespan to monetize before someone upstages you.

There are huge AI economic opportunities, and maybe one or two “engine” companies can manage to get a captive audience. But the tech itself doesn’t seem to entail an opportunity for a sustainable advantage — and even if it did in theory, people who have other things to offer are willing to not just give it away as a service, but open source the work product that is expensive to accrue (which would have otherwise seemed to be the main moat).

If the engine has a strong value proposition but not actual economic value (because it is commoditized and insufficiently enduring as state of the art, nor advantaged to build on for the next version of state of the art — and therefore an upstart can leapfrog you, your investments are not cumulative), then the monetizable value proposition will be an engine (maybe it doesn’t matter which one) deployed in some particular context.

In that world, the folks who already have or can build that context, that vertical integration, are the ones who will make a sustainable business.

And if you are in the engine business, they’ll just keep giving it away, pulling the rug out from underneath you.

The AI engine quickly becomes part of the social commons, in the way all innovations do — but this time at a much faster pace than we’re used to. This is because of their inherent asymmetry in cost to make vs implement and replicate, and also because the work product is itself a cognition accelerant, so it’s self-feeding.

This isn’t entirely unfamiliar. It took a huge amount of logical depth, of R&D, to make the human genome. But now, any group of people can make more people very easily and cheaply. The differentiating value those new people can and will contribute to the world economy isn’t based on how extraordinary it is to be a person — which actually is the amazing thing and 99% of the “work” — but rather the tiny difference in add-on that depends on their different contexts, which in the end makes all the difference between being a modern-day hunter-gatherer, no matter how wise or smart, and a genius entrepreneur building a global productivity explosion worth a trillion dollars.

[Darren Shepherd @ibuildthecloud]

git diff --staged | catgpt -m gpt-4 "Generate a git commit message"

https://t.co/8yZwgQvluG

[mkwld @mkwld]

“battle-tested model” - they are not from air. So really what he observed were mini-steps - those “linear approximations” around a small region

[Arnaud Benard @arnaudai]

When i started in AI, I thought that complex model and sophisticated math will lead to better results. I was so wrong.

The true key to success is using battle-tested model architectures and grinding on data quality.

[Greg Brockman @gdb]

A nice approach to library design is to first write the bones of a new abstraction standalone, and then flesh it out while refactoring existing code using it. One good sign an abstraction will be easy to extend later is if it’s easy to add existing functionality you’ve forgotten.

[Arnaud Benard @arnaudai]

if you think open-source models will beat gpt-4 this year, you're wrong.

i worked at top ai research labs (google ai) and built open-source libraries with > 5M monthly downloads.

gpt-4 is one year old and so far, no model matches it, here's why:

1. talent - openai recruited top ai engineers with salaries > $1M

2. data - massive proprietary chatgpt dataset and human annotated data

3. team structure - in-person centralized teams work better than decentralized open-source teams

4. model vs product - gpt-4 is not just a model, it's a product, you can't beat it with a better model

5. infrastructure - public cloud infra is terrible compared to what google/deepmind/openai has. hard for open-source teams to iterate at the same speed.

disagree?

[Chris Sev @chris__sev]

DALLE made me this cool website design

What’s the best way to get the assets for this so I can make it reality?

Mainly the background image/svg for the hero

[Yann LeCun @ylecun]

Genuine question about image generation:
If someone uses a generative AI tool to produce an image that is substantially similar to a copyrighted piece (a drawing, painting, movie screenshot, etc), who should be liable for copyright infringement?
Should it be:
A. the company producing the tool?
B. the (human) creator of the piece?
C. the person or entity posting/publishing the piece?
D. the communication platform through which the piece is distributed?

1. Copyright law protects against unauthorized exact or near-exact copies of a painting, photo, movie, or other visual piece.
2. When a person distributes a sufficiently similar copy of an art piece, it's a violation of copyright regardless of the tools and process used to produce it.
3. The liable person is the person *distributing* the piece, not the artist, and not makers of the tools.
4. Image generation systems are trained to generate images that are on the "manifold" of nice-looking images. Obviously, the training images are on this manifold.
5. Hence, sufficiently detailed prompt will produce images that are substantially similar to images from the training set. It is not at all surprising that a prompt like "The Batman movie, rooftop scene, screenshot, 4k..." will produce an image very similar to an actual screenshot from the movie.
6. Whether using publicly available-yet-copyrighted screenshots and other materials as part of the training set constitutes a violation of copyright is a separate question. As far as I can tell, this question is not legally settled in the US.

[Andrej Karpathy @karpathy]

There's too much happening right now, so here's just a bunch of links

GPT-4 + Medprompt -> SOTA MMLU
https://t.co/Jkp96izfec

Mixtral 8x7B @ MLX nice and clean
https://t.co/75StzY5AHe

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
https://t.co/gOCWjfY7ec

Phi-2 (2.7B), the smallest most impressive model
https://t.co/Fps8tI5QVi

LLM360: Towards Fully Transparent Open-Source LLMs
https://t.co/l6E16GfdIN

Honorable mentions
https://t.co/7GQqiCGHRH
https://t.co/3GZrYPp9KP
https://t.co/Su8iiDksMZ

[Andrej Karpathy @karpathy]

The hottest new programming language is English

[Andrew Yang🧢⬆️🇺🇸 @AndrewYang]

We’re going to need a bigger space . . .