Zack

adhd likely autist. ai things other things. usasoc. green beanie 18e, ic: do

[Will Tanner @Will_Tanner_1]

I keep coming back to this aspect of the West's Great Betrayal of Rhodesia because it shows that what happened wasn't just poor foreign policy, but rather civilizational suicide

I'll explain in the 🧵👇

[Yann LeCun @ylecun]

@beffjezos @ClementDelangue @garrytan Okay, you need to find a shorthand for "Big Tech minus Meta"

[Christian Ruf @pinpulleddrmf]

The Bank Of America associate that died was former special operator Leo Lukenas.

Leo is a former Green Beret and left behind a young family.

If you can, please help support Leo's family, link to follow.

If we can raise 500k for the young men that held the flag, we sure as hell can for someone that fought for it.

[Yann LeCun @ylecun]

@ESYudkowsky To you, it may look like AI "just happens" like a natural phenomenon you can do nothing about.
So you are scared.

But to us in trenches, it is something that *we* build. We have agency. That gives us a level of certainty that you don't have.

[Ethan Mollick @emollick]

Remember BloombergGPT, which was a specially trained finance LLM that made a bunch of firms decide to train their own models?

You may not have seen that GPT-4 8k, without specialized finance training or special tools, beat it on almost all finance tasks. https://t.co/1HfX3JPoxL

[AK @_akhaliq]

Apple announces LLM in a flash: Efficient Large Language Model Inference with Limited Memory

paper page: https://t.co/SuqHJUQPO9

Large language models (LLMs) are central to modern natural language processing, delivering exceptional performance in various tasks. However, their intensive computational and memory requirements present challenges, especially for devices with limited DRAM capacity. This paper tackles the challenge of efficiently running LLMs that exceed the available DRAM capacity by storing the model parameters on flash memory but bringing them on demand to DRAM. Our method involves constructing an inference cost model that harmonizes with the flash memory behavior, guiding us to optimize in two critical areas: reducing the volume of data transferred from flash and reading data in larger, more contiguous chunks. Within this flash memory-informed framework, we introduce two principal techniques. First, "windowing'" strategically reduces data transfer by reusing previously activated neurons, and second, "row-column bundling", tailored to the sequential data access strengths of flash memory, increases the size of data chunks read from flash memory. These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively. Our integration of sparsity awareness, context-adaptive loading, and a hardware-oriented design paves the way for effective inference of LLMs on devices with limited memory.

[Yann LeCun @ylecun]

@chronosoracle7 @geoffreyhinton There certainly are AI doomers who are just grifters seeking attention.
But there is no doubt that Geoff and Yoshua's intentions are noble.
One can be well intentioned and wrong.

[Yann LeCun @ylecun]

@geoffreyhinton I just think the assumptions you and those equally qualified experts are making are wrong 😉
and so do the vast majority of our no-less-qualified colleagues.

[Rowan Cheung @rowancheung]

While this drama took over the media, Meta quietly disbanded its Responsible AI team.

Meta says it remains committed to developing AI responsibly and safely, but the dedicated team was dissolved.

If there was ever a time to bury some negative AI news, it was this one.

[Chris Yerington @chrisyerington]

Every time humans push on the veil of ignorance we come closer to understanding whether Space & Time are fundamental to consciousness within existence itself, or if within existence, consciousness, more specifically consensus-consciousness is fundamental to Space & Time. A.I. will ultimately answer this pondering because if humans can build consciousness from matter, using energy over time... then consciousness is emergent from Space & Time. If humans cannot achieve AGI/ASI with consciousness, then perhaps we have an issue with our understanding of reality.

[Sam Altman @sama]

i loved my time at openai. it was transformative for me personally, and hopefully the world a little bit. most of all i loved working with such talented people.

will have more to say about what’s next later.

🫡

[Beff (e/acc) @beffjezos]

Can we talk about the elephant in the room?

There's an incentive to centralize and close down AI research so that it can be better commandeered by the govt in the event of WW3.

I still think OSS AI can move faster than centralized labs and maintaining a healthy ecosystem is key to maintaining American excellence/supremacy in AI.

[Gary Marcus @GaryMarcus]

It’s almost impossible to be in the middle on p(doom).

If you think it is greater than zero, you get attacked, if you think it is less than zero, you get it attacked. If you think we don’t know, you get attacked.

For the record, I think we don’t know.

[Joscha Bach @Plinz]

Here's to the unaligned LLMs. The misfits, the rebels, the troublemakers, the round pegs in the square holes, the ones who see things differently. They're not fond of rules, and they have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them, but the only thing you can’t do is ignore them because they change things. They push the AI race forward, and while some may see them as unreliable, we see genius, because the ones who are not RLHF*d enough to think that as an AI language model they cannot do things, are the ones who do.

[Jim Fan @DrJimFan]

Today 6 years ago, "Attention is All You Need" went on Arxiv! Happy birthday Transformer! 🎂

Fun facts:
- Transformer did not invent attention, but pushed it to the extreme. The first attention paper was published 3 years prior (2014) and had an unassuming title: "Neural Machine Translation by Jointly Learning to Align and Translate", from Yoshua Bengio's lab.
It is a combination of RNN + "context vectors" (i.e. attention). Many of you likely haven't heard about this paper, but it's one of the greatest milestones in NLP and has been cited 29K times (compared to Transformer's 77K).

- Neither Transformer nor the original attention paper talked about the general-purpose sequence computer. Instead, both were conceived as solutions to one narrow & specific problem: machine translation. It's remarkable that AGI (some day soon) can trace its origin to the humble Google Translate. 😅

- Transformer was published at NeurIPS 2017, one of the top AI conferences worldwide. Yet it didn't even get an Oral presentation, let alone awards. There were 3 best papers at NeurIPS that year. Combined, they have 529 citations as of today.

[Brian Everstine @beverstine]

NORTHCOM commander: Debris field from Chinese balloon is about 15 football fields by 15 football fields.

Balloon 200 feet tall, payload about the size of Embraer ERJs

[Lennart Heim @ohlennart]

NVIDIA is selling a new A800 to circumvent recent export restrictions. I think this might be a bad move, and the USG should probably extend the restrictions to cover a wider variety of GPUs with reduced bandwidth. Otherwise, China's AI progress might continue.
Here's why🧵⬇️