Marco Fedele

[Nicolas Fulghum @nicolasfulghum]

An economy based on electricity is a massive leap over the current fossil based economy

⚡️ In the UK, powering a car with electricity needs only a third of the energy input of a petrol car

🏭 Going electric also reduces the amount of fossil fuels needed by a factor of six!

[Marco Fedele @fedele_marco]

Clean energy vs fossil fuels investment worldwide

[Simon Evans @DrSimEvans]

Very eye-opening to see who's leading the way on clean energy investment – and who's leading on fossil fuels

China: Clean energy investment 3.7x fossil fuels
Europe: 5.5x more in clean energy
N America: Biggest investor in fossil fuels
Middle East: 5.4x more on fossil fuels

[Marco Fedele @fedele_marco]

Last 200 years in a picture

[Max Roser @MaxCRoser]

I've updated my Short History of Global Living Conditions.

here it is: https://t.co/6MEsVpjfTX

[Massimo @Rainmaker1973]

Valeriya Gogunskaya is an internationally known longboard-dancer: it's all about flow and speed.

[📹 valeriya_gogunskaya]

[Marco Fedele @fedele_marco]

AI avatar 🤖

[Marco Fedele @fedele_marco]

2023 - > The year of the LLM
2024 - > Copyright VS LLM legal war?

[Matthew Berman @MatthewBerman]

The New York Times just sued OpenAI and Microsoft for copyright infringement for using its content to train ChatGPT.

Some of the allegations are STUNNING!

I read all 69 pages of the lawsuit, here are the most 🔥 parts:

[Marco Fedele @fedele_marco]

Infinity percent of normality 😅

[Eliezer Yudkowsky ⏹️ @ESYudkowsky]

Me: Can you draw a very normal image?

ChatGPT: Here is a very normal image depicting a tranquil suburban street scene during the daytime.

Me: Not bad, but can you go more normal than that?

(cont.)

[Marco Fedele @fedele_marco]

Einstein the plumber

[Paul Halpern @phalpern]

In 1954, Albert Einstein was so upset by the Oppenheimer hearing and the McCarthy era attacks on the loyalty of scientists, that he said that if he were just starting out he'd become a plumber or peddler, instead of a physicist.

He soon got an invitation to join a plumbing firm.

[Marco Fedele @fedele_marco]

Parametric DALL•E 3 pictures!

[Alvaro Cintas @dr_cintas]

Yesterday, I showed you how DALL•E 3 can take your pictures to the next level.

Now, with just ONE prompt, you can create four images that range from normal to the weirdest level!

Here is the prompt:

“Another text to image model has a parameter called “weird”. The higher this parameter is, the weirder the image gets, ranging from 0 (normal) to 3000 (the weirdest).

I want you to recreate this feature. You will create four images, one at a weird level 0, one at 1000, one at 2000, and another one at 3000.

The image I want you to create is of: [desired image].”

You can also specify things you want to exclude in your prompt. For example, you can write that you don’t want your images to have a cartoonish vibe.

Below you can find more examples. I would love to see your creations too 👀

[Marco Fedele @fedele_marco]

La luna e le difficoltà radioattive

[Andrea Ferrero @a_a_f_t]

Come hanno fatto le missioni Apollo ad attraversare le fasce di Van Allen? Le fasce di Van Allen sono prodotte dall’interazione tra vento solare e campo magnetico terrestre e contengono particelle ad alta energia che possono essere pericolose per gli esseri umani.

(continua)

[Marco Fedele @fedele_marco]

Why China’s renewable energy exponential growth is not yet sufficient to decarbonize Its electricity

[Lauri Myllyvirta @laurimyllyvirta]

It's become a bit of a cliche to call it a paradox or a contradiction that China is building staggering amounts of both new clean power generation and new coal power plants.

It only is one if you fail to grasp the scale of China's electricity consumption growth, as most people inevitably do.

China today generates enough power from clean sources to power Germany SIX times over, up from two times a decade ago. In a few years, China's clean power generation will be equal to U.S.' total electricity consumption. China's power generation from wind&solar alone is about to hit three times Germany's total electricity consumption next year.

If China's power demand had stayed on 2009 level, the massive increase in CO2-free power generation would have made the grid 80% clean, instead of the current 33%. CO2 emissions from the power sector would have fallen by 75%.

What happened instead is that electricity demand doubled and clean power generation didn't keep up, while it did manage to increase its share from 20% to 33%. Predictably, the difference was delivered from coal, and CO2 emissions went up by 90%.

China now uses twice as much power as the U.S., after overtaking the country only in 2010. Put another way, China's additional power demand since 2010 is equal to the total consumption of the U.S.

60% of that electricity demand growth went to industry, mainly basic manufacturing like steel and other metals, cement, glass and chemicals, so it's really a function of China's extremely investment- and construction intensive growth model.

And no, it's not just about the size of China's population: per capita electricity consumption in China overtook Germany in 2022.

The good news is that China's clean energy growth is finally reaching the scale where it can cover all of the growth in power demand. This is happening even as power demand growth likely slows down with the economic slowdown.

These trends have been masked in the past year by the collapse in hydropower generation which is readily visible in the graph, but will likely become apparent in the year or two.

Most of the data for the post and the graph is from the amazing datasets of @EmberClimate.

[Marco Fedele @fedele_marco]

Impressive changes in the leading cause of death in the US.

[John LeFevre @JohnLeFevre]

The 18-44 age bucket is shocking….

Within two decades.

[Marco Fedele @fedele_marco]

The role of open source LLMs

[Bindu Reddy @bindureddy]

The Ongoing Case For Open Source LLMs
Custom LLMs, long context, and efficient inference

Some folks believe that training open-source LLMs is a losing battle and a complete waste of time.

They argue that the gap between closed models like GPT-4 and open models like Llama will widen and these open-source models may never catch up.

Yes, closed models like Google's Gemini or Open AI's Gobi promise to be way more powerful than GPT-4, so what hope does open source have?

To start with inference on GPT-4 is very expensive. These very large models may be performant but aren't cost-effective. At Abacus, we routinely use fine-tuned versions of LLama-2 and smaller models when we need to run 1M+ API calls a day for standard enterprise applications Q/A, summarization, and NLP at scale. GPT-4 would cost > $100K a day in these cases.

Instruct-tuned LLMs can match the performance of GPT-4 for a specific task. Instruction tuning is a technique that aims to improve the capabilities and controllability of LLMs. It involves further training these models on a dataset consisting of (instruction, output) pairs in a supervised manner. This bridges the gap between the next-word prediction objective of LLMs and the users' objective of having LLMs adhere to human instructions.

For example, we have instruct-tuned open-source models for tasks like Q/A, NER, and classification. These instruct-tuned models are better at generalizing the task to new data and can do so in a resource-efficient manner.

Another shortcoming of currently available closed models like GPT-4 is that they have relatively short context lengths. 8K tokens are default and this means that you can't pass it large documents and ask it to extract the results from there.

Luckily the open-source community has been busy solving practical problems like this. Earlier this week, the paper LongLoRA introduced an ultra-efficient fine-tuning approach to significantly extend the context windows of pre-trained LLMs.

LongLoRA adopts LLaMA2 7B from 4k context to 100k, or LLaMA2 70B to 32k on a single 8x A100 machine, and basic implementation only takes 2 lines of code.

Open-source LLMs have been the focus of the GPU-poor and constraining resources typically have magical effects - efficient, elegant, and simple innovations that solve the problem!

Open-source LLMs have emerged as cheap and efficient alternatives for enterprise AI use cases and will continue to play an important role in the space.

Some have argued that open-sourcing LLMs is dangerous and they may be misused by bad actors.

There is no historical precedent for this argument.

Traditionally, open-source technology has spurred innovation, transparency, and the creation of safe and robust systems. Linux, triumphed over Unix in the OS world, largely because it is open-source and has a huge developer community.

Open source promotes collaboration, community oversight, rapid iteration, and benchmarking all essential for responsible AI development. Open-source developer communities tend to be great at detecting and plugging vulnerabilities.

Disappointingly, big players like OpenAI (despite their name) and Google, haven't open-sourced a lot of their technology. Luckily for the open-source community, Meta has created accessible open-source LLMs. In spite of Meta open-sourcing the powerful 70B LLama-2., the doomsday scenarios outlined by the anti-open-source crowd haven't come true.

Finally, multimodal LLMs (MLLM) are around the corner and if the GPU-rich won't outsource a MLLM, we can always enhance an existing open-source LLM and convert it into a multi-modal model.

In summary, open-source LLMs play a role in the real-world application of AI and are crucial for the democratization of this technology, transparency, and AI alignment

[Marco Fedele @fedele_marco]

LLMs and Basic Logic

[Owain Evans @OwainEvans_UK]

Does a language model trained on “A is B” generalize to “B is A”?
E.g. When trained only on “George Washington was the first US president”, can models automatically answer “Who was the first US president?”
Our new paper shows they cannot!

[Marco Fedele @fedele_marco]

DALL•E 3 VS Midjourney

[Nick St. Pierre @nickfloats]

Side-by-sides of DALL•E 3 and Midjourney

Since Midjourney currently doesn't support text (that's coming in the v6 update) I removed the text call-outs from the prompt when running in MJ.

Here's 7 comparisons:

[Marco Fedele @fedele_marco]

Will we still ski in Europe?

[Carbon Brief @CarbonBrief]

Guest post: How climate change will hit snow levels across Europe’s ski resorts | @smlmrn @hugues_fr Raphaëlle Samacoïts David Neil Bird @JudithKoberl @FPrettenthaler

Read here: https://t.co/e47KnJUwIT

[Colin Mckerracher @colinmckerrache]

Great quote. There won't be enough synthetic fuels for everyone:

“Oliver can maybe have some for his 911, but we really need the volumes,” Spohr said, referring to Porsche AG CEO Oliver Blume’s push to get exemptions from combustion engine bans."

https://t.co/jIfd3gsEk2

[Beatrice Brignone @beabri]

Semplice e preciso. Dieci anni dopo siamo ancora qui a doverlo ribadire.

#scuola #margheritahack

[isabelle 🪐 @isabelleboemeke]

“Greenpeace is stuck in the past fighting clean, carbon-free nuclear energy while the world is literally burning. We need to be using all the tools available to address climate change and nuclear is one of them. I’m tired of having to fight my fellow environmentalists about this when we should be fighting fossil fuels together.”

[Marco Fedele @fedele_marco]

A thread on China 🇨🇳 solar mega boom 💥

[Science Is Strategic @scienceisstrat1]

China 🇨🇳 solar mega boom 💥

➡️ China installed 3x solar capacity in Q1 than in the same period in 2022

➡️ China is on track to add more solar in 2023 than the entire cumulative total in the US

(1/4)