Michael Fromm

Im a research scientist in NLP, currently working on the OpenGPT-X, EuroLingua-GPT, and TrustLLM to build open-source multilingual LLMs for Europe.

[Michael Fromm @effi288]

🌟 𝐓𝐞𝐮𝐤𝐞𝐧-7𝐁-𝐢𝐧𝐬𝐭𝐫𝐮𝐜𝐭 𝐢𝐬 𝐡𝐞𝐫𝐞!

The first LLM from OpenGPT-X is now available free of charge on Hugging Face.
For me, OpenGPT-X represents a significant milestone in Germany’s NLP research landscape, demonstrating how 𝐩𝐫𝐚𝐠𝐦𝐚𝐭𝐢𝐬𝐦 and 𝐬𝐜𝐢𝐞𝐧𝐭𝐢𝐟𝐢𝐜 𝐫𝐢𝐠𝐨𝐫 can come together to create impactful results.

🚀 𝐖𝐡𝐲 𝐭𝐡𝐢𝐬 𝐞𝐱𝐜𝐢𝐭𝐞𝐬 𝐦𝐞:

- International Benchmark: OpenGPT-X shows that Germany can deliver projects of international caliber. This is crucial for retaining the highly skilled professionals trained here.

- Beacon for Innovation: Projects like this inspire and highlight what’s possible. They act as magnets for talent in computer science.

👩‍💻 𝐌𝐲 𝐑𝐨𝐥𝐞 𝐢𝐧 𝐭𝐡𝐞 𝐏𝐫𝐨𝐣𝐞𝐜𝐭 (𝐚𝐧𝐝 𝐋𝐢𝐟𝐞!):
For about a year, we’ve been closely monitoring the training of Teuken-7B, overseeing progress daily, and adjusting processes based on new insights. This intensive but rewarding work has laid the foundation for future LLMs in EuroLingua.

At the same time, I’ve been raising two little humans at home—monitoring their progress, navigating surprises, and making adjustments as needed! Let’s just say, whether it’s AI models or toddlers, both require patience, consistency, and a good sense of humor. 😊

🏗️ 𝐈𝐧𝐯𝐞𝐬𝐭𝐢𝐧𝐠 𝐢𝐧 𝐄𝐮𝐫𝐨𝐩𝐞’𝐬 𝐀𝐈 𝐅𝐮𝐭𝐮𝐫𝐞:
Over time, we’ve built a robust, future-ready framework for Europe:

•Multilingual Evaluation: We created benchmarks and a leaderboard that covers 21 European languages to systematically assess AI models.
•Custom Training Framework: Starting from scratch, we developed “Modalities,” an open-source training framework that will power upcoming models like EuroLingua.
•Data Pipeline: We are building a European data pipeline capable of processing multiple petabytes of data following the latest insights in research, ensuring scalability for future demands.

💡 𝐖𝐡𝐲 𝐎𝐩𝐞𝐧 𝐒𝐨𝐮𝐫𝐜𝐞 𝐌𝐚𝐭𝐭𝐞𝐫𝐬 𝐟𝐨𝐫 𝐀𝐈:
Open Source removes barriers to learning, sharing, and improving systems. It provides the essential freedoms to:

1. Use the system for any purpose.
2. Study how it works.
3. Modify it as needed.
4. Share it freely.

By opening up Teuken-7B, we’re fostering collaboration, transparency, and innovation to ensure Europe’s digital sovereignty.

📣 𝐓𝐞𝐮𝐤𝐞𝐧-7𝐁-𝐢𝐧𝐬𝐭𝐫𝐮𝐜𝐭 𝐢𝐬 𝐣𝐮𝐬𝐭 𝐭𝐡𝐞 𝐛𝐞𝐠𝐢𝐧𝐧𝐢𝐧𝐠!
OpenGPT-X and this model represent the foundation for even more groundbreaking work.

👉 𝐄𝐱𝐩𝐥𝐨𝐫𝐞 𝐚𝐧𝐝 𝐠𝐞𝐭 𝐢𝐧𝐯𝐨𝐥𝐯𝐞𝐝:
- Model Card and Technical Information https://t.co/RXmiIIn8Du
- Leaderboards https://t.co/y7e7n0Zdvt
- OpenGPT-X Discord https://t.co/onLfmJ8DcH
- Modalities https://t.co/oxBjzYoaz7

A big thank you to the entire team, our partners, and the BMWK for supporting this project!

#OpenSource #AI #DigitalSovereignty #Teuken7B #EuroLingua #OpenGPTX

[TrustLLM @TrustLLM_]

How do we make LLMs more factually reliable?
Join our TrustLLM webinar on 14 April, 10–11 CET
👉 Register here: https://t.co/iVeNqaAXmR
📌 Please note that the webinar will be recorded.

[xAI @xai]

Introducing Grok 4, the world's most powerful AI model. Watch the livestream now: https://t.co/59iDX5s2ck

[Munich🥨NLP @MunichNlp]

🚀 𝙎𝙝𝙖𝙥𝙞𝙣𝙜 𝙩𝙝𝙚 𝙁𝙪𝙩𝙪𝙧𝙚 𝙤𝙛 𝙈𝙪𝙡𝙩𝙞𝙡𝙞𝙣𝙜𝙪𝙖𝙡 𝘼𝙄 𝙬𝙞𝙩𝙝 𝙏𝙚𝙪𝙠𝙚𝙣-7𝘽
Join us for a talk with Dr. Michael Fromm (@fraunhofer.bsky.social) on June 21st (3pm CEST) as he shares insights into the Teuken-7B project.
#AI #NLP #Teuken7B #Teuken #OpenGPTX

[Manuel Brack @MBrack_AIML]

🚀 New Preprint

We introduce JQL: a highly efficient, modular pipeline for multilingual pre-training data curation.

📄 𝐀𝐫𝐗𝐢𝐯: https://t.co/uqVll54EIP
🤗 𝐇𝐮𝐠𝐠𝐢𝐧𝐠 𝐅𝐚𝐜𝐞: https://t.co/wK38Wu8aRS
🔧 𝐆𝐢𝐭𝐇𝐮𝐛: https://t.co/wjav57dXsc

[TrustLLM @TrustLLM_]

TrustLLM secures 500K node hours on EuroHPC Leonardo BOOSTER for AI Act-compliant LLM training! 👉 Read more: https://t.co/gxrsv1D9R2 #TrustLLM #EuroHPC #trustworthyAI

[Charles University @CharlesUniPRG]

🤩 The OpenEuroLLM project, led by Charles University, was launched today at the Carolinum, bringing together 20 of Europe's top institutions, companies and computing centres to create powerful, open and multilingual Language Learning Models (LLMs) for European languages.

🌍 "The OpenEuroLLM project and the use of open language models will help companies to increase their global competitiveness while contributing to Europe's digital sovereignty,“ underlined Professor Jan Hajič, the project's lead coordinator from the Faculty of Mathematics and Physics at Charles University.

🤝 The project OpenEuroLLM is funded by the European Commission under the Digital Europe programme and co-financed by industry and providers in individual countries, including the Ministry of Education of the Czech Republic.

[Michael Fromm @effi288]

Happy to be a (small) part of it!

[OpenEuroLLM @OpenEuroLLM]

Kick-off successfully completed. Go OpenEuroLLM team!
https://t.co/XCaoRHehHc

[Michael Fromm @effi288]

@ClementDelangue There is already an European:

https://t.co/CvO72comwv

[Steve Jurvetson @FutureJurvetson]

The Moore's Law Update
NOTE: this is a semi-log graph, so a straight line is an exponential; each y-axis tick is 100x. This graph covers a 1,000,000,000,000,000,000,000x improvement in computation/$. Pause to let that sink in.

Humanity’s capacity to compute has compounded for as long as we can measure it, exogenous to the economy, and starting long before Intel co-founder Gordon Moore noticed a refraction of the longer-term trend in the belly of the fledgling semiconductor industry in 1965.

I have color coded it to show the transition among the integrated circuit architectures. You can see how the mantle of Moore's Law has transitioned most recently from the GPU (green dots) to the ASIC (yellow and orange dots), and the NVIDIA Hopper architecture itself is a transitionary species — from GPU to ASIC, with 8-bit performance optimized for AI models, the majority of new compute cycles.

There are thousands of invisible dots below the line, the frontier of humanity's capacity to compute (e.g., everything from Intel in the past 15 years). The computational frontier has shifted across many technology substrates over the past 128 years. Intel ceded leadership to NVIDIA 15 years ago, and further handoffs are inevitable.

Why the transition within the integrated circuit era? Intel lost to NVIDIA for neural networks because the fine-grained parallel compute architecture of a GPU maps better to the needs of deep learning. There is a poetic beauty to the computational similarity of a processor optimized for graphics processing and the computational needs of a sensory cortex, as commonly seen in the neural networks of 2014. A custom ASIC chip optimized for neural networks extends that trend to its inevitable future in the digital domain. Further advances are possible with analog in-memory compute, an even closer biomimicry of the human cortex. The best business planning assumption is that Moore’s Law, as depicted here, will continue for the next 20 years as it has for the past 128. (Note: the top right dot for Mythic is a prediction for 2026 showing the effect of a simple process shrink from an ancient 40nm process node)

----
For those unfamiliar with this chart, here is a more detailed description:

Moore's Law is both a prediction and an abstraction. It is commonly reported as a doubling of transistor density every 18 months. But this is not something the co-founder of Intel, Gordon Moore, has ever said. It is a nice blending of his two predictions; in 1965, he predicted an annual doubling of transistor counts in the most cost effective chip and revised it in 1975 to every 24 months. With a little hand waving, most reports attribute 18 months to Moore’s Law, but there is quite a bit of variability. The popular perception of Moore’s Law is that computer chips are compounding in their complexity at near constant per unit cost. This is one of the many abstractions of Moore’s Law, and it relates to the compounding of transistor density in two dimensions. Others relate to speed (the signals have less distance to travel) and computational power (speed x density).

Unless you work for a chip company and focus on fab-yield optimization, you do not care about transistor counts. Integrated circuit customers do not buy transistors. Consumers of technology purchase computational speed and data storage density. When recast in these terms, Moore’s Law is no longer a transistor-centric metric, and this abstraction allows for longer-term analysis.

What Moore observed in the belly of the early IC industry was a derivative metric, a refracted signal, from a longer-term trend, a trend that begs various philosophical questions and predicts mind-bending AI futures.

In the modern era of accelerating change in the tech industry, it is hard to find even five-year trends with any predictive value, let alone trends that span the centuries.

I would go further and assert that this is the most important graph ever conceived. A large and growing set of industries depends on continued exponential cost declines in computational power and storage density. Moore’s Law drives electronics, communications and computers and has become a primary driver in drug discovery, biotech and bioinformatics, medical imaging and diagnostics. As Moore’s Law crosses critical thresholds, a formerly lab science of trial and error experimentation becomes a simulation science, and the pace of progress accelerates dramatically, creating opportunities for new entrants in new industries. Consider the autonomous software stack for Tesla and SpaceX and the impact that is having on the automotive and aerospace sectors.

Every industry on our planet is going to become an information business. Consider agriculture. If you ask a farmer in 20 years’ time about how they compete, it will depend on how they use information — from satellite imagery driving robotic field optimization to the code in their seeds. It will have nothing to do with workmanship or labor. That will eventually percolate through every industry as IT innervates the economy.

Non-linear shifts in the marketplace are also essential for entrepreneurship and meaningful change. Technology’s exponential pace of progress has been the primary juggernaut of perpetual market disruption, spawning wave after wave of opportunities for new companies. Without disruption, entrepreneurs would not exist.

Moore’s Law is not just exogenous to the economy; it is why we have economic growth and an accelerating pace of progress. At Future Ventures, we see that in the growing diversity and global impact of the entrepreneurial ideas that we see each year — from automobiles and aerospace to energy and chemicals.

We live in interesting times, at the cusp of the frontiers of the unknown and breathtaking advances. But, it should always feel that way, engendering a perpetual sense of future shock.

[Andrej Karpathy @karpathy]

The reality of the Turing test

[Michael Fromm @effi288]

@Elaina43114880 @FraunhoferIAIS @OpenGPTX Can you explain what OpenRouter is about?

[Mr. Roth @RaconteurR2D2]

The European research project OpenGPT-X has released the language model “Teuken-7B”, specifically designed to align with European values, data protection standards, and linguistic diversity. It was trained with the 24 official languages of the EU and consists of 7 billion parameters. The model is freely available on the Hugging Face platform and can also be used for commercial projects.

The project began in 2022 to create an alternative to the dominant AI models from the US (such as GPT-4, Llama, or Gemini). Its goal is to promote European independence in AI technology and support scientific as well as commercial applications.

OpenGPT-X is led by the Fraunhofer Institutes IAIS and IIS, with contributions from other research institutions and companies. The model aims to drive the development of transparent and adaptable AI solutions for science and industry.

[AshutoshShrivastava @ai_for_success]

We have new model Teuken-7B-instruct, Multilingual, Open Source, Made in Europe 🇪🇺

[Chubby♨️ @kimmonismus]

Teuken 7B Instruct: an European model released

Finally some good news from Europe. The Frauenhofer Institute has trained its own 7b model and it can keep up with the “big players” such as Llama 3.1 8b.

This is so important for Europe's survival in the AI era. In this respect, I expressly welcome the fact that with Teuken 7B Instruct, a European model is finally being released that can at least keep up in the SLM league.

[Michael Fromm @effi288]

@LifeIsGr8M8 @kimmonismus 7 billion parameters in the neural net.

[Michael Fromm @effi288]

@FraunhoferIAIS @OpenGPTX I'm one of the developers, happy about hearing feedback or answering questions!

[Michael Fromm @effi288]

@RaconteurR2D2 Thanks for sharing!
I'm one of the developers, happy about hearing feedback or answering questions!

[Michael Fromm @effi288]

@lordofborg Thanks for sharing!
I'm one of the developers, happy about hearing feedback or answering questions!

[Michael Fromm @effi288]

@kimmonismus Thanks for sharing!
I'm one of the developers, happy about hearing feedback or answering questions!

[Michael Fromm @effi288]

@Oli82817545 @kimmonismus Mistral was not trained in all 24 official EU languages.
I would try the model first before judging