Mirko Torrisi

Self-supervised southern hacking OMICS via shallow and deep learning @openchip_. Previously @bmsnews @UCDdublin @UCIrvine #Bioinformatics #AI #sustainability

[SemiAnalysis @SemiAnalysis_]

JUNE 1, 2001
🚨MICROSOFT CEO: OPEN-SOURCE OPERATING SYSTEMS ARE DANGEROUS

$MSFT CEO Bill Gates told lawmakers that open-source operating systems such as Linux are "going down a very dangerous path." Transcript below ⬇️

"""
The scaling of open source operating systems, I think it's going down a very dangerous path. And again, if the path continues, I think we could get to a very dangerous place.

I think it's worth saying some things on Linux that are clear to all the experts, but I want to make sure is understood by this committee, which is when you control the operating system and you're shipping it, you have the ability to monitor its usage. It might be misused at one point, but then you can push an update. You can revoke a user's license. You can change what the system is willing to run.

When an operating system is released in an uncontrolled manner, by some guy compiling his own kernel in his basement, there's no ability to do that. It's entirely out of your hands. And so I think that should be attended to carefully.

There may be ways to release software open source so that it's harder to circumvent the licensing, but that's a much harder problem, and we should confront the advocates of this with that problem and challenge them to solve it.

Finally, I'd say open source is a little bit of a misnomer here, right? Open source normally refers to smaller developers who are iterating quickly, and I think that's a good thing. But here we're talking about something a little bit different, which is a more uncontrolled release of larger systems by, again, to your point, Senator Hawley, like much larger entities that pay tens or even hundreds of millions of dollars to develop them. I think we should think of that in a little bit of a different category, and their obligations in a little bit of a different category.
"""

[Mirko Torrisi @Mirko_Torrisi]

@rasbt Looks too good to be true, and the original post is from a suspended account..?!

[Mirko Torrisi @Mirko_Torrisi]

@alex_prompter Would be interesting to see a similar evaluation for agent harnesses optimized for scientific discovery instead of prompting base models. Testing codex isn't the same thing as testing ChatGPT for coding.

[Alex Rives @alexrives]

Today we're announcing ESMFold2, an open scientific engine to power prediction, design, and discovery across protein biology.

The new model delivers state of the art performance on protein interactions, especially antibodies, a critical modality for therapeutics.

We have designed and validated miniprotein binders and single chain antibodies across five therapeutic targets that are important in cancer and immunology. We are seeing very high success rates, and affinities at levels consistent with therapeutic activity.

We’re also releasing an atlas of 6.8 billion proteins, and 1.1 billion predicted structures.

ESMFold2 is built on a state of the art language model that has been trained on billions of protein sequences.

A world model of protein biology emerges through language modeling.

We’ve used the techniques of mechanistic interpretability developed to understand large language models to understand the concepts ESM uses to represent proteins.

The model’s representation space has a compositional organization of features across scales, levels of complexity, and abstraction, that reflects and mirrors the understanding of protein biology developed through a century of empirical science.

This understanding emerges without prior knowledge, just from language modeling of protein sequences.

Language models are becoming a powerful substrate to understand and program biology.

The design of protein interactions is one of the most fundamental problems in biophysics, and has critical implications for the discovery of new medicines. A simple gradient based search with the model was able to discover high-affinity protein binders.

I'm excited by the potential this has to accelerate basic science and the understanding of proteins. And especially for the new avenues it opens up for therapeutic design and medicine.

[Max Welling @wellingmax]

Today @cusp_ai and @KemiraGroup announce a milestone in AI-driven materials discovery.

We have used generative AI to design new materials targeting PFAS removal from drinking and process water at trace concentrations.

[Eric Topol @EricTopol]

The largest randomized trial of medical A.I.
—Over 100,000 women in Sweden
—radiologist + AI vs 2 radiologists, in follow-up
—AI added led to 29% more cancer detected, 44% reduced workload, and
—Less cancer dx in subsequent 2 years, and, when found, less aggressive
https://t.co/e1hY3F0cGo

[Alexia Jolicoeur-Martineau @jm_alexia]

I wanna grab your attention one moment to thank @k_schuerholt and the team at @arcprize for allowing me to participate in this historic competition normally only reserved for the giants.

Thank you @arcprize !!

[Christian Dallago @sacdallago]

Are you passionate about leading collaborative, fast moving, applied bioinformatics research projects that help the entire community move forward?

Apply to work in my team at NVIDIA: https://t.co/9IKbz5eN08

[Nando de Freitas @NandoDF]

The only bitter lesson is that LLMs have succeeded beyond any expert expectations.

Underpinning LLMs is the idea of scaling, which is too often misunderstood as more parameters. Scaling is about using massive compute effectively to maximise the throughput of data ingestion into the learning process to obtain more capable models.

We are still far from hitting the limits in this. We are still compute hungry because there is a ton more we could achieve if only we had more compute, from experimental ablations to data acquisition and curation.

Scaling is largely about data and evals. The models are now trained on almost all the web and equally large (but growing) self generated synthetic data. sifting through such vasts quantities of data (the whole of the human creation) requires formidable engineering and intelligent ideas. This is what differentiates most models.

AI is finally in the hands of billions of users, and with it come billions of tasks - every reasonable user need. This scaling in tasks and evaluations is many orders of magnitude larger than pre-LLMs.

Having the right architecture matters, but we know several alternatives could all work well, eg replacing attention in Transformers for RNNs and interleaving such layers with local layers. What matters is fine ablations to maximise hardware usage. This is the realm of sophisticated high-precision engineering. It encompasses semiconductor design, datacenter design, distributed systems, MFU, etc. There is fascinating work on flow matching, JEPA, sparser MoEs, etc, that is all consistent with scaling.

I’m terrible at predictions, but in this we have stayed the course. There’s been pleasant surprises like the effectiveness of reasoning, which while allowing for less parameters, still demands even more compute.

Sparser multimodal MoEs also will allow for better continual learning. This is an old idea, eg https://t.co/ZjqVwwoy5L, which is finally being done at scale.

Successful scaling is mostly about organising people into effective teams for research, development and production. They have to be teams of happy and ambitious people who put the team first. Yes, tech VCs and CEOs: work life balance matters to achieve prologued success, something I think @demishassabis did really well at @GoogleDeepMind and which I promote at @MicrosoftAI.

Bitter lesson: it really is all about scaling and hard work by thousands of amazing people. Hardly bitter, but hopeful and inspiring.

[Sergey Ovchinnikov @sokrypton]

CASP is getting cut by NIH... 😢

(Anyone with extra funds wanna help support perhaps the most important competition of the century?)
https://t.co/mOuROZYJLh

[Mirko Torrisi @Mirko_Torrisi]

@MarcoEffe74 @grok @espressonline No, dimostra che anche grok puo' essere sviato da ragionamenti fuorvianti di un decelebrato.

[Mirko Torrisi @Mirko_Torrisi]

@MarcoEffe74 @espressonline @grok verifica la correttezza di questi 3 ragionamenti rispetto al post iniziale

[Thomas Wolf @Thom_Wolf]

Finally took time to go over Dario's essay on DeepSeek and export control and to be honest it was quite painful to read. And I say this as a great admirer of Anthropic and big user of Claude*

The first half of the essay reads like a lengthy attempt to justify that closed-source models are still significantly ahead of DeepSeek. However, it mostly refers to internal unpublished evals which limit the credit you can give it, and statements like « DeepSeek-V3 is close to SOTA models and stronger on some very narrow tasks » transforming in a general conclusion « DeepSeek-V3 is actually worse than those US frontier models — let’s say by ~2x on the scaling curve » left me generally doubtful. The same applies to the takeaway that all discoveries and efficiency improvements of DeepSeek have been discovered long ago by closed-models companies, this statement mostly resulting from a comparison of DeepSeek openly published $6M training numbers with some vague « few $10M » on Anthropic side without providing much more details. I have no doubts the Anthropic team is extremely talented and I’ve regularly shared how impressed I am with Sonnet 3.5 but this longwinded comparison of open research with vague closed research and undisclosed evals has left me less convinced of their lead than I was before I reading it.

Even more frustrating was the second half of the essay which dive into the US-China race scenario and totally misses the point that the DeepSeek model is open-weights, and largely open-knowledge due to its detailed tech report (and feel free to follow Hugging Face’s open-r1 reproduction project for the remaining non-public part: the synthetic dataset). If both DeepSeek and Anthropic models had been closed source, yes the arm-race interpretation could have make sense but having one of the model freely widely available for download and with detailed scientific report renders the whole « close-source arm-race competition » argument artificial and unconvincing in my opinion.

Here is the thing: open-source knows no border. Both in its usage and its creation.

Every company in the world, be it in Europe, Africa, South-America or the USA can now directly download and use DeepSeek without sending data to a specific country (China for instance) or depending on a specific company or server for running the core part of its technology.

And just like most open-source library in the world are typically built by contributors from all over the world, we’ve already seen several hundred derivative models on the Hugging Face hub created everywhere in the world by teams adapting the original model to their specific use cases and explorations.

What's more, with the open-r1 reproduction and the DeepSeek paper, the coming months will clearly see many open-source reasoning models being released by teams from all over the world. Just today, two other teams, AllenAI in Seattle and Mistral in Paris both independently released open-source base models (Tülu and Small3) which are already challenging the new state-of-the-art (with AllenAI indicating that its Tülu model surpasses the performance of DeepSeek-V3).

And the scope is even much broader than this geographical aspect. Here is the thing we don’t talk nearly enough about: open-source will be more and more essential for our… safety!

As AI becomes central to our lives, resiliency will increasingly become a very important element of this technology. Today we’re dependent on internet access for almost everything. Without access to the internet, we lose all our social media/news feeds, can’t order a taxi, book a restaurant, or reach someone on WhatsApp. Now imagine an alternate world to ours where all the data transiting through the internet would have to go through a single company’s data centers. The day this company suffers a single outage, the whole world would basically stop spinning (picture the recent CrowdStrike outage magnified a millionfold).

Soon, as AI assistants and AI technology permeate our whole life to simplify many of our online and offline tasks, we (and companies using AI) will start to depend more on more on this technology for our daily activities and we will similarly start to find annoying or even painful any downtime in these AI assistants from outages.

The most optimal way to avoid future downtime situations will be to build resilience deep in our technological chain.

Open-source has many advantages like shared training costs, tunability, control, ownership, privacy but one of its most fundamental virtue in the long term –as AI becomes deeply embedded in our world– will likely be its strong resilience. It is one of the most straightforward and cost-effective ways to easily distribute compute across many independent providers and to even run models locally and on device with minimal complexity.

More than national prides and competitions, I think it’s time to start thinking globally about the challenges and social changes that AI will bring everywhere in the world. And open-source technology is likely our most important asset for safely transitioning to a resilient digital future where AI is integrated into all aspects of society.

*Claude is my default LLM for complex coding. I also love its character with hesitations and pondering, like a prelude to the chain-of-thoughts of more recent reasoning models like DeepSeek generations.

[Dario Amodei @DarioAmodei]

My thoughts on China, export controls and two possible futures https://t.co/LZAANEcZJW

[DeepSeek @deepseek_ai]

🚀 DeepSeek-R1 is here!

⚡ Performance on par with OpenAI-o1
📖 Fully open-source model & technical report
🏆 MIT licensed: Distill & commercialize freely!

🌐 Website & API are live now! Try DeepThink at https://t.co/v1TFy7LHNy today!

🐋 1/n

[Mirko Torrisi @Mirko_Torrisi]

For more details, check out the job posting or feel free to reach out to us with any questions. We look forward to hearing from potential candidates!

https://t.co/ZEbFxLxPAS

[Mirko Torrisi @Mirko_Torrisi]

Are you a PhD student interested in ML & Drug Discovery problems in a industry setting? Apply by Dec 10th to join the Visiting Scientists program at BMS CITRE in Seville, Spain!
https://t.co/ZEbFxLxPAS

#MachineLearning #DrugDiscovery #visiting #internship

[Mirko Torrisi @Mirko_Torrisi]

- Integrate drug screening data with RNAseq and scRNAseq data, apply dimensionality reduction, perform differential expression analysis, use clustering and classification methods, and conduct pathway and network analysis to understand biological mechanisms