Frank Dondelinger

Practicing machine learning by day, perfecting my baking skills by night. Everything else fits in between.

[Frank Dondelinger @don_frank]

Steven Scott, 2008-2018 https://t.co/FuNti5Vje1

[signüll @signulll]

i have never come across anyone who worked at google from the beginning to ~2012 being anything other than a great human being.

google trained people to be non assholes cuz the founders were very much nice dudes & really cared about everyone that worked there.

e.g. they literally handed envelopes of pure cash during christmas to employees.

[Frank Dondelinger @don_frank]

@algobaker @dalloliogm Agree, this is just the same thing we saw many times in the DREAM competitions and on Kaggle.

[Frank Dondelinger @don_frank]

No value judgment in that by the way, there is a lot of value in thinking deeply and for a long time to understand e.g. sociological phenomena. It just won't lead to high publication output without an external incentive.

[Frank Dondelinger @don_frank]

CS is hardly lab based, unless you're building robots... My theory about this is that this is a difference between fields where the emphasis is on theories (which you can refine indefinitely) vs experiments (which have to finish at some point).

[Florian Ederer @florianederer]

Research output after tenure drops off a cliff for business, economics, sociology, and other non-lab fields.

But it remains high post-tenure in lab-based fields such as chemistry, physics, computer science, and engineering.

[Frank Dondelinger @don_frank]

@JKSteinberger TIL there were two Austrians whose names start with H who tried to conquer the world -- and one may have succeeded.

[Frank Dondelinger @don_frank]

This seems to go against academic principles of freedom of ideas - - would like to hear ETHs justification.

[Daniel Paleka @dpaleka]

It has not been reported much, but I believe ETH Zurich has, as of last week, banned new Master and PhD students who attended a long list of universities in China, Russia, and Iran. 🧵

[Daniel Paleka @dpaleka]

It has not been reported much, but I believe ETH Zurich has, as of last week, banned new Master and PhD students who attended a long list of universities in China, Russia, and Iran. 🧵

[Frank Dondelinger @don_frank]

I wonder if a 3D-aware encoding (whether Rope or another approach) in a sequence to expression model could use information from HiC, since we generally don't know the full 3D structure of DNA, nor is it static...

[François Chollet @fchollet]

A common misconception about Transformers is to believe that they're a sequence-processing architecture. They're not.

They're a *set-processing* architecture. Transformers are 100% order-agnostic (which was the big innovation compared to RNNs, back in late 2016 -- you compute the full matrix of pairwise token interactions instead of processing one token at a time).

The way you add order awareness in a Transformer is at the *feature* level. You literally add to your token embeddings a position embedding / encoding that corresponds to its place in a sequence. The architecture itself just treats the input tokens as a set.

There's nothing that prevents you from applying the same trick to 2D grid of tokens or a 3D cube of tokens instead! Instead of adding an embedding for (i * N + j), the position of (i, j) in a flattened grid, just embed i and j and concat the result.

Now you have a 2D position embedding that makes your Transformer natively process grids of tokens! This can be useful for ARC-AGI, since ARC-AGI grids are of course 2D grids of discrete tokens, not sequences.

[Frank Dondelinger @don_frank]

In fact one of the main achievements of LLMs should be too teach us that much of what we admire in successful people (being articulate, formulating convincing arguments, accessing knowledge in their head quickly) is pattern matching rather than reasoning.

[Frank Dondelinger @don_frank]

This also applies to humans by the way...

[Yann LeCun @ylecun]

Worth repeating:
Do not confuse retrieval with reasoning.
Do not confuse rote learning with understanding.
Do not confuse accumulated knowledge with intelligence.

[Frank Dondelinger @don_frank]

I won't do as good a job as @NandoDF, but thanks to the @NobelPrize for giving me a decent chance at explaining both the Chemistry and Physics Prize. Computational biologist or polymath, you decide...

[Nando de Freitas @NandoDF]

For the first time in my life I can explain what the physics @NobelPrize is about!

In fact, if you’d like to learn what is a Hopfield net and how it relates to NP hard satisfiability, Boltzmann machines, autoencoders, score matching, Maxwell demons, maximum likelihood, generative AI, quantum computing, unsupervised learning and neural networks, see these slides and video lectures from a course I taught at @ipam_ucla 2012

https://t.co/4kcR14kSuz

https://t.co/PueyqN9lsn

https://t.co/L523CfjNcg

https://t.co/iURLQM8HRb

[Nando de Freitas @NandoDF]

For the first time in my life I can explain what the physics @NobelPrize is about!

In fact, if you’d like to learn what is a Hopfield net and how it relates to NP hard satisfiability, Boltzmann machines, autoencoders, score matching, Maxwell demons, maximum likelihood, generative AI, quantum computing, unsupervised learning and neural networks, see these slides and video lectures from a course I taught at @ipam_ucla 2012

https://t.co/4kcR14kSuz

https://t.co/PueyqN9lsn

https://t.co/L523CfjNcg

https://t.co/iURLQM8HRb

[Frank Dondelinger @don_frank]

I would have used TikTok instead of Tinder as the example, but otherwise pretty spot on. We all used to be somewhere between Wall-E and C3PO for a definition of AI, now the best you can hope for is "not Magic 8 Ball".

[Richard McElreath 🐈‍⬛ @rlmcelreath]

I told a colleague that logistic regression is AI and they got mad at me, so I made a chart. Find yourself. I am "Tinder is AI".

[Richard McElreath 🐈‍⬛ @rlmcelreath]

I told a colleague that logistic regression is AI and they got mad at me, so I made a chart. Find yourself. I am "Tinder is AI".

[Frank Dondelinger @don_frank]

@anshulkundaje Agreed - - this is part of my long-running argument that what is holding back deep learning in biology is at least in part the lack of custom architectures that address biological processes.

[Anshul Kundaje @anshulkundaje]

Nice work. The issue with current seq-> expression models in terms of their very poor performance on variant effect prediction is not that they need to see more variation data. The problem is they don't learn long range effects of enhancers effectively. 1/

[Frank Dondelinger @don_frank]

Great place to start a group!

[Paul DW Kirk @pauldwkirk]

We are now hiring into our new Biostatistical Machine Learning theme @MRC_BSU ! We are looking for: (1) a Junior Group Leader in AI/ML for Biomedicine (w/ startup package including an RA) who will help shape the theme; and (2) an RA to join my group. Details below. Please RT :)

[Frank Dondelinger @don_frank]

@m_elantkowski @XRAIGlass Disappointed they didn't interview you for the video 😉

[Frank Dondelinger @don_frank]

We have an open position for a Data Science Innovation Fellow in my group at Novartis Biomedical Research in Oncology! If you are interested in doing AI research with direct impact on oncology drug discovery, please see below. Feel free to reply or DM me. https://t.co/U6Uot3Fzuj

[Laura Luebbert, PhD @NeuroLuebbert]

Lior and I wrote a blog post about what it was like for me to find occurrences of duplicated (and seemingly manipulated) data as a first-year graduate student at @Caltech. I am speaking openly because I believe that we, as the scientific community, can do better.