David Zhang

ML research at Qualcomm AI

[Derek Lim @dereklim_lzh]

Our new workshop at ICLR 2025: Weight Space Learning: https://t.co/1CiTQXl3G1

Weights are data.

We can learn from weights.

Learning can outperform human-designed methods for optimization, interpretability, model merging, and more.

[Eliahu Horwitz @EliahuHorwitz]

📢Thrilled to announce our ICLR 2025 @iclr_conf workshop on Weight Space Learning, exploring model weights as a new data modality! 📢

Stay tuned for submission instructions and deadlines.

https://t.co/DUag8P01JS

#weightspace #weightspacelearning #ICLR2025 #iclr

[Gabriel Synnaeve @syhw]

Want to do research in code generation with LLMs and wonky deep learning from the 90s? We're recruiting one Master student (M2) intern for 2025 at FAIR Paris in my team https://t.co/5uWO1BxZrE

[Guan-Horng Liu @guanhorng_liu]

📢Interested in #interning at #FAIR NY? Excited to share that I have one internship position available for #Summer2025 🙂!

Looking for PhD interested in flow/diffusion models, optimal transport/control for structural problems.

🙌Send me your CV, website & GScholar by #Oct16th!

[Taco Cohen @TacoCohen]

🚨 Attention aspiring PhD students: Meta / FAIR is looking for candidates for a joint academic/industry PhD! 🚨

Among others, the CodeGen team is looking for candidates to work on world models for code, discrete search & continuous optimization methods for long-term planning, large-scale RL, and related topics.

A joint industry / academic PhD gives you the best of both worlds: academic freedom, strong support for open science & open source, a gigantic compute cluster, and amazing colleagues working as a team. Oh, and FREE FOOD. Truly a great opportunity 😄

Ideal candidates should have strong engineering skills and a solid understanding of ML & RL foundations.

We want to move fast so apply ASAP at the link below!

[Boris Knyazev @BorisAKnyazev]

Optimization can be sped up by 50% using our NiNo model! It takes a history of parameter values and predicts future parameters leveraging "neural graphs".
Accelerating Training with Neuron Interaction and Nowcasting Networks: https://t.co/hd9GOXA3rH
code: https://t.co/hdQMTfxECd

[@levelsio @levelsio]

🇪🇺 eu/acc

A few weeks ago Mario Draghi asked my recommendations for his report that came out today about European competitiveness

I had a call with him and summarized my problems with doing business in the EU

I wrote this which is included in the report presented to the European Union today:

1. Minimum revenue cut offs for current and new regulation
Exempt small businesses with annual revenues below €10 million from complex regulations like VATMOSS, GDPR, the EU AI Act, and certain labor laws. This approach encourages innovation and growth by allowing startups to focus on product development and market validation without the heavy burden of regulatory compliance. Once these businesses surpass €10 million, they will have the resources to comply with regulations, ensuring that growth is not stifled.

2. Simplify starting a pan-EU business with an EU-wide Incorporation (Inc.) business form
Currently, starting and operating a business across the EU is complex due to 27 member states, each with its own company registration requirements. To streamline this process and make it easier for entrepreneurs to operate across Europe, there should be a single, standardized business entity that applies uniformly across all EU countries. I call this the European Inc.

3. Start an EU business fully online, no physical offices, notaries, lawyers etc
To continue, right now starting a business in most EU member states it’s complicated, very time and resource intensive, and often involves lawyers and notaries. Instead, it should be as simple as going online to a centralized EU website, where entrepreneurs can register their business and details in just a few clicks. The entire process should be streamlined and efficient, allowing businesses to start operating immediately.

The EU government taxes and bookkeeping of this business should also be fully online in an EU portal/dashboard.

4. 0% corporate tax for first 3 years of any new business
Countries like Singapore have successfully attracted new businesses from around the world by giving them a massive tax discount during the first 3 years of business. Because they know that’s the most difficult time of a business: figuring out what product it makes and if there’s a market for it. That takes pressure off startups and business founders that they can focus on creating a great product and innovating.

5. Change tax on stock options: don't tax when a stock option is exercised, but tax it when the stock is sold
The current tax policy in the EU taxes stock options at the time they are exercised, creating a significant financial burden on employees who have not yet realized any tangible financial gain. This approach stifles innovation, discourages entrepreneurship, and places the EU at a competitive disadvantage compared to other regions like the United States.

I propose a simple change: Tax stock options when the stock is sold, not when the option is exercised.

6. Don’t see tech or AI as an enemy, but as a burgeoning and essential industry
The most popular companies in tech are focused on AI right now for a reason. It’s the next frontier of computing. The European Union seems to consider AI the enemy. Any technology can be used for good or bad. By regulating it even before Europe has made much contributions (Europe has almost no tech companies leading in AI), it has stifled any potential innovation in AI from the start.

Apart from the regulation itself, the optics of it make the EU look bad on a global scale. Why would tech founders move to Europe to start a business if the EU is actively positioning itself as Anti-AI?

AI has gigantic potential to be used for good: think of the medical field for diagnosis of diseases, generally in programming (it helps programmers to create software faster/better), etc.

This goes further than AI. The same applies to tech in general. It seems the EU is on a crusade against technology while not being able to compete in it itself. It feels a case of sour grapes: if we can’t build great technology in EU, nobody is allowed to do so!

7. Teach tech/coding/AI topics in all schools and unis
It would help a lot if the EU has a focus on teaching AI and tech in schools and universities. Making the new generation competitive in this field instead. To secure the future prosperity of the European Union, we must prioritize education in technology, coding, and AI across all levels of schooling, from primary education to universities. This strategic focus is not just an educational reform—it’s a critical investment in the future competitiveness, innovation, and economic resilience of the EU.

[Markus Nagel @mnagel87]

Are you pursuing a PhD and are you interested in working on efficiency of LLMs/LVMs? Then join our model efficiency team in #QualcommAIResearch for an internship!

Apply below, we have openings for 2025 as well as autumn/winter 2024.
https://t.co/vNrUT9wzrB

[Damian Borth @damianborth]

🚀 Exciting News! 🚀

I am more than happy to share that I have officially begun my research sabbatical this week!

Over the coming months, I am fortunate to be working as @TUeEAISI Visiting Professor at @TUeindhoven collaborating with @joavanschoren on some exciting ideas around #AutoML and #WeightSpaceLearning.

I’m also thrilled to be joining the @uwcse at the @UW in Seattle as a Visiting Professor. At UW, I’ll be working with @timalthoff at the intersection of #WeightSpaceLearning and #LargeLanguageModels.

I’m looking forward to diving into different research directions, to explore new ideas, and to learn!

#Sabbatical #TUe #UWAllen #MachineLearning #AI #WeightSpaceLearning

[Damian Borth @damianborth]

Pleasant surprise of @icmlconf: There is a growing Weight Space Learning Community out there - it was great to meet you all:

Haggai Maron (@HaggaiMaron), Gal Chechik (@GalChechik), Konstantin Schürholt (@k_schuerholt), Eliahu Horwitz (@EliahuHorwitz), Derek Lim (@dereklim_lzh), David Zhang (@davwzha), Yan Zhang (@Cyanogenoid), Fabrizio Frasca (@ffabffrasca), Guy Bar Shalom, Giorgos Bouritsas (@gbouritsas), and Vincent Herrmann (@idivinci), who is unfortunately missing in this picture (he had to give his talk).

#weightspace #weightspacelearning #icml2024

[Natasha Butt @NatashaEve4]

Come see our poster #715 on CodeIt today at #ICML2024 13.30-15.00 Halle C. We approach ARC by self-improving LLMs with prioritized hindsight replay. @blazejmanczak @aukejw Corrado Rainone @davwzha @m_deff @TacoCohen

[David Zhang @davwzha]

@tydsh Very interesting work! Do you expect this to enable/improve general planning capabilities for a pre-trained LLM if done at larger scale (and if yes, at what conditions)? With general I mean planning in OOD domains (not mazes or sokoban) or for less formal problem specifications.

[Lewis Tunstall @_lewtun]

Our solution write-up for the 1st AIMO Progress Prize is now out ✍️!

https://t.co/q9NSmfZJwW

In it, we share technical details on:

♾️💻 The 2-stage MuMathCode recipe we used to train NuminaMath 7B TIR with iterative SFT

⚖️ Evals on MATH - 56.3% for Stage 1 & 68.2% for Stage 2 (a new SOTA for 7B models?)

🚀 Promising results from applying on-policy KTO, which unfortunately didn't make the final submission due to time constraints 😭

🧭 The Self-Consistency with Tool-Integrated Reasoning decoding algorithm (SC-TIR or "scooter" for short) we developed to boost performance and reduce variance

🐭 Quantising our models with GPTQ to make them run fast under some rather trying hardware constraints (T4s were not made for LLMs 😅)

🙈 The many, many, ideas we tried, but didn't work out (RL, model merging and more)

[Haggai Maron @HaggaiMaron]

Thanks, @KostasPenn, @CongyueD, and the team for inviting me. It's an honor to speak alongside this esteemed group of researchers! My talk will focus on our recent work on *Equivariant Weight Space Learning*: designing neural networks that can process other neural networks.

[Auke Wiggers @aukejw]

ARC is a tough reasoning benchmark where modern LLMs far underperform humans still. Great to see that there's serious additional backing!

Coincidentally, we just open-sourced CodeIt, our LLM-improvement approach for ARC: https://t.co/iQnsV5YFVj

[Hazel Doughty @doughty_hazel]

I'm hiring for a fully funded #PhD position in #ComputerVision on 'Detailed Video Understanding' at @LIACS @UniLeiden.

Apply before 22nd June. More info👇
https://t.co/QafgWY9co5

[Morning Brew ☕️ @MorningBrew]

Dating a model Dating a model
(2004) (2024)

[Mina Ghadimi @GhadimiAtigMina]

Don't miss our survey on "Hyperbolic Deep Learning in Computer Vision: A Survey"!🙌 #IJCV

[Natasha Butt @NatashaEve4]

Come checkout our poster on CodeIt today at the #ICLR2024 #AGIWorkshop 4pm Halle A7. We tackle @fchollet’s ARC by self-improving LLMs with prioritized hindsight replay.
@blazejmanczak @aukejw Corrado Rainone @davwzha @m_deff @TacoCohen

[Allan Zhou @AllanZhou17]

ilya knows that it's time for weight-space architectures to shine :>