Alex Shtoff

Ph.D. Principal Scientist @ TII.
Ex @YahooResearch.
I do machine learning ∩ numerical methods ∩ SW dev.
Author of

[Alex Shtoff @AlexShtf]

New post in my "Eigenvalues as models" series.

This one asks a practical question: can eigenvalue-based models be made much cheaper to train and evaluate without collapsing into something too simple to be interesting?

Dense matrices are expressive but expensive. Fully diagonal ones are cheap but too restrictive. In this post I explore a middle ground that turned out to be much more useful than I expected.

It is probably the most implementation-focused entry in the series so far: structured matrices, PyTorch/SciPy plumbing, and experiments.

If you care about spectral methods, differentiable numerical linear algebra, or unusual tabular model classes, this is post and the entire series is for you:

https://t.co/8JdgeEG5R9

[Alex Shtoff @AlexShtf]

What's the story behind this?

[Alex Shtoff @AlexShtf]

@adamnemecek1 Yes. Let's do ADMM per layer and remove autidiff altogether.

[Alex Shtoff @AlexShtf]

@DeepDishEnjoyer "Works" = converges to a point where the model generalizes.

[Alex Shtoff @AlexShtf]

@NVIDIAHealth There are unconstrained function approximation techniques that are **not** splines and are well-known in the numerical analysis community, such as orthogonal rational functions, and perhaps others. Why "hack" splines, if you can use natively unbounded approximators?

[Alex Shtoff @AlexShtf]

@piersmorgan @realDonaldTrump His duty is to serve the citizens of Israel, not the president of the US.

[Alex Shtoff @AlexShtf]

@omerdank @meiretingr בכניעה של האוייב.

[Alex Shtoff @AlexShtf]

@FeedTechILUncen כי בארץ אין תרבות של התאגדות "בריאה". יש מדינות בהן איגוד עובדים עוזר לעובד מפוטר עם הכשרות וחיפוש עבודה בעזרת רשת קשרים, ויש מדינות בהן איגודי עובדים הורסים כל חלקה טובה.

ישראל נמנית עם הסוג השני.

[Alex Shtoff @AlexShtf]

I want to present a counter-point. The Linux kernel, Firefox, and many other great innovative open-source projects are developed in a distributed WFH style. Their success is unprecedented, and their software is deployed in unimaginable scale for-profit businesses can only dream of.

This is also not for everybody, but it shows you that different people are built differently. There are people who flourish in a WFH environment that see each other once a year in a community event, and there are those who flourish in an in-person environment.

It's not about what is better, but what is the kind of people you currently have in your company - those you hire should flourish under similar conditions to those you already have.

[Alex Shtoff @AlexShtf]

@eiranoirx Zed did it better.

[Alex Shtoff @AlexShtf]

@_Opi_p @FeedTechILUncen חלק מהשוק החופשי היא החלפת מסרים בין השחקנים בשוק. בין השאר הפוסט הזה, והמסרים שעוברים בתגובות. *לשכנע שחקנים אחרים בשוק הוא חלק אינטגרלי מהחופש בשוק*

כל מי שחושב שהשוק החופשי מתחיל ונגמר ב"מתאים לך - קח, לא מתאים - עבור הלאה" ח0לא מבין מה זה שוק חופשי.

[Alex Shtoff @AlexShtf]

@BarhounNabil Certainly isn't measured in Muqawama and Sumud. If you are in need of muqawama and sumud, you've already lost.

[Alex Shtoff @AlexShtf]

@argofowl I'm still waiting for the Linux app.

[Alex Shtoff @AlexShtf]

@hhh_ggg78355 @ChShersh No

[Alex Shtoff @AlexShtf]

@genzforreadonly @ChShersh This was a standard rage bait :)

[Alex Shtoff @AlexShtf]

@ChShersh :)

[Alex Shtoff @AlexShtf]

"Come work with us. You'll have cookies, but your kids won't have a parent."

[Alex Shtoff @AlexShtf]

@FeedTechILUncen בלינקד-אין לא עבר בשקט.

[Alex Shtoff @AlexShtf]

@haha_girrrl Codex is just smarter.

[Alex Shtoff @AlexShtf]

@AcevedoHector @MashaGabriel @elmundoes These are the terms of the ceasefire. Dont like them? Dont agree to a ceasefire and dont complain the other side is violating.