Franz Nowak

PhD Student at @CSatETH, Natural Language Processing enthusiast

[Mario Giulianelli @ICML @glnmario]

What if the way we communicate is shaped by a hidden harmonic rhythm?

In our #ACL2025NLP paper, we explore the hypothesis that the components of the information contour of a document vary periodically, with periods that correspond to the boundaries of structural units.

[Franz Nowak @franz_nowak]

It was great to talk to Letitia on the AI Coffee Break channel about our research on the expressivity of LLMs. Thanks for having me on the podcast!

[AI Coffee Break with Letitia @AICoffeeBreak]

Are transformer LLMs Turing complete or not?
Spoiler: they are! @franz_nowak @AnejSvete @butoialexandra @RyanCotterell proved it for CoT LLMs in their paper.
We talk with Franz about the expressivity of Transformer encoders, decoders with CoT, and RNNs.
📺https://t.co/Cn6HIgn0Lp

[Franz Nowak @franz_nowak]

Check out our #NAACL2024 paper showing how RNN Language Models can simulate Probabilistic Finite State Automata! https://t.co/iAUomQqpvj

With @AnejSvete, Anisha Mohamed Sahabdeen, and @ryandcotterell

[Jaap Jumelet @JumeletJ]

✨How do LMs learn grammatical phenomena? Do they rely on co-occurrence patterns during training, or can they make grammatical judgments based on indirect evidence as well?

Through Filtered Corpus Training (FiCT) we are able to answer these questions at scale!

🧵

[Nadav Borenstein @NadavBorenstein]

I am excited to share our #ACL2024 main paper, "What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages".

📖 Link: https://t.co/wsI5asBQpQ

[Franz Nowak @franz_nowak]

@djpr Da ist wohl jemand bei der Search Engine Optimization für die CSU Website mit der Maus ausgerutscht 🤷‍♂️

[Nick Pinkston 🏭 🏗️ 🏙️ ⏭️ @NickPinkston]

This guide to picking STEM majors is too good.

Especially true about chemistry and feelings on safety haha

[Franz Nowak @franz_nowak]

He did pull it off!

[Ethan Gotlieb Wilcox @weGotlieb]

Join me today, at #EMNLP2023 11:00 in PS1, where I will pull off an impossible feat – presenting three posters at the same time! 🎩🌟🎩🌟

[Jaap Jumelet @JumeletJ]

How can we faithfully test interpretability methods in an environment that's reflective of natural language?

We approach this via large-scale PCFGs from which we sample corpora that provide full control over the source distribution!

I'll present this Findings work @BlackboxNLP!

[Franz Nowak @franz_nowak]

@_florianmai Yes, the main idea of having an adaptive number of steps between outputs is the same, if no hard upper limit is set. However, note that ACT interpolates outputs and hidden states by "halting probabilities" in a way that is at odds with simulating PTMs using our construction.

[Franz Nowak @franz_nowak]

🤖 Ever wondered what RNN-based language models are truly capable of? Check out our #EMNLP2023 paper which places formal bounds on their capabilities! With @AnejSvete, @leoduw, and @RyanCotterell (1/7)
https://t.co/CwBl9Klyhh

[Franz Nowak @franz_nowak]

📊 We show that a lower bound of the capabilities of RNN language models is that they can simulate multi-tape real-time Turing machines. With this, we delimit the capabilities of RNN language models in terms of theoretical models of computation. (7/7)

[Franz Nowak @franz_nowak]

🔬 Since, in practice, this is not how most language models operate, we treat this as an upper bound on their expressivity.  We also investigate the more realistic case of RNN language models having to output a word at each time step. (6/7)

[Franz Nowak @franz_nowak]

@Storch_i @Tagesspiegel Irgendwie kommt mir das Bild bekannt vor...

[AI Coffee Break with Letitia @AICoffeeBreak]

We summarized the #acl2023nlp Toronto conference for you with some poster recordings and author interviews! 👇
🎬 https://t.co/gLDbCt8IAA

Featuring @s_tworkowski @jasivan_s @kundan_official @ebugliarello @leoduw @_florianmai @franz_nowak @PaulDarm @MoritzPlenz and @JayAlammar 👏

[juri @nlopitz]

most recent #nlproc developments #MachineLearning