Our paper is now out in PNAS!๐ก
Are LLMs developing human-like concepts that are central to human cognition? If so, how are such concepts represented, organized, and related to behavior?
https://t.co/ZnLIevdN4j
1/N
Is your LM secretly an SAE?
Most circuit-finding interpretability methods use learned features rather than raw activations, based on the belief that neurons do not cleanly decompose computation. In our new work, we show MLP neurons actually do support sparse, faithful circuits!
An investigation into representations formed by LLMs during inference tasks concludes that human-like conceptual representations can emerge purely from language prediction without real-world grounding. In PNAS: https://t.co/7UiSllQ3VW
Finally, thanks to the reviewers and the PNAS editors for their constructive and thoughtful feedback, and for helping shape the final version of the paper into what it is today.
11/N = 11
Our paper is now out in PNAS!๐ก
Are LLMs developing human-like concepts that are central to human cognition? If so, how are such concepts represented, organized, and related to behavior?
https://t.co/ZnLIevdN4j
1/N
๐ฅณ Introducing SpeechGPT 2.0-preview: A GPT-4o-level, real-time spoken dialogue system! (Only Chinese for now)
๐ Highlights:
๏ฝโก๏ธ Real-time speech-to-speech dialogue with latency under 200ms
๏ฝ๐ Rich in emotion and diverse in style, with strong speech style generalization
๏ฝ๐ฆ Strong role-playing capabilities
๐ค๏ธ Try it out:
Online system: https://t.co/HQvMpCcGg0
Github: https://t.co/uATnZUTJ4v
More demos: https://t.co/d2JGvsSFOJ
@GenBench Our work builds upon our earlier discoveries, showing that cross-linguistic syntactic difference is captured by LLMs within their representation space and significantly impacts their cross-lingual transfer performance. https://t.co/KeVI3XTQFL
Had an incredible week at #EMNLP2023!
I gave my 1st in-person presentation at the @GenBench workshop (on aligning conceptual correspondence for cross-lingual generalization) and had so many interesting conversations! You can check our paper at https://t.co/dWAe54ZegT.