Sriram V

[AI Club IITM @aiclubiitm]

In Project Viveka 1.0, we explored the Simplex framework. Extending "Constrained Belief Updates Explain Geometric Structures in Transformer Representations," we tested how MLPs convert constrained belief updates to full belief updates. 1/8

[AI Club IITM @aiclubiitm]

Background - We trained one layer transformers on the Mess3 Mealy Markov process (non-unifilar over token transition representation) across various parametrizations of (x,alpha) of the token-wise transition matrices. 2/8

[AI Club IITM @aiclubiitm]

How can we interpret the internal activations of LLMs? Logit lens is a popular technique to translate internal activations of an LLM to token space. In this blog, written during Project Viveka ,we examined where logit lens fails.
#AI #LLM #mechanisticinterpretability #logitlens

[AI Club IITM @aiclubiitm]

Logit lens applies the unembedding matrix to internal activations to decode them as tokens. While effective on small models like gpt2, it yields garbage tokens in some layers for models like gemma-2-2b-it.

[AI Club IITM @aiclubiitm]

Ever wanted to start learning AI and ML but didn't know where to begin? Yeah, we've all been there.

We compiled our Knowledge Transfer Sessions into Translating Tensors, a from scratch intro covering regression, classification, core ML models, deep learning and a lot more .

[AI Club IITM @aiclubiitm]

It comes with the math behind each concept, code for every part of it.

So you don't just understand ML, you can practice it alongside.

Find it here: https://t.co/AzviAMTYzl

[AI Club IITM @aiclubiitm]

Meet our project Viveka 2.0, in which we dig into whether AI models solving ARC-AGI actually learn abstract rules, and advance non LLM approaches from our interpretability results.
https://t.co/rbr3zz5sQy

#ARCAGI #AI #Interpretability #MachineLearning #AIResearch #IITMadras

[AI Club IITM @aiclubiitm]

Tiny Recursive Model (TRM) approaches to solve ARC-AGI by packing a puzzle's entire transformation logic into one embedding vector, using a tiny 7M-param model as a pure "executor."

We tested if that's actually true. It's not that clean.

[AI Club IITM @aiclubiitm]

@SMITali @v_sriram_v Image references - https://t.co/F79vHQvVLF
https://t.co/abjvPcV2yL

[AI Club IITM @aiclubiitm]

@SMITali @v_sriram_v We will post the entire report with code soon. We tried various other things in the Simplex framework and will share them in coming weeks! Reach out if interested! 8/8 Authors -
@SmitaliBhandari, @v_sriram_v, Prof. Avishek Chatterjee, IITM
#AI #mechinterp #Simplex #transformer