Aadhav Thaniga

Neuroscience @ Georgia Tech | Learning neurotech through papers and projects | Building toward neural interfaces & robotic control

[Aadhav Thaniga @AadhavThaniga]

Paper: https://t.co/pLAHmHmzp7

[Aadhav Thaniga @AadhavThaniga]

Day 11/365 of learning neurotech.

Back to reading papers today. I spent some time reading "Neural constraints on learning" by Sadtler et al. (2014) to better understand how existing patterns of neural activity constrain learning.

The researchers used a BCI to map monkeys' motor cortex activity to cursor velocity. They used factor analysis to identify an intrinsic manifold, a low-dimensional space capturing how the recorded neurons varied together.

They then introduced two types of decoder changes. Within-manifold changes altered how existing population patterns mapped to movement. Outside-manifold changes altered how individual neurons contributed to the decoder's latent factors, requiring different patterns of coordination for proficient control.

Both initially disrupted performance, but the monkeys learned the within-manifold changes much more readily over a few hours.

My main takeaway is that population geometry can help predict learning. Identifying a low-dimensional representation tells us something about the activity a network already produces, and potentially which new control mappings it can learn quickly.

[Aadhav Thaniga @AadhavThaniga]

@HKallioGoblin There's still so much we don't understand about the brain, but the progress in neural decoding has been incredible. I think scaling and generalization across individuals still have a long way to go before something like that becomes possible. Such an exciting time for BCI!

[Aadhav Thaniga @AadhavThaniga]

Day 10/365

This is me playing Snake using my brain.

The goal was to build a brain-computer interface using steady-state visually evoked potentials (SSVEPs). When you look at a flickering visual stimulus, your visual cortex produces activity at the corresponding frequency. By assigning different frequencies to four directional targets, I wanted to decode which one I was looking at and use it to control Snake.

I built a real-time pipeline to stream EEG from my NPG Lite over Bluetooth at 500 Hz, process the signals, and use CCA/FBCCA to classify the four directions.

Actually getting it to work was much harder than I expected.

My first calibrations failed because of intermittent electrode contact through my hair and a persistent 16.67 Hz interference signal. One of my original targets flickered at 17 Hz, so the decoder kept predicting that direction regardless of where I looked. I’m still working out where the 16.67 Hz signal comes from.

I spent hours analyzing recordings, improving signal-quality checks, adjusting electrode placement, and changing the stimulus frequencies to avoid the interference.

Eventually, I managed to get the gameplay shown in the video using real EEG. However, this was one of very few successful attempts, and the accuracy is very far from perfect.

The biggest thing I learned was how different implementing a neural decoder is from getting one to work with real hardware. Electrode contact, electrical interference, signal quality, and stimulus timing can matter just as much as the decoding algorithm itself.

Next, I plan to invest in cup electrodes with conductive paste and move from one to three occipital EEG channels. Once decoding is reliable, I can start optimizing the accuracy-latency tradeoff to make the controls more responsive.

[Aadhav Thaniga @AadhavThaniga]

Thanks for the suggestions! I'm using a 60 Hz MacBook display and initially had a 17 Hz target, which I later changed to 19 Hz to avoid interference.

Interestingly, the 16.67 Hz interference persisted without any screen flickering and repeated every 30 samples at the board's 500 Hz sampling rate. Since I'm in the US with 60 Hz power lines, I suspect it may originate from the hardware or acquisition process, though I haven't confirmed that yet.

I'll investigate further and try frequencies better suited to my display next time.

[Aadhav Thaniga @AadhavThaniga]

@praneethsamine The recent ones have just been Matplotlib, but I'll probably use manim again soon for some more explanatory animations

[Aadhav Thaniga @AadhavThaniga]

Day 9/365 of learning neurotech

In computational neuroscience, studying neural population dynamics helps us understand how patterns of activity evolve over time. Yesterday, I used this perspective to examine memory in a transformer trained on a 3-bit flip-flop task.

The transformer's hidden representations formed eight memory-state clusters along roughly three bit-specific directions. Today, I wanted to go beyond their geometry and understand how those representations evolve as new commands arrive.

Dynamical systems describe how a system's state evolves based on its current state and inputs. Applying this framework to the transformer let me explore its memory updates while practicing methods relevant to studying neural population dynamics and developing BCI decoders.

I projected the hidden representations onto three principal components and trained models to predict the next representation from the current state and input command. I then tested them on unseen sequences, allowing each model to generate trajectories using its own previous predictions.

The results (final-horizon R²):
1) Shared linear dynamics (0.490): A single linear transition rule captured some structure but accumulated substantial error.
2) Switching linear dynamics (0.915): A separate linear operator for each command substantially improved predictions.
2) Nonlinear dynamics (0.992): A small MLP closely reproduced the projected trajectories.

Interestingly, these three dimensions explained only approximately 63% of hidden-state variance across sequence positions, yet supported highly accurate nonlinear predictions.

However, accurate predictions don't establish the transformer's underlying mechanism. It can attend to its preceding context, and these models may simply be learning the task's memory-update rules.

Next, I want to test whether these three dimensions are sufficient to predict future representations, or whether earlier inputs provide additional information.

The animation compares the transformer's actual trajectory with each model's free-running predictions.

[Aadhav Thaniga @AadhavThaniga]

@anshulcreates Thanks! I used Matplotlib's FuncAnimation for the video

[Aadhav Thaniga @AadhavThaniga]

Day 8/365 of learning neurotech

I've been reading Sussillo and Barak's Opening the Black Box, which investigates how RNNs implement computations through their internal dynamics.

I wanted to explore whether similar representational structure emerges in a different architecture, so I trained a small causal transformer on a 3-bit flip-flop task.

The model had to track three independently updated memory bits. The eight possible memory states formed distinct clusters in PCA space, with the first three PCs explaining 99.46% of the variance. Given that the task involves three independent binary variables, this low-dimensional structure makes sense.

I then tested whether the eight memory states could be reconstructed by combining three learned vectors, one for each bit. The idea is that changing one bit should shift the hidden representation along a consistent direction, regardless of the other two bits.

This additive model explained 99.43% of held-out hidden-state variance, suggesting the transformer learned an approximately compositional representation of memory rather than eight unrelated representations.

The next question is whether these representations follow a consistent state-transition rule: can we predict how the transformer's hidden state changes from its current representation and the next input, across unseen sequences and histories?

The animation shows the hidden representations projected into three principal components (left), alongside the predicted memory values (right), as individual bits are updated.

[Aadhav Thaniga @AadhavThaniga]

@praneethsamine Day 21/365 of building a terminator

[Aadhav Thaniga @AadhavThaniga]

@aarav_kothari_x Excited to follow along and see what you build!

[Aadhav Thaniga @AadhavThaniga]

Paper: https://t.co/MEV1jk5WtC

[Aadhav Thaniga @AadhavThaniga]

Day 7/365 of learning neurotech

Today, I looked into Neural Encoding and Decoding at Scale (NEDS), a multimodal transformer that jointly learns neural encoding and behavioral decoding through multi-task masking. Learning both directions could potentially help build representations useful for more than a single prediction task.

NEDS does this through four masking strategies:

1. Neural masking: Hide neural activity and predict it from behavior (learn encoding).

2. Behavioral masking: Hide behavior and predict it from neural activity (learn decoding).

3. Within-modality masking: Hide parts of one modality and reconstruct them using only the remaining observations from that same modality (learn structure within neural or behavioral data).

4. Cross-modal masking: Hide parts of both modalities and reconstruct them using the available neural and behavioral data (learn relationships across modalities).

Each modality is tokenized separately, then processed by a shared transformer with modality-specific output heads. Session-specific mappings allow recordings with different neural populations to use the same shared model.

The authors pretrained on 74 recording sessions from 73 mice, then fine-tuned on 10 held-out animals. In this evaluation, NEDS outperformed POYO+ and NDT2 across four decoding tasks while also supporting neural encoding. The gains over POYO+ were small for choice prediction and larger for continuous movement variables.

The learned neuron embeddings also supported approximately 83% brain-region classification accuracy across five regions, even though anatomical labels were not used to train NEDS.

I think the main takeaway is that the pretraining objective matters alongside architecture and dataset size. Combining encoding, decoding, and reconstruction provides the model with several ways to learn relationships between neural activity and behavior.

[Aadhav Thaniga @AadhavThaniga]

Here is the paper: https://t.co/PSgAjucSXx

[Aadhav Thaniga @AadhavThaniga]

Day 6/365 of learning neurotech

On Day 3, I reproduced a POYO-inspired neural decoding model. Today, I read POYO+ (Azabou et al., 2025), which extends the original architecture toward more generalizable neural decoding.

The main challenge POYO addressed was that recordings from different sessions typically contain different populations of neurons, making it difficult to train a shared model. Its tokenization approach and Perceiver architecture allowed it to learn across sessions without requiring neuron-to-neuron correspondence.

However, POYO primarily focused on spike-based motor decoding. It didn’t test whether the same approach could generalize across different recording modalities, brain regions, cell types, and behavioral tasks.

POYO+ extends the architecture in several ways:

1) Calcium imaging support: Instead of relying only on discrete spike events, POYO+ introduces tokenization for continuous calcium fluorescence recordings.

2) Task-conditioned decoding: Learned task embeddings allow the same model to make predictions across multiple tasks, including running speed, pupil position, and visual stimulus orientation, using a shared neural representation.

3) Multitask training: By jointly training across different neural populations and decoding objectives, the model can learn representations that transfer between tasks and brain regions.

Here are the results:

Multitask training improved performance on several decoding tasks compared to single-task models. Pretraining across multiple visual cortical regions also improved average orientation-decoding accuracy on held-out sessions.

More importantly, representations pretrained on visual cortex transferred to hippocampal position decoding, reducing mean prediction error from 10.33 cm to 9.58 cm after adaptation. The model's learned representations also contained information about anatomical regions and cell types, despite not being explicitly trained to classify them.

The broader implication is that diverse neural recordings can provide useful shared training signals, even when the underlying populations and tasks differ.

However, transfer after fine-tuning is not the same as zero-shot generalization. POYO+ still requires adaptation to new recordings, and it remains unclear how well these learned representations will generalize across more diverse neural populations, recording conditions, and tasks.

I want to continue exploring neural foundation models, particularly how well their learned representations generalize across different neural populations, recording modalities, and tasks.

[Aadhav Thaniga @AadhavThaniga]

Day 5/365 of learning neurotech

I had limited time for project work today, so this is a brief overview of neural population dynamics, a topic I’m researching at Georgia Tech.

A recording from N neurons represents population activity in an N-dimensional space. Because neurons exhibit coordinated activity, much of the task-related variation can often be represented in a lower-dimensional latent space. Each latent dimension describes a pattern of activity distributed across neurons.

The population’s latent state changes over time, forming a trajectory. Dynamics describe how the current state and external inputs determine its subsequent evolution.

Trajectory geometry, timing, and responses to inputs provide constraints on possible computational mechanisms. Models can test whether proposed dynamics reproduce these features and predict behavior under new conditions.

However, neither dimensionality reduction nor a well-fitting model alone identifies a mechanism. Similar trajectories can arise from different circuit structures or inputs, and the inferred representation depends on the model and recorded neurons. Behavioral comparisons and perturbation experiments help distinguish competing explanations.

The goal is to understand which features of population activity support a computation and how the underlying circuit produces them.

[Aadhav Thaniga @AadhavThaniga]

Paper: https://t.co/DaW3LmHDx3

[Aadhav Thaniga @AadhavThaniga]

Day 4/365 of learning neurotech

In a closed-loop neural interface, the brain doesn't just control the decoder. The decoder's behavior also influences how the brain learns to control it.

When both the user and decoder adapt simultaneously, their learning processes become coupled. This raises an important question: how should we optimize a decoder when its updates also change the behavior of the system it's trying to decode?

Today, I read Computational Framework to Predict and Shape Human–Machine Interactions in Closed-Loop, Co-Adaptive Neural Interfaces (Madduri et al., 2026), which explores exactly this problem.

The researchers developed a framework using control theory and game theory to model this interaction. They tested it with 14 participants controlling a 2D cursor through a 64-channel forearm EMG interface.

Rather than optimizing the decoder in isolation, they modeled the user and decoder as two learners, each balancing tracking accuracy with their own effort.

I found two of their results particularly interesting.

First, faster decoder adaptation actually made performance worse in their experiments. When the decoder updated too quickly, users struggled to adapt alongside it, disrupting the co-adaptation process.

Second, changing the decoder's regularization altered the user's estimated control effort, even when tracking performance remained similar.

This suggests that we can influence how users learn to control an interface through the decoder's optimization objectives.

I think this has important implications for adaptive BCIs and robotic control. Instead of focusing solely on decoding accuracy, we should consider how decoder updates affect user learning, closed-loop stability, and the behavior of the combined system.

I'm especially interested in how these ideas could extend to generalizable, closed-loop neurotechnology, where interfaces adapt to the user while accounting for how the user adapts in return.

[Praneeth Samineni @praneethsamine]

Day 17/365 of robot learning.

We won 2nd place at the @ycombinator Agent Hackathon in the @insforge track!

We built a Cursor for training real robot arms, where agents manage the entire pipeline from data collection through deployment. We’re planning to continue experimenting with this as a side project.

Huge shoutout to my teammates @AneeshDan and @suhasdev23 for building this with me! Thanks to @nemild for trying to help us find a robot arm.