Aadhav Thaniga

Neuroscience @ Georgia Tech | Learning neurotech through papers and projects | Building toward neural interfaces & robotic control

[Aadhav Thaniga @AadhavThaniga]

Day 10/365

This is me playing Snake using my brain.

The goal was to build a brain-computer interface using steady-state visually evoked potentials (SSVEPs). When you look at a flickering visual stimulus, your visual cortex produces activity at the corresponding frequency. By assigning different frequencies to four directional targets, I wanted to decode which one I was looking at and use it to control Snake.

I built a real-time pipeline to stream EEG from my NPG Lite over Bluetooth at 500 Hz, process the signals, and use CCA/FBCCA to classify the four directions.

Actually getting it to work was much harder than I expected.

My first calibrations failed because of intermittent electrode contact through my hair and a persistent 16.67 Hz interference signal. One of my original targets flickered at 17 Hz, so the decoder kept predicting that direction regardless of where I looked. I’m still working out where the 16.67 Hz signal comes from.

I spent hours analyzing recordings, improving signal-quality checks, adjusting electrode placement, and changing the stimulus frequencies to avoid the interference.

Eventually, I managed to get the gameplay shown in the video using real EEG. However, this was one of very few successful attempts, and the accuracy is very far from perfect.

The biggest thing I learned was how different implementing a neural decoder is from getting one to work with real hardware. Electrode contact, electrical interference, signal quality, and stimulus timing can matter just as much as the decoding algorithm itself.

Next, I plan to invest in cup electrodes with conductive paste and move from one to three occipital EEG channels. Once decoding is reliable, I can start optimizing the accuracy-latency tradeoff to make the controls more responsive.

[Aadhav Thaniga @AadhavThaniga]

@praneethsamine The recent ones have just been Matplotlib, but I'll probably use manim again soon for some more explanatory animations

[Aadhav Thaniga @AadhavThaniga]

Day 9/365 of learning neurotech

In computational neuroscience, studying neural population dynamics helps us understand how patterns of activity evolve over time. Yesterday, I used this perspective to examine memory in a transformer trained on a 3-bit flip-flop task.

The transformer's hidden representations formed eight memory-state clusters along roughly three bit-specific directions. Today, I wanted to go beyond their geometry and understand how those representations evolve as new commands arrive.

Dynamical systems describe how a system's state evolves based on its current state and inputs. Applying this framework to the transformer let me explore its memory updates while practicing methods relevant to studying neural population dynamics and developing BCI decoders.

I projected the hidden representations onto three principal components and trained models to predict the next representation from the current state and input command. I then tested them on unseen sequences, allowing each model to generate trajectories using its own previous predictions.

The results (final-horizon R²):
1) Shared linear dynamics (0.490): A single linear transition rule captured some structure but accumulated substantial error.
2) Switching linear dynamics (0.915): A separate linear operator for each command substantially improved predictions.
2) Nonlinear dynamics (0.992): A small MLP closely reproduced the projected trajectories.

Interestingly, these three dimensions explained only approximately 63% of hidden-state variance across sequence positions, yet supported highly accurate nonlinear predictions.

However, accurate predictions don't establish the transformer's underlying mechanism. It can attend to its preceding context, and these models may simply be learning the task's memory-update rules.

Next, I want to test whether these three dimensions are sufficient to predict future representations, or whether earlier inputs provide additional information.

The animation compares the transformer's actual trajectory with each model's free-running predictions.

[Aadhav Thaniga @AadhavThaniga]

@anshulcreates Thanks! I used Matplotlib's FuncAnimation for the video

[Aadhav Thaniga @AadhavThaniga]

Day 8/365 of learning neurotech

I've been reading Sussillo and Barak's Opening the Black Box, which investigates how RNNs implement computations through their internal dynamics.

I wanted to explore whether similar representational structure emerges in a different architecture, so I trained a small causal transformer on a 3-bit flip-flop task.

The model had to track three independently updated memory bits. The eight possible memory states formed distinct clusters in PCA space, with the first three PCs explaining 99.46% of the variance. Given that the task involves three independent binary variables, this low-dimensional structure makes sense.

I then tested whether the eight memory states could be reconstructed by combining three learned vectors, one for each bit. The idea is that changing one bit should shift the hidden representation along a consistent direction, regardless of the other two bits.

This additive model explained 99.43% of held-out hidden-state variance, suggesting the transformer learned an approximately compositional representation of memory rather than eight unrelated representations.

The next question is whether these representations follow a consistent state-transition rule: can we predict how the transformer's hidden state changes from its current representation and the next input, across unseen sequences and histories?

The animation shows the hidden representations projected into three principal components (left), alongside the predicted memory values (right), as individual bits are updated.

[Aadhav Thaniga @AadhavThaniga]

@praneethsamine Day 21/365 of building a terminator

[Aadhav Thaniga @AadhavThaniga]

@aarav_kothari_x Excited to follow along and see what you build!

[Aadhav Thaniga @AadhavThaniga]

Paper: https://t.co/MEV1jk5WtC

[Aadhav Thaniga @AadhavThaniga]

Day 7/365 of learning neurotech

Today, I looked into Neural Encoding and Decoding at Scale (NEDS), a multimodal transformer that jointly learns neural encoding and behavioral decoding through multi-task masking. Learning both directions could potentially help build representations useful for more than a single prediction task.

NEDS does this through four masking strategies:

1. Neural masking: Hide neural activity and predict it from behavior (learn encoding).

2. Behavioral masking: Hide behavior and predict it from neural activity (learn decoding).

3. Within-modality masking: Hide parts of one modality and reconstruct them using only the remaining observations from that same modality (learn structure within neural or behavioral data).

4. Cross-modal masking: Hide parts of both modalities and reconstruct them using the available neural and behavioral data (learn relationships across modalities).

Each modality is tokenized separately, then processed by a shared transformer with modality-specific output heads. Session-specific mappings allow recordings with different neural populations to use the same shared model.

The authors pretrained on 74 recording sessions from 73 mice, then fine-tuned on 10 held-out animals. In this evaluation, NEDS outperformed POYO+ and NDT2 across four decoding tasks while also supporting neural encoding. The gains over POYO+ were small for choice prediction and larger for continuous movement variables.

The learned neuron embeddings also supported approximately 83% brain-region classification accuracy across five regions, even though anatomical labels were not used to train NEDS.

I think the main takeaway is that the pretraining objective matters alongside architecture and dataset size. Combining encoding, decoding, and reconstruction provides the model with several ways to learn relationships between neural activity and behavior.

[Aadhav Thaniga @AadhavThaniga]

Here is the paper: https://t.co/PSgAjucSXx

[Aadhav Thaniga @AadhavThaniga]

Day 6/365 of learning neurotech

On Day 3, I reproduced a POYO-inspired neural decoding model. Today, I read POYO+ (Azabou et al., 2025), which extends the original architecture toward more generalizable neural decoding.

The main challenge POYO addressed was that recordings from different sessions typically contain different populations of neurons, making it difficult to train a shared model. Its tokenization approach and Perceiver architecture allowed it to learn across sessions without requiring neuron-to-neuron correspondence.

However, POYO primarily focused on spike-based motor decoding. It didn’t test whether the same approach could generalize across different recording modalities, brain regions, cell types, and behavioral tasks.

POYO+ extends the architecture in several ways:

1) Calcium imaging support: Instead of relying only on discrete spike events, POYO+ introduces tokenization for continuous calcium fluorescence recordings.

2) Task-conditioned decoding: Learned task embeddings allow the same model to make predictions across multiple tasks, including running speed, pupil position, and visual stimulus orientation, using a shared neural representation.

3) Multitask training: By jointly training across different neural populations and decoding objectives, the model can learn representations that transfer between tasks and brain regions.

Here are the results:

Multitask training improved performance on several decoding tasks compared to single-task models. Pretraining across multiple visual cortical regions also improved average orientation-decoding accuracy on held-out sessions.

More importantly, representations pretrained on visual cortex transferred to hippocampal position decoding, reducing mean prediction error from 10.33 cm to 9.58 cm after adaptation. The model's learned representations also contained information about anatomical regions and cell types, despite not being explicitly trained to classify them.

The broader implication is that diverse neural recordings can provide useful shared training signals, even when the underlying populations and tasks differ.

However, transfer after fine-tuning is not the same as zero-shot generalization. POYO+ still requires adaptation to new recordings, and it remains unclear how well these learned representations will generalize across more diverse neural populations, recording conditions, and tasks.

I want to continue exploring neural foundation models, particularly how well their learned representations generalize across different neural populations, recording modalities, and tasks.

[Aadhav Thaniga @AadhavThaniga]

Day 5/365 of learning neurotech

I had limited time for project work today, so this is a brief overview of neural population dynamics, a topic I’m researching at Georgia Tech.

A recording from N neurons represents population activity in an N-dimensional space. Because neurons exhibit coordinated activity, much of the task-related variation can often be represented in a lower-dimensional latent space. Each latent dimension describes a pattern of activity distributed across neurons.

The population’s latent state changes over time, forming a trajectory. Dynamics describe how the current state and external inputs determine its subsequent evolution.

Trajectory geometry, timing, and responses to inputs provide constraints on possible computational mechanisms. Models can test whether proposed dynamics reproduce these features and predict behavior under new conditions.

However, neither dimensionality reduction nor a well-fitting model alone identifies a mechanism. Similar trajectories can arise from different circuit structures or inputs, and the inferred representation depends on the model and recorded neurons. Behavioral comparisons and perturbation experiments help distinguish competing explanations.

The goal is to understand which features of population activity support a computation and how the underlying circuit produces them.

[Aadhav Thaniga @AadhavThaniga]

Paper: https://t.co/DaW3LmHDx3

[Aadhav Thaniga @AadhavThaniga]

Day 4/365 of learning neurotech

In a closed-loop neural interface, the brain doesn't just control the decoder. The decoder's behavior also influences how the brain learns to control it.

When both the user and decoder adapt simultaneously, their learning processes become coupled. This raises an important question: how should we optimize a decoder when its updates also change the behavior of the system it's trying to decode?

Today, I read Computational Framework to Predict and Shape Human–Machine Interactions in Closed-Loop, Co-Adaptive Neural Interfaces (Madduri et al., 2026), which explores exactly this problem.

The researchers developed a framework using control theory and game theory to model this interaction. They tested it with 14 participants controlling a 2D cursor through a 64-channel forearm EMG interface.

Rather than optimizing the decoder in isolation, they modeled the user and decoder as two learners, each balancing tracking accuracy with their own effort.

I found two of their results particularly interesting.

First, faster decoder adaptation actually made performance worse in their experiments. When the decoder updated too quickly, users struggled to adapt alongside it, disrupting the co-adaptation process.

Second, changing the decoder's regularization altered the user's estimated control effort, even when tracking performance remained similar.

This suggests that we can influence how users learn to control an interface through the decoder's optimization objectives.

I think this has important implications for adaptive BCIs and robotic control. Instead of focusing solely on decoding accuracy, we should consider how decoder updates affect user learning, closed-loop stability, and the behavior of the combined system.

I'm especially interested in how these ideas could extend to generalizable, closed-loop neurotechnology, where interfaces adapt to the user while accounting for how the user adapts in return.

[Praneeth Samineni @praneethsamine]

Day 17/365 of robot learning.

We won 2nd place at the @ycombinator Agent Hackathon in the @insforge track!

We built a Cursor for training real robot arms, where agents manage the entire pipeline from data collection through deployment. We’re planning to continue experimenting with this as a side project.

Huge shoutout to my teammates @AneeshDan and @suhasdev23 for building this with me! Thanks to @nemild for trying to help us find a robot arm.

[Aadhav Thaniga @AadhavThaniga]

A visual walkthrough of my causal Mini-POYO implementation.

If you’re interested in the architecture, give the original POYO paper by Azabou et al. a read:
https://t.co/GZV8F48bdY

[Aadhav Thaniga @AadhavThaniga]

Day 3/365 of learning neurotech

Today, I implemented a small causal decoder inspired by POYO.

My broader goal is to explore generalizable neural dynamics and embodied control for adaptive, bidirectional neurotechnology. I’m looking into foundation models because learning shared structure across recordings could help neural interfaces adapt to new sessions or users with less calibration.

For this first experiment, I simulated 32 neurons whose firing rates depended on a 2D movement signal, then sampled spike events from those rates. The task was to recover horizontal and vertical velocity from the spikes, giving me a simple continuous decoding problem with known ground truth.

Each spike is represented by its neuron identity and timestamp. Through cross-attention, learned latent vectors gather information from these events. Two self-attention blocks process the resulting representations.

Output queries then read from the latents at specific timestamps, and a prediction head produces the two velocity components. I utilized causal masking throughout to ensure each prediction only uses spikes available at that time.

The animation below walks through the full pipeline.

After training on 60 minutes of synthetic activity, the model achieved a mean held-out R^2 of 0.872 versus 0.850 for causal ridge regression. That’s a modest absolute improvement, corresponding to about 15% lower MSE. The comparison is preliminary, since the transformer could access a longer spike history than ridge.

The initial version, trained on much less data, actually lost to ridge. Next, I’d like to vary training-set size while keeping evaluation fixed to understand how performance scales with data.

This experiment uses the same simulated neural population throughout. I’d also like to test whether the model can adapt to a different set of neurons using only a small amount of calibration data.

Stay tuned!

[Aadhav Thaniga @AadhavThaniga]

Day 2/365 of learning neurotech

Yesterday, I built a simplified action-chunking transformer in JAX for 2D reaching. Today, I connected recorded muscle activity to a simulated robot arm that grasps, lifts, and releases a block.

This is another step toward MyoAct, my project exploring EMG-to-action transformers for robotic control.

I used 16 forearm EMG channels from GRABMyo, causally filtered at 20-450 Hz. From 200 ms windows updated every 50 ms, I extracted signal features and trained an LDA classifier to recognize hand open, hand close, and rest. Commands require three consecutive predictions of the same gesture above a validation-selected confidence threshold. The decoder reached 92.3% window-level balanced accuracy on held-out recordings from the same participant and session.

Those commands guide a small deterministic ACT-style transformer trained to imitate a scripted Fetch-arm controller in MuJoCo. It takes eight recent observations of the robot, block, and command, predicts eight future actions, and executes the first before observing and replanning.

My first version had low validation loss but failed every grasp evaluation. Small movement errors pushed the robot into states absent from its demonstrations, and it kept drifting. Previous-action inputs also appeared to encourage copying the last action.

I added noise during demonstration collection so the robot encountered off-path states while learning the clean controller’s corrections. I also removed previous actions from the inputs and adjusted normalization. After these changes, the policy completed grasp-and-release runs at block positions absent from training.

The demo below shows the learned controller alongside scrolling EMG traces. Four held-out recordings play back-to-back through rest, close, rest, and open. A decoded close requests grasping and lifting, while open requests lowering and release. In this example, the first accepted open command takes about two seconds to arrive.

For now, EMG provides discrete requests and the transformer generates the motion. This uses one participant’s recorded data replayed in simulation, with live human control still ahead.

Next, I want to test the decoder across sessions and participants, moving toward MyoAct’s goal of generalizable control. I also plan to explore direct conditioning on EMG features, temporal ensembling, and eventually a CVAE.

I’ll spend the next couple of posts sharing papers I’m reading while I continue building. Stay tuned.