Dr Gareth Roberts

Research Scientist focused on creating products and solving problems in "hard-tech" (brains, minds, AI safety, space, mining).

[Dr Gareth Roberts @gareth1]

Lots of excitement around loop engineering
on x at the moment! Here's my thoughts that go a bit deeper into the mechanisms than most.

Appreciate feedback.

https://t.co/qc90nxZcGD

[Dr Gareth Roberts @gareth1]

https://t.co/0QtVd0UVWV

[Dr Gareth Roberts @gareth1]

https://t.co/0QtVd0UVWV

[Dr Gareth Roberts @gareth1]

https://t.co/DN7EgU7lpN

[Dr Gareth Roberts @gareth1]

Remember the number one way of fixing picture quality on old tvs? Give it a big smack - voila! Lawn mower not starting? Give it a kick. Shit have I been abusing entities for longer than I thought?

Of course, these models can predict statistical regularities associated with text that contains human emotion...something conveniently left out from this video is that being nice to the model and including calming words increases sycophancy dramatically...

[Dr Gareth Roberts @gareth1]

Check out my latest article: Redis as the Stateful Coherence Layer for Complex LLM Systems https://t.co/IMl3Tp7ifC via @LinkedIn

[Dr Gareth Roberts @gareth1]

@matt_elevenlabs 11B

[Dr Gareth Roberts @gareth1]

Check out my latest article: Architecture Wars: How Physics Shapes AI Strategy in 2025 https://t.co/82nlGOQVW1 via @LinkedIn

[Ruben Hassid @rubenhassid]

NVIDIA just dropped paper exposing a $57 billion AI industry mistake.

While Big Tech keeps pushing expensive LLMs like ChatGPT & Claude...

Small language models handle 70% of AI agent work at 1/30th the cost.

Here's why this changes everything:

(hint: less is more)

[Dr Gareth Roberts @gareth1]

Claude Opus 4 for the win! Just coded an agent-based brain simulation complete with neural firing noises in JavaScript. After producing a pretty cool artefact, it then responds "so what's next want me to add neurotransmitter visualisation, brain electrophysiology, inter-hemis.."

[Dr Gareth Roberts @gareth1]

Check out my latest article: Continuous "Thought" Machines https://t.co/eEh9nWogFa via @LinkedIn

[Dr Gareth Roberts @gareth1]

@sama Future pic of Stargate after training the first quintillion parameter LLM

[Rohan Paul @rohanpaul_ai]

Current AI models, like Deep Neural Networks (DNNs), typically use static structures during training and use.

This contrasts with the human brain's dynamic nature.

The paper suggests AI can mimic brain processes like neurogenesis (neuron creation) and neuroapoptosis (neuron death) for more adaptive models.

It introduces 'dropin' (adding neurons) and revisits 'dropout' and 'structural pruning' (removing neurons), combining them to simulate neuroplasticity in AI.

📌 Dropin dynamically adds neurons, letting models adapt capacity during training based on performance needs.

📌 Combining dropin/dropout simulates neuroplasticity, auto-adjusting model complexity for better task adaptation.

📌 This plasticity balances adding capacity (dropin) and regularization (dropout) for potentially better lifelong learning.

----------

Methods Explored in this Paper 🔧:

→ The paper proposes "dropin" learning, analogous to neurogenesis, where new neurons or connections are added to a network during training, potentially triggered by performance convergence (loss changes less than a threshold delta).

→ It contrasts this with existing techniques like "dropout" (temporary random neuron deactivation during training to prevent overfitting) and "structural pruning" (permanent removal of neurons or connections based on criteria like low magnitude or activation).

→ Combining dropin with dropout/pruning allows for dynamic network adjustments, mimicking biological neuroplasticity, where networks can grow or shrink based on task demands or data changes.

----------------------------

Paper - arxiv. org/abs/2503.21419

Paper Title: "Neuroplasticity in AI -- An Overview and Inspirations on Drop In & Out Learning"

[Dr Gareth Roberts @gareth1]

Check out my latest article: The AI Whisperers: Inside the DAN Attack That Transformed AI Safety https://t.co/fZ7e1OVutF via @LinkedIn

[Dr Gareth Roberts @gareth1]

@newsystems_ brampton

[Dr Gareth Roberts @gareth1]

Rethinking Corrigibility: Architectural Solutions to the AI Alignment Paradox https://t.co/Y2SV2l8YS4

[Dr Gareth Roberts @gareth1]

@AmandaAskell If he's a bad doggie - he responds well to punishment too!

[Riley Goodside @goodside]

ARC-AGI scores for past five years of OpenAI models (updated w/ release dates)

[Mike Knoop @mikeknoop]

ARC Prize remains undefeated. Six months ago @fchollet and I launched this crazy $1M @arcprize experiment. Are new ideas needed for AGI? We can now unequivocally say yes. Today we're thrilled to announce 2024 winners, verified SOTAs, new open source code, and technical report!

[Dr Gareth Roberts @gareth1]

I just published The Psychology of AI Alignment: Lessons from Clinical Mental Health https://t.co/DAW82sRPJ6