Augustas Macijauskas

Senior AI/ML engineer at Spike Technologies working on agentic voice systems. Previously ML at @Cambridge_Uni.

[Augustas Macijauskas @augustasio]

I'm excited to announce that I have graduated with distinction from the MPhil in Machine Learning and Machine Intelligence at the University of Cambridge! 📚

[Augustas Macijauskas @augustasio]

6/6 A month of weekends and almost 300 commits later, it's live. If there's a film or play you'd like explained, you can request it on the site.

[Augustas Macijauskas @augustasio]

1/6 For the past month I've been building a passion project: a site where AI agents explain films from anywhere, and the plays staged at Lithuanian theatres, with every claim checked against its sources.

Give it a look: https://t.co/OeZCapHSul

[Augustas Macijauskas @augustasio]

5/6 What surprised me most: before showing me a UI change, the agents open the site in a browser, shrink it to phone size and fix whatever looks off.

[Augustas Macijauskas @augustasio]

Early step: small models and multiple-choice evals, and the gates weaken dangerous knowledge rather than erase it. My first paper, done alongside a full-time job, with Patrikas Vanagas and @CS_Laurynas.
Paper: https://t.co/YSYV1G6E1m

[Augustas Macijauskas @augustasio]

Our paper "Capability-Gated Language Models: Security Composes, Utility Does Not" has been accepted at the FLMSec workshop at NeurIPS 2026. One model, different permission levels per user: stacking restrictions keeps safety intact, but can quietly break usefulness.

[Augustas Macijauskas @augustasio]

Combining restrictions never significantly weakened suppression on held-out data, and usually strengthened it. But restrictions that each cost at most 1.6 points of everyday accuracy cost 17.1 together, and we prove no general bound on that is possible.

[Augustas Macijauskas @augustasio]

@karpathy Would be curious to know the actual data mixture used for training and try to fully replicate it 🤔

[Augustas Macijauskas @augustasio]

@karpathy The speed comes from the fact that it is written in Rust and the library also allows to easily assemble and train a new tokenizer from scratch. Took ~40 lines of code to replicate the Llama 3 tokenizer (including training code): https://t.co/bBIHDtxz6P

[Augustas Macijauskas @augustasio]

As it turns out, @huggingface has great tooling around building tokenizers and running them in inference mode. I am already doing a deep dive with the end goal of replicating the LLaMA 3 tokenizer as closely as possible. Stay tuned!

Read the article: https://t.co/drm0GC6oC5

[Augustas Macijauskas @augustasio]

Excited to share another article! This time, it's about the various problems and peculiar behaviours that can occur if one is not careful about the tokenization process.

Towards the end, I challenge @karpathy's claim that building tokenizers from scratch is not possible.

[Augustas Macijauskas @augustasio]

It uses dimensionality reduction techniques, such as t-SNE and UMAP. It turns out that the embeddings contain a very nice semantic structure. Read the blog post to find out more!

URL: https://t.co/hKjXhEZWlD

[Augustas Macijauskas @augustasio]

After a recent post announcing a tool for visualising the LLM tokenization process, I was asked about ways to visualise the embeddings learnt during the LLM pre-training process. The blog post below implements one possible way to visualise the embeddings.

[Augustas Macijauskas @augustasio]

We introduce a tool to visualize various @huggingface tokenizers, making NLP understanding more accessible to all!

GitHub repo: https://t.co/wDGbndHvDM

#NLPTokenizers #OpenSource

[Augustas Macijauskas @augustasio]

Excited to announce the release of an open-source project I built with @CS_Laurynas. It was inspired by @karpathy insightful video on LLM tokenizers.

URL: https://t.co/S2CJudptwt
Video: https://t.co/w01uAlD419

[Augustas Macijauskas @augustasio]

I'm open to new collaborations and experiences as I navigate my path beyond graduation.

[Augustas Macijauskas @augustasio]

I'm excited to announce that I have graduated with distinction from the MPhil in Machine Learning and Machine Intelligence at the University of Cambridge! 📚

[Augustas Macijauskas @augustasio]

I enjoyed working on working on the interpretability and alignment of LLMs. If you have any exciting opportunities or projects related to these topics, please don't hesitate to contact me.