Mercity AI

Building research-led capabilities for product and enterprise teams.

Synthetic data · Custom training · Evals

Solving problems generic APIs cannot solve.

[Mercity AI @MercityAI]

Today we introduce Mercity.

A research-first AI firm building at the intersection of synthetic data, custom training, evals and safety

We believe that the future of safe enterprise AI lies in enabling research led capabilities

Here's our work so far :
https://t.co/Do0u6W9x7t

[Mercity AI @MercityAI]

Today we're open-sourcing Heckler.

Heckler uses simulated users to test your agent.

It is purpose built to push your agent's system rules to the limit and tells you where exactly your agent messed up.

It not only catches outright jailbreaks, but also subtle policy violations and rule breaking behaviour.

This makes testing your agents and prompts before they hit production way easier and simpler!

Try it here :
https://t.co/OHoQgEBzYS

[Harsh. @freshlimesofa]

Today we introduce Heckler by @MercityAI

Heckler stress tests your AI agents with simulated users and shows you exactly where your agent breaks the rules you gave it.

We ran 3 agents (DeepSeek V4.1 Flash, GLM 5.3 and GPT-6 Luna) on Heckler and over 80% of chats broke a policy.

[Mercity AI @MercityAI]

@MrrrOzi Absolutely

[Mercity AI @MercityAI]

Our Latest Research:

We pretrained four 1B-parameter models on 6B FineWeb tokens to test two ideas from frontier MoE labs, Kimi Delta Attention and LongCat n-gram embeddings, at small scale.

We then compared grafting, pruning, 4-bit quantization, and downstream NER fine-tuning

[Mercity AI @MercityAI]

Research done by @_notapenguin
Tweet authored by @freshlimesofa

[Mercity AI @MercityAI]

We then operated on trained checkpoints

Removing layer 0 cut the score by 1/3rd while most layers past layer 8 could go almost for free

4-bit quant cost under a point

Attention could not be moved between models without collapsing

Full research log: https://t.co/RIyshOBzeQ

[Mercity AI @MercityAI]

Find the full research log here:

https://t.co/R2wJT775xd

[Mercity AI @MercityAI]

We REMOVED upto 10 WHOLE layers from Qwen3-8B-Base using LaCo (Layer Collapse).

Upon deleting only 6 layers, the model got 16.7% smaller, keeping 78-94% of its reasoning scores at a 1.68x perplexity ratio.

Code and pruned weights are up on our Hugging Face.

Here are insights from our research :

[Mercity AI @MercityAI]

The paper's Llama settings didn't transfer to Qwen3

A 0.65 similarity threshold sent MMLU to random, 0.85 worked

Merging near the output broke the model, so we stopped at layer 28

Merging LayerNorms gave gibberish, so we left them alone

There is probably an element of how many tokens the model is trained on! We will explore this in the coming times!

[Mercity AI @MercityAI]

Find out more about our Models, Research, Datasets and work here :

https://t.co/smhTqMspVX
https://t.co/s1E2Lvz3Bi
https://t.co/LfbRSWBuKW

[Mercity AI @MercityAI]

Today we introduce Mercity.

A research-first AI firm building at the intersection of synthetic data, custom training, evals and safety

We believe that the future of safe enterprise AI lies in enabling research led capabilities

Here's our work so far :
https://t.co/Do0u6W9x7t

[Mercity AI @MercityAI]

We build custom data, models and evaluation for problems an API cannot solve.

If that is the problem in front of you, our DMs are open, or start a conversation at https://t.co/smhTqMspVX