Martha

PMO Lead → AI Red Teamer. 10+ yrs governance & control. AI Red Teamer | Prompt Injection, LLM attacks, AI evasion. Governance meets adversarial AI.

[Martha @nishogoza]

I was just awarded the ShadowGuard badge on HTB Academy https://t.co/RN6NE6D3Ct #hackthebox #htbacademy #cybersecurity

[Martha @nishogoza]

Very interesting, thank you

[Machine Learning Mastery @TeachTheMachine]

Fine-Tuning Agentic AI: A Practical Guide

https://t.co/bB2cApDMnL

[Martha @nishogoza]

Module 14. Attack half proves the risk is real. Defense half proves it's controllable, not just avoidable through hope.

[Martha @nishogoza]

I was just awarded the ShadowGuard badge on HTB Academy https://t.co/RN6NE6D3Ct #hackthebox #htbacademy #cybersecurity

[Martha @nishogoza]

Of everything in this pathway, this is the module I'd hand to a DPO with the least translation required. Membership inference is the technical proof behind an intuition GDPR has encoded for years: aggregation isn't automatically anonymisation.

If your organisation trains models on personal data and calls it "anonymised" because names were removed this module is the reason to check that claim again. 👇

[Martha @nishogoza]

Here's the uncomfortable part there's no dial setting that gets you full accuracy and zero privacy risk simultaneously.

Every real project trained on real records is choosing a point on that dial, whether anyone frames it that way or not. Most just don't measure where the dial is currently set.

[Martha @nishogoza]

Here's the sentence from this module's conclusion that's worth sitting with, because it's really a governance sentence wearing an ML label:

"How much utility are you willing to trade for a measurable reduction in what an adversary can learn about any one person?"

That's not a technical question. That's the exact trade-off risk teams make every time they decide how granular a report gets, or how long data gets retained. This module just gives it a mathematically precise version.

[Martha @nishogoza]

DP-SGD is that dial implemented directly in training: clip each individual's gradient contribution, inject calibrated noise, repeat every step.

[Martha @nishogoza]

DP-SGD: bakes that guarantee into training itself, through per-sample gradient clipping and calibrated noise injection, so no individual's data can dominate what the model learns.

This is a hard cap on any one data point's influence, enforced at the point of processing rather than checked after the fact.

[Martha @nishogoza]

Turn the dial: differential privacy adds mathematical noise, calibrated to guarantee that no individual's data can be confidently identified from the output.

[Martha @nishogoza]

The mechanism: models behave differently on data they were trained on versus data they weren't. That gap is caused by overfitting, and it creates what the module calls a "membership fingerprint" detectable just from prediction confidence patterns.

Governance parallel: this is metadata leakage. Nobody published the sensitive fact directly. The pattern of how the system responds gives it away anyway.

[Martha @nishogoza]

Zero privacy protection looks like this: train the model normally, optimise purely for accuracy.

Result: the model quietly memorises outliers and edge cases better than it should, and that over-memorisation is exactly what makes membership inference possible in the first place.

[Martha @nishogoza]

Read that again slowly, because the example in the module is the one that makes it land.

If a model was trained exclusively on cancer patients, successfully proving someone was a training member reveals their medical status. No data leak required. No breach. Just the model behaving slightly differently on data it has seen before.

[Martha @nishogoza]

Here's the frame I want to use this time: privacy protection in ML isn't a switch. It's a dial. And every technique in this module is really about deciding how far to turn it.

[Martha @nishogoza]

AI Privacy, Membership Inference Attacks (MIA), differential privacy, DP-SGD, and PATE.

The core finding: a trained model can reveal which specific individuals were in its training data. Not what the data said. Whether a specific person's record was used at all.

[Martha @nishogoza]

"The membership inference attacks you built revealed a pretty uncomfortable but useful truth about AI: a model can betray exactly which records shaped it."

That's not my line. That's how HTB closes Module 14. It's the most quietly damning sentence in this whole pathway

[Martha @nishogoza]

Every module so far has been "here's how to attack it."

This one is different. Half attack, half defense and the attack half proves something GDPR has been assuming for years without most people fully understanding why it's true. 🧵

[Martha @nishogoza]

https://t.co/NPbfVK5N17 Module 8 was about changing every input value by a tiny, imperceptible amount.

Module 9 asks a completely different question: what if you don't touch most of the input at all and change only a handful of features, but change them more?

Same goal. Opposite strategy. 🧵