Data For Science

Take Control Of Your Data. Join our Data Science Briefing newsletter for the best in #DataScience and #MachineLearning

[Data For Science @data4sci]

17 days until the LLM Eval Harness workshop.

Build a production eval harness live in 4 hours. Calibrated judges, error bars, CI gates.

Engineers who measure ship faster.

https://t.co/xzIQiz5adG

[Data For Science @data4sci]

Recording + notebook stay up after. Part 1 of 3 → full course Nov 18.

[Data For Science @data4sci]

Tomorrow, 12pm ET: Jev in One Call. 30 min, free, live on Zoom.

Meet Choice, Score, Noul on a real support ticket — a dozen questions in one pass.

Register: https://t.co/UY9kfEUusO

[Data For Science @data4sci]

If you need a framework to make your AI agent "safe," your agent isn't safe.

Safety is engineered into the system. Not imported from a package.

Curious what others think — am I wrong?

(Building this live on Oct 31 → https://t.co/IyzqO83Tpa)

[Data For Science @data4sci]

30 min, live on Zoom, recording + notebook yours after. Part 1 of 3 feeding the full bakeoff course, Nov 18.

[Data For Science @data4sci]

Free lesson — Wed Oct 21, 12pm ET.

Meet the 3 primitives of Jev on a real ticket. One ~100ms call, $0.042/1M tokens, output free.

Register: https://t.co/UY9kfEUusO

[Data For Science @data4sci]

4/ Problem 3: There's no retry logic, no cost ceiling, no audit trail.

One malformed query. One infinite loop. One $400 API bill later — you wish you'd built run supervision from the start.

I'm teaching this on Oct 31. 4 hours, hands-on, Python only.
→ https://t.co/IyzqO83Tpa

[Data For Science @data4sci]

1/ Most AI agents fail in production not because the model is bad — but because nobody engineered the system around it.

Here's what actually breaks (and how to fix it):

🧵

[Data For Science @data4sci]

3/ Problem 2: The context window fills up and the agent starts forgetting — silently.

Token budgets, memory tiers, and conversation compaction aren't optional. They're how you keep the agent from going off the rails 45 minutes into a session.

[Data For Science @data4sci]

I'm running a live 4-hour workshop on Oct 31: Build an AI Data Analyst you can actually trust — MCP tools, guardrails, context management, fault tolerance. Laptop-only, framework-free Python. 🎃

Details + tickets: https://t.co/IyzqO83Tpa

[Data For Science @data4sci]

4/
The rest of the harness does the same job for the other failure modes:
permissioned tools, context budgets, injection defense, retries, traces.

It's a 4-hour hands-on build, framework-free, on a laptop. Sat Oct 31 - 19 days out:

Register: https://t.co/IyzqO83Tpa

[Data For Science @data4sci]

1/
Deeper on the one failure that actually ships:
fabricated numbers.

How an agent reports a 12% revenue drop that never exists, and the exact guardrail that stops it -

[Data For Science @data4sci]

3/
Then it verifies:
- value matches the query output
- unit is right (not $12M reported as 12)
- reporting period is complete

LLM judging is reserved for clarity - not for 'is this number real.'

[Data For Science @data4sci]

19 days.

Sat Oct 31 - 4 hours, live - we build an AI data analyst you can actually trust, in plain Python. No GPU, no cloud, no API key.

$149.99, seats are limited.

Register: https://t.co/IyzqO83Tpa

[Data For Science @data4sci]

Last call — free LLM eval gate session is TOMORROW (Oct 7, 2 PM EDT).

Regression thresholds + GitHub Actions wiring + merge blocking, all in 30 min.

Register: https://t.co/7m9XF1CPD2

[Data For Science @data4sci]

Hot take: most of your AI agent's reliability problem isn't the model.

It's that you're letting the model pick its own tools and trust its own numbers.

The model should SUGGEST. A harness should DECIDE.

Am I wrong?

[Data For Science @data4sci]

https://t.co/7m9XF1CPD2

[Data For Science @data4sci]

Hot take: if your LLM app doesn’t have a CI eval gate, you don’t actually know if it works.

You know if it worked last time someone manually checked. That’s not the same thing.

[Data For Science @data4sci]

What does your team’s current LLM quality process look like? Be honest 👇

(Replying with a link to a free session on fixing this — drop your answer first)