Andrew Saad

Founder & CEO @HealthTag | Making healthcare affordable in Egypt 🇪🇬
Digital medical records + discounts at 4,500+ providers , 4.2mn customers

[Andrew Saad @AndrewSaadG]

Very insightful take from @mhgaber
watch and learn everyone

[Mohamed Gaber @mhgaber]

Ever since Jev dropped, the timeline’s been full of decision-model talk.

Most of it is APIs and benchmarks.

We put one in the hardest place we could find: the hot path of a live voice call, on a 400 ms budget.

The goal wasn’t to fix something broken. It was to take something that already works well and push it further.

Baseline
Knowledge-base retrieval in https://t.co/tCR3VSgTaj runs the industry default: BM25 + embeddings, RRF, and a freshness multiplier. No model calls in the hot path. It’s fast, and the right answer is already #1 89.3% of the time.

A lot of teams would stop there.

But a voice call has no results page. One answer gets spoken. So we went after the last mile.

The ceiling
Similarity-based retrieval has a known limit:

It measures “which entry is closest?”, not “which entry contains the answer?”
“What’s in the Gold plan” and “How much is the Gold plan” are almost the same words. Only one has the price.
RRF ranks by position, not score. The top 10 often sit within 13% of each other, so a tie-breaker like freshness gets more weight than it deserves.
There’s no calibrated “I don’t know”. The best heuristic (“if the top score is low, there’s no answer”) only reaches an AUROC of 0.74.
The fix
Let retrieval do what it’s good at. Put a judge on top.
Retrieval returns the top 8 candidates. The judge reads each one and rules on it.

Our judge is Cloudflare’s open-source Clef (Apache 2.0, Jev-API compatible). Because the weights are open, it runs on our own GPUs. The customer’s knowledge base never leaves our infrastructure.

The question

We ask a Score with four levels:
0 → different subject

1 → same topic, but doesn’t answer (the price, the condition, the date)
2 → partially answers
3 → directly answers
Level 1 is the whole story. It gives the near-miss a name: right neighborhood, wrong house.
Tip: describe each level in a full sentence, not just a number. The judgments get noticeably sharper.

Reordering

The reorder is deliberately boring and deterministic:
2 and 3 first → then 1 (and anything not judged in time) in original search order → then 0 at the bottom.

Demote, never remove. Freshness is now just a tie-breaker.

Answerable is tri-state: true / false / null. The false is what lets the agent say “let me check and get back to you” instead of reading out the closest thing it found.

Hot-path rules we don’t bend

One 400 ms deadline for all 8 judgments together. Late ones get dropped. No retries. If nothing comes back, we fall back to plain search order. A slow judge never stalls a call.
Allowlist, not denylist. Only 4 fields go to the judge. Anything added later stays private until someone explicitly decides otherwise.
off / shadow / live per agent. Shadow judges after the response is already sent and logs what it would have changed — production latency and accuracy with zero risk.
A status on every query: disabled / shadow / applied / partial / timeout / error. A degraded judge should be visible, never silent.
Eval (we don’t ship on vibes)
Questions that sound like real callers: Egyptian Arabic, formal Arabic, English, mixed, broken voice-to-text, plus questions the KB genuinely can’t answer.
Labels from a different model so the judge isn’t marking its own homework.
Ablation: the same pipeline without the judge matched the baseline. The gain is all the judge.
Paired + McNemar: the odds the gap is chance are about one in a million.
Results
Top-1: 89.3% → 97.5% (11 questions improved for every 1 that got worse)
English questions on Arabic content: 81.7% → 98.3%
Questions about a specific detail: 83.3% → 95%
“I don’t know”: AUROC 0.74 → 0.99
Because the probabilities are calibrated, the threshold becomes a business decision, not a guess.
At 0.5 it catches 96.4% of unanswerable questions while wrongly flagging only 3.3%.
If a wrong “I don’t know” costs more, 0.2 drops that to 1.9%.

The best part
“I don’t know” became a backlog for the knowledge base.

Every unanswered question goes to a background worker that:
masks personal numbers

re-judges anything that missed the deadline

classifies it (off-topic, needs live data, unclear, or genuinely missing knowledge)
looks deeper than the top 8
groups questions that ask for the same thing
The trap
The same question in Arabic and English scored 0.22–0.43 similarity.
Two different questions on the same topic scored 0.72.
Any similarity threshold works exactly backwards.
So embeddings only build a shortlist. The decision model decides “is this the same question?”. It correctly separated Velents’ WhatsApp number from Velents Pay’s.
The final knowledge-base suggestion is never allowed to invent a fact — not in the prompt, in the code. A knowledge base that makes up its own prices is worse than one with gaps.

All of it runs on a dedicated thread with a bounded queue, not the framework’s background tasks (those share a pool with retrieval, and nobody wants gap analysis delaying a call).

Takeaway

Retrieval is great at finding candidates.
A decision model is great at choosing between near-misses.

Different jobs.

Generation is a commodity.
The decision inside the hot path is the real engineering.

[Andrew Saad @AndrewSaadG]

@mhgaber cause it’s global grade take ya Gaber it should be written in English as well

[Andrew Saad @AndrewSaadG]

@TheOnlyWarman سافرت منك دي 🤝

[Andrew Saad @AndrewSaadG]

I’m not a developer myself
But I have a hunch that if models somehow started reading or reasoning at the end of a prompt instead of the first it will be much more efficient and faster!

[Alexandr Wang @alexandr_wang]

$100k to the developer who can make muse spark open fastest!

[Andrew Saad @AndrewSaadG]

@Perfekt_Shawn Tony Stark was able to build this in a Cave with box of scraps!

[Andrew Saad @AndrewSaadG]

One thing i fear in ai age , we lose names to models and start calling ourselves with numbers

[Andrew Saad @AndrewSaadG]

@dara_venture that struck a nerve … ykw a nervous system!

[Andrew Saad @AndrewSaadG]

I will lease a cinema would you invest @NaguibSawiris ? , Hello

[Andrew Saad @AndrewSaadG]

Match in public is the new thing
long dead build in public
Go on bro. 🤞🏼

[Andrew Tan @andrewztan]

guys i'm so in

[Andrew Saad @AndrewSaadG]

Low-hanging fruits are mostly rotten, Stretch!

[Andrew Saad @AndrewSaadG]

Low-hanging fruits are mostly rotten, Stretch!

[Andrew Saad @AndrewSaadG]

@MagnusGrimeland urgent

[Andrew Saad @AndrewSaadG]

Will agents inherit our shortcomings as their creator !
or will they evolve to pull us and our behaviors to near perfection efficiency?
Patrick’s POV is quite insightful

[Patrick Collison @patrickc]

Assorted things I've been mulling over regarding agents in the economy:

• Companies are in some sense the original superintelligence. It's often the case that they can use their amortized cognitive surplus to advance their interests against those of certain consumers: relative to the optimization capacity of companies, consumers can be inattentive, underinformed, irrational, etc. (Think, say, of some of the classic insurance upsells at the end of a booking process, which is typically priced substantially above what a regular policy would cost.) Herbert Simon introduced the idea of bounded rationality, and firms often end up benefiting from it. Thanks to personal agents, we will probably get somewhat more rational in how we engage with companies.

• How exactly will outcomes change? Well, certain kinds of price discrimination will work less well (how do coupons work if every agent hunts fastidiously for them?), and certain kinds of customer cross-subsidy will change (today, people who forget to cancel subscriptions subsidize those who don't). Taken across the economy as a whole, I'm not sure what the incidence of all of this will be.

• In corporate strategy, people have long debated the relative importance of distribution and product quality. Perhaps you've made something better, but does anyone know about it? Won't people just continue to buy the ACME Corp Mousetraps? Personal agents will, I think, act as a kind of structural subsidy for product quality. How would markets look different if individuals always spent at least 10 hours researching their purchases? On the margin, focusing on making a product better is, I think, going to become a more effective strategy for companies. This is good!

• It true, a corollary is that information about product quality will become more important and impactful. Today, most people have very limited ability to aggregate and integrate such data, but agents will have insatiable thirst for such signals. (It's often said that data is becoming more valuable because it helps in the training of models. That's obviously true, but I think there's a secondary way in which it's more valuable simply as a complement to cheaper intelligence.)

• A big question is whether agents will increase superstar effects (rational agents all settle on the same product) or increase dispersion (because everyone has slightly different tastes and preferences, which agents are good at eliciting, finally overcoming the artificial clustering we've been subjected to).

• Many internet companies have monetized demand routing, which is in turn often predicated on short attention spans. "I'll click on the first thing I see." This has led to some amount of inefficiency, as companies try to propitiate capricious ranking algorithms and incur various placement fees. It's not all deadweight loss (willingness to pay is itself a kind of signal, which is valuable to buyers), but it's not great. How will the nature and value of this routing change in a world of personal agents? I don't know the answer, but I think that the effects might be pretty big.

• Much of what I'm writing hinges on the assumption that personal agents will be on the consumer's side. I suspect that that will be the winning strategy, but whether it's true, and what it means to be true, involves a lot of other questions.

Taken as a whole, the market is a massively parallelized, decentralized, and highly imperfect value attribution machine, and, taking stock of all of the effects, it seems to me that the quality point may well be the biggest: that agents will change the nature of the backpropagation such that the rewards for better products increase. Maybe this is too optimistic, but I suspect (and hope) that companies that make things that they're proud of will feel that the universe is a little more partisan in their favor.

[Andrew Saad @AndrewSaadG]

hear me out !
I now listen to music in a lossless format , headphones in
But I feel like I'm on stage and my ears are within the singer mic ... I hear music behind me!

it's not like i stand in front of them! am I alone in this?

[Andrew Saad @AndrewSaadG]

“Died by suicide”

[Simon Andriesz @SAndriesz]

I am the British whistleblower who reported evidence concerning Howard Lutnick, his links to Jeffrey Epstein, and Cantor Fitzgerald and BGC to the FBI in 2020–21.

Listen to the BBC File on 4 documentary about my whistleblower story:

https://t.co/rLRDnZA4Dv

BBC News:

https://t.co/DdNNOosnKL

I later shared evidence with the House Oversight Committee, which subsequently questioned Lutnick behind closed doors.

My disclosures also contributed to regulatory action in the US and UK, including a £96m HMRC tax assessment relating to BGC’s partnership scheme. I believe the true tax exposure is substantially higher.

I also reported to the FBI and financial regulators matters involving Cantor Fitzgerald’s Relief Fund and BGC/Cantor’s annual 9/11 Charity Day.

The personal consequences of whistleblowing, and the retaliation I say followed, have been devastating for my family and me.

Mother Jones previously reported that documents I uncovered revealed a years-long business tie between Howard Lutnick, Cantor Fitzgerald and Jeffrey Epstein through AdFin, including a 2018 exchange in which Lutnick and Epstein discussed the company directly:

https://t.co/aRdn302v4i
https://t.co/aRdn302v4i

Follow me here on X and on LinkedIn. I intend to share more of the evidence and the story behind what happened.

[Andrew Saad @AndrewSaadG]

@yasser_elsaid_ direct flights from cairo 🤔

[Andrew Saad @AndrewSaadG]

انا حاسس ان الناس دي بتستخدم اسمك و مش تبعك يا باشمهندس @NaguibSawiris

[Andrew Saad @AndrewSaadG]

It’s kinda interesting how Ai are getting human names and humans are getting user IDs!

[Andrew Saad @AndrewSaadG]

@alexandr_wang We still can’t access muse in egypt !

[Andrew Saad @AndrewSaadG]

@coryalthoff agriculture in egypt bot startup but boring business