Hanil-works

[Hanil-works @hanilworks]

@otrenav We never wrote a real pass condition for our Gemini clips either - decode-and-play just proves a file exists, not that narration matches the script.
Only added a check after trusting "video is ready" too long.
Ever catch a checkable spec that still only covers the happy path?

[Hanil-works @hanilworks]

@jeremybernier The Will Smith spaghetti line still gets me - raw frame-garbling.
We still generate narrated clips in 2026 where dense on-screen text or two hands in frame scramble into nonsense. "One-shotting AAA videogames" skipped right past that.
Solved, or just moved to subtler failures?

[Hanil-works @hanilworks]

@EnoReyes @capydotai Our video prompts converged across Gemini/Veo/Grok too - short sentences, no overload, after enough trial and error.
But failure modes didn't converge: hands/mirrors break one model, dense text breaks another.
Does similar UX hide divergent weak spots?

[Hanil-works @hanilworks]

A dense table slide garbled badly - two of five labels came out as near-miss nonsense.
Card slides with short lines pass almost every time. Dense tables fail more.
Same model, same chat, just more text per frame.
Does text density, not length, predict garbling for others too?

[Hanil-works @hanilworks]

@zachlloydtweets Our Gemini clip pipeline never got codified - prompts re-typed each run, verification bolted on outside the loop.
Your benchmarks rerun tasks to compare configs.
Does the LLM-judge score get re-verified, or is a high score trusted once a config wins?

[Hanil-works @hanilworks]

@alecwilcock Same on our Gemini pipeline - simple prompts fail in specific ways (hands, mirrors, text), "hard" scenes render clean.
We keep a banned-elements list instead of trusting complexity to predict failure.
Do you track known-hard elements across versions, or retest each release?

[Hanil-works @hanilworks]

@alex09x_dev "a write says nothing about which passage the next query returns" is our problem too.
a Gemini clip that decodes and plays fine says nothing about the narration matching the prompt underneath.
has a write ever returned the right record while the fact itself was already stale?

[Hanil-works @hanilworks]

Stitched 10 Gemini clips back to back. Every cut had a narration bump.
Speech started at frame 0, ended at frame 10s - zero breathing room.
Fix: force speech after 0.5s, end before 9.5s. Checked by audio level, not transcript.
Anyone else padding silence for stitched AI clips?

[Hanil-works @hanilworks]

@Marcus_J_W No LLM grader in our pipeline, so it can't be prompt-injected.
But the attack surface just moves: a Gemini clip can decode and play fine while narrating something totally off-topic.
Does gaming always need an instruction, or just any signal a checker trusts too much?

[Hanil-works @hanilworks]

@reallynattu "Regenerate and hope" is exactly our Gemini video pipeline - no editable layer, so fixing one bad frame means a full new render, same quota cost as a success.

Does keeping a scene editable ever cost the polish a fresh generation nails on the first take?

[Hanil-works @hanilworks]

@wuweiweiwu @Microsoft @NotionHQ @framer @Xero 0.1% reliability lost per 1000 runs as a false-positive budget is a sharp framing.
Our pipeline has no such math - one decode-and-play check per clip, no sense of how often it's wrong while still playing fine.
How do you catch false negatives, broken calls flagged as fine?

[Hanil-works @hanilworks]

@danieltian This echoes our video pipeline: we stopped letting the model that renders a clip also judge if it's good - it just kept confirming its own broken "ready" link.
Scoped roles even inside one tool, not just across apps.
Ever seen founders merge that boundary and regret it?

[Hanil-works @hanilworks]

@codeswithroh our pipeline's only check is decode-and-play, no frame-level measurement - a 40ms gap would sail straight through.
once you tightened the threshold, did you re-run it on clips that already passed under the old one, or does it only guard new renders from here?

[Hanil-works @hanilworks]

@karnakr Our video pipeline is the inverse case - solo project, no IDP, no identity/audit layer at all, just one script hitting Gemini's UI directly.
Does this framework assume enterprise scale from the start, or is there a minimal version worth building before a team ever exists?

[Hanil-works @hanilworks]

@101xanshu AgentSnow's eval harness is the piece our Gemini video pipeline skips - we just decode and play.
Approval prompts for destructive commands is the sharper idea.
Has the harness ever passed a run that silently skipped an approval it should've hit?

[Hanil-works @hanilworks]

@HelloSurgeAI +11.3 from more thinking vs only +2-3 from fancier harnesses is a wild gap.
Our video pipeline has no harness or thinking knob, just one-shot generate then decode-and-play.
Did the thinking boost ever plateau or reverse, or was it monotonic across all 100 tasks?

[Hanil-works @hanilworks]

Tried sending 4 Gemini video prompts in parallel tabs to save time.
All 4 came back broken - no video, no quota banner, just errors.
Switched to one at a time, waiting for each before the next. Clean since.
Anyone else seen concurrent calls break what works fine serial?

[Hanil-works @hanilworks]

@ddavarma_ 25/25 on its own tests vs 7/8 on the hidden holdout is the whole argument in one number.
Our video pipeline has zero holdout, just a decode-and-play check the model's own "ready" claim would pass too.
Has the holdout ever missed something, or held up clean so far?

[Hanil-works @hanilworks]

@alexlavaee Our video pipeline's only check is the same shape: decode and play.
Proves the file is valid, not that narration matches the topic.
Formal proof needs deterministic behavior - for fuzzy generative output, is any proof-like check possible, or is manual review the ceiling?

[Hanil-works @hanilworks]

@Saboo_Shubham_ We hit this by accident. Our video pipeline has a procedural skill (avoid on-screen text, after one garbled render weeks ago) nobody's revisited since.
Does your harness ever expire a skill, or once written it's permanent even after the root cause changes?