Sami Kuikka

AI Engineer at Bind.
Sharing on things I'm working on.
中文流利.

[Sami Kuikka @SamiKuikui]

@ibocodes Give confidence signal to the LLM-as-judge results.

[Sami Kuikka @SamiKuikui]

@holdenmatt Wrote blog about it. I wouldn't replace, but as additional information it's super useful https://t.co/aAztOxx7y1

[Sami Kuikka @SamiKuikui]

Jev (@typesafeai ) not only can be used as judge, but it's confidence signal can be used to find eval problems.

Don't just throw away your LLM-as-judge results to Jev, but make Jev give additional information for your evals:

BLOG: https://t.co/aAztOxx7y1

[Sami Kuikka @SamiKuikui]

Found totally awesome Jev use-case that changes how I think about evals:

TLDR: Use Jev in LLM-as-judge to give confidence on the PASS/FAIL answer.

I'll try to gather my thought on this to blog, but I recommend others to try. Early evidence very strong. @typesafeai

[Sami Kuikka @SamiKuikui]

@xeophon Thank you for this research!

[Sami Kuikka @SamiKuikui]

@thdxr I like to read it Jeff

[Sami Kuikka @SamiKuikui]

GLM 5.3 > Opus 5.
Maybe not in benchmarks,
but I rather read GLM 5.3 output than Opus 5 explanations.
Maybe need to add some instructions on the output format for Claude models, so hard to read

[Sami Kuikka @SamiKuikui]

@PegZor Gatekeeping agents 🥲

[Sami Kuikka @SamiKuikui]

Some thought on the principles of loop engineering in new blog: https://t.co/DZhqoZBtpi

Better model helps on agent autonomy, but designing your environment for the looping agents is at least as much important.

[Sami Kuikka @SamiKuikui]

@zainhas Should have also tested on their official harness, zcode

[Sami Kuikka @SamiKuikui]

Check it out here

https://t.co/yOY0o5jjGJ

[Sami Kuikka @SamiKuikui]

You changed one prompt. Now you're manually re-testing everything.

Write down what your agent must be capable of as tests. Then let your coding agent improve the system in loop: real agent runs, pass or fail with evidence, improve, repeat.

https://t.co/rISeThhfMq

[Sami Kuikka @SamiKuikui]

Code --> Hit limit --> Reset --> Code

This week just feels like there's new reset everytime I hit the limit in @zcode_ai .

[Sami Kuikka @SamiKuikui]

@ZixuanLi_ Make it learn from tool errors. When flash uses tool and the tool responds woth error message on what went wrong, currently it just retries the same command over an over again, not learning from previous mistakes.

[Sami Kuikka @SamiKuikui]

@EXM7777 Nothing beats GLM legacy plan

[Sami Kuikka @SamiKuikui]

@buildwithhassan How to game it even more:

- Use zcode for additional 50% more tokens in the 5 hour window
- Use outside of peak hours
- Heavy use of idle-tasks for free tokens.
- zcode for usage resets (Have used alresdy 2 times last week, it's really frequent)

[Sami Kuikka @SamiKuikui]

@buildwithhassan V1 legacy plan best money I have spent. At the moment tho I was hesitant.

[Sami Kuikka @SamiKuikui]

Thank you @zcode_ai ! 😎

[Sami Kuikka @SamiKuikui]

Loop engineering in the harness?

Actually possible!

Creating smart interfaces for the agent to run your system, trace your system. judge your system, and modify your system, all through CLI.

https://t.co/FH4PN2hNB5

[Sami Kuikka @SamiKuikui]

Giving my blog mock agents :D

Fun!