do we have any benchmarks to compare different harnesses?
like take N coding tasks, run the same model with the same prompts across claude code, codex-cli, omp, etc. and compare success rates and token usage?
looks simple, yet i don't remember seeing any benchmark.
@can1357 pdf preview seems broken on 0.9.23 (35c63ab). opening a pdf from the files section shows a corrupted render, and it’s also broken when opening a pdf in the tab view. screenshot attached
I first made this semantic font recommender using Carbon + Antigravity and then later updated it using Gemini’s Decision API.
Best believe it works! :D
Type in whatever you need and it’ll scan 1984 typefaces across Google Fonts to give you the best results. It’ll even extend out your search into deeper branches. Deceptively simple yet so much fun!
🔗 https://t.co/0ZnWWvJ6xc
do we have any benchmarks to compare different harnesses?
like take N coding tasks, run the same model with the same prompts across claude code, codex-cli, omp, etc. and compare success rates and token usage?
looks simple, yet i don't remember seeing any benchmark.
yeah i know these, but i haven't seen a comparison of different harnesses using the exact same model. like opus 5.5 running on claude code vs pi vs omp vs other harnesses, on the same tasks.
these benchmarks mostly use the default harness for each model and focus on comparing models rather than harnesses.
do we have any benchmarks to compare different harnesses?
like take N coding tasks, run the same model with the same prompts across claude code, codex-cli, omp, etc. and compare success rates and token usage?
looks simple, yet i don't remember seeing any benchmark.
day 2/∞
- attended my film art class. this week's film was The Wizard of Oz, and we discussed the hero's journey and related concepts. [p1]
- continued experiments for our CVPR paper, deadline is in a couple of weeks! [p2]
- slept from 8-11pm so i could drink monster & pull an all-nighter
will be here tomorrow
trying to be more active here, so i'll start sharing what i work on every day.
day 1/∞
- studied optimal control theory (especially unconstrained optimization part) [p1]
- benchmarked timings and some error metrics of STT models on NVIDIA Jetson Orin (part of my internship task) [p2]
- reviewed some papers for the kaggle 'name that llm' competition and made a few submissions
let's see how far this goes.
trying to be more active here, so i'll start sharing what i work on every day.
day 1/∞
- studied optimal control theory (especially unconstrained optimization part) [p1]
- benchmarked timings and some error metrics of STT models on NVIDIA Jetson Orin (part of my internship task) [p2]
- reviewed some papers for the kaggle 'name that llm' competition and made a few submissions
let's see how far this goes.