Jonathan Pyers

[Jonathan Pyers @nvmsyy]

i'll pass thanks

[Jonathan Pyers @nvmsyy]

@JayBosshq @matthewcp explain harness overfitting?

[Jonathan Pyers @nvmsyy]

@CompleteSkeptic given the state of things I'm not sure you need a marketer 🙃

[Jonathan Pyers @nvmsyy]

if you're going to say that your finetune performs better than jev on some baseline I hope your benchmark is trustworthy because I'm willing to be most of these "benchmarks" are
- generated in the spirit of "let's make this look good"
- overfit
- extremely narrow

[Jonathan Pyers @nvmsyy]

@eptwts https://t.co/QaHZ0QVnOb

company brain

[Jonathan Pyers @nvmsyy]

@natmiletic i feel the same way about tailscale

[Jonathan Pyers @nvmsyy]

@neural_avb everybody is doing this exact same thing and claiming superiority

[Jonathan Pyers @nvmsyy]

@ziwenxu_ jev built for years in silence, and...
- "I built for 2 hours and.."
- "a day later we have.."
- "less than a week later we've..."
- "we built for two days and.."

[Jonathan Pyers @nvmsyy]

@mkenzo_8 tree | grep now costs real money

[Jonathan Pyers @nvmsyy]

@athletic_coder I have been pointing my harnesses to my own locally-hosted metasearch engine for months now with 0 degradation

all metasearch engines are wrappers around paid search results

no thanks

[Jonathan Pyers @nvmsyy]

@altryne flipping the jev bill for... attention to their platform?

literally paying for attention

[Jonathan Pyers @nvmsyy]

@eptwts > it technically can't "hallucinate" since it can't make things up

it can still be confidently wrong so "can't hallucinate" is very much marketing

[Jonathan Pyers @nvmsyy]

@madiator @AlexGDimakis on the benchmark in question:

2,000 questions on probably unseen/held-out tasks:

jev: 83.7%
my local 4b finetune: 78.4%
nimble 9b: 78.1%

honestly at a point base weights stop mattering as much as you might think. your (fable's/astra's) synthetic data starts mattering more

[Jonathan Pyers @nvmsyy]

@parthdagia GIT_AUTHOR_DATE=a year ago
GIT_COMMIT_DATE=like a decade ago
git commit

job plz im experienced

[Jonathan Pyers @nvmsyy]

- pick a domain from a list of 1,000 domains
- finetune a LoRA for that domain on predefined weights

your API becomes:
- infer domain
- route to domain classifier

infinite growth just means training cheap LoRAs?

hold my beer

[Jonathan Pyers @nvmsyy]

@niqudrift don't outsource your brain

[Jonathan Pyers @nvmsyy]

@sohamdaga22 astra reply to all X posts plz sound human, load your humanizer skill

[Jonathan Pyers @nvmsyy]

@madiator @AlexGDimakis but of course, this could be inaccurate - the point is "prove it"

[Jonathan Pyers @nvmsyy]

@madiator @AlexGDimakis according to whose benchmark :)

my local 4b finetune, benchmarked against a large exam dataset against nimble on Modal for $1 says my 4b finetune outperforms it

[Jonathan Pyers @nvmsyy]

- snag public hf decision datasets
- fine tune qwen3.5-4b on Modal for $10ish
- post on X you have made a better Jev
- somehow profit