Calaveras AI

Over 100 B novel code tokens and realistic+robust RL envs

[Calaveras AI @CalaverasAI]

come hunt tokens with us

[near @nearcyan]

My job? I'm a rare token hunter. I track down dead languages in Tibetan monasteries, decrypt Tesla's private journals, chase whispers of pre-contact Amazonian dialects. The AIs pay top credit for tokens they've never tasted, you know. Work is work, even if it's for the machines.

[Calaveras AI @CalaverasAI]

@TimFarrelly8 wouldnt u like to know 😉

[Calaveras AI @CalaverasAI]

https://t.co/Xk3ru7g3xb

[Calaveras AI @CalaverasAI]

@avaitopiper so true

[Calaveras AI @CalaverasAI]

there is a lot of alpha in actually looking at the data

[Epoch AI @EpochAIResearch]

We looked at OSWorld, a popular evaluation of AI computer use capabilities.

Our findings: tasks are simple, many don't require GUIs, and success often hinges on interpreting ambiguous instructions. The benchmark is also not stable over time.

See thread for details!

[Calaveras AI @CalaverasAI]

making a new dataset with 6 - 7 trillion tokens

[Calaveras AI @CalaverasAI]

Beware narratives!

[Miles Brundage @Miles_Brundage]

Seems like the whole "RL just surfaces intelligence, it doesn't increase it" series of papers is just an artifact of RL being a small fraction of compute in most LM contexts still, no?

AlphaGo (etc.) shows quite clearly that there is nothing to this as a general matter

[Calaveras AI @CalaverasAI]

@mattfreed @cis_female is working on this

[Calaveras AI @CalaverasAI]

@cuijiaxun Hi! Let's chat :)

[Calaveras AI @CalaverasAI]

@uzpg_ @KaivuHariharan Let's meet!

[Calaveras AI @CalaverasAI]

Startups refusing to pay competitively for top engineers when they have the ability to are just punching themselves in the face over and over again

[Calaveras AI @CalaverasAI]

@nikolaj2030 @Miles_Brundage it's unreal how much better the Inspect codebase is than other open source evals frameworks

[Calaveras AI @CalaverasAI]

If you want top talent, you need to be willing to pay for it

[Calaveras AI @CalaverasAI]

@e869120 ❤️❤️

[Calaveras AI @CalaverasAI]

@avaitopiper @joodalooped thanks bhavit :)

[Calaveras AI @CalaverasAI]

How's that for a makeover?

Thanks @joodalooped

Check us out at calaveras dot ai

[Calaveras AI @CalaverasAI]

@tkskri_kypr おめでとうございます！もしAI業界をリードする企業でRL環境の未来を築くことに興味がある方がいらっしゃいましたら、DMをください。ランチをご馳走させていただきます ^_^

[Calaveras AI @CalaverasAI]

@leilavclark @joodalooped thanks leila :)

[Calaveras AI @CalaverasAI]

I've been interviewing so many impressive programmers who deeply understand the systems they work with, are driven, and have grit.

We work on hard problems, pay good comp, and have an exciting mission.

If you think you'd like it here, please DM me!

[Zy @ZyMazza]

U should be required to poast the salary you’re offering when you say stuff like this