mekaneeky - e/acc

Building Decentralized AI @bitsota_ai

[mekaneeky - e/acc @GPLv6]

I have been looking at ways to run Trillion scale models using consumer devices in a distributed manner. I was looking at low-rank iterative distillation as a way to allow lower memory/compute devices to run/train large scale models. 1/n

[mekaneeky - e/acc @GPLv6]

@0xSero Looks like merging is an interesting direction. @SakanaAILabs proved this on a smaller example before, but apparently it scales

[mekaneeky - e/acc @GPLv6]

@TheAhmadOsman Also dataset and RL environment quality is an issue. GIGO.

[mekaneeky - e/acc @GPLv6]

@TheAhmadOsman It isn't if you do decentralized pretraining and use diloco like approaches. But I think with a different focus (composability, hard routing, embarrassingly parallel inference and PEFT) and some R&D it should be feasible.

[mekaneeky - e/acc @GPLv6]

@ai_bond_connery To be fair we need a frontier model coming out of bittensor asap.

[mekaneeky - e/acc @GPLv6]

@ericwang1215 @_0xghost_ @TheseusNetwork You think it can deliver frontier grade AI and not $TAO? How come?

[mekaneeky - e/acc @GPLv6]

Fuck anthropic Fuck fearmongering
Fuck censorship Lets build an open source alternative https://t.co/IJ11O7wQv5

[mekaneeky - e/acc @GPLv6]

https://t.co/IJ11O7wQv5

[mekaneeky - e/acc @GPLv6]

My attempts to fix my dopamine/brain by avoiding twitter are on break to bitch about anthropic and highlight that we've been building decentralized AI on bitsota ! Mine with us https://t.co/kBAX693ABX or buy our token https://t.co/h4rKuxH2hX

[mekaneeky - e/acc @GPLv6]

@Teknium https://t.co/tl58OoSsUX !

[mekaneeky - e/acc @GPLv6]

@Teknium You can participate in decentralized autoresearch on https://t.co/uqmKcv9Hii using hermes

We're currently trying to find the best binary and ternary quantized model up to 27b size

With plug and play skills ! https://t.co/9UiFJjEMOX

[mekaneeky - e/acc @GPLv6]

@skirano openAI did that when they didn't share the weights of GPT 3.5. This is extra though !

[mekaneeky - e/acc @GPLv6]

@DavidSacks Note to self: Never do AI in the US.

[mekaneeky - e/acc @GPLv6]

@karpathy ...

[mekaneeky - e/acc @GPLv6]

We are doing similar work as well on @bitsota_ai ! We were actually inspired by your implementation in how we ended up doing ours. We were using Automl zero like evolutionary search before @karpathy's tweet highlighted that the days of non-LLM Automl are over then we switched to LLMs. Happy to connect.

[Kat ⊷ the Poet Engineer @poetengineer__]

been working on moonfish, an obsidian writing companion. i see it as a small experiment in ambient/calm computing:
keystrokes ripple a pond; sentences become fish; moons keep the weeks writing.

[Teknium 🪽 @Teknium]

I literally run 12 hermes agent instances every day in parallel to build Hermes Agent, and its now a top 100 GitHub repositories of all time. Agents do bring value and do create substantive software and work.

[mekaneeky - e/acc @GPLv6]

@tongzhi_mate @ChunCuiJingYan @areyouawitchor2 So "formalizing" it would be having more things taxed, banked and "registered"? How would that fix anything?

[mekaneeky - e/acc @GPLv6]

@d0rc EGGROLL?

[mekaneeky - e/acc @GPLv6]

@SimoneSyed @DarioAmodei Is TFR always expected to be stable or grow? Can we not solve for X? What is the best X population count for meeting civilizational goals for community Y over a time frame of N years, including quality of life metrics, GDP/capita, scientific breakthroughs, galactic colonies, etc