Jorks93

another day, another dollar....

[Jorks93 @criptoludada]

@ORamosBets Los que dicen y si empata yo creo que no han entendido nada

[Andrej Karpathy @karpathy]

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project.

This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.:

- It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work.
- It found that the Value Embeddings really like regularization and I wasn't applying any (oops).
- It found that my banded attention was too conservative (i forgot to tune it).
- It found that AdamW betas were all messed up.
- It tuned the weight decay schedule.
- It tuned the network initialization.

This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism.
https://t.co/WAz8aIztKT

All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges.

And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

[Jorks93 @criptoludada]

@SecretoDefi @MegaCorpHQ si si pero en brave tienes que quitar el bloqueo

[Jorks93 @criptoludada]

Job market is so cooked I just got a job piloting remote demolition drones for @megacorphq

[Jorks93 @criptoludada]

@SecretoDefi @MegaCorpHQ

[Jorks93 @criptoludada]

@mattlefun @solanamobile lfg

[MattleFun @mattlefun]

🚀 Seeker Game Boost!

Special rewards for @solanamobile users

All Seeker & Saga Genesis wallets can claim: Points, Energy, Mattle Keys, up to 500 $MATTLE, and PENGU Hero.

Drop your .skr wallet below and follow the steps 👇

[MattleFun @mattlefun]

MATTLE Public Sale – Official Launch

The Public Sale will officially start on October 6
Time: Oct 6, 8 AM UTC
Purchase range: 50 – 5000 USDC
Vesting: 100% unlocked at TGE

More 👇

[MattleFun @mattlefun]

Mattle Run Seeker Season

To amplify Seeker Season from Solana and celebrate our Grand Prize at the @SolanaMobile Hackathon, Mattle Run Season 6 is named Seeker Season.

Reward pool: $10,000 — the exact prize we won, now fully given back to the community!

Details👇

[AquaSave @AquaSaveCrypto]

🌍💧 AquaSave at #TOKEN2049 Singapore 🌍💧

Our CEO @sergio__aqua met with key investors and leaders in DePIN and Web3! The future of decentralized infrastructure and sustainable water management looks brighter than ever 🌊.

Exciting talks with potential ventures interested in AquaSave's blend of IoT, blockchain, and data monetization 💼. But that’s not all… something big is cooking 😏.

Stay tuned for a game-changing announcement coming soon! 🚀

#TOKEN2049 #DePIN #Web3 #IoT #Blockchain #AquaSave #Sustainability #AquaBank

[AquaSave @AquaSaveCrypto]

Invest in AquaBank with total confidence 🔒

Not only will you save water and energy, but you’ll also quickly recover your investment 💸

This is the first DePIN product that guarantees earnings with every use 📌

AquaBank is an investment in your future and the planet. Saving has never been this easy!💧
#SmartInvestment #AquaBankSaver #AquaSave #DePIN

[AquaSave @AquaSaveCrypto]

With AquaBank, saving isn’t just good for the planet—it’s good for you too! 🌍

Earn rewards in AQC$ for your sustainable consumption. 

Every time you use AquaBank, you’re one step closer to real benefits while protecting the environment. It’s a win-win for everyone! 💰

#AQCRewards #AquaBankSaver#AquaSave#DePIN

[AquaSave @AquaSaveCrypto]

Concerned about water quality in your home? 📊

With AquaBank, you can monitor quality levels directly from your smartphone📱

Get real-time notifications and ensure your family always has access to the purest water. Manage your home’s health with just a tap💧

#WaterQuality #AquaBankSaver #AquaSave

[AquaSave @AquaSaveCrypto]

Revolutionize your water savings with AquaSave! 🌊 

Discover the AquaBank, the device of the future designed to conserve water efficiently and sustainably.🚀

Click to learn how you can contribute to a greener planet! 🌍💧 

#SustainableInnovation #AquaSave #GreenTechnology

[AquaSave @AquaSaveCrypto]

We're glad to be finally on DePIN Sector Map from @MessariCrypto 🙌

#DePIN will be the leading sector in the bull run and we are more than proud to be part of this.

https://t.co/OuNojgxjic

[AquaSave @AquaSaveCrypto]

DePIN Hub Podcast 🎙️

Learn how you can earn crypto while saving water! 😱

Listen to our CEO Sergio Marquez now 👇

https://t.co/oqlr1tc6iL

[The Crypto Professor @TheCryptoProfes]

Decrypting Crypto - DePIN Roundtable

Looking forward to hosting my 1st official Decrypting Crypto Roundtable next Tuesday where I will be welcoming the following projects to discuss all things #DePIN:

@OVRtheReality
@ClusterProtocol
@Posemesh
@arkreen_network
@wicrypt
@AquaSaveCrypto

Make sure to like, share, and follow the projects involved as we bring education and utility to Twitter 🤝

Listen here: https://t.co/040q7wKR50

[AquaSave @AquaSaveCrypto]

Water scarcity doesn't just mean dry taps—it hits our wallets too. 💸 

It drives up costs in agriculture, disrupts industries, and makes daily life more expensive. 

From higher food prices to increased energy bills, the ripple effects are felt globally.

[AquaSave @AquaSaveCrypto]

Water scarcity is rising, affecting millions.

AquaSave offers a revolutionary solution, empowering you and your community to manage water resources smartly and sustainably.

Together, we can make a difference! 🌱