I also believe @SonicLabs/@SonicEcosystem will manage to construct a showcase example - like high visibility/ high virality meme of dapp - which will demonstrate capabilities of Sonic at large scale to massive audience.
… and this will change the world
Leading cryptos are “only” about 20-30% down …
$FTM/$S - much more because it’s a change of the chain.
There is and will be steady inflow of capital and transactions to @SonicLabs - which along with continuation of bull run should drive the price higher.
Excited to release new repo: nanochat!
(it's among the most unhinged I've written).
Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.
It weighs ~8,000 lines of imo quite clean code to:
- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.
Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.
My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.
Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.
Coinbase will add support for Sonic (S) on the Sonic network. Do not send this asset over other networks or your funds may be lost. Transfers for this asset are available on @Coinbase & @CoinbaseExch in the regions where trading is supported.
While we wait for Crypto to reset or find direction I'll be using that free time to help people that have lost it all, who want to start a business, and connecting real builders together.
I've rebranded my alt to @IncomeStarts to keep the content separate. Trading vs Building.
Some context for everyone…
Tech stocks are currently the most oversold since COVID, and only seven other times since 1988 have we seen more than 55% of stocks within the NASDAQ 100 trading with a 14-day RSI below 30:
August 1990 – Iraq War/Oil Spike
August 1998 – Russian Default/LTCM
September 2001 – 9/11
October 2008 – Lehman Brothers
August 2011 – US Debt Downgrade/Eurozone Debt Crisis
August 2015 – RMB Devaluation
March 2020 – COVID
This week, I am releasing a conversation with @AndreCronjeTech from @SonicLabs !
We go deep into:
- South Africa
- How he finished 3 years of school computer classes in 8 months
- Building Fantom
- How Yearn Finance started as a solution that Andre built for himself
- Dealing with FUD and Depression
- How it felt to receive 10 SEC letters
- Why Andre left Twitter for a while but never really left crypto
- Why Andre doesn’t believe anymore that people are good
- Coming back to bring Sonic to the next level
And much more
Can’t wait to share this one with ya’ll
Out on Thursday! 🚨
Algorand is bringing real-world assets on-chain.
✈️ Airline Tickets
💰 Currencies
📈 Stocks
⚽️ FIFA World Cup Tickets
🏡 Real Estate
☕ Coffee
🏦 Financial Assets
🏺 Historical Events
🖼️ Art
⚡ Solar Panels
🌊 Water Credits
🏅 Precious Metals
🎶 Music Royalties
🌍 Carbon Credits
🌾 Agriculture
...and more.
Algorand is powering real-world adoption, one asset at a time.