Kappa run is now stopped, not too bad overall! We passed llama 3.2 1b on ARC-C, OBQA, TQA, etc., nearly matched on ARC-E, SciQ, etc. Considering that was 9T tokens and we hit those numbers on the 576B checkpoint, AND we did so around 90% cheaper per token, I'd call it a solid preliminary win!
Issues with the run:
1. In one of the Gated DeltaNet-2 heads, the per-channel decay gate underflowed, causing state wipe, and ourput RMSNorm amplified near-zero outputs ~1000x into fleet wide grad spikes.
2. Very late/slow nodes were so far behind their evidence was discarded, not a huge effect but we should have had better tolerance for stale data and partial updates.
3. We packed samples but didn't do masking/etc. which is sub-optimal for pretraining (not a huge diff but worth fixing anyways).
4. The first and last layers being GDN2 may be suboptimal also, vs. SWA, no clear evidence but might be worth switching up the order a bit for those two.
We have a fix for this GDN2 issue, but while that was still running I also tested EDA (https://t.co/2baRkRGnDA) and it actually addresses some of the root cause directly (and seems to have better loss than GDN2 anyways) so we'll probably switch to EDA once all tests come back clean and pass the kappa's instability regime (and waiting until we have highly optimized tilelang kernels for EDA to keep MFU/throughput high).
Onwards and upwards 🚀
@Axel_bitblaze69 A brilliant aspect of Zama's design is that liquidity doesn't need to be bridged to an entirely new chain, it stays right where it is. This makes it effortless for users to interact with and seamless for DeFi apps to integrate
Privacy should be a feature available on every chain. If you build a chain focused solely on privacy and expect people to move their capital there, it’s going to be very difficult.
That’s also why most L2s struggle to survive, with exceptions like Robinhood and Base, L2s backed by major exchanges.
If you want a truly programmable blockchain, then to me, it’s ethereum:0xa27ec0006e59f245217ff08cd52a7e8b169e62d2 , not Starknet (ethereum:0xca14007eff0db1f8135f4c25b34de49ab0d42766 ).
And if you’re looking for a privacy layer on EVM, leveraging the existing EVM ecosystem and liquidity, then the answer is solana:4Zp52aF4hZi9fzH19xpbWKYKQvgLyCN67KFbrQDqeTKh .
bought $740k of starknet this week (yes, seriously).
here's the thinking, and i'll say up front that this is one of the grossest charts i've ever seen and makes me want to throw up.
it's down about 99% from the launch highs and it's been down only for 2.5 years. i'd guess 99 out of 100 people who know what it is, hate it, and based on the pa i don't blame them at all. price leads narrative, so when a chart looks like that nobody has anything nice to say about it.
that's kind of why i'm here.
chart starting to look bulla:
i'm seeing big bullish div on the weekly as well as rsi back above 50 and is currently retesting. price is curling up and building a base. to me that has the makings of a long term major bottom.
narrative: "programmable zcash"
zcash led this cycle and near ran after it as the next privacy play, and i think starknet could be third in line. there are deep connections between zcash cofounders and the starknet team. starknet shipped native privacy in june, so you can shield basically any token on the chain, and there's a private version of bitcoin on there too. the proofs it runs on are quantum resistant (nice buzzword for the future).
there's also a proposal called ztarknet to build a starknet style layer 2 directly on top of zcash. so the simple version is it is "programmable zcash", zcash's little brother sitting at 315m mcap.
this reminds me a lot of near. i bought about $2m of near around a $1.40 back in march and april. literally no one was talking about it, let alone buying it and most of my replies were some version of:
"why near? it's a 2021 shitcoin that's down only, you're throwing your money away"
now everyone loves near. the chart then looked a lot like the strk chart does now. it went on to roughly 3x from there in 6 months. too bad i didn't hold it. i got shaken out, which was bad execution on my part, but the thesis played out.
the big difference is that near had intents, a product people actually used and adds value. starknet doesn't have that yet. hardly anyone uses the chain, revenue is basically nothing, and there's no app people care about. but these guys are well funded and seem to be shipping new things.
where i could see a real product gain traction is private trading. on hyperliquid every wallet and every trade is public, so people get copy traded, whales get hunted, wallets get flagged. i think there's a market for a dex where you keep your own keys and nobody can see your positions. some of that is already live on starknet, private swaps and private perps, it just doesn't have users yet. and every private transaction costs 4 strk, so if that ever gets real usage the token is right in the path of it. i also played around with an options app on starknet called supervega for a few minutes. haven't placed a trade but the ui was clean.
the token unlocks have been the main reason to hate the coin. 127m tokens a month. here's the good news, there are only 6 unlocks left and the last one is march of 2027. each one is around $5.5m at current prices and about 74% of supply is already out. the september one got absorbed without a problem. feels to me like the sellers are mostly done. the market usually front runs these things.
last thing. if this does get going it has the potential to be a hated rally, and i kind of like that. a lot of people will want to fade it or short it because of what it did to them, and that kind of thing gets a coin talked about and can push a move further than it would normally go.
the risks are obvious (mainly, no real usage).
time will tell if i have gone full schizo, or if this will be a nice play.
we'll see.
@Franacc_ The most important thing in crypto is liquidity. Any project that has liquidity will win, and any L2 except those backed by big CEXs will die
Most AI runs in a datacenter, and someone else holds the off switch.
At Exploit Summit (@ExploitSummit ), Jon Durbin showed Parallax running on a phone. CPU only, no GPU. The trained 8B model decoded at 59.6 tokens per second. A 40.75B model ran at 26.9 tokens per second with 12 GB peak process memory.
Where that leads:
• You could use AI in a country that blocks it.
• A doctor could run a model on a tablet in the exam room, and no patient data leaves the building.
• A law firm could keep one on a laptop in a locked office and never worry about prompts leaking to a vendor.
• You could use it on a plane or through an outage.
• No monthly bill and no rate limits. No company deciding tomorrow that your use case violates its policy.
A datacenter has an off switch. A billion personal devices don't.
Training and inference will run on the edge, by miners, myopically seeking energy wells like they do with Bitcoin mining.
@jon_durbin presenting the future and how humanity gets there.
Introducing Zakura Common.
We just shipped 14x faster proving, 21x faster hashing, 1.5x faster trial decryption, and 4-8x faster verification.
New releases of Zakura and wallets that use Zakura Common (such as @vizorwallet) will benefit immediately.
https://t.co/Xp02qyJXhk