I'm one of the people most directly responsible for this outage. I take Sui outages extremely personally and am incredibly disappointed that this happened.
I'll echo what @EvanWeb3 said: Moving fast is no excuse for outages. This outage was caused primarily by an edge case in gas smashing introduced by address balances. However, we did not ship this feature carelessly. We spent over 6 months testing address balances and tried as hard as we could to exercise every edge case and combination of features we could think of. Unfortunately, we were bitten by an edge case that we did not discover during testing.
Again, this is not an excuse - it shows that our testing strategy was inadequate, despite how much effort we put into it. We did not have a systematic way to enumerate or discover all possible edge cases. We will be re-examining our approach in light of this. But I do want to emphasize that we did not ship this feature carelessly or in a rush. We made mistakes, but we were absolutely not operating in "move fast and break things" mode.
One more thing: Address Balances is by far the biggest change to Sui's execution model that we have shipped, and we don't have any plans to ship anything this large and disruptive again. It was a necessary change to make Sui into what it should be, but we don't seek out huge disruptive changes like this for the sake of it.
Now that the feature is out we will be redoubling our efforts to make Sui the most reliable L1. We have a lot of work to do, I'm excited to start on it!
Moving fast is not a valid excuse for the outages. They are unacceptable, period. You have my sincerest apologies for the outages.
We are focused on restoring the level of reliability and security to the absolute maximum.
@Lidlmedewerker1@EvanWeb3 Confidential transactions are implemented entirely in Move and don't require any core level changes, so it's highly unlikely they will introduce any instability.
@MrBreadSmith@kucoincom@BC_KuCoin@0xd34th Hey there - the funds are not lost. If you look at the transaction in an explorer (https://t.co/M0ArHK9EzW) you will see the transfer, and if you look at the destination account you will see the funds. Once KuCoin fixes their issue you will be able to see the funds
Even ignoring the formal verification angle, "AI is good at finding bugs so software will always be insecure" is fundamentally a very weird idea that stops making sense if you think about it for 2 minutes
Many people have claimed that with AI-assisted bug finding, secure code (and hence trustless anything) will be impossible.
I have a much more optimistic take, and AI-assisted formal verification is a major part of the reason why:
https://t.co/0ceMBZ6uqj
Postel's Law (Be conservative in what you emit, liberal in what you accept) is bad advice. It's infinitely harder to make your software stricter in the future than to make it looser, so start out as strict as you possible can be.
Something people don't understand about LLMs and security: It's not an arms race. Once you fix all the bugs, you win. Defender has the advantage! There is no AI model powerful enough to find bugs that don't exist.
We've been working on a more scalable read and write layer for @SuiNetwork. Our solution, codenamed Hikari, is a near-parity drop-in replacement for Sui gRPC nodes, and unsupported requests automatically fall back to a full node.
On the read side, we designed a custom storage backend that's fronted by multiple layers of caching, each serving a unique purpose. In our benchmarks for GetCheckpoint and GetTransaction requests conducted in 5 regions around the world, we're seeing the following p90 results:
- Baseline Full Node: 400ms
- Hikari (Origin): 54ms
- Hikari (L3 Cache): 42ms
- Hikari (L2 cache): 37ms
- Hikari (L1 cache): 28ms
- Hikari (L0 cache): 20ms
That's a 22x improvement in latency.
For writes (transaction submission), we implemented a lightweight service that's edge-deployable. Furthermore, unlike full nodes, Hikari can be horizontally scaled up in seconds. We benchmarked Hikari against https://t.co/1JpUq6KBIj across 5 regions, and achieved a 15-25% faster p90.
We are working closely with the @CodaNetwork team to pilot Hikari. After a successful testing period, we will make Hikari available for the broader ecosystem.
@FFmpeg The real problem with GC is that it is non-local. If you have code that is slow because of calling malloc(), you can remove the malloc(). If you have GC pauses, the cause of that is the _entire rest of the program_
I love that I can't tell if in advance if the socialist position is going to ban the chain grocery stores from opening or ban them from closing. Could be either. Could be both!