𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛

Anarchist • Coder • Cypherpunk • Free Software • PGP 0x304952407A6E5C38 • I use Arch, btw.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

My little contribution to Bitcoin Core is waiting for the sharp eyes of code reviewers 👀 #Bitcoin #BitcoinCore #CodeReview

https://t.co/Puzdwahj8J

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@4moonsettler @L0RINC @andrew_s_toth It’ll take a few more days to finish the re-testing, but it seems to me that this slowdown has nothing to do with LevelDB. The disk reads are coming from the files blk/rev*.dat. For some reason, those files aren’t staying in RAM the way they do with the default dbcache setting.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

Can someone explain this?

IBD with a +10GB dbcache runs fine until ~820k blocks, then disk reads suddenly explode and I/O becomes the bottleneck.

If the cache is supposed to prevent this, why does it happen right there?

#Bitcoin #Benchmark #Performance

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@4moonsettler @L0RINC @andrew_s_toth I'm rerunning all the tests, and this time I'm logging all the activity (vmstat, iostat, and iotop) with timestamps so I can create several graphs showing disk and memory usage.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@L0RINC @murchandamus @andrew_s_toth The performance level I'm using is Balanced. Let's assume that's the cause - in that case, shouldn't I also see the same bottleneck during IBD with dbcache=1024 starting from block ~820,000 onwards, like I do with higher dbcache values?

https://t.co/rEX8zjUDVq

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@L0RINC @murchandamus @andrew_s_toth As shown in the chart, IBD with a dbcache=20480 had the same behavior as the dbcache=10240 case.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@murchandamus @L0RINC @L0RINC Does this screenshot clear up your question? It was taken while the bottleneck was happening.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@L0RINC @murchandamus It’s a freshly created Oracle VM instance with only bitcoind running.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@L0RINC @andrew_s_toth @4moonsettler I'm trying to understand this performance discrepancy between different dbcache values and why this bottleneck occurs with dbcache=10240 when there's still plenty of free memory available.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@andrew_s_toth @4moonsettler Based on that theory, shouldn't the bottleneck have started around block 661,907 instead?

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@ProofOfCash I already expected it wouldn't perform better (faster) than with dbcache=4096, but I wasn't expecting such a huge bottleneck.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@andrew_s_toth @4moonsettler Take a look at the disk I/O at the beginning (around block ~450,000).

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@andrew_s_toth @4moonsettler I don't think it's a disk performance limitation. If it were, I'd expect it to be present throughout the entire process, not only starting around block ~820,000.

Also, with more memory available for "dbcache", shouldn't disk usage be significantly lower?

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@L0RINC I assume that warning only appears when "dbcache" is configured higher than the total amount of RAM available. With 24 GB of RAM, "dbcache=10 GB" should work fine.

The only warnings I ran into were these:

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@4moonsettler @andrew_s_toth I didn't get a chance to generate the disk I/O graph, but I do have this screenshot from when I first started noticing the excessive disk usage.

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

See the IBD chart @L0RINC @lopp @murchandamus @0xB10C @stickies_v @r0ckstardev @sedited_ @KevKevPal @darosior @fanquake @ajtowns @TheBlueMatt @andrew_s_toth

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@MrHodl New GNOBAN update dropping soon... 👀

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

@MrHodl New GNOBAN update dropping soon... 👀

[The Rage @theragetech]

🚨NEW: MINED IN AMERICA ACT WOULD PUT BITCOIN NETWORK AT RISK

The Mined in America Act would distort the overall economic incentives of the Bitcoin network and capture U.S. based miners in a situation where dependence on associated loan or grant programs makes it economically non-viable for them to deny government demands.

Effectively, the bill would turn miners into their own surveillance teams, collecting and reporting information to the government in exchange for economic favoritism.

⬇️ Full story by @brian_trollz

[𝙲𝚊𝚎𝚜𝚊𝚛𝙲𝚘𝚍𝚎𝚛 @caesrcd]

#Bitcoin Core v31.0

IBD completed in ~24h50m on:

- Ubuntu 24.04 ARM64
- Oracle Cloud Ampere A1
- 4 OCPU / 24GB RAM
- 200GB SSD

600+ peers during sync.

Obs: Tor/I2P peers rarely stayed connected for more than ~10 min during IBD due to lower throughput compared to IPv4 peers.

[John Cena @JohnCena]

Minimize envy. Just because someone else wins doesn’t mean you lose.