Soko

Lead Developer @MonkeyTiltPlay

Keen interest in large scale distributed systems, data intensive systems and low level programming.

[Soko @SokoDev]

Predictable1 with a MASSIVE run @monkeytilt

[Soko @SokoDev]

Pussymonster extending his lead on the @monkeytilt leaderboard, with a surprise entry and meteoric rise from Mattdeuce32

[Soko @SokoDev]

@Prediction03 Will check it out!

[Soko @SokoDev]

Website translations / localization is wayyy harder than it should be. We've routinely had to wait days for our translation teams.

Has anyone built an LLM app that accomplishes this quick? Translation files are fairly well structured

[Soko @SokoDev]

@dan_Ops96 @SunWukong_MT Dan is best monkey mover

[Tilt Tens @TiltTens]

Don’t miss zeee godfather gambling tonight on https://t.co/rrzxnoreYO

https://t.co/zqskUd05S3

[Soko @SokoDev]

I've seen multiple interesting solutions to this over my career. We're going to try some interesting stuff and I'll write back soon (stay tuned)

Only after we're at the limit of what we can do will we scale up resources + replicas

[Soko @SokoDev]

Scaling problems 101 : @monkeytilt is experiencing a large growth in DAUs, and we spent a lot of today dealing with classic issues in database scaling for our backend services

Small 🧵on how we think about scaling

[Soko @SokoDev]

You can actually see exactly where this Cache Stampede occurs below. All new requests, finding invalidated cache entries, blast the DB

[Soko @SokoDev]

Quite surprised that there hasn't been more work done in large scale recommender systems for online gaming. Not one product I've used does anything more sophisticated then some broad local profiling + category collaborative filtering / popularity ranking

[Soko @SokoDev]

(9/9) Some other misc points:
- We have a LOT of images in game cards, sports logos and more. Lazy loading all images below the fold once the user started scrolling was helpful
- Server side compute : for data intensive calls such as fetching 100s of games, we keep those components on the server (NextJS has everything on server by default)
- Recurrent timers run in the background on web workers
- NextJS image optimization comes for free

[Soko @SokoDev]

(7/9) API calls can be optimized in a bunch of interesting ways. One thing we found success with was matching the API re-computation to the data access pattern. How often do the trending games really change? Every 24h? We just spun up server tasks that push this data onto a CDN. Client side work = close to 0

[Soko @SokoDev]

(8/9) For visual stability (CLS), properly sizing images and adding Suspense boundaries while they loaded made sure things stayed put. We also tried invisible text during font loading so there were no flashes of old font

[Soko @SokoDev]

(6/9) Browser work depends heavily on what kind of interactivity you want to offer.

Browsers do a LOT of work from rendering DOMs, executing your JS code and more. Since JS uses a run to completion model, there's multiple thread starvation opps that come from poorly designed React Components

A hacky thing we did early on was place an upper limit on the amount of stuff each React Component could do. Small components are your friends, because the browser can most likely inject higher prio work

React Suspense is a godsend here as well, as Next streams HTML to the browser to amortize work

[Soko @SokoDev]

(5/9) What we did to make X as small as possible:
- Preload / prefetching comes for free with Next, but you can provide hints with tags as well. This amortizes X
- Asset optimization : webp is a modern format we use, but we also played with image sizes + resolutions
- Moved all fonts to the CDN (we use custom fonts, but this is static data)
- Lots and lots of bundle profiling. NextJS 14 really doesn't require many external deps
- Heavy use of React Suspense : our infra automatically streams these to the browser

[Soko @SokoDev]

(4/9) Resource Delivery is concerned with how quickly you can move X bytes from point A to point B. Intuitively, you can improve this by:
- Making X as small as possible : more on this later
- Making the distance from A to B as small as possible : choice of infrastructure provider matters a LOT here. You want one that has many points of presence for both compute and data. We use Cloudflare for both edge compute and CDN data

[Soko @SokoDev]

(3/9) Each of these are fairly intricate tasks in and of themselves, but (as expected) follow a Pareto distribution in the effort required to fix

Will talk about production measurement in a separate thread, but this is a must to make sure changes are actually working

[Soko @SokoDev]

(1/9) Online casinos are "content heavy / content catalogue" apps (think Netflix, Spotify) with thousands of games, sports and lines. Users expect snappy interactivity, in spite of a high volume of data

A 🧵 on what @monkeytilt is doing to keep FE performance up

[Soko @SokoDev]

(2/9) Generally, FE performance is measured on three axes:
- Resource delivery : how quickly are bytes shipped from your server to my browser (metrics such as TTFB, FCP and LCP)
- Browser work : how much work does my browser have to do once bytes are shipped (metrics such as INP)
- Visual stability : is it easy on the eyes (metrics such as CLS)