Brent Frederick

ml infra @meta, former principal engineer @twitter

[Brent Frederick @brentcodes]

Product Mixer - Twitter's product development platform that I created with @NegatedVoid - was open sourced! Big shoutout to contributors @chrislewis_aus, @AzizDoesThings, @JesseFeinman, and all of the amazing Twitter engineers 💙

https://t.co/NzqKF5LHMD

[Brent Frederick @brentcodes]

@PeilinYang Congrats on the epic run! Hope to work together again in the future 🫡

[Brent Frederick @brentcodes]

@kchro3 @neha01mittal Congrats Jeff!

[Brent Frederick @brentcodes]

@saurabhverma @NegatedVoid Thanks for the kind words!

[Brent Frederick @brentcodes]

First two 🫡
@NegatedVoid I guess this means our Product Mixer vision was brought to fruition. Thanks and congrats to the remaining Tweeps who finished the job 💙

[Engineering @Engineering]

This has been a year full of engineering excellence that sometimes can go unnoticed. Besides all the visible changes you see on our app, here are some of the most important improvements we have made under the hood.

- Consolidated the tech stacks for For you, Following, Search, Profiles, Lists, Communities and Explore around a singular product framework.

- Completely rebuilt the For you serving and ranking systems from the ground up, resulting in a decrease 90% reduction in lines of code from 700K to 70K, a 50% decrease in our compute footprint, and an 80% increase in the throughput of posts scored per request.

- Unified the For you and video personalization and ranking models, which significantly improved video recommendation quality.

- Refactored the API middleware layer of our tech stack and in doing so simplified the architecture by removing more than 100K lines of code and thousands of unused internal endpoints and eliminating unadopted client services.

- Reduced post metadata sourcing latency by 50%, and global API timeout errors by 90%.

- Blocked bots and content scrapers at a rate +37% greater than 2022. On average, we prevent more than 1M bots signup attacks each day and we’ve reduced DM spam by 95%.

- Shutdown the Sacramento data center and re-provisioned the 5,200 racks and 148,000 servers, which generated more than $100M in annual savings. In total, we freed up 48 MW of capacity and tore down 60k lbs. of network ladder rack before re-provisioning it to other data centers.

- Optimized our usage of cloud service providers and began doing much more on-prem. This shift has reduced our monthly cloud costs by 60%. Among the changes we made was a shift of all media/blob artifacts out of the cloud, which reduced our overall cloud data storage size by 60%, and separately, we succeeded in reducing cloud data processing costs by 75%.

- Built on-prem GPU Supercompute clusters and designed, developed, and delivered 43.2Tbps of new network fabric architecture to support the clusters.

- Scaled network backbone capacity and redundancy, which resulted in $13.9M/year in savings.

- Started automated peak traffic failover tests to validate the scalability and availability of the entire platform continuously.

[Brent Frederick @brentcodes]

@chrislewis_aus Little Bobby tables would like a word

[Brent Frederick @brentcodes]

@enriquebrgn @NegatedVoid Happy to hear!

[Brent Frederick @brentcodes]

@fredloh Thanks for the kind words on behalf of @NegatedVoid and I 💙 I had finished the first draft of the Product Mixer paper, but I guess it will die on the vine. At least source is public https://t.co/ktdfCSH72a
Strato paper by @jakedonham https://t.co/e1LBn8f57M

[Brent Frederick @brentcodes]

@aqueel @enriquebrgn 💙

[Brent Frederick @brentcodes]

@fredloh Thanks for the kind words on behalf of @NegatedVoid and I 💙 I had finished the first draft of the Product Mixer paper, but I guess it will die on the vine. At least source is public https://t.co/ktdfCSH72a
Strato paper by @jakedonham https://t.co/e1LBn8f57M

[Brent Frederick @brentcodes]

@fredloh Thanks for the kind words on behalf of @NegatedVoid and I 💙 I had finished the first draft of the Product Mixer paper, but I guess it will die on the vine. At least source is public https://t.co/ktdfCSH72a
Strato paper by @jakedonham https://t.co/e1LBn8f57M

[Brent Frederick @brentcodes]

@imPrachiPoddar Congrats on a great run Prachi!

[Brent Frederick @brentcodes]

@jpd Congrats on an amazing run JP 💙

[Brent Frederick @brentcodes]

@fredloh Yes! Can't wait for pattern matching for switch and more record pattern matching. Now if they would just add `val` on top of the `var` support...

[Brent Frederick @brentcodes]

@ChristianSelig 🫡

[Brent Frederick @brentcodes]

@AadityaLandge @Twitter Congrats on a great run Aaditya! 💙

[Brent Frederick @brentcodes]

@TheGerardTaylor @chrislewis_aus @arcalinea @jack I’m working on an unrelated project if you’re interested :) monorepo typescript (t3) stack

[Brent Frederick @brentcodes]

@chrislewis_aus @arcalinea @jack Here’s one! https://t.co/NzqKF5LHMD

Although as much as I love the language, Scala would be a poor choice tbh

[Brent Frederick @brentcodes]

@RohitSSJain @Twitter Congrats on a great run Rohit 💙

[Brent Frederick @brentcodes]

@chrislewis_aus Haha good guess

[Brent Frederick @brentcodes]

@tjdharamsi @Twitter Congrats on a great run Tejas 💙