lyhourchhen

[Bytebytego @bytebytego]

Netflix's Overall Architecture.

This post is based on research from many Netflix engineering blogs and open-source projects. If you come across any inaccuracies, please feel free to inform us.

Mobile and web: Netflix has adopted Swift and Kotlin to build native mobile apps. For its web application, it uses React.

Frontend/server communication: Netflix uses GraphQL.

Backend services: Netflix relies on ZUUL, Eureka, the Spring Boot framework, and other technologies.

Databases: Netflix utilizes EV cache, Cassandra, CockroachDB, and other databases.

Messaging/streaming: Netflix employs Apache Kafka and Fink for messaging and streaming purposes.

Video storage: Netflix uses S3 and Open Connect for video storage.

Data processing: Netflix utilizes Flink and Spark for data processing, which is then visualized using Tableau. Redshift is used for processing structured data warehouse information.

CI/CD: Netflix employs various tools such as JIRA, Confluence, PagerDuty, Jenkins, Gradle, Chaos Monkey, Spinnaker, Altas, and more for CI/CD processes.

–
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/FIzCeaWsZV

[lyhourchhen @LyhourChhen]

Welcome to year 2024 !

[George Moller @_georgemoller]

❌ Avoid Provider wrapping hell in React.

✅ Instead combine all your Providers using composition.

[Alex Xu @alexxubyte]

A handy cheat sheet for the most popular cloud services (2023 edition).

The method to download the high-resolution PDF is available at the end.

What’s included?
- AWS, Azure, Google Cloud, Oracle Cloud, Alibaba Cloud
- Cloud servers
- Databases
- Message queues and streaming platforms
- Load balancing, DNS routing software
- Security
- Monitoring

Over to you - which company is the best at naming things?

Subscribe to our newsletter to download the 𝐡𝐢𝐠𝐡-𝐫𝐞𝐬𝐨𝐥𝐮𝐭𝐢𝐨𝐧 𝐜𝐡𝐞𝐚𝐭 𝐬𝐡𝐞𝐞𝐭. After signing up, find the download link on the success page: https://t.co/X5JI3eHYUy

[Matt Pocock @mattpocockuk]

🔥 TypeScript Tip 🔥

You can use 'as const' on a template literal to force TypeScript to infer it.

Without the as const, it defaults to string. With it, it goes DEEP.

[Ruurtjan Pul @Ruurtjan]

I ❤️ DNS

I’ve spent 2 years full-time building https://t.co/LYem2r1Vyh.

Now, I’m teaching everything I know in this course.

[Seb ⚛️ ThisWeekInReact.com @sebastienlorber]

This Week In React 164 - Back after a long break! 👋

⚛️
- Next.js
- Remix
- Taint API
- <Activity>
- React Server Components
- React Forget
- Docusaurus
- Astro
- MDX
- Redux Toolkit
- Storybook

📱
- Expo Orbit
- Ignite
- Victory XL
- Reanimated

... more by ✉️

Details 👇

[Alex Xu @alexxubyte]

Almost every software engineer has used Linux before, but only a handful know how its Boot Process works :) Let's dive in.

The diagram below shows the steps.

Step 1 - When we turn on the power, BIOS (Basic Input/Output System) or UEFI (Unified Extensible Firmware Interface) firmware is loaded from non-volatile memory, and executes POST (Power On Self Test).

Step 2 - BIOS/UEFI detects the devices connected to the system, including CPU, RAM, and storage.

Step 3 - Choose a booting device to boot the OS from. This can be the hard drive, the network server, or CD ROM.

Step 4 - BIOS/UEFI runs the boot loader (GRUB), which provides a menu to choose the OS or the kernel functions.

Step 5 - After the kernel is ready, we now switch to the user space. The kernel starts up systemd as the first user-space process, which manages the processes and services, probes all remaining hardware, mounts filesystems, and runs a desktop environment.

Step 6 - systemd activates the default. target unit by default when the system boots. Other analysis units are executed as well.

Step 7 - The system runs a set of startup scripts and configure the environment.

Step 8 - The users are presented with a login window. The system is now ready.

–
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/uc5M7CdXXC

[Seb ⚛️ ThisWeekInReact.com @sebastienlorber]

Hmmm ok 😄

[Elon Musk @elonmusk]

All design & engineering of the original @Tesla Roadster is now fully open source.

Whatever we have, you now have.

https://t.co/5d10soAYr8

[Alex Xu @alexxubyte]

Open-sourcing over 100 byte-sized system design concepts with high-resolution diagrams.

Goals:
- Become a better engineer by understanding how systems work.
- Prepare for system design interviews.

What's included in the GitHub repository:
- 100 byte-sized system concepts with visuals.
- Real-world case studies.
- Tips on how to prepare for system design interviews.

Topics included (and many many more):
- SOAP vs. REST vs. GraphQL vs. RPC
- HTTP 1.0 -> HTTP 1.1 -> HTTP 2.0 -> HTTP 3.0 (QUIC)
- CI/CD Pipeline Explained in Simple Terms
- 8 Data Structures That Power Your Databases
- Top caching strategies
- What does a typical microservice architecture look like?

Start exploring the repository here: https://t.co/keZF9CNuye

If you find it useful, please RETWEET to spread the word. Thank you.

[Aurimas Griciūnas @Aurimas_Gr]

Why is 𝗠𝗼𝗱𝗲𝗹 𝗥𝗲𝗴𝗶𝘀𝘁𝗿𝘆 so important in your 𝗠𝗟𝗢𝗽𝘀 𝗦𝘁𝗮𝗰𝗸?

Let's look again into the Machine Learning Model Training Lifecycle.

𝗟𝗲𝘁’𝘀 𝗿𝗲𝘃𝗶𝗲𝘄 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝘀𝘁𝗲𝗽𝘀:

𝟭: Version Control: Machine Learning Training Pipeline is defined in code, once merged to the main branch it is built and triggered.
𝟮: Feature Preprocessing: Features are retrieved from the Feature Store, validated and passed to the next stage. Any feature related metadata that is tightly coupled to the Model being trained is saved to the Experiment Tracking System.
𝟯: Model is trained and validated on Preprocessed Data, Model related metadata is saved to the Experiment Tracking System.
𝟰: If Model Validation passes all checks - Model Artifact is passed to a Model Registry. The model is stored and ready to be packaged.

✅ 𝗧𝗵𝗶𝘀 𝗶𝘀 𝗶𝘁 - regardless of what deployment type will follow, the model is stored in The Model Registry. Model Registry is what glues Training and Deployment Pipelines together and this is where handover of the Model Artifact happens. 

𝟱: The same model can be packaged as a container for different deployment types by implementing a respective interface. E.g.

👉 Flink application for Stream Processing Deployment.
👉 gRPC API for Request-Response.
👉 Plain model pointing to the Batch Serving Feature Store API for Batch.

✅ 𝗧𝗵𝗶𝘀 𝗶𝘀 𝗴𝗼𝗼𝗱 𝗻𝗲𝘄𝘀 - as long as you are training your models using historical data the training pipeline is the same for any type of deployment.
❗️ It’s a different story when it comes to online training but more on it in the future episodes so 𝘀𝘁𝗮𝘆 𝘁𝘂𝗻𝗲𝗱 𝗶𝗻!

You can find a more detailed explanation of different Model Deployment procedures in one of my previous Newsletter releases.

Leave any thoughts in the comment section 👇

--------

Follow me to upskill in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space.

Also hit 🔔to stay notified about new content.
𝗗𝗼𝗻’𝘁 𝗳𝗼𝗿𝗴𝗲𝘁 𝘁𝗼 𝗹𝗶𝗸𝗲 💙, 𝘀𝗵𝗮𝗿𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗺𝗲𝗻𝘁!

Join a growing community of Data Professionals by subscribing to my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: https://t.co/qgNCnGtF4A

[Steven Tey @steventey]

We just shipped a quality-of-life improvement to the @nextjs and @vercel docs 🎉

You can now zoom in on images that have detailed illustrations in them for better legibility – and it automatically zooms out on scroll 😁

Try it out for yourself: https://t.co/TlXqqL75Od

[AI at Meta @AIatMeta]

To better enable the community to build on our work — and contribute to the responsible development of LLMs — we've published further details about the architecture, training compute, approach to fine-tuning & more for Llama 2 in a new paper.

Full paper➡️ https://t.co/GlY2a1wKMk

[Dara Mech @MechDara1]

microfinance life is risk...😄😂

[Thai PBS World @ThaiPBSWorld]

The swift decision by Samdech Prime Minister Hun Sen to appoint his son, Hun Manet, as the next leader of Cambodia was a well-calculated and Machiavellian move to create a fresh image for the once war-torn county, as the Southeast Asian nation with the youngest leader.
The fresh optic will be important, to dilute the longstanding negative actions and memories associated with the previous leader. Hun Manet is lucky that he does not have a so-called “legacy trap”, in which his father was, and thus has a unique opportunity to be a transformational Cambodian leader. It is also notable that it is the first time in 500 years of Khmer history that there has been a peaceful transition of leadership.
So far, the Western world and media have exclusively focused on the dark side of the current leader’s governance. In the coming months, there will certainly be a big change in the Western rhetoric about Cambodia and the country’s leadership style.
Cambodia used to be one of the world’s poorest countries. Now, the nation of 13 million people is booming as never before. Phnom Penh is no longer disconnected from the international community, as it was thirty years ago. Truth be told, within the ASEAN context, Cambodia is also considered one of the freer members with a dynamic society.
While Manet’s father’s human rights record is checkered, the local voters still support him and his party, the Cambodia People’s Party. The turnout of nearly 85% was impressive for the seventh election since 1993. Hun Manet has been able to draw on support from young voters, which was not the case for his father. Nonetheless, even though Hun Sen is stepping down, he will remain a powerful figure and oversee the country’s overall development. He is still a popular leader. After all, he has brought peace and stability to Cambodia, which has led to prosperity and wealth.
Throughout Hun Sen’s leadership, both political and economic factors have earned him the most credibility and credit, despite the restrictive measures he employed to cripple his political opponents. In the coming weeks, Hun Manet needs to show, especially during the first 100 days, his governing ability. Among the top priorities will be improving the living standards of Khmer people, to reduce the rate of poverty. Furthermore, he must make his country an international hub of more diversified foreign investment and negotiate more free trade agreements. Later, he will need to carry out some deep institutional reforms, inherited from the previous administration. Younger leaders should be brought into the Cabinet. It will be a tall order.
As far as international cooperation and perception are concerned, however, the new prime minister faces a huge challenge. He must promote a free and open society in Cambodia in tangible ways. The country still needs foreign assistance from countries that set a high standard of governance and rule of law.
Hun Manet, 45, is young and Western-educated with a knowledge of Western military and economic doctrines. That will come in handy when he actually takes up the premiership. Not to overstate the new leadership, he could be a catalyst to propel Cambodia into a new and vibrant developing country. Cambodia, under Hun Sen, has not shied away from the international limelight, whether on the Russia-Ukraine war, the quagmire in Myanmar or US-China rivalries, among others. His son is likely to stay on this trajectory in engaging with powerful external powers, regardless of their attitude towards the country.
As a young cadet, he showed interest in regional and international affairs. At one of the lectures given by this writer in 2005 on ASEAN and its global roles, Hun Manet was extremely interested in seeing the formation and joint operation of ASEAN peacekeeping forces in the future. In a related development, in the past two months, after the electoral triumph of Thailand’s Move Forward Party, Hun Sen was the only leader to comment on the progressive party’s demeanor. Indeed, one of the purposes of anointing his son so quickly could also stem from a desire for Cambodia to have a young and fresh leader as well. Originally, Hun Sen planned to appoint his son later next year.
Now that the baton has been passed to him, Hun Manet has to move quickly to promote Cambodia’s national interest and follow regional and international rules-based norms. He is lucky to enter the ring now, as Cambodia has emerged with fewer bruises than other ASEAN countries from the havoc and hazards of the COVID-19 pandemic.
As the youngest leader in ASEAN, in the beginning, Hun Manet will be in listening mode, trying to absorb lessons learned from the ASEAN elders, in particular his father. Closer ties with ASEAN and active engagement in economic diplomacy and people-to-people exchange, especially with the young, could be his unique forte. Now Cambodia can show the international community what kind of transformation he can bring about, to generate a new, fresh and respectable image for his country.
By Kavi Chongkittavorn
https://t.co/vsoWNnwzYU
#ThaiPBSWorld
#Cambodia
#HunSen
#Hunmanet

[Alex Xu @alexxubyte]

Top 7 Ways to 10x Your API Performance

1. Pagination
This is a common optimization when the size of the result is large. The results are streaming back to the client to improve the service responsiveness.

2. Asynchronous Logging
Synchronous logging deals with the disk for every call and can slow down the system. Asynchronous logging sends logs to a lock-free buffer first and immediately returns. The logs will be flushed to the disk periodically. This significantly reduces the I/O overhead.

3. Caching
We can cache frequently accessed data into a cache. The client can query the cache first instead of visiting the database directly. If there is a cache miss, the client can query from the database. Caches like Redis store data in memory, so the data access is much faster than the database.

4. Payload Compression
The requests and responses can be compressed using gzip etc so that the transmitted data size is much smaller. This speeds up the upload and download.

5. Connection Pool
When accessing resources, we often need to load data from the database. Opening the closing db connections add significant overhead. So we should connect to the db via a pool of open connections. The connection pool is responsible for managing the connection lifecycle.

On 6 and 7, we created a video about this topic and included 2 more ways to improve performance.

Watch and subscribe here: https://t.co/mdeHOu0ozB

[Oversight Board @OversightBoard]

The Oversight Board has overturned Meta’s decision to leave up a video in which Cambodian PM Hun Sen threatens political opponents with violence.

Due to the severity of the violation, the Board recommends that Meta suspend Hun Sen’s FB page and IG account for 6 months.