In the 🌍 of Web3, ChatGPT is your MVP! 🏆 Whether you're tackling blockchain tech or high-fiving over smart contracts, make your digital playbooks shine!
You might not need to cluster your @elixirlang nodes. If all you need is pub/sub across nodes, it's really easy to use a combo of Postgres listen/notify and Phoenix.PubSub for this:
The strategy is to use Phoenix.PubSub to handle subscriptions and broadcasts on the local node. Then use Postgres listen/notify to broadcast messages between nodes.
Your PubSub module has three functions:
`subscribe` uses regular Phoenix.PubSub.subscribe/2.
`broadcast_local` likewise uses regular Phoenix.PubSub.broadcast/3.
`broadcast` is where things get interesting. This function encodes the topic and the message. Then broadcasts it via `pg_notify($1, $2)`.
Each node runs a singleton GenServer. On boot, that GenServer subscribes to the broadcast channel:
Now, every node will receive broadcast notifications from each other. Here's what the handler looks like:
The handler relies on Phoenix.PubSub to handle all the subscriptions on the local node.
We don't broadcast locally in `broadcast/2` because it would result in two Phoenix.PubSub.broadcast/3 calls on every node. An optimization is to include a source node's ID in the notify request – then a source node could ignore its own Postgres notify calls.
What makes this approach so simple is that all nodes already share Postgres as a common connection. If a node can't reach Postgres, it has no business operating anyway. No split brain/network partition issues.
I used to really struggle to learn math.
I spent many years trying to learn math by reading textbooks. I would generally get part-way through, get stuck, and then give up.
It took a lot of trial and error to find the right set of books and videos that worked for my brain.
But eventually, I found a list of resources that (at least opinion) explained the concepts clearly and effortlessly.
I've collated them in this document: "the ultimate self-study guide to learning pure math".
It's a list of all my favorite textbooks, videos, and exercises. It covers the undergrad math curriculum from start to finish: all the way from real analysis to more advanced topics like algebraic topology.
https://t.co/e6gIh7Ipra
SSO (Single Sign-On) Explained.
SSO can be thought of as a master key to open all different locks. It allows a user to log in to different systems using a single set of credentials.
In a time where we are accessing more applications than ever before, this is a big help to mitigate password fatigue and streamlines user experience.
To fully understand the SSO process, let’s take a look at how a user would log into LinkedIn using Google as the identity provider:
1) User requests access
First, the user would attempt to access the Service Provider (LinkedIn). At this point, a user would be presented with login options, and in this example, they would select "Sign in with Google".
2) Authentication request
From here, the Service Provider (LinkedIn) will redirect the user to the Identity Provider (Google) with an authentication request.
3) IdP checks for active session
Once the Identity Provider (Google) has received the request, it will check for an active session. If it doesn't find one, authentication will be requested.
4) User submits credentials
At this stage, the user will submit their login credentials (username and password) to the Identity Provider (IdP).
5) IdP verifies credentials
The Identity Provider will then verify the submitted credentials against its User Directory (database). If the credentials are correct, the IdP will create an authentication token or assertion.
6) IdP sends token to Service Provider
Once the token or assertion has been created, the IdP sends it back to the Service Provider confirming the user's identity. The user is now authenticated and can access the Service Provier (LinkedIn).
7) Access granted using existing session
Since the Identity Provider has established a session, when the user goes to access a different Service Provider (eg; GitHub), they won't need to re-enter their credentials. Future service providers will request authentication from the Identity Provider, recognize the existing session, and grant access to the user based on the previously authenticated session.
SSO workflows like the above operate on SSO protocols, which are a set of rules that govern how the IdP and SP communicate and trust each other. Common protocols include Security Assertion Markup Language (SAML), OpenID Connect, and OAuth.
💭 What's your favorite way to go about authentication? 💬
~~
Thanks to our partner Kestra who keeps our content free to the community.
How much easier would it be if you could define all your workflows from simple YAML files, and visualize them all from a UI?
Kestra makes that possible.
Check it out: https://t.co/qR5j8TKEaW
Algorithms you should know before taking System Design Interviews
What are some of the algorithms you should know before taking system design interviews?
I put together a list and explained why they are important. Those algorithms are not only useful for interviews but good to understand for any software engineer.
One thing to keep in mind is that understanding “how those algorithms are used in real-world systems” is generally more important than the implementation details in a system design interview.
What do the stars mean in the diagram?
It’s very difficult to rank algorithms by importance objectively. I’m open to suggestions and making adjustments.
Five-star: Very important. Try to understand how it works and why.
Three-star: Important to some extent. You may not need to know the implementation details.
One-star: Advanced. Good to know for senior candidates.
Over to you: did I miss anything important on the list? Which ones do you know and which you don’t?
--
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/xOtKy03EXx
How do we design secure and safe APIs?
The rise in API-related security breaches highlights the necessity for robust API security.
Let's look at 12 essential tips for improving API security:
🔹 HTTPS
Enforcing HTTPS for all API connections is a critical step in securing sensitive data since it ensures data encryption in transit, preventing attacks and interceptions.
🔹 Rate limiting and throttling
Throttling and rate limiting are vital for reducing API abuse and protecting against DDoS attacks as they manage request rates, which keeps our API available for legitimate users.
🔹 Authentication
Authentication is another must-have. Leverage strong authentication mechanisms, such as OAuth, to verify user or system identities.
Learn more about API authentication here: https://t.co/2cWp0UZwoX
🔹 Authorization
After authentication comes authorization. Follow the least privilege principle to ensure users access only role-relevant data and actions, reducing unauthorized access risks.
🔹 Input validation
Validating API inputs is crucial to safeguard against vulnerabilities like SQL injection and XSS. Whitelisting can also be useful here to ensure only valid data is processed.
🔹 API gateway
Deploy an API Gateway as a security layer, managing authentication, monitoring traffic, and enforcing policies like rate limits.
🔹 Regular security audits
Regular security audits and penetration testing are advisable to identify and fix vulnerabilities, preventing exploitation and maintaining API security.
🔹Dependency management
Regularly updating software dependencies is important to mitigate risks from vulnerabilities in external libraries.
🔹 Logging and monitoring
Investing in comprehensive logging and real-time monitoring is vital for early detection of suspicious activities, enabling swift incident response to mitigate security breaches.
🔹API versioning
To manage changes and updates securely, utilize proper API versioning, which prevents compatibility and security issues.
Learn more about API versioning here: https://t.co/6L1SrPClME
🔹Data encryption at rest
Encrypting sensitive data at rest is crucial to prevent unauthorized access and comply with data protection regulations.
API security isn’t a nice to have; it’s a must. Following the techniques and best practices I’ve shared above will take you a long way, they are the foundations of building safe and secure APIs.
A great way to automate checking the health, security, and performance of your APIs is Postman Monitors.
You can learn more about it here: https://t.co/cdPITqkdPd
Kubernetes Service Types
A Kubernetes service provides an abstraction to expose a set of pods over the network so users can interact with them.
The common Kubernetes service types are:
🔹 ClusterIP: The default service type that exposes the service on an internal cluster IP address, making it reachable only from within the cluster.
🔹 NodePort: Exposes the service on each node's IP address at a static port (the NodePort). The service becomes accessible externally at <NodeIP>:<NodePort>.
🔹 LoadBalancer: Exposes the service externally using a cloud provider’s load balancer.
🔹 ExternalName: Maps the service to a DNS name, commonly used to represent an external resource like a database inside the Kubernetes cluster.
–
Subscribe to our weekly newsletter to get a Free System Design PDF (158 pages): https://t.co/kNfv0DVDdf
Free Certification Courses to Learn Machine Learning in 2024:
1. Python
🔗 https://t.co/gTbLYAFUY9
2. SQL
🔗 https://t.co/SzslJwZz89
3. Statistics and R
🔗 https://t.co/XV3xAFiwaK
4. Data Science: R Basics
🔗https://t.co/yxTYxuzOET
5. Excel and PowerBI
🔗 https://t.co/iyV7Y6FIV0
6. Data Science: Visualization
🔗https://t.co/jd6FoFYBui
7. Data Science: Machine Learning
🔗https://t.co/N2BRbNWAvt
8. R
🔗https://t.co/evPpMndDuz
9. Tableau
🔗https://t.co/VJVrM9Wfov
10. PowerBI
🔗 https://t.co/OzBt3UXW25
11. Data Science: Productivity Tools
🔗 https://t.co/DqBWoG5oEm
12. Data Science: Probability
🔗https://t.co/vrO9BL1xYz
13. Mathematics
🔗https://t.co/TKLSpOUdSg
14. Statistics
🔗 https://t.co/wx610e63Ov
15. Data Visualization
🔗https://t.co/ewaLFSBKRE
16. Machine Learning
🔗 https://t.co/k6YTXO23dM
17. Deep Learning
🔗 https://t.co/aurTSW681x
18. Data Science: Linear Regression
🔗https://t.co/645OsptZvv
19. Data Science: Wrangling
🔗https://t.co/7rQcGUllaS
20. Linear Algebra
🔗 https://t.co/MzvYzT5CPC
21. Probability
🔗 https://t.co/dlMoVjIA8u
22. Introduction to Linear Models and Matrix Algebra
🔗https://t.co/cFlZ7cJSt3
23. Data Science: Capstone
🔗 https://t.co/0TbD8v3WjX
24. Data Analysis
🔗 https://t.co/CBFaIe3O4f
25. IBM Data Science Professional Certificate
https://t.co/vX16mR7vZJ
26. Neural Networks and Deep Learning
https://t.co/oZCiqT7Xzx
27. Supervised Machine Learning: Regression and Classification
https://t.co/JFxb8hdD6C
Happy Learning!🌟
Follow @manishkumar_dev for more such content.