@ETSInsights
My GRE at-home test (ETS ID: 2SG76B66), scheduled on Feb 13, 2025, was terminated before the exam began.
The proctor asked me to disconnect and contact CSR. I’ve sent many emails and made several calls but got no proper response.
This is delaying my academic plans.
Hackathon Mode: "We developed a full-fledged application in just three days!"
Post-Hackathon Reality: "Adding a single icon? That'll take at least three weeks."
Are you struggling to pay a huge amount on paid courses?
I'm giving you access to 20+ FREE Courses 🦑
1. Artificial Intelligence
2. Machine Learning
3. Cloud Computing
4. Ethical Hacking
5. Data Analytics
6. AWS Certified
7. Data Science
8. BIG DATA
9. Python
10. MBA
To get, just:
- Like & Retweet
- Comment "Free"
- MUST be Following (so that I can dm)
A minimal study plan for Machine Learning 📈
- Python
- Maths for ML
- Machine Learning
- Machine Learning Ops (MLOps)
(links in next tweet)
Check this out👇
xLSTM: Extended Long Short-Term Memory
Attempts to scale LSTMs to billions of parameters using the latest techniques from modern LLMs and mitigating common limitations of LSTMs.
To enable LSTMs the ability to revise storage decisions, they introduce exponential gating and a new memory mixing mechanism (termed sLSTM).
To enhance the storage capacities of LSTMs, they add a matrix memory and a covariance update rule (termed mLSTM).
Both the sLSTM and xLSTM cells stabilize their exponential gates using the same technique.
These extensions lead to xLSTM blocks that are residually stacked into the final xLSTM architecture.
Compared to Transformers, xLSTMs have a linear computation and constant memory complexity concerning the sequence length.
The xLSTM architecture is shown to be efficient at handling different aspects of long context problems.
Achieves better validation perplexities when compared to different model classes like Transformers, SSMs, and RNNs.
Then the xLSTM was used to train an LLM with 300B tokens from SlimPajama, with model sizes ranging from 125M to 1.3B parameters.
xLSTMs maintain low perplexities for longer contexts when compared to other methods. xLSTMs also broadly outperform all other models when evaluated on a set of downstream tasks.
Check the paper for more interesting insights and results such as assessing power-law scaling behavior.
Overall, my impression is that this enhanced xLSTM architecture has a lot of potential when they are scaled to billions of parameters. It shows training and performance stability when scaled which was a huge barrier. Looking forward to seeing more variants of this new LLM architecture.
If you are looking for something to code & read this weekend, I uploaded a notebook to finetune a small GPT model to classify SPAM messages with ~96% accuracy: https://t.co/9SGciqJfqJ
(Fun fact: it's small enough to train it on your laptop; ~5 min on my M3 MacBook Air!)