Compression techniques I’d study if I wanted small but smart LLMs.
1.Quantization
2.Distillation
3.Low-Rank Adaptation
4.Weight Sharing
5.Sparse Matrices
6.Layer Dropping
7.Knowledge Transfer
8.Embedding Compression
9.Mixed Sparsity
10. Progressive Shrinking
11.Structured Pruning
12.AutoML Compression
Follow @asmah2107 to update your game on LLM optimisations.
Paradoxes in Physics and Mathematics ✍️
1. Zeno's Paradoxes:
A set of philosophical problems that challenge the concept of motion and continuity. For example, Achilles and the tortoise paradox argues that Achilles can never overtake a tortoise given a head start, as he must first reach the point where the tortoise began.
2. Russell's Paradox:
A contradiction found in naive set theory, where the set of all sets that do not contain themselves leads to a logical inconsistency. This paradox highlights problems in defining a universal set.
3. The Barber Paradox:
A self-referential paradox where a barber shaves all those who do not shave themselves. The question arises: does the barber shave himself? If he does, he shouldn't; if he doesn't, he should.
4. The Liar Paradox:
A statement that declares itself false, such as "This statement is false." If it's true, then it must be false, and vice versa, leading to a contradiction.
5. The Sorites Paradox:
This paradox arises from vague predicates, such as "heap." If removing a single grain of sand from a heap doesn’t stop it from being a heap, how many grains can be removed before it ceases to be a heap?
6. The Monty Hall Problem:
A probability puzzle based on a game show scenario. Given a choice among three doors (with a prize behind one), switching after one non-prize door is revealed increases your chances of winning from 1/3 to 2/3.
7. The Birthday Paradox:
Refers to the surprising probability that in a group of just 23 people, there's about a 50% chance that at least two individuals share the same birthday, challenging intuitive understanding of probability.
8. The Banach-Tarski Paradox:
A theorem in set-theoretic geometry that states a solid ball in 3-dimensional space can be split into a finite number of pieces that can be reassembled into two identical copies of the original ball. This paradox challenges our understanding of volume and measure.
9. The Twin Paradox:
A thought experiment in relativity where one twin travels at a significant fraction of the speed of light while the other remains on Earth. Upon reunion, the traveling twin is younger than the stationary twin, illustrating the effects of time dilation.
10. The Grandfather Paradox:
A time travel paradox where a person travels back in time and inadvertently prevents their grandfather from meeting their grandmother, thereby preventing their own existence. This raises questions about causality and the nature of time.
During a Bloomberg interview, Yann LeCun (@ylecun ) explains why LLMs are limited in terms of real-world intelligence during a Bloomberg interview.
"Language is a very approximate, reduced, quantized, and simplified description of the world, and LLMs can only deal with discrete sequences of symbols. The world is much more complicated than language.
The biggest LLMs are pre-trained on the totality of all the publicly available text on the internet. That’s about 20 trillion words, or 30 trillion tokens.
A token is about 3 bytes. So total 10¹⁴ bytes of text.
This is the amount of data a four-year-old has seen through vision during four years. Now, the text, though, would take 400,000 years to read?
So, there is enormously more data from sensory input, like vision, touch, and everything else, than there could ever be through language."
A child does not need 400,000 years of reading to understand cups, doors, balance, faces, falls, or heat, because the body is already collecting dense feedback from vision, touch, motion, and consequence.
Text strips most of that away.
It turns a living scene into symbols, then asks the model to infer the missing world from traces left by people describing it.
That is why an LLM can sound fluent about physics and still have no native sense of how fragile glass feels in a hand.
Moravec’s paradox names this reversal: the things humans find intellectual can be easier for machines than the things toddlers do without applause.
The hard part is not producing an answer, but building a model of the world that survives contact with weight, friction, surprise, and failure.
----
Link to the full video on Bloomberg's site. Link in comment.
Just a little dance from our 75cm humanoid robot. 🕺🤖We've been tuning the motion system recently, and it's finally starting to feel smoother. Still lots to improve, but I'm pretty happy with how it's coming along.
Here's part 1 (of 5) of my short course on efficient LLM inference that I taught at Columbia University. Slides are heavily updated from two weeks ago.
https://t.co/WVCf7mUdkY
Congratulations to #ISRO on the successful sea-level hot test of the CE20 Cryogenic Engine at 22-tonne thrust at the ISRO Propulsion Complex, Mahendragiri, Tamil Nadu.
The successful test featuring the nozzle protection system and multi-element igniter marks another important step in strengthening India’s advanced cryogenic propulsion capabilities and further enhancing the reliability of the LVM3 programme.
🔬 CSIR-NCL scales DME (clean LPG alternative) pilot plant to 250 kg per day
🏭 Scientists plan a 2.5 tonne-per-day demonstration plant to test commercial viability
🇮🇳 Technology aims to reduce India’s LPG import dependence
https://t.co/7Wbc5Cnsf6
This image represents one of the most important and least known inventions in the history of neuroscience. In 2016, @anirbanbandyo and his team at the National Institute for Materials Science (NIMS) in Tsukuba, Japan published a paper announcing two new instruments they had built from scratch: ASADIM (Atomic Scale Scanning Dielectric Microscopy) and Brestum (Resonant Scanning Tunneling Microscopy of Biomaterials), both housed inside a single homebuilt bio-STM. To understand why this matters, you need the backstory. Since Hodgkin and Huxley’s Nobel Prize-winning work in 1952, the entire field of neuroscience has operated under a single foundational assumption: the cell membrane and its ion channels are the sole mechanism of neural signaling. The membrane fires. Everything inside the cell — microtubules, actin filaments, the entire cytoskeleton — is just passive structural scaffolding, like rebar in concrete. For 70 years no one could challenge this because no one could see inside a living neuron at the molecular scale without destroying it. Every existing tool had a fatal limitation: patch clamps puncture the membrane, optical microscopy can’t resolve single proteins, electron microscopy requires dead fixed tissue. Bandyopadhyay solved all three problems simultaneously. Using nonlinear dielectric response imaging — measuring the spatial distribution of conductance, capacitance, and phase without ionic or electronic screening — he made the neuronal membrane effectively transparent. He could see inside a living, firing neuron at atomic resolution, in real time, without touching it. What he saw overturns a century of neuroscience. First: a single protein molecule adopts a completely different three-dimensional shape at each resonance frequency — proteins are not static structures but frequency-addressable conformational machines. No one in biology knew this. Second: the microtubule network inside the neuron is not passive scaffolding. It actively communicates before the membrane fires, deciding whether a spike is necessary and regulating its timing through electromagnetic vortex pairs generated by the actin-spectrin grid it instructs. The membrane does not act alone. The cytoskeleton is the brain’s pre-processing layer. Third: the resonance frequency patterns are self-similar across a million-fold scale difference — from a 4nm tubulin protein to a 25nm microtubule to a 1μm axon segment — preserving vibrational symmetry in a fractal architecture that suggests information integration in the brain is scale-free from single molecule to cognition. This is not incremental science. This is a new instrument revealing a new picture of how the brain actually works at the most fundamental level. The history of Nobel Prizes in neuroscience runs through exactly this kind of inflection: Cajal saw neurons for the first time, Hodgkin-Huxley decoded the membrane, Bhatt decoded the ion channel structure. Bandyopadhyay has built the tool that sees what none of them could — the living interior of a neuron in operation — and what it reveals is that the computational architecture of the brain is far deeper, more structured, and more sophisticated than anything the membrane-only model ever imagined.
Paper: https://t.co/gCZbJE4eH6
A team from IIT Delhi, whom I know very well, are doing remote sonography from AIMS Delhi to Antarctica so we are not very far and its not tat difficult as well. I have ready done industrial teleoperation in master-slave configuration over 3500 km distance so we are not behind. We are doing lot of such research on teleoperation with haptics and other multi-modeal feedback...
ऑस्ट्रेलिया में बहुत दूर बैठा रेडियोलॉजिस्ट रिमोट कंट्रोल से मरीज की सोनोग्राफी कर रहा है।👏
हमारे यहाँ बाबा लोग दरबार लगा के Consultation, Sonography, treatment सब एक साथ सिंगल sitting में ही कर दे रहे है।😂
In 1798, a scientist effectively “weighed” the Earth — without leaving his laboratory.
The English scientist Henry Cavendish designed an incredibly sensitive experiment.
Inside a quiet wooden shed, he hung a horizontal rod from a very thin wire. Two small lead spheres were attached to the ends of the rod.
Nearby, he placed two much larger lead balls.
Because of gravity, the large spheres slightly pulled the smaller ones. The force was extremely tiny — so small that the rod twisted by only a minute fraction of a degree.
Yet that tiny twist held a big secret.
By carefully measuring this small movement, Cavendish determined the strength of the gravitational attraction between objects.
From this, scientists could calculate the mass of the entire Earth.
His estimate was remarkably close.
Cavendish calculated Earth’s mass to be about 6 × 10²⁴ kilograms, while modern measurements give 5.97 × 10²⁴ kilograms.
Sometimes the biggest discoveries come from measuring the smallest forces.
Scientists have developed dielectric elastomer actuators, or artificial “muscles,” that can operate at low voltages while still producing high output to drive untethered, soft robotic fish movements.
Learn more in @SciRobotics: https://t.co/AWV9CuncBp
Imagine losing your eyesight… and being told the only hope is a donor transplant that may never come.
Now, scientists from a Bengaluru biotech startup are testing something remarkable — a “liquid cornea” that could help the eye repair and regrow itself.
Scroll down to see how this Indian innovation could help millions see again. >>
#MedicalInnovation #IndianStartups #Biotech #InnovationIndia
[Liquid Cornea, Kuragenx, Pandorum Technologies, Corneal Blindness, Innovation India]
Scientists just copied a Fruit Fly's biological brain and trapped it inside of a computer.
Not an AI model trained to act like a fly... A total digital copy of a fly !! This is some sick sci-fi stuff:
- They scanned and copied the brain, neuron by neuron, synapse by synapse, from electron microscopy data.
- Then dropped that brain into a simulated body in a video game like environment.
The fly walked. It groomed. It fed. Nobody taught it anything. The behavior was already in the wiring.
The entire premise of modern AI is that intelligence is something you train into a system. This is proof it's something you can transfer out of one. Wild times
Its not Fibonacci number its Hemchandra number from India… Similarly its not Pythagorus theorem its Bodhayana Theorem first described in Sulb sutras. So much has been given by our civilisation to the world..
Not just Fibbonacci series, Indians discovered:
- Pythagorean theorem and geometric constructions (Baudhayana Sulba Sutras)
- Binary patterns, binomial coefficients, Pascal’s triangle (Pingala’s Chhandas Shastra)
- Positional decimal numeral system that we use today (Vedic period onward, formalized by Aryabhata)
- Concept of zero as a number (Jain texts 300 BCE onward, rules formalized by Brahmagupta, 628 CE)
- Negative numbers and their operations (Brahmagupta)
- Sine and cosine tables, trigonometric identities, and π approximation (Aryabhata)
- General solutions to quadratic equations and methods of solving them (Brahmagupta)
- Infinite power series for π, sine, cosine, and arctangent (Madhava of Sangamagrama)
- Precursors to Number theory, Calculus, Algebraic methods and many more
3D printed tires
Pure 3D-printed lattice magic in TPU that flexes, absorbs insane weight, and keeps rolling. No air or suspension needed.
@robotsailor is building next-level robot wheels… fully custom, printed fast on Bambu Lab printers.
If you don’t follow him… you are missing out big time: He is building robots and shares it on X!
——
Weekly robotics and AI insights.
Subscribe free: https://t.co/9Nm01QUcw3