Devendra Desale

I am a Data Enthusiast & Big Data Developer. Currently a Data science grad student. @KDNuggets contributing editor. #BigData #DataScience #MachineLearning

[Santiago @svpino]

Large Language Models don't reason.

Thank you,
Apple.

[Rohan Paul @rohanpaul_ai]

Nice Paper for a long weekend read - "A Primer on the Inner Workings of Transformer-based Language Models"

📌 Provides a concise intro focusing on the generative decoder-only architecture.

📌 Introduces the Transformer layer components, including the attention block (QK and OV circuits) and feedforward network block, and explains the residual stream perspective. It then categorizes LM interpretability approaches into two dimensions: localizing inputs or model components responsible for a prediction (behavior localization) and decoding information stored in learned representations to understand its usage across network components (information decoding).

📌 For behavior localization, the paper covers input attribution methods (gradient-based, perturbation-based, context mixing) and model component importance techniques (logit attribution, causal interventions, circuits analysis). Causal interventions involve patching activations during the forward pass to estimate component influence, while circuits analysis aims to reverse-engineer neural networks into human-understandable algorithms by uncovering subsets of model components interacting together to solve a task.

📌 Information decoding methods aim to understand what features are represented in the network. Probing trains supervised models to predict input properties from representations, while the linear representation hypothesis states that features are encoded as linear subspaces. Sparse autoencoders (SAEs) can disentangle superimposed features by learning overcomplete feature bases. Decoding in vocabulary space involves projecting intermediate representations and model weights using the unembedding matrix.

📌 Then summarizes discovered inner behaviors in Transformers, including interpretable attention patterns (positional, subword joiner, syntactic heads) and circuits (copying, induction, copy suppression, successor heads), neuron input/output behaviors (concept-specific, language-specific neurons), and the high-level structure mirroring sensory/motor neurons. Emergent multi-component behaviors are exemplified by the IOI task circuit in GPT2-Small. Insights on factuality and hallucinations highlight the competition between grounded and memorized recall mechanisms.

[Greg Brockman @gdb]

Introducing GPT-4o, our new model which can reason across text, audio, and video in real time.

It's extremely versatile, fun to play with, and is a step towards a much more natural form of human-computer interaction (and even human-computer-computer interaction):

[Abhishek @HeyAbhishek]

ChatGPT can now create Mind Maps.

No more wasting hundreds of hours making visuals for studying or simplifying complex ideas.

Here’s how to do it for free in a few seconds:

[Andrew Ng @AndrewYNg]

Chatting with @GroqInc’s CEO @JonathanRoss321. Groq has super fast token generation capabilities now. And, I was excited also to hear about his plans to scale up capacity aggressively and also expand this to other models than just LLMs! This is a good time to be building AI applications.

[Swapnil @swapnilkate04]

OpenAI'a Sora is the best example of Synthetic data example.
Hard to replicate such a moat in an enterprise but if we can get the right distribution of the data and its attributes, I think we can see better models for the basic use cases of the enterprise.
cc @DevendraDesale

[Devendra Desale @DevendraDesale]

Finally free from stranglehold of Facebook products. Moved over 40 friends to signal over last weekend and finally successful in #Deletewhatsapp.

[Devendra Desale @DevendraDesale]

Excited to be networking from home on @lunchclubai! Use my invite link to skip the waitlist and meet interesting people over video: https://t.co/6U623Jralu

[Vala Afshar @ValaAfshar]

The fastest route is not always a straight line. https://t.co/A2dLNWNZph

[Shreyas Doshi @shreyas]

People can be:

1) How-first
2) What-first
3) Why-first

How-first people execute well.

What-first people create well.

Why-first people lead well.

[andrew chen @andrewchen]

Guy on the right is a Growth PM

[Richard Eudes, PhD @RichardEudes]

Drones, data analytics, smart seeds: How to reforest x1,000 faster after wildfires https://t.co/G3laXlyE9d #bigdata, #datascience #ds

[Devendra Desale @DevendraDesale]

@flyspicejet Thanks for nothing :/

[Devendra Desale @DevendraDesale]

@IndiGo6E He reached the airport at 4.35. When contacted from Pune airport, someone called Mr.Kiran from customer service told him that he has informed the Hyd staff of the situation.

[Devendra Desale @DevendraDesale]

@flyspicejet How I am supposed to take an alternate flight when the existing one got delayed? And canceling was not an option since he is on defence duty as already mentioned earlier.

[Devendra Desale @DevendraDesale]

@IndiGo6E We called up customer service while at Pune airport and immediately after reaching Hyderabad as well. The flight had not taken off and last min boarding was being done. He could have been easily accommodated. And this was a genuine case for gods sake

[Devendra Desale @DevendraDesale]

@IndiGo6E Dont state the obvious, we know the rules. In this case his incoming flight got delayed by 3 hrs and we had called up customer service well in advance to inform that he might be late. He literally begged several of your staff but he was not let on the flight. 6E773 pnr-TKDZ3B

[Devendra Desale @DevendraDesale]

@flyspicejet we KNOW the flight was delayed. We need you to clean the mess you made. Either reimburse him for next flight or arrange to accommodate him on it. You realize he is on defence duty and needs to report to work tomorrow.

[Devendra Desale @DevendraDesale]

@flyspicejet PNR - ZFZ4KK