Pravesh Goyal

[Pravesh Goyal @prvlabs]

Soft Labels =>

Each class is assigned a probability between 0 and 1

Example:
Next word prediction: "My teddy bear is..."
😊 Cute → 60%
🎁 New → 25%
🧸 Soft → 15%
✅ Captures uncertainty
✅ Common in NLP and knowledge distillation

#AI #Machinelearning #LLM

[Pravesh Goyal @prvlabs]

A Classification Label is the ground truth a model tries to predict.

Two common types:

Hard Labels =>
Each class is either present or absent
Example:
🧸 Teddy Bear = 1
🚗 Car = 0
🏐 Volleyball = 0

✅ Clear, binary answers
✅ Common in image classification

#AI #MachineLearning

[Pravesh Goyal @prvlabs]

Low Loss → Predictions are close to the target values.

High Loss → Predictions are far from the target values.

In simple terms: the loss function tells the model how wrong it is so it can learn to become more accurate.

#AI #MachineLearning #DeepLearning #NeuralNetworks

[Pravesh Goyal @prvlabs]

A Loss Function measures how far a model's predictions are from the true target values.

Input → Model → Prediction (ŷ)
⬇
Loss Function L(y, ŷ)
⬇
Prediction Error

Goal:
Quantify model performance during training.
#AI #MachineLearning #DeepLearning #NeuralNetworks

[Pravesh Goyal @prvlabs]

👣 Steps per Epoch (s)
Number of iterations needed to see all training examples.

Relationship:
N = b × s

Example:
If N = 10,000 and b = 100, then s = 100 steps per epoch.
After 1 epoch, the model has seen every training example exactly once.
#AI #LLM #Machinelearning

[Pravesh Goyal @prvlabs]

An Epoch is one complete pass through the entire training dataset during model training.

Key terms:

📊 Training Size (N)
Total number of training examples.

📦 Batch Size (b)
Number of examples processed at once.

#AI #MachineLearning #LLM

[Pravesh Goyal @prvlabs]

🔹 He Initialization =>
Best for: ReLU, Leaky ReLU, ELU, GELU
Goal: Preserve signal strength in deep networks and improve gradient flow.

➡️ Poor initialization can lead to vanishing or exploding gradients, making training difficult or even impossible.
#AI #MachineLearning

[Pravesh Goyal @prvlabs]

Weight Initialization is the first step before training a neural network.

Every weight starts with an initial value, and choosing these values wisely can significantly affect:
⚡ Training speed
📈 Convergence stability
🎯 Final model performance

#AI #LLM #MachineLearning

[Pravesh Goyal @prvlabs]

Popular initialization methods:

🔹 Xavier (Glorot) =>
Best for: Sigmoid & Tanh activations
Goal: Keep activations from becoming too large or too small as they flow through the network.

[Pravesh Goyal @prvlabs]

Regression =>
The network outputs numerical values directly without Softmax.

Examples:
🏠 House price prediction
🌡️ Temperature forecasting
📉 Stock price estimation

#AI #MachineLearning #DeepLearning #NeuralNetworks #LLM

[Pravesh Goyal @prvlabs]

The Output Layer is the final stage of a neural network, where predictions are produced.

Two common types of outputs:

➡️ Classification answers "Which category?"
➡️ Regression answers "What value?"

#AI #MachineLearning #DeepLearning #NeuralNetworks #LLM

[Pravesh Goyal @prvlabs]

Classification=>
The network predicts probabilities for each class using the Softmax function:

pᵢ = eᵒⁱ / Σⱼ eᵒʲ

All probabilities sum to 1
The class with the highest probability becomes the prediction

Example:
🐱 Cat: 0.85
🐶 Dog: 0.10
🐰 Rabbit: 0.05

#AI #LLM

[Pravesh Goyal @prvlabs]

ELU
A(z) = z, if z > 0
α(eᶻ − 1), if z ≤ 0
Range: (-α, ∞)
Use: Faster convergence and improved learning stability.
Shape: Smooth negative curve, linear positive side.

ReLU powers most CNNs, while GELU dominates modern Transformer architectures.
#AI #DeepLearning #LLM

[Pravesh Goyal @prvlabs]

Activation Functions determine how neurons transform information and are key to learning complex patterns.

Sigmoid =>
A(z) = 1 / (1 + e⁻ᶻ)
Range: (0, 1)
Use: Binary classification outputs and probabilities.
Shape: S-curve.

[Pravesh Goyal @prvlabs]

Leaky ReLU =>
A(z) = max(αz, z), α ≈ 0.01
Range: (-∞, ∞)
Use: Avoids dead neurons by allowing a small negative gradient
Shape: Slight slope for negative values

GELU =>
A(z) = z · Φ(z)
Range: (-∞, ∞)
Use: Default activation in many Transformer models and LLMs

[Pravesh Goyal @prvlabs]

Input × Weights + Bias → Activation → Learned Features
Without hidden layers, neural networks would struggle to solve complex real-world problems.
#AI #MachineLearning #DeepLearning #NeuralNetworks

[Pravesh Goyal @prvlabs]

Hidden Layers are where a neural network learns patterns from data.
Each hidden layer uses:
⚖️ Weights → determine the importance of inputs
➕ Biases → shift the computation
⚡ Activation Functions → introduce nonlinearity, allowing the model to learn complex relationships

[Pravesh Goyal @prvlabs]

Examples:
📝Text → words/tokens converted into vectors
🎵 Audio → sound represented by extracted features 🖼️ Images → pixels represented by RGB values
Different data types, same goal: transform real-world information into numbers that neural networks can understand.
#AI

[Pravesh Goyal @prvlabs]

Input is the starting point of a Feed-Forward Neural Network (FFNN).
The network receives data as a vector of numbers (an embedding), which represents information in a machine-readable format.
#AI #MachineLearning #DeepLearning #NeuralNetworks