Questions

Machine Learning

Machine Learning Concepts

Tip

Answer

Tip

Key challenges in machine learning include 

- insufficient or low-quality data
- poor feature selection
- non-representative samples
- models that either underfit (too simple) or overfit (too complex)

Answer

LLM Fundamentals

Answer

Tip

Bias: Error from incorrect assumption in the model
- High bias leads to underfitting, where the model fails to capture patterns in the training data
Variance: Error from sensitivity to small fluctuations in the training data
- High variance leads to overfitting, where the model memorizes noise instead of learning generalization patterns

The bias-variance tradeoff in ML describe the tension between ability to fit training data and ability to generalize the new data

bias-variance in LLM:

Model Parameters: Capacity vs. Overfitting
- Too few parameters: A model with insufficient (e.g. small transformer) cannot capture complex patterns in the data, leading to high bias.
A small LLM might fail to understand language or generate coherent long texts.
- Too many parameters: A model with excessive capacity risks overfitting to training data, memorizing noise and details instead of learning generalizable patterns
A large LLM fine-tuned on a small dataset may generate text that is statistically similar to the training data but lack coherence and factual accuracy. (e.g. hallucinations)
- Balancing Act:
More parameters reduce bias by enabling the model to capture complex patterns but increase variance if not regularized. Regularization techniques: (e.g dropout, weight decay) help mitigate overfitting in high-parameter models
Training Epochs: Learning Duration vs. Overfitting
- Too few epochs: The model hasn’t learned enough from the data, leading to high bias.
A transformer trained for only 1 epoch may fail to capture meaningful relationships in the text.
- Too many epochs: The model starts memorizing training data, increasing variance. This is common in transformer with high capacity and small datasets
A transformer fine-tuned on a medical dataset for 100 epochs may overfit to rare cases, leading to poor generalization.
- Tradeoff in Transformers
Training loss decreases with epochs (low bias), but validation loss eventually increase (high variance).

Early stopping is critical for transformers to avoid overfitting, especially when training on small or noisy datasets.
Noise vs Representativeness
- Low-quality data: Noisy, biased, or incomplete data prevents the model from learning accurate patterns, increasing biase.
A transformer trained on a dataset with limited examples of rare diseases may fail to diagnose them accurately
- Noisy/unrepresentative data: The model learns inconsistent patterns, increasing variance.
A dataset with duplicate or corrupted text may cause the model to overfit. A transformer trained on a dataset with biased political content may generate polarized outputs. Data augmentation (e.g. paraphrasing, back-translation) increases diversity, mitigating overfitting

Answer

Tip

Regularization adds a penalty term to the loss function so that the optimizer favours simpler or smoother solutions. In practice it is usually added to a model‑level loss (cross‑entropy, MSE, …) as a separate scalar that scales with the weights.

[ \text{Loss}{\text{regularized}} = \text{Loss}{\text{original}} + \lambda \cdot \text{Penalty}(w) ]

Feature	L1 (Lasso)	L2 (Ridge)
Weight Behavior	Many → 0 (sparse)	“All → small, non-zero”
Feature Selection	Yes	No
Solution	Not always unique	Always unique
Robust to Outliers	Less	More

Key Insight:

L1 regularization is more robust to outliers in the data (target outliers)
L2 regularization is more robust to outliers in the features (collinearity)

L1/L2 in LLM:

Use L2 by default. Use L1 if you want sparse, interpretable updates.
L2 keeps updates smooth. L1 keeps updates minimal — and that’s often better for deployment.
Use L2 to win benchmarks. Use L1 to ship to users.

Sparse LoRA = Tiny Adapters

Faster Inference (Real Speedup!)

Better Generalization (Less Overfitting)

Interpretable Fine-Tuning

Clean Model Merging

  ┌──────────────────────┐
  │ Fine-tuning an LLM?  │
  └────────┬─────────────┘
          │
          ▼
  ┌──────────────────────┐     YES → Use L2 (weight_decay=0.01)
  │ Large, clean data?   │───NO──►┐
  └────────┬─────────────┘        │
          │                    │
          ▼                    ▼
  ┌──────────────────────┐ ┌──────────────────────────┐
  │ Need max accuracy?   │ │ Want small/fast model?   │
  └────────┬─────────────┘ └────────────┬─────────────┘
          │                          │
          YES                        YES
          │                          │
          ▼                          ▼
      Use L2                     Use L1 (+ pruning)

Answer

Tip

Where dropout appears in a Transformer
- Attention dropout
- Feedforward dropout
- Residual dropout
The ensemble view of dropout in a Transformer
- Each layer (and even each neuron) may be dropped independently.
- A particular dropout mask defines one specific subnetwork (one “member” of the ensemble).

During training: Randomly turn off some neurons (like flipping a coin for each one). This forces the network to learn many different “sub-networks” — each time you train, a different combination of neurons is active.

During testing (inference): Instead of picking one sub-network, we use all neurons, but scale down their strength (usually by half if dropout rate is 50%). This is the “mean network.”

Why this is like an ensemble: Imagine you could run the model 1,000 times (or times for N neurons), each time with a different random set of neurons turned off, and then average all their predictions. That would be a huge ensemble of sub-networks — very accurate, but way too slow. Dropout’s trick: Using the scaled “mean network” at test time gives exactly the same prediction as if you had averaged the geometric mean of all those possible sub-networks.

Dropout = training lots of sub-networks, inference = using their collective average — fast and smart.

Answer

Tip

Higher dimensions → sparser data → harder to learn meaningful relationships.

The curse of dimensionality refers to the set of problems that arise when data or model representations exist in high-dimensional spaces.

Data sparsity: Points become exponentially sparse — distances between points tend to concentrate, making similarity less meaningful.
Combinatorial explosion: The volume of the space grows exponentially , so covering it requires exponentially more data.
Poor generalization: Models struggle to learn smooth mappings because there’s too little data to constrain the high-dimensional space.

Token in Transformers Transformers process tokens as vectors in a high-dimensional embedding space (e.g., 768 or 4096 dimensions). However — self-attention treats each token as a set element rather than a sequence element. The attention mechanism itself has no built-in sense of order. The model only knows “content similarity,” not which token came first or last.

Without order, the model would need to learn positional relationships implicitly across high-dimensional embeddings. That’s hard — and it exacerbates the curse of dimensionality because:

There’s no geometric bias for position.
Each token embedding can drift freely in a massive space.
The model must infer ordering purely from statistical co-occurrence — requiring more data and more parameters.

How Positional Encodings Help

Positional encodings (PEs) inject structured, low-dimensional information about sequence order directly into the embeddings. - Adds a geometric bias to embeddings — nearby positions have nearby encodings. - Reduces the effective search space — positions are no longer independent random vectors. - Enables extrapolation: the sinusoidal pattern generalizes beyond training positions. - The model can compute relative positions via linear operations (e.g., dot products of PEs reflect distance).

Answer

Tip

Cross-Entropy: Cross-entropy measures how well a probabilistic model predicts a target distribution — in LM, how well the model assigns high probability to the correct next tokens.
Perplexity: Perplexity (PPL) is simply the exponentiation of the cross-entropy
BLEU (Bilingual Evaluation Understudy): BLEU is an n-gram overlap metric for evaluating machine translation or text generation quality against reference texts

Perplexity is rephrasing cross-entropy in a more intuitive, more human-readable.

Perplexity = “How predictable is the language?”

BLEU = “How much does the output match a reference?”

Example:

Reference: “The cat is on the mat.”

Model output: “The dog is on the mat.”

→ Low perplexity (grammatical, fluent)

→ Low BLEU (wrong content) BLEU is non-probabilistic and reference-based — unlike cross-entropy and perplexity.

⚠️ When Perplexity Is Misleading???

Perplexity only measures how well the model predicts tokens probabilistically — not how meaningful or correct the generated text is.

Different tokenizations or vocabularies
- A model with smaller tokens or subwords might have lower perplexity just because predictions are more granular, not actually better linguistically.
Domain mismatch
- A model trained on Wikipedia might have low perplexity on Wikipedia text but produce incoherent answers to questions — it knows probabilities, not task structure.
Human-aligned vs statistical objectives
- A model can assign high likelihood to grammatical but dull continuations (e.g., “The cat sat on the mat”) while rejecting creative or rare but correct continuations — good perplexity, poor real-world usefulness.
Non-autoregressive or non-likelihood models
- For encoder-decoder or retrieval-augmented systems, perplexity may not correlate with generation quality because these models are not optimized purely for next-token prediction.
Overfitting
- A model with very low perplexity on training data may memorize text, but generalize poorly (BLEU or human eval drops).

Answer

Fundamentals of Large Language Models (LLMs)

Question Bank

Fundamentals of Large Language Models (LLMs)

LLM Basic

Tip

Answer

Tip

Causal Decoder (Decoder-Only): Autoregressive model that generates text left-to-right, attending only to previous tokens.
Prefix Decoder (PrefixLM): Causal decoder with a bidirectional prefix (input context) followed by autoregressive generation.
Encoder-Decoder (Seq2Seq): Two separate Transformer stacks(Encoder & Decoder)

Causal Decoder

Prompt

Translate to French: The cat is on the mat.

Generation (autoregressive, causal mask):

Le [only sees “Le”]

Le chat [sees “Le chat”]

Le chat est [sees “Le chat est”]

Le chat est sur [sees up to “sur”]

Le chat est sur le [sees up to “le”]

Le chat est sur le tapis. [final]

Summary

Cannot see future tokens

Cannot see full input bidirectionally — but works via prompt engineering

Prefix Decoder

Input Format

[Prefix] The cat is on the mat. [SEP] Translate to French: [Generate] Le chat est sur le tapis.

Attention

Prefix (The cat is on the mat. [SEP] Translate to French:) → bidirectional

Encoder-Decoder

Answer

Tip

Causal LM: every token can only attend to previous tokens.
Prefix LM: the prefix can be fully bidirectional, while the rest is generated causally.

Feature	Causal LM	Prefix LM
Attention	Strictly left-to-right	Prefix: full; Generation: causal
Use case	Free-form generation	Conditional generation, prefix tuning
Examples	GPT, Llama, Mixtral	T5 (prefix mode), UL2, some prompt-tuning models
Future access?	No	Only inside prefix
Mask complexity	Simple	Mixed masks

Layer Normalization Variants

Tip

Answer

Tip

Norm	Formula	Pros	Cons
BatchNorm	Normalize across batch	Great for CNNs	Bad for variable batch / autoregressive decoding
LayerNorm	Normalize across hidden dim	Stable for Transformers	Slightly more compute than RMSNorm
RMSNorm	Normalize only scale	Faster, more stable in LLMs	No centering → sometimes slightly less expressive

Answer

Tip

~~Pre-NormPost-Norm~~ (Original Transformer, 2017): Normalizes after adding the residual.
- Pros:
  - Fairly stable for shallow models (<12 layers)
  - Works well in classic NMT models
- Cons:
  - Fails to train deep models (vanishing/exploding gradients)
  - Poor gradient flow
  - Not used in modern LLMs
Pre-Norm (Current Standard in GPT/LLaMA): Normalize before attention or feed-forward
- Pros:
  - Much more stable for deep Transformers
  - Great training stability up to hundreds of layers
  - Works well with small batch sizes
  - Default in GPT-2/3, LLaMA, Mistral, Gemma, Phi-3, Qwen2
- Cons:
  - Residual stream grows in magnitude unless controlled (→ RMSNorm or DeepNorm often added)
  - Slightly diminished expressive capacity compared to Post-Norm (but negligible in practice)
Sandwich-Norm: LayerNorm applied before AND after sublayers.
- Pros:
  - Extra stability & smoothness
  - Improved optimization in some NMT models
- Cons:
  - Expensive (two norms per sublayer)
  - Rarely used in large decoder-only LLMs

🧠 Why LayerNorm position matters

1. Training Stability
    •	Pre-Norm prevents exploding residuals
    •	Post-Norm accumulates errors → unstable for deep models
2. Gradient Flow
    - Residuals in Pre-Norm allow gradients to bypass the sublayers directly.

Answer

Activation Functions in LLMs

Attention Mechanisms — Advanced Topics

Tip

Answer

Cross-Attention

Transformer Operations

LLM Loss Functions

Similarity & Contrastive Learning

Advanced Topics in LLMs

Advanced LLM

Fine-Tuning Large Models

General Fine-Tuning

Tip

Answer

SFT Tricks

Training Experience

LangChain and Agent-Based Systems

LangChain Core

Long-Term Memory in Multi-Turn Conversations

Practical RAG Q&A using LangChain

Retrieval-Augmented Generation (RAG)

RAG Basics

RAG Concepts

RAG Layout Analysis

RAG Retrieval Strategies

RAG Evaluation

RAG Optimization

Graph RAG

Parameter-Efficient Fine-Tuning (PEFT)

PEFT Fundamentals

Adapter Tuning

::::