+91 96452 47165

Lessons > Positional Encoding (Sinusoidal / Learnable)

Positional Encoding (Sinusoidal / Learnable)

Course Content
Day 1 – LLM Foundations & Math Prerequisites
0/2
What is an LLM and How Does It Work?
Mathematical Essentials for Transformers
Day 2 – Tokenization and Vocabulary Building
0/3
Build a Character-Level Tokenizer
Prepare Input/Output Sequences for Training
Hands-on: Build your own tokenizer.py
Day 3 – Attention Mechanism from Scratch
0/2
Self-Attention Explained Intuitively & Mathematically
Implement Scaled Dot-Product Attention in NumPy
Day 4 – Multi-Head Attention & Positional Encoding
0/3
Why Multi-Head Attention?
Positional Encoding (Sinusoidal / Learnable)
Hands-on: Implement multi-head attention and positional encodings.
Day 5 – Build a Mini Transformer Block
0/3
Layer Norm, Residual Connections, and Feedforward Layers
Stack Your First Decoder Block
Hands-on: transformer_block.py with full forward pass
Day 6 – Create Training Data & Setup Training Loop
0/3
Generate a Tiny Dataset (e.g., Tiny Shakespeare)
Define Training Objectives
Hands-on: Build training loop using only PyTorch/NumPy
Day 7 – Train Your First LLM (TinyGPT)
0/3
Training a Transformer from Scratch
Text Generation with Sampling/Greedy Search
Hands-on: train.py + generate.py complete
Day 8 – Add Masking & Padding for Efficiency
0/2
Causal Masking for Decoder-Only LLMs
Batching & Efficient Training
Day 9 – Model Optimization + Parameter Scaling
0/2
Scaling Tips: Batch Size, Layers, Token Count
Model Checkpointing + Saving to Disk
Day 10 – Capstone: Build & Showcase a Working Mini-LLM
0/3
Train on Your Own Dataset (Poetry, Code, Tweets)
Deploy as a CLI or Gradio App
Hands-on: Final project + 5-min demo presentation
LLM – Build From Scratch (Foundations to Minimal Working LLM)

Incorporate sequence order without recurrence.

Previous
Next