+91 96452 47165

Courses > LLM – Build From Scratch (Foundations to Minimal Working LLM) – Zoom

LLM – Build From Scratch (Foundations to Minimal Working LLM) – Zoom

Students

Last Updated: Sep 02, 2025

About Course

This hands-on course is designed to demystify the inner workings of Large Language Models (LLMs) by guiding you through building one from the ground up — without relying on HuggingFace or pre-trained models. Over 2 weeks, you’ll implement every core component using Python, NumPy, and PyTorch (or JAX), gaining a deep understanding of tokenization, attention mechanisms, training loops, and inference logic. By the end, you’ll have your very own TinyGPT model that you can train, generate text with, and deploy!

Course Content

Day 1 – LLM Foundations & Math Prerequisites

What is an LLM and How Does It Work?
Mathematical Essentials for Transformers

Day 2 – Tokenization and Vocabulary Building

Day 3 – Attention Mechanism from Scratch

Day 4 – Multi-Head Attention & Positional Encoding

Day 5 – Build a Mini Transformer Block

Day 6 – Create Training Data & Setup Training Loop

Day 7 – Train Your First LLM (TinyGPT)

Day 8 – Add Masking & Padding for Efficiency

Day 9 – Model Optimization + Parameter Scaling

Day 10 – Capstone: Build & Showcase a Working Mini-LLM

Available in:

+91 96452 47165

About Course

What Will You Learn?

Course Content

Day 1 – LLM Foundations & Math Prerequisites

What is an LLM and How Does It Work?

Mathematical Essentials for Transformers

Day 2 – Tokenization and Vocabulary Building

Build a Character-Level Tokenizer

Prepare Input/Output Sequences for Training

Hands-on: Build your own tokenizer.py

Day 3 – Attention Mechanism from Scratch

Self-Attention Explained Intuitively & Mathematically

Implement Scaled Dot-Product Attention in NumPy

Day 4 – Multi-Head Attention & Positional Encoding

Why Multi-Head Attention?

Positional Encoding (Sinusoidal / Learnable)

Hands-on: Implement multi-head attention and positional encodings.

Day 5 – Build a Mini Transformer Block

Layer Norm, Residual Connections, and Feedforward Layers

Stack Your First Decoder Block

Hands-on: transformer_block.py with full forward pass

Day 6 – Create Training Data & Setup Training Loop

Generate a Tiny Dataset (e.g., Tiny Shakespeare)

Define Training Objectives

Hands-on: Build training loop using only PyTorch/NumPy

Day 7 – Train Your First LLM (TinyGPT)

Training a Transformer from Scratch

Text Generation with Sampling/Greedy Search

Hands-on: train.py + generate.py complete

Day 8 – Add Masking & Padding for Efficiency

Causal Masking for Decoder-Only LLMs

Batching & Efficient Training

Day 9 – Model Optimization + Parameter Scaling

Scaling Tips: Batch Size, Layers, Token Count

Model Checkpointing + Saving to Disk

Day 10 – Capstone: Build & Showcase a Working Mini-LLM

Train on Your Own Dataset (Poetry, Code, Tweets)

Deploy as a CLI or Gradio App

Hands-on: Final project + 5-min demo presentation

More Courses

Performance Marketing for eCommerce

KC Jagadeep

HuggingFace for Developers

KC Jagadeep

Other Courses In This Category:

This Course Includes:

Material Includes

Requirements

Share

Audience

Quick Links

Explore

Newsletter