Sunset Learning Turns 30! Discover our story, exclusives, and celebrations →
→
- COURSE
Fine-Tuning Your Own LLM with LoRA & QLoRA (AI-FINE-TUNE)
Price:
Duration:
Certification:
Exam:
Continuing Education Credits:
Learning Credits:
The definitive 3-day bootcamp that turns data scientists and ML engineers into owners of production-grade, proprietary LLMs. Using ClearML as the single source of truth and LoRA/QLoRA as the fastest path to custom performance, you will run 30+ real fine-tuning experiments, beat public models on your domain, and leave with a registered, reproducible, ready-to-deploy checkpoint. Ten 30-minute labs per day – no fluff, no downtime.
Upcoming Class Dates and Times
All Sunset Learning courses are guaranteed to run
Course Outline and Details
Prerequisites
- Solid Python and basic Linux
- Familiarity with Hugging Face Transformers
- GPU labs provided (A100/H100)
Target Audience
- Data scientists & ML engineers ready to stop renting LLMs
- Teams building internal agents, RAG, or vertical AI solutions
Course Objectives
- 30 tracked ClearML experiments (minimum)
- LoRA & QLoRA on 8B–70B models with single-GPU budgets
- Unsloth + Flash-Attention-2 speed tricks
- Automated evaluation and model selection
- ClearML Model Registry mastery
Course Outline
Foundations of Fine-Tuning & ClearML Setup
- Renting vs. Owning: The Case for Private Models
- ClearML for LLM Ops: Tracking the Chaos
- Lab: Environment Setup (CUDA, PyTorch, ClearML)
- Lab: Pulling & Caching Base Models (Llama 3 / Mistral)
- Lab: Initializing Your First ClearML Experiment
Data Engineering for Instruction Tuning
- Data Formats: Alpaca, ShareGPT, and Chat Templates
- Tokenization Strategies & Context Windows
- Lab: Preparing Raw Text for Instruction Tuning
- Lab: Applying Chat Templates Correctly
- Lab: Versioning Datasets in ClearML Data
PEFT: Low-Rank Adaptation (LoRA)
- LoRA Intuition: Rank, Alpha, and Matrices
- Target Modules: What to Fine-Tune?
- Lab: Defining LoRA Configurations with PEFT
- Lab: Your First LoRA Run (Small Model)
- Lab: Inspecting Trainable Parameters vs. Frozen Weights
QLoRA: Quantization & Efficiency
- The QLoRA Breakthrough: 4-bit Normal Float
- Double Quantization & Paged Optimizers
- Lab: Loading Base Models in 4-bit Precision
- Lab: Running a QLoRA Fine-Tune on Single GPU
- Lab: Monitoring GPU VRAM Usage in ClearML
High-Performance Tuning with Unsloth
- Unsloth Architecture: Manual Autograd & Speed
- Flash Attention 2: The Physics of Speed
- Lab: Refactoring for Unsloth Integration
- Lab: Benchmarking Training Speed (Vanilla vs. Unsloth)
- Lab: Handling Long Contexts efficiently
Hyperparameter Optimization (HPO)
- The Geometry of Loss: LR, Batch Size, and Epochs
- Overfitting in LLMs: The "Catastrophic Forgetting" Trap
- Lab: HPO Sweep: Varying LoRA Rank (r)
- Lab: HPO Sweep: Learning Rate Schedulers
- Lab: Comparing Loss Curves in ClearML Dashboard
Evaluation & Selection
- Beyond Loss: Perplexity vs. Real-World Task Evals
- LLM-as-a-Judge Patterns
- Lab: Running a Perplexity Evaluation
- Lab: Implementing an LLM-Based Grader
- Lab: Visualize & Compare Generations in ClearML
Model Management & Production Export
- The Model Registry: From Experiment to Artifact
- Merging Weights: Folding LoRA into Base
- Lab: Merging Adapters for Inference
- Lab: Registering the Golden Model in ClearML
- Lab: Exporting to GGUF / Ollama (Optional)
Capstone: Domain-Specific Fine-Tune
- Capstone Brief: The Medical/Legal Specialist
- Lab: Ingest & Format Domain Data
- Lab: Execute Optimized QLoRA Run
- Lab: Evaluate Against Base Model
- Lab: Publish Final Model Checkpoint
Continuing Education
- Continuing Education (Advanced Quantization)
- Curriculum Path: Building Agents with Your Model