Sunset Learning Turns 30! Discover our story, exclusives, and celebrations →
→
- COURSE
Fine-Tuning Large Language Models (FT-LLM)
Price:
Duration:
Certification:
Exam:
Continuing Education Credits:
Learning Credits:
You will develop the skills to gather, clean, and organize data for fine-tuning pre-trained LLMs and Generative AI models. Through a combination of lectures and hands-on labs, you will use Python to fine-tune open-source Transformer models. Gain practical experience with LLM frameworks, learn essential training techniques, and explore advanced topics such as quantization. During the hands-on labs, you will access a GPU-accelerated server for practical experience with industry-standard tools and frameworks.
Upcoming Class Dates and Times
All Sunset Learning courses are guaranteed to run
Course Outline and Details
Prerequisites
- Python – PCEP Certification or Equivalent Experience
- Familiarity with Linux
Target Audience
- Project Managers
- Architects
- Developers
- Data Acquisition Specialists
Course Objectives
- Clean and Curate Data for AI Fine-Tuning
- Establish guidelines for obtaining RAW Data
- Go from Drowning in Data to Clean Data
- Fine-Tune AI Models with PyTorch
- Understand AI architecture: Transformer model
- Describe tokenization and word embeddings
- Install and use AI frameworks like Llama-3
- Perform LoRA and QLoRA Fine-Tuning
- Explore model quantization and fine-tuning
- Deploy and Maximize AI Model Performance
Course Outline
Learning Your Environment
- Using Vim
- Tmux
- VScode Integration
- Revision Control with GitHub
Data Curation for AI
- Curating Data for AI
- Gathering Raw Data
- Data Cleaning and Preparation
- Data Labeling
- Data Organization
- Premade Datasets for Fine Tuning
- Obtain and Prepare Premade Datasets
Deep Learning
- What is Intelligence?
- Generative AI
- The Transformer Model
- Feed Forward Neural Networks
- Tokenization
- Word Embeddings
- Positional Encoding
Pre-trained LLM
- A History of Neural Network Architectures
- Introduction to the LLaMa.cpp Interface
- Preparing A100 for Server Operations
- Operate LLaMa3 Models with LLaMa.cpp
- Selecting Quantization Level to Meet Performance and Perplexity Requirements
Fine Tuning
- Fine-Tuning a Pre-Trained LLM
- PyTorch
- Basic Fine Tuning with PyTorch
- LoRA Fine-Tuning LLaMa3 8B
- QLoRA Fine-Tuning LLaMa3 8B
Operating Fine-Tuned Model
- Running the llama.cpp Package
- Deploy Llama API Server
- Develop LLaMa Client Application
- Write a Real-World AI Application using the Llama API
Course Delivery Options
Train face-to-face with the live instructor. (Please note, not all classes will have this option)
Attend the live class from the comfort of your home or office.
Join us in person at our Denver or Reston training facilities! Learn alongside a live, remote instructor in our HD-equipped classrooms. We love having students on-site! An SLI sales rep can confirm availability and reserve your seat.
Access to on-demand training content anytime, anywhere. (Please note, not all classes will have this option)