PyTorch in Practice: An Applications-First Approach (LFD473)

The course begins with an overview of PyTorch, including model classes, datasets, data loaders and the training loop. Next the role and power of transfer learning is addressed along with how to use it with pretrained models. Practical lab exercises cover multiple topics including image classification, object detection, sentiment analysis, text classification, and text generation/completion. Learners also will use their data to fine-tune existing models and leverage third-party APIs.
 

This course is designed for machine learning practitioners who want to add deep learning models in PyTorch - especially pretraining models for Computer Vision and Natural Language Processing - to quickly protype and deploy applications.
 

While there are no formal prerequisites, students should have some knowledge of Python (notions of object-oriented programming), PyData Stack (Numpy, Pandas, Matplotlib, Scikit-Learn), and Machine Learning concepts (supervised learning, loss functions, train-validation-test split, evaluation metrics).
 

Introduction

• Who You Are
• Who we are
• Copyright and No Confidential Information
• Training
• Certification Programs and Digital Badging

PyTorch, Datasets, and Models

• What is PyTorch
• The PyTorch Ecosystem
• Supervised vs Unsupervised Learning
• Software Development vs Machine and Deep Learning
• ``Hello Model'
• Naming Is Hard
• Setup and Environment

Building Your First Dataset

• Tensors, Devices, and CUDA
• Datasets
• Dataloaders
• Datapipes
• Lab 1A: Non-Linear Regression

Training Your First Model

• Recap
• Models
• Loss Functions
• Gradients and Autograd
• Optimizers
• The Raw Training Loop
• Evaluation
• Saving and Loading Models
• NonLinearities
• Lab 1B: Non-Linear Regression

Building Your First Datapipe

• A New Dataset
• Lab 2: Price Prediction
• Tour of High Level Libraries

Transfer Learning and Pretrained Models

• What is Transfer Learning?
• Torch Hub
• Computer Vision
• Dropout
• ImageFolder Dataset
• Lab 3: Classifying Images

Pretrained Models for Computer Vision

• PyTorch Image Models
• HuggingFace

Natural Language Processing

• Natural Language Processing
• One Logit or Two Logits?
• Cross-Entropy Loss
• TensorBoard
• Lab 4: Sentiment Analysis
• Hugging Face Pipelines
• Generative Models

Image Classification with Torchvision

• Torchvision
• Pretrained Models as Feature Extractors

Fine-Tuning Pretrained Models for Computer Vision

• Fine Tuning Pretained Models
• Zero-shot Image Classification

Serving Models with TorchServe

• Archiving and Serving Models
• TorchServe

Datasets and Transformations for Object Detection and Image Segmentation

• Object Detection, Image Segmentation, and Keypoint Detection
• Bounding Boxes
• Torchvision Operators
• Transforms (V2)
• Custom Dataset for Object Detection
• ab 5A: Fine-Tuning Object Detection Models

Models for Object Detection and Image Segmentation

• Models
• Lab 5B: Fine-Tuning Object Detection Models

Models for Object Detection Evaluation

• Recap
• Making Predictions
• Evaluation
• YOLO
• HuggingFace Pipelines for Object Detection
• Zero-Shot Object Detection

Word Embeddings and Text Classification

• Torchtext
• AG News Dataset
• Tokenization
• Embeddings
• Vector Databases
• Zero-Shot Text Classification
• Chunking Strategies
• Lab 6: Text Classification using Embeddings

Contextual Word Embeddings with Transformers

• Attention is All You Need
• Transformer
• An Encoder-Based Model for Classification
• Contextual Embeddings

Huggingface Pipelines for NLP Tasks

• HuggingFace Pipelines
• Lab 7: Document Q&A

Question and Answer, Summarization, and LLMs

• EDGAR Dataset
• Hallucinations
• Asymmetric Semantic Search
• ROUGE Score
• Decoder-Based Models
• Large Language Models (LLMs)

Closing and Evaluation Survey

• Evaluation Survey