Introduction to Machine and Deep Learning
Date: October 24, 2024
Lecture Duration: 2.5 hours
Topic Overview: This lecture serves as a foundational bridge into modern AI, introducing the core concepts of Machine Learning (ML) and Deep Learning (DL). We explore how systems learn from data to make predictions and decisions without explicit programming, laying the groundwork for advanced computer vision architectures.
1. Introduction to Machine Learning
We discussed the fundamental paradigms and key concepts that define classical machine learning:
- Learning Paradigms:
- Supervised Learning: Training with labeled data to predict outcomes (Classification and Regression).
- Unsupervised Learning: Finding hidden structures in unlabeled data (Clustering and Dimensionality Reduction).
- Reinforcement Learning: Learning optimal actions through interactions and reward signals.
- Key ML Concepts: We defined essential terminology including Features, Labels, Training/Validation/Test splits, and the common pitfall of Overfitting.
2. Introduction to Deep Learning
We transitioned from classical ML to Deep Learning, focusing on the mechanics of artificial neural networks:
- Basic Network Architecture:
- Layers: Understanding the Input Layer, Hidden Layers (which extract complex patterns), and the Output Layer.
- Forward Propagation: The flow of data through the network, applying weights, biases, and activation functions to generate predictions.
- Backward Propagation (Backprop): The process of calculating prediction error using a loss function and updating network weights via optimization algorithms like Gradient Descent to minimize that error.
- Convolutional Neural Networks (CNNs):
- Transitioning from fully connected layers to convolutional layers designed specifically for structured grid data like images.
- Understanding Convolutional Filters, Pooling for down-sampling, and feature hierarchy.
3. Training a Neural Network
To optimize the network, several mathematical components must interact:
- Loss Function & Optimizer: Quantifying the difference between predictions and labels, and defining how to update weights.
- Learning Rate: A crucial hyperparameter determining the step size taken during optimization.
- Common Training Pathologies: Mitigating issues like Overfitting, Underfitting, and Vanishing/Exploding Gradients.
4. Deep Learning Implementation in PyTorch
We put theory into practice by introducing the PyTorch framework.
- PyTorch Workflow: Moving from data preparation (
DatasetandDataLoader) to model definition (nn.Module). - Training Loop: We wrote a full training routine encompassing the forward pass, loss calculation, backpropagation, and optimizer steps.
- Classical ML Implementations: In addition to deep learning, the notebook also covers fundamental machine learning code implementations such as Classification, Regression, Evaluation Metrics, and Hyperparameter Optimization with Optuna.
Interactive Demonstration
Below is the complete Jupyter Notebook used in class. It features foundational code snippets demonstrating image dataset loading, CNN architecture definition, and a complete training loop using PyTorch.