18. PyTorch --- Big Picture
PyTorch lets us build and train neural networks.
Core flow:
Data
↓
Tensor
↓
Dataset
↓
DataLoader
↓
Neural Network
↓
Prediction
↓
Loss
↓
Backpropagation
↓
Optimizer
↓
Updated parameters
We have currently covered through Neural Network + basic ReLU.
19. PyTorch Tensors
A tensor is similar to a NumPy array but is designed for deep learning, GPU/device computation, and automatic differentiation.
import torch
x = torch.tensor([1, 2, 3])
Dimensions
Scalar:
torch.tensor(5)
0D
Vector:
torch.tensor([1, 2, 3])
1D
Matrix:
torch.tensor([
[1, 2],
[3, 4]
])
2D
20. Tensor Shape
X = torch.tensor([
[1, 2, 3],
[4, 5, 6]
])
print(X.shape)
torch.Size([2, 3])
For ML:
(batch_size / samples, features)
Example:
[32, 10]
32 samples
10 features each
Important: When PyTorch gives a matrix multiplication error, check tensor shapes first.
21. Tensor dtype
Neural networks commonly use floating-point tensors:
x = torch.tensor(
[1, 2, 3],
dtype=torch.float32
)
Common:
torch.float32
Labels may require different types depending on the loss function being used.
22. Creating Tensors
torch.tensor([1, 2, 3])
torch.zeros(3, 4)
torch.ones(2, 3)
torch.rand(2, 3)
Random values are important because neural-network parameters are typically initialized automatically before training.
23. Tensor Indexing
x = torch.tensor([
[10, 20, 30],
[40, 50, 60]
])
x[1, 1] # 50
x[0] # first row
x[:, 0] # first column
Very similar to NumPy.
24. Tensor Operations
x * 2
x + 5
x.sum()
x.mean()
x.min()
x.max()
Operations are generally vectorized.
25. Element-wise vs Matrix Multiplication
a * b
performs element-wise multiplication.
a @ b
or:
torch.matmul(a, b)
performs matrix multiplication/dot-product behavior depending on dimensions.
Neural networks heavily rely on matrix multiplication.
26. Reshaping
x = torch.tensor([1, 2, 3, 4, 5, 6])
x = x.reshape(2, 3)
Becomes:
[[1, 2, 3],
[4, 5, 6]]
Same values, new arrangement.
27. NumPy ↔ PyTorch
NumPy to PyTorch:
tensor = torch.from_numpy(array)
PyTorch to NumPy:
array = tensor.numpy()
Common workflow:
Pandas
↓
NumPy / preprocessing
↓
PyTorch tensors
28. CPU / GPU / MPS
Choose device:
device = torch.device(
"cuda" if torch.cuda.is_available()
else "cpu"
)
Move tensor:
x = x.to(device)
On supported Apple hardware:
if torch.backends.mps.is_available():
device = torch.device("mps")
else:
device = torch.device("cpu")
Important: Model and input tensors generally need to be on compatible/same devices.
For small learning projects, CPU is fine.
29. Dataset
A PyTorch Dataset represents your samples and labels.
from torch.utils.data import TensorDataset
dataset = TensorDataset(X, y)
Access sample:
X_sample, y_sample = dataset[0]
Think:
Dataset = WHAT data do I have?
30. Custom Dataset
from torch.utils.data import Dataset
class StudentDataset(Dataset):
def __init__(self, X, y):
self.X = X
self.y = y
def __len__(self):
return len(self.X)
def __getitem__(self, index):
return self.X[index], self.y[index]
Important methods:
__init__ → store/setup data
__len__ → number of samples
__getitem__ → retrieve one sample
Real-world use
A scam-text Dataset might:
Read text
↓
Tokenize/vectorize
↓
Create tensor
↓
Return (input, label)
31. DataLoader
The DataLoader feeds Dataset samples in batches.
from torch.utils.data import DataLoader
loader = DataLoader(
dataset,
batch_size=32,
shuffle=True
)
Loop:
for X_batch, y_batch in loader:
...
Think:
Dataset = WHAT is the data?
DataLoader = HOW is it fed to the model?
32. Batch Size
Instead of processing the entire dataset at once:
10,000 samples
we might process:
32
↓
32
↓
32
↓
...
with:
batch_size=32
This is mini-batch training.
Benefits include manageable memory use and frequent parameter updates.
Common batch sizes:
16
32
64
128
There is no universally best batch size.
33. Shuffling
Training:
DataLoader(
train_dataset,
batch_size=32,
shuffle=True
)
Evaluation:
DataLoader(
test_dataset,
batch_size=32,
shuffle=False
)
Shuffling prevents the model from repeatedly receiving training samples in a potentially problematic fixed ordering.
34. Epoch
One epoch = one full pass through the training dataset.
Example:
1,000 samples
batch size = 100
Approximately:
10 batches per epoch
If:
epochs = 5
then approximately:
50 batch iterations
35. Neural Networks --- One Neuron
Simplified neuron:
inputs
↓
multiply by weights
↓
sum
↓
add bias
↓
output
Conceptually:
output = x₁w₁ + x₂w₂ + b
Where:
x = inputs
w = weights
b = bias
36. Weights
Weights are learnable parameters controlling how inputs influence outputs.
Initially, PyTorch initializes them.
During training:
prediction
↓
error/loss
↓
backpropagation
↓
adjust weights
↓
new prediction
Training is largely the process of finding better parameter values.
37. Bias
Bias is another learnable parameter.
Without bias:
x₁w₁ + x₂w₂
With bias:
x₁w₁ + x₂w₂ + b
It gives the layer additional flexibility rather than forcing outputs solely through weighted input combinations.
38. Layers --- nn.Linear
from torch import nn
layer = nn.Linear(
in_features=2,
out_features=4
)
Meaning:
2 input features
↓
4 neurons
↓
4 outputs
For:
nn.Linear(2, 4)
parameter shapes:
weights → (4, 2)
bias → (4,)
Each of the 4 neurons receives the 2 input features.
39. nn.Module
Base class for PyTorch neural-network modules.
class NeuralNetwork(nn.Module):
def __init__(self):
super().__init__()
It gives PyTorch the machinery needed to track parameters, move models between devices, save/load state, and more.
40. forward()
Defines how input travels through the network.
def forward(self, x):
x = self.layer1(x)
x = self.layer2(x)
return x
Normally call:
output = model(x)
rather than directly calling:
model.forward(x)
41. First Neural Network
class NeuralNetwork(nn.Module):
def __init__(self):
super().__init__()
self.layer1 = nn.Linear(2, 4)
self.relu = nn.ReLU()
self.layer2 = nn.Linear(4, 1)
def forward(self, x):
x = self.layer1(x)
x = self.relu(x)
x = self.layer2(x)
return x
Architecture:
2 input features
↓
Linear(2 → 4)
↓
ReLU
↓
Linear(4 → 1)
↓
1 output
42. Activation Functions --- Current Introduction
A network containing only stacked linear transformations still behaves like an overall linear transformation.
Activation functions introduce nonlinearity, allowing neural networks to model more complex relationships.
ReLU
ReLU(x) = max(0, x)
Examples:
-5 → 0
-1 → 0
0 → 0
3 → 3
8 → 8
PyTorch:
relu = nn.ReLU()
We will study activation functions more deeply in the next lesson.
43. nn.Sequential
A concise way to chain straightforward layers:
self.network = nn.Sequential(
nn.Linear(2, 4),
nn.ReLU(),
nn.Linear(4, 1)
)
Then:
def forward(self, x):
return self.network(x)
Useful when the model follows a simple sequential flow.
44. Learnable Parameters
For:
Linear(2 → 4)
ReLU
Linear(4 → 1)
First layer:
4 × 2 weights = 8
4 biases = 4
total = 12
Second layer:
1 × 4 weights = 4
1 bias = 1
total = 5
Total:
17 learnable parameters
Inspect:
for name, parameter in model.named_parameters():
print(name, parameter.shape)
ReLU has no weights or biases here.
45. What Training Really Means
The architecture:
2 → 4 → 1
usually stays fixed during ordinary training.
The learnable parameters change.
Random/initial parameters
↓
Forward pass
↓
Prediction
↓
Compare with correct answer
↓
Loss
↓
Backpropagation
↓
Optimizer updates parameters
↓
Repeat
This is the central neural-network training process.
46. Logistic Regression → Neural Network Connection
Classical logistic regression:
Features
↓
weighted combination
↓
classification
Neural network:
Features
↓
Linear layer
↓
Activation
↓
Linear layer
↓
...
↓
classification
The neural network can learn more complex nonlinear relationships.
Important: A more complex model is not automatically better. For some datasets, a simple classical model can outperform a neural network.