WEBVTT

1
00:00:00.000 --> 00:00:02.711
Part 6 of ML Libraries with a Bot.

2
00:00:02.711 --> 00:00:06.139
PyTorch on the CPU: First Steps with a
Bot.

3
00:00:06.139 --> 00:00:12.835
PyTorch is one of the most widely used
deep learning libraries, and you don't
need a

4
00:00:12.835 --> 00:00:14.828
GPU to start learning it.

5
00:00:14.828 --> 00:00:21.445
In this guide we install the CPU-only
build, check it works, and train the
smallest

6
00:00:21.445 --> 00:00:25.671
useful model: a straight line fitted to
noisy points.

7
00:00:25.671 --> 00:00:31.491
A bot explains the code, and we compare
what it says with PyTorch's docs.

8
00:00:32.250 --> 00:00:34.872
What the bot can and cannot do.

9
00:00:34.872 --> 00:00:38.170
Explain the training loop line by line.

10
00:00:38.170 --> 00:00:42.568
Spot a missing zero_grad() or a wrong loss
function.

11
00:00:42.568 --> 00:00:45.613
Translate a NumPy idea into PyTorch.

12
00:00:45.613 --> 00:00:48.657
Know which wheel suits your machine.

13
00:00:48.657 --> 00:00:52.802
Use the selector on PyTorch's Start
Locally page.

14
00:00:52.802 --> 00:00:57.707
Guarantee identical numbers on another
machine or version.

15
00:00:57.707 --> 00:01:04.473
PyTorch's Reproducibility notes say
results aren't guaranteed across releases
or

16
00:01:04.473 --> 00:01:05.319
platforms.

17
00:01:05.319 --> 00:01:06.503
Promise speed.

18
00:01:06.503 --> 00:01:07.940
Time it yourself.

19
00:01:08.625 --> 00:01:16.569
Train something tiny and check the result
is right before you train something big.

20
00:01:16.569 --> 00:01:21.025
A model that can't learn y = 2x + 1 has a
bug.

21
00:01:21.625 --> 00:01:28.247
On PyTorch's Start Locally page, choose
your OS, Pip, Python and CPU, then run the

22
00:01:28.247 --> 00:01:29.620
command it shows.

23
00:01:29.620 --> 00:01:36.243
When we checked, the stable version was
2.15.0 and the page said the latest stable

24
00:01:36.243 --> 00:01:39.312
release requires Python 3.11 or later.

25
00:01:39.312 --> 00:01:42.138
Our box installed torch 2.15.0+cpu.

26
00:01:42.138 --> 00:01:45.530
The +cpu tag tells you it's the CPU build.

27
00:01:46.292 --> 00:01:47.983
Real output on our box:

28
00:01:48.583 --> 00:01:55.543
The tensor line confirms PyTorch works: a
5 by 3 tensor of float32 on the CPU.

29
00:01:55.543 --> 00:02:02.056
The data is 100 points on the line y = 2x
+ 1, with a little noise added.

30
00:02:02.056 --> 00:02:08.569
The loss starts at 6.79 and drops to
0.0027 by step 50, then stays there.

31
00:02:08.569 --> 00:02:12.317
What's left is roughly the noise we added.

32
00:02:12.917 --> 00:02:17.397
A good prompt: Ask the bot what happens
without zero_grad(), then try it.

33
00:02:17.397 --> 00:02:22.245
Comparing its prediction with the real run
is the best way to learn how much to

34
00:02:22.245 --> 00:02:22.797
trust it.

35
00:02:23.458 --> 00:02:28.107
torch.manual_seed(0) makes the random
numbers repeatable.

36
00:02:28.107 --> 00:02:34.142
PyTorch's Reproducibility notes add that
fully reproducible results aren't

37
00:02:34.142 --> 00:02:40.830
guaranteed across PyTorch releases,
commits or platforms, and that some
operations

38
00:02:40.830 --> 00:02:44.092
need extra settings to be deterministic.

39
00:02:44.092 --> 00:02:47.029
Record your versions with the model.

40
00:02:47.792 --> 00:02:53.360
PyTorch's Serialization semantics notes
recommend saving a model's state_dict.

41
00:02:53.360 --> 00:02:59.142
The torch.load reference shows
weights_only=True as the default in the
version we

42
00:02:59.142 --> 00:03:02.640
checked, which limits what loading a file
can do.

43
00:03:02.640 --> 00:03:06.495
Keep it that way for files you didn't
create yourself.

44
00:03:07.167 --> 00:03:11.209
Change the learning rate to 5.0 and watch
the loss.

45
00:03:11.209 --> 00:03:14.538
Ask the bot to predict what happens first.

46
00:03:14.538 --> 00:03:20.324
Follow PyTorch's official Quickstart
tutorial, which trains a small image

47
00:03:20.324 --> 00:03:21.196
classifier.

48
00:03:21.875 --> 00:03:28.579
Tensor: PyTorch's array type, which can
live on a CPU or GPU and track gradients.

49
00:03:28.579 --> 00:03:32.220
Module: a model or layer, such as
nn.Linear.

50
00:03:32.220 --> 00:03:34.786
Its weights are its parameters.

51
00:03:34.786 --> 00:03:38.758
Loss: a number that says how wrong the
model is.

52
00:03:38.758 --> 00:03:41.572
Training tries to make it smaller.

53
00:03:41.572 --> 00:03:48.442
Optimizer: the object that updates
parameters using gradients, such as SGD or
Adam.

54
00:03:49.042 --> 00:03:55.758
When you have a GPU build, the code above
needs two changes: choose a device once

55
00:03:55.758 --> 00:03:59.986
and move the model and data to it with
.to(device).

56
00:03:59.986 --> 00:04:06.951
Check torch.cuda.is_available() first, and
run Part 1's report so you know the wheel

57
00:04:06.951 --> 00:04:08.609
matches your driver.

58
00:04:09.375 --> 00:04:11.637
Tested vs documented.

59
00:04:11.637 --> 00:04:19.823
Tested by us: the install, the script and
its output, on PyTorch 2.15.0 CPU.

60
00:04:19.823 --> 00:04:28.010
Documented, not tested by us:
reproducibility and serialization advice,
from

61
00:04:28.010 --> 00:04:29.626
PyTorch's docs.

62
00:04:30.333 --> 00:04:37.623
Tomorrow brings TensorFlow and Keras, JAX
on the CPU, and reading Hugging Face model

63
00:04:37.623 --> 00:04:39.272
cards and licenses.

64
00:04:39.958 --> 00:04:46.002
Before you run anything: check every bot
answer against the official docs, work in
a

65
00:04:46.002 --> 00:04:51.902
virtual environment, load model files only
from sources you trust, respect dataset

66
00:04:51.902 --> 00:04:57.154
and model licenses, and keep personal data
out of bots and training sets.

67
00:04:57.154 --> 00:04:59.673
Every source is linked on the page.