WEBVTT

1
00:00:00.000 --> 00:00:02.641
Part 1 of ML Libraries with a Bot.

2
00:00:02.641 --> 00:00:07.069
GPU, Driver and CUDA Compatibility: A Bot-
Assisted Check.

3
00:00:07.069 --> 00:00:13.517
Most "my GPU isn't working" problems in
machine learning come down to three
version

4
00:00:13.517 --> 00:00:19.888
numbers that don't fit together: the
NVIDIA driver, the CUDA version a library
was

5
00:00:19.888 --> 00:00:23.150
built for, and the library version itself.

6
00:00:23.150 --> 00:00:29.210
Bots are good at explaining how these fit,
but they're bad at remembering this

7
00:00:29.210 --> 00:00:30.375
month's tables.

8
00:00:31.208 --> 00:00:33.752
What the bot can and cannot do.

9
00:00:33.752 --> 00:00:40.318
Explain the difference between the driver,
the CUDA Toolkit and the CUDA runtime

10
00:00:40.318 --> 00:00:42.780
that ships inside a pip wheel.

11
00:00:42.780 --> 00:00:48.606
Read your environment report and point out
the number that doesn't fit.

12
00:00:48.606 --> 00:00:53.776
Turn a compatibility table you paste in
into a short checklist.

13
00:00:53.776 --> 00:00:56.731
Know the current tables from memory.

14
00:00:56.731 --> 00:01:02.147
NVIDIA, PyTorch, TensorFlow and JAX update
them with each release.

15
00:01:02.147 --> 00:01:03.542
See your machine.

16
00:01:03.542 --> 00:01:05.922
It only knows what you paste.

17
00:01:05.922 --> 00:01:10.600
Promise that a GPU will be fast enough for
your workload.

18
00:01:10.600 --> 00:01:12.651
That needs a measurement.

19
00:01:13.333 --> 00:01:16.479
Collect facts before you change anything.

20
00:01:16.479 --> 00:01:22.769
Installing a new driver or a different
library build to "see if it helps" is how
a

21
00:01:22.769 --> 00:01:25.071
working machine stops working.

22
00:01:25.071 --> 00:01:31.438
Write down what you have, check it against
the docs, and only then change one thing

23
00:01:31.438 --> 00:01:32.205
at a time.

24
00:01:32.958 --> 00:01:39.524
This read-only script prints the Python
version, the versions of the main
libraries,

25
00:01:39.524 --> 00:01:45.778
the driver reported by nvidia-smi if there
is one, and what PyTorch thinks about

26
00:01:45.778 --> 00:01:51.874
CUDA: Here is the real output from our
box, which has no NVIDIA GPU: Two lines

27
00:01:51.874 --> 00:01:58.440
matter most. nvidia-smi: not found means
there's no NVIDIA driver at all, so no
CUDA

28
00:01:58.440 --> 00:02:00.629
library will see a GPU here.

29
00:02:01.542 --> 00:02:07.989
NVIDIA's CUDA Compatibility page describes
minor version compatibility, available

30
00:02:07.989 --> 00:02:14.596
since CUDA 11: an application built with
any toolkit in a CUDA major release family

31
00:02:14.596 --> 00:02:21.044
can run on a driver from that family
that's new enough, with some feature
limits.

32
00:02:21.044 --> 00:02:27.731
It also describes forward compatibility
packages for running newer toolkits on
older

33
00:02:27.731 --> 00:02:30.995
drivers, on supported platforms and GPUs.

34
00:02:31.833 --> 00:02:37.862
Each library publishes its own: Your GPU's
compute capability matters too.

35
00:02:37.862 --> 00:02:41.692
NVIDIA's CUDA GPUs page lists it for each
card.

36
00:02:42.292 --> 00:02:48.140
We turned NVIDIA's table into a function,
so the rule lives in code instead of

37
00:02:48.140 --> 00:02:54.288
memory: Its output on three example pairs:
And the tests, which include both sides

38
00:02:54.288 --> 00:02:58.411
of each boundary: Result on our box: 4
passed in 0.01s.

39
00:02:58.411 --> 00:03:01.485
The checker only covers the driver floor.

40
00:03:01.485 --> 00:03:07.708
It doesn't know about compute capability,
cuDNN versions or a library's own limits.

41
00:03:08.458 --> 00:03:10.377
The last sentence matters.

42
00:03:10.377 --> 00:03:16.207
Without it, a bot will often fill a gap
with a version number from its training

43
00:03:16.207 --> 00:03:16.576
data.

44
00:03:17.250 --> 00:03:23.687
CPU wheel on a GPU machine.
torch.version.cuda is None even though
nvidia-smi works.

45
00:03:23.687 --> 00:03:27.443
Reinstall using the selector on the
PyTorch page.

46
00:03:27.443 --> 00:03:29.665
Driver too old for the wheel.

47
00:03:29.665 --> 00:03:33.037
For example a CUDA 13 build on a 550
driver.

48
00:03:33.037 --> 00:03:35.796
Our checker flags exactly this case.

49
00:03:35.796 --> 00:03:37.022
Two CUDA stacks.

50
00:03:37.022 --> 00:03:42.233
A system CUDA install and pip-installed
CUDA libraries can conflict.

51
00:03:42.833 --> 00:03:44.403
Tested vs documented.

52
00:03:44.403 --> 00:03:50.534
Tested by us: the environment report and
the checker, on a CPU-only Linux box with

53
00:03:50.534 --> 00:03:53.898
Python 3.13.5, and four passing pytest
tests.

54
00:03:53.898 --> 00:04:00.029
Documented, not tested by us: everything
about real GPUs, drivers and CUDA builds,

55
00:04:00.029 --> 00:04:05.786
taken from the NVIDIA, PyTorch,
TensorFlow, JAX and ONNX Runtime pages in
the

56
00:04:05.786 --> 00:04:06.384
sources.

57
00:04:07.208 --> 00:04:13.403
The next part today looks at the other
half of the hardware question: how much

58
00:04:13.403 --> 00:04:15.071
memory a model needs.

59
00:04:15.750 --> 00:04:21.794
Before you run anything: check every bot
answer against the official docs, work in
a

60
00:04:21.794 --> 00:04:27.694
virtual environment, load model files only
from sources you trust, respect dataset

61
00:04:27.694 --> 00:04:32.946
and model licenses, and keep personal data
out of bots and training sets.

62
00:04:32.946 --> 00:04:35.464
Every source is linked on the page.