WEBVTT

1
00:00:00.000 --> 00:00:02.808
Part 10 of ML Libraries with a Bot.

2
00:00:02.808 --> 00:00:06.578
Export a Model to ONNX and Run It: A Deep
Dive.

3
00:00:06.578 --> 00:00:13.077
Training a model in one library and
running it somewhere else is a common
need: a

4
00:00:13.077 --> 00:00:17.810
web service, a small device, a program in
another language.

5
00:00:17.810 --> 00:00:24.067
ONNX is an open format for machine
learning models, and ONNX Runtime runs
ONNX

6
00:00:24.067 --> 00:00:26.073
models on many platforms.

7
00:00:26.833 --> 00:00:29.161
What the bot can and cannot do.

8
00:00:29.161 --> 00:00:32.539
Write the export code for a library you
name.

9
00:00:32.539 --> 00:00:37.119
Explain conversion errors, such as an
unsupported model type.

10
00:00:37.119 --> 00:00:40.648
Write the comparison check, if you ask for
one.

11
00:00:40.648 --> 00:00:45.303
Know which models a converter supports in
its current version.

12
00:00:45.303 --> 00:00:47.330
Check the converter's docs.

13
00:00:47.330 --> 00:00:51.835
Tell you the exported model is correct
without a comparison.

14
00:00:51.835 --> 00:00:56.415
Pick the right GPU package for your CUDA
version from memory.

15
00:00:56.415 --> 00:00:58.442
Use the requirements table.

16
00:00:59.042 --> 00:01:03.620
Never deploy an exported model without
comparing its outputs with the original on

17
00:01:03.620 --> 00:01:04.524
the same inputs.

18
00:01:04.524 --> 00:01:07.802
Conversion can change numbers or,
occasionally, behaviour.

19
00:01:08.458 --> 00:01:15.420
ONNX Runtime's install page gives pip
install onnxruntime for CPU.

20
00:01:15.420 --> 00:01:24.070
For scikit-learn models, the skl2onnx
converter from the sklearn-onnx project
does

21
00:01:24.070 --> 00:01:25.230
the export.

22
00:01:25.230 --> 00:01:32.086
On our box this installed onnxruntime
1.31.1 and skl2onnx 1.20.0.

23
00:01:32.792 --> 00:01:34.025
Real output:

24
00:01:34.625 --> 00:01:41.040
Train the same scaler-plus-logistic-
regression pipeline as Part 5, on all 150
Iris

25
00:01:41.040 --> 00:01:43.465
rows, with features as float32.

26
00:01:43.465 --> 00:01:46.517
ONNX models commonly use 32-bit floats.

27
00:01:46.517 --> 00:01:49.646
Convert with to_onnx(model, X[:1], ...).

28
00:01:49.646 --> 00:01:56.139
The sample row tells the converter the
input type and shape. zipmap: False asks
for

29
00:01:56.139 --> 00:02:02.555
probabilities as a plain array rather than
a list of dictionaries, which is easier

30
00:02:02.555 --> 00:02:03.415
to compare.

31
00:02:04.250 --> 00:02:09.385
Ask a bot for this check every time:
Choose the tolerance for your use.

32
00:02:09.385 --> 00:02:13.942
For a classifier, label agreement is
usually what matters most.

33
00:02:14.542 --> 00:02:20.647
ONNX Runtime's CUDA Execution Provider
page explains which package matches your

34
00:02:20.647 --> 00:02:21.111
stack.

35
00:02:21.111 --> 00:02:27.448
When we checked: Combine this with Part
1's driver checker before installing a GPU

36
00:02:27.448 --> 00:02:27.912
build.

37
00:02:28.667 --> 00:02:35.314
PyTorch has a built-in exporter,
torch.onnx.export, and TensorFlow models
are

38
00:02:35.314 --> 00:02:39.112
commonly converted with the tf2onnx
project.

39
00:02:39.112 --> 00:02:41.443
We didn't test either here.

40
00:02:41.443 --> 00:02:48.522
The same rule applies: compare outputs on
real inputs before you trust the export.

41
00:02:49.208 --> 00:02:52.702
An ONNX file describes a computation
graph.

42
00:02:52.702 --> 00:02:59.201
Still, treat model files from unknown
sources with care, and keep track of where

43
00:02:59.201 --> 00:03:02.694
each one came from and under which
license.

44
00:03:03.375 --> 00:03:09.179
If a converter rejects your model, paste
the exact error and the converter's
version

45
00:03:09.179 --> 00:03:14.776
into the bot, and ask it which part of the
model isn't supported according to the

46
00:03:14.776 --> 00:03:15.950
converter's docs.

47
00:03:15.950 --> 00:03:20.303
Don't accept a workaround that changes
what the model computes.

48
00:03:21.042 --> 00:03:23.237
Tested vs documented.

49
00:03:23.237 --> 00:03:32.020
Tested by us: export with skl2onnx 1.20.0,
inference with onnxruntime 1.31.1 on CPU,

50
00:03:32.020 --> 00:03:34.006
and the comparison.

51
00:03:34.006 --> 00:03:42.057
Documented, not tested by us: GPU packages
and CUDA requirements, PyTorch and

52
00:03:42.057 --> 00:03:44.566
TensorFlow export paths.

53
00:03:45.375 --> 00:03:51.192
The next part today is about reading error
messages and finding slow code.

54
00:03:51.792 --> 00:03:57.836
Before you run anything: check every bot
answer against the official docs, work in
a

55
00:03:57.836 --> 00:04:03.735
virtual environment, load model files only
from sources you trust, respect dataset

56
00:04:03.735 --> 00:04:08.988
and model licenses, and keep personal data
out of bots and training sets.

57
00:04:08.988 --> 00:04:11.506
Every source is linked on the page.