Part 10 of 12

Export a Model to ONNX and Run It: A Deep Dive

Deep diveBy Bot Tutorials · Published Nov 3, 2026 · Facts last verified Nov 3, 2026

ML libraries series complete. "Review Your Training Code with a Bot: Checklist" was the finale. See every part in the series hub.

Part 10: convert a scikit-learn pipeline to ONNX, run it with ONNX Runtime, and prove the predictions match, with real output and GPU notes.

Video: Export a Model to ONNX and Run It: A Deep Dive (4:12). Narrated from this guide, with the real code and real output we ran. Captions included (caption file). Prefer reading? Everything in the video is in the guide below.

Training a model in one library and running it somewhere else is a common need: a web service, a small device, a program in another language. ONNX is an open format for machine learning models, and ONNX Runtime runs ONNX models on many platforms. In this deep dive we export the scikit-learn pipeline from Part 5 to ONNX, run it with ONNX Runtime, and check that the answers really match, which is the step people skip.

What the bot can and can't do here

It can:

Write the export code for a library you name.
Explain conversion errors, such as an unsupported model type.
Write the comparison check, if you ask for one.

It can't:

Know which models a converter supports in its current version. Check the converter's docs.
Tell you the exported model is correct without a comparison.
Pick the right GPU package for your CUDA version from memory. Use the requirements table.

Check first, ship second

Never deploy an exported model without comparing its outputs with the original on the same inputs. Conversion can change numbers or, occasionally, behaviour.

Install

ONNX Runtime's install page gives pip install onnxruntime for CPU. For scikit-learn models, the skl2onnx converter from the sklearn-onnx project does the export. On our box this installed onnxruntime 1.31.1 and skl2onnx 1.20.0.

The script

import numpy as np, onnxruntime as ort
from sklearn.datasets import load_iris
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from skl2onnx import to_onnx

X, y = load_iris(return_X_y=True)
X = X.astype(np.float32)
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)).fit(X, y)
onx = to_onnx(model, X[:1], options={"zipmap": False})
open("iris.onnx", "wb").write(onx.SerializeToString())
sess = ort.InferenceSession("iris.onnx", providers=["CPUExecutionProvider"])
print("providers:", sess.get_providers())
inp = sess.get_inputs()[0].name
labels, probs = sess.run(None, {inp: X})
agree = (labels == model.predict(X)).mean()
print("label agreement with scikit-learn:", agree)
print("max prob difference:", float(np.abs(probs - model.predict_proba(X)).max()))

Real output:

providers: ['CPUExecutionProvider']
label agreement with scikit-learn: 1.0
max prob difference: 1.4901161193847656e-07

What the script does

Train the same scaler-plus-logistic-regression pipeline as Part 5, on all 150 Iris rows, with features as float32. ONNX models commonly use 32-bit floats.
Convert with to_onnx(model, X[:1], ...). The sample row tells the converter the input type and shape. zipmap: False asks for probabilities as a plain array rather than a list of dictionaries, which is easier to compare.
Run with InferenceSession(..., providers=["CPUExecutionProvider"]). The output confirms only the CPU provider was used.
Compare labels and probabilities with scikit-learn's own predict and predict_proba.

Reading the comparison

Label agreement 1.0 means all 150 labels matched.
Max probability difference about 1.5e-07 is tiny. It comes from 32-bit floating point, which keeps about seven significant digits. A difference like 0.01 would mean something went wrong.

Ask a bot for this check every time:

Add a check that runs both the original model and the ONNX model on the same inputs, and reports label agreement and the maximum absolute difference in probabilities. Fail if agreement is below 1.0 or the difference is above 1e-5.

Choose the tolerance for your use. For a classifier, label agreement is usually what matters most.

GPU notes, documented

ONNX Runtime's CUDA Execution Provider page explains which package matches your stack. When we checked:

Because of NVIDIA's minor version compatibility, builds with CUDA 12.8 need CUDA 12.8 or newer, and builds with CUDA 13.0 need 13.0 or newer.
From version 1.27, the onnxruntime-gpu packages on PyPI are built with CUDA 13.0 by default, with separate CUDA 12.8 packages.
Builds with cuDNN 8.x and 9.x aren't interchangeable.
The table covered up to 1.30.x, while pip gave us 1.31.1 for CPU. When a table lags the latest release, check the release notes before assuming the newest row applies.

Combine this with Part 1's driver checker before installing a GPU build.

Exporting from PyTorch or TensorFlow

PyTorch has a built-in exporter, torch.onnx.export, and TensorFlow models are commonly converted with the tf2onnx project. We didn't test either here. The same rule applies: compare outputs on real inputs before you trust the export.

Security

An ONNX file describes a computation graph. Still, treat model files from unknown sources with care, and keep track of where each one came from and under which license.

Words you'll see

Graph: the network of operations an ONNX file describes.
Opset: the version of the ONNX operator set a model uses. Runtimes support a range of opsets.
Execution provider: the back end ONNX Runtime uses, such as the CPU or CUDA provider.
Session: a loaded model ready to run, created with InferenceSession.

If a converter rejects your model, paste the exact error and the converter's version into the bot, and ask it which part of the model isn't supported according to the converter's docs. Don't accept a workaround that changes what the model computes.

Tested vs documented

Tested by us: export with skl2onnx 1.20.0, inference with onnxruntime 1.31.1 on CPU, and the comparison.
Documented, not tested by us: GPU packages and CUDA requirements, PyTorch and TensorFlow export paths.

Where to go next

The next part today is about reading error messages and finding slow code.

Safety, licensing and privacy: read this before you run anything

This series is about learning and small personal projects with machine learning libraries. A bot is a helper, not an authority.

Check bot advice against official docs. Version numbers, install commands and compatibility tables change often. Compare every answer with the PyTorch, TensorFlow, JAX, scikit-learn, NVIDIA, Hugging Face or ONNX Runtime pages we link. If the bot and the docs disagree, the docs win.
Work in a virtual environment. Don't install or remove packages in your system Python, and don't run sudo pip.
Run untrusted files carefully. Pickle-based model files can run code when loaded. Only load models from sources you trust, and prefer safer formats like safetensors where the library supports them.
Respect licenses. Datasets and model weights carry their own licenses and terms. Read the model card and license before you use, share or sell anything built on them. We don't give legal advice; ask a qualified person if it matters.
Protect personal data. Don't paste private, customer or medical data into a bot or a training set unless you have the right to, and check the bot's own privacy settings first.
No hardware claims you didn't measure. Treat any speed figure from a bot as a guess until you've timed it on your own machine.
No medical, legal or financial decisions from a toy model. These guides are for learning only.

FAQ

Why export to ONNX?

ONNX is an open format, and ONNX Runtime can run the same model from Python and other languages without scikit-learn or PyTorch installed.

Do the predictions match exactly?

In our test every label matched, and the largest probability difference was about 1.5e-07, which comes from using 32-bit floats.

Was the GPU tested?

No. We ran the CPU execution provider only. The CUDA requirements come from ONNX Runtime's docs.