WEBVTT

1
00:00:00.000 --> 00:00:02.454
Part 5 of ML Libraries with a Bot.

2
00:00:02.454 --> 00:00:08.229
Your First scikit-learn Model, Built with
a Bot. scikit-learn is the friendliest

3
00:00:08.229 --> 00:00:11.621
place to start with machine learning in
Python.

4
00:00:11.621 --> 00:00:17.540
Every model has the same fit and predict
methods, the library ships small datasets

5
00:00:17.540 --> 00:00:21.798
for practice, and its docs are clear about
common mistakes.

6
00:00:21.798 --> 00:00:27.573
In this walkthrough a bot helps write a
first model, we run it and show the real

7
00:00:27.573 --> 00:00:33.058
numbers, and then we check the bot's code
against scikit-learn's own advice.

8
00:00:33.875 --> 00:00:36.395
What the bot can and cannot do.

9
00:00:36.395 --> 00:00:40.215
Write a train-and-test script in a few
seconds.

10
00:00:40.215 --> 00:00:43.142
Explain what each step does and why.

11
00:00:43.142 --> 00:00:45.824
Suggest a different model to try.

12
00:00:45.824 --> 00:00:52.246
Know whether its code leaks test data into
training unless you ask it to check.

13
00:00:52.246 --> 00:00:55.823
Tell you a score is "good" for your
problem.

14
00:00:55.823 --> 00:00:58.099
That depends on the problem.

15
00:00:58.099 --> 00:01:01.025
Replace reading the output yourself.

16
00:01:01.625 --> 00:01:07.055
Run the code, read the numbers, and check
them against a second measure such as

17
00:01:07.055 --> 00:01:09.873
cross-validation before you believe them.

18
00:01:10.542 --> 00:01:16.686
scikit-learn's install page gives pip
install -U scikit-learn inside a virtual

19
00:01:16.686 --> 00:01:21.727
environment, and says recent versions need
Python 3.10 or newer.

20
00:01:21.727 --> 00:01:23.696
Use the venv from Part 3.

21
00:01:24.375 --> 00:01:30.184
The prompt: Write a scikit-learn script
that loads the Iris dataset, splits it
into

22
00:01:30.184 --> 00:01:36.062
train and test with stratification and a
fixed random_state, puts StandardScaler
and

23
00:01:36.062 --> 00:01:41.661
LogisticRegression in a pipeline, prints
test accuracy, and prints 5-fold cross-

24
00:01:41.661 --> 00:01:44.670
validation scores on the training set
only.

25
00:01:44.670 --> 00:01:45.510
No plotting.

26
00:01:46.250 --> 00:01:49.733
Real output with scikit-learn 1.9.1:

27
00:01:50.333 --> 00:01:56.313
Test accuracy 0.974 means the model
labelled 37 of the 38 held-out flowers

28
00:01:56.313 --> 00:01:57.121
correctly.

29
00:01:57.121 --> 00:02:01.566
With test_size=0.25, Iris's 150 rows give
38 test rows.

30
00:02:01.566 --> 00:02:08.112
Cross-validation trains and scores five
times on different slices of the training

31
00:02:08.112 --> 00:02:08.435
set.

32
00:02:08.435 --> 00:02:15.223
The scores sit close together, around
0.955, which suggests the result isn't a
fluke

33
00:02:15.223 --> 00:02:16.274
of one split.

34
00:02:16.274 --> 00:02:18.860
The two numbers differ slightly.

35
00:02:18.860 --> 00:02:19.991
That's normal.

36
00:02:20.833 --> 00:02:26.435
scikit-learn's Common pitfalls page warns
about data leakage: if you fit a scaler or

37
00:02:26.435 --> 00:02:32.037
other preprocessing on the whole dataset
before splitting, information from the
test

38
00:02:32.037 --> 00:02:36.439
set leaks into training and the score
looks better than it should.

39
00:02:36.439 --> 00:02:41.640
Putting StandardScaler inside
make_pipeline means it's fitted only on
the data

40
00:02:41.640 --> 00:02:45.575
passed to fit, including inside each
cross-validation fold.

41
00:02:46.375 --> 00:02:53.154
load_iris(return_X_y=True) gives the
features X (150 rows, 4 measurements) and

42
00:02:53.154 --> 00:02:59.845
labels y (3 species).
train_test_split(..., stratify=y) keeps
the same mix of

43
00:02:59.845 --> 00:03:07.058
species in train and test.
make_pipeline(StandardScaler(),
LogisticRegression(...))

44
00:03:07.058 --> 00:03:14.185
scales each feature, then fits a linear
classifier. max_iter=1000 gives the solver

45
00:03:14.185 --> 00:03:21.485
enough steps to converge on this data.
cross_val_score(model, X_tr, y_tr, cv=5)
runs

46
00:03:21.485 --> 00:03:26.090
five-fold cross-validation on the training
rows only.

47
00:03:27.125 --> 00:03:31.401
scikit-learn's Model persistence page
compares formats.

48
00:03:31.401 --> 00:03:37.930
Pickle-based formats are fast but can run
code when loaded, so the page asks whether

49
00:03:37.930 --> 00:03:40.962
you trust where a model file came from.

50
00:03:40.962 --> 00:03:46.248
It also describes skops.io as more secure
than pickle-based formats.

51
00:03:46.248 --> 00:03:52.078
Only load model files from sources you
trust, and save the library versions

52
00:03:52.078 --> 00:03:53.245
alongside them.

53
00:03:54.000 --> 00:04:00.477
Swap LogisticRegression for
RandomForestClassifier and compare cross-
validation

54
00:04:00.477 --> 00:04:03.101
scores, not just one test score.

55
00:04:03.101 --> 00:04:09.988
Ask the bot to explain the confusion
matrix for your model, after you've
printed it.

56
00:04:09.988 --> 00:04:15.235
Try a dataset of your own, with only data
you're allowed to use.

57
00:04:15.917 --> 00:04:22.359
Features and labels: the inputs (X) and
the answers you want to predict (y).

58
00:04:22.359 --> 00:04:26.088
Estimator: any scikit-learn object with
fit.

59
00:04:26.088 --> 00:04:29.394
Models and scalers are both estimators.

60
00:04:29.394 --> 00:04:35.158
Pipeline: a chain of steps that is fitted
and used as one estimator.

61
00:04:35.158 --> 00:04:39.650
Stratify: keep the same mix of classes in
each split.

62
00:04:40.250 --> 00:04:46.778
Bots sometimes produce older scikit-learn
code, such as importing from modules that

63
00:04:46.778 --> 00:04:47.643
have moved.

64
00:04:47.643 --> 00:04:54.092
If an import fails, paste the error and
your scikit-learn version, and ask the bot

65
00:04:54.092 --> 00:05:00.462
to check the current API reference.
scikit-learn's Getting Started page is a
good

66
00:05:00.462 --> 00:05:02.664
place to compare the basics.

67
00:05:03.500 --> 00:05:05.677
Tested vs documented.

68
00:05:05.677 --> 00:05:14.178
Tested by us: the script and its output,
on scikit-learn 1.9.1 with Python 3.13.5.

69
00:05:14.178 --> 00:05:22.886
Documented, not tested by us: the leakage
and model persistence advice, from scikit-

70
00:05:22.886 --> 00:05:24.233
learn's docs.

71
00:05:24.833 --> 00:05:28.483
The last part today does a similar job in
PyTorch, on the CPU.

72
00:05:29.083 --> 00:05:35.127
Before you run anything: check every bot
answer against the official docs, work in
a

73
00:05:35.127 --> 00:05:41.027
virtual environment, load model files only
from sources you trust, respect dataset

74
00:05:41.027 --> 00:05:46.279
and model licenses, and keep personal data
out of bots and training sets.

75
00:05:46.279 --> 00:05:48.798
Every source is linked on the page.