Handson-ml: ch3: multilabel classification

[jiehou]

I run the following code:
y_train_knn_pred = cross_val_predict(knn_clf, X_train, y_multilabel, cv=3, n_jobs=-1)
f1_score(y_multilabel, y_train_knn_pred, average="macro")
Finally, i got a totally different result, which is only 0.23978719361580844. However, on the book the value is 0.97709078477525002. Why is the difference so large? Thanks in advance.

[ageron]

Hi @jiehou ,
Thanks for your feedback. It might be due to a change in Scikit-Learn 0.20. The last time I ran and checked all the notebooks, I was using 0.19. Could you please try with Scikit-Learn 0.19 just to confirm that this is the problem?

[jiehou]

Hi @ageron ,
thanks very much for your quick reply. I used scikit-learn 0.19.2. And I got the following result:
0.23978719361580844, which is the same as before.
So do you have some other suggestions? Thanks

[jiehou]

By the way, it took really long time to get this result. :-)

[ageron]

Thanks for double-checking, this is really odd. I just ran the whole notebook, using Python 3.6, and the latest versions of NumPy and Scikit-Learn, and I got 0.97709078477525. I cannot reproduce your 0.23978 value. Can you please double-check that your Jupyter notebook was not modified in any way? Also make sure that you restart the kernel and run the cells in order. If you still get the same result, then perhaps your dataset is corrupted? In this case, you may want to try getting a fresh copy?

[ageron]

This cell is indeed pretty long. KNN does not scale very well... sorry about that.

[r-ichi]

It is probably a dataset version thing, I got 0.9689456219392814 using the following code:

y_train_sup6 = (y_train >=7)
y_train_odd = (y_train % 2 == 1)

y_multilabel = np.c_[y_train_sup6, y_train_odd]

knn_clasif = KNeighborsClassifier()
knn_clasif.fit(X_train, y_multilabel)
knn_clasif.predict([X[36000]]) # predict a single instance's result

y_train_knn_pred = cross_val_predict(knn_clasif, X_train, y_train, cv = 3, n_jobs=-1)
f1_score(y_train, y_train_knn_pred, average='macro')

sklearn.__version__ : '0.19.1'

[ageron]

Thanks for your feedback, this looks much better indeed. 👍

Side note: when pasting Python code in github comments, you want to use the following syntax:

```python
print("this will give you")
print("proper syntax highlighting")
```

The result will look like this:

print("this will give you")
print("proper syntax highlighting")

[r-ichi]

omg, I had no idea, thank you! :)!!