Handson-ml: from sklearn.datasets import fetch_openml

[hamzza-K]

mnist = fetch_openml('mnist_784', version=1, cache=True)
mnist.target = mnist.target.astype(np.int8)

whenever i run this piece of snippet in my terminal, my OS gets stuck for reasons unknown to me.
I'm using Ubuntu 18.04.

P.S All this code doing is fetching the mnist dataset which is also provided in keras dataset. As an alternative, can one use this dataset for this exercise?

[ageron]

Hi @hamzza-K ,
Thanks for your feedback. I noticed that fetch_openml() is very slow, so perhaps your OS is not stuck, you just need to wait for a few minutes. If that does not work, then perhaps you can try clearing Scikit-Learn's cache: just remove the $HOME/scikit_learn_data/openml/openml.org/ directory and try again. If that still doesn't work, then perhaps there is a proxy or firewall blocking the download. Also, perhaps it's using a lot of RAM, so you don't run out of it (e.g., not too many programs open, and close some tabs in your navigator).
If nothing works, then indeed you can use any other method for downloading MNIST, including using Keras:

from tensorflow import keras
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()

However, X_train will be a 3D array (shape [60000, 28, 28]), whereas fetch_openml() returns a 2D array (shape [70000, 784]), so you need to reshape X_train and X_test. Plus, it contains uint8 (unsigned bytes) from 0 to 255, instead of float64, so no need to cast to byte (astype(np.int8)).

X_train = X_train.reshape(-1, 784)
X_test = X_test.reshape(-1, 784)

Also note that fetch_openml() returns the full dataset (70,000 images), whereas load_data() returns two sets (train=60,000 images + test=10,000 images).
Hope this helps.

[hamzza-K]

Thanks ! that clarified a lot and also resolved the issue i had.
Much appreciated.

[ageron]

Hi @hamzza-K ,
I'm glad this fixed your issue. In case someone else has the same problem, could you please indicate what exactly the problem was, and what fixed it?
Thanks!
Aurélien

[hamzza-K]

from tensorflow import keras
(X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()

X_train = X_train.reshape(-1, 784)
X_test = X_test.reshape(-1, 784)

Since, I had tensorflow installed on my machine, I used keras to
extract the mnist dataset. However, I was negligent of the concept of
arrays being

returned in 3D whilst the code in the book works on 2D. Reshaping the
train and test matrices solved my problem. Sorry I did not try other
two suggestions

that you gave because this seemed more plausible to me.

Thanks again though !

On Sat, May 4, 2019 at 2:32 PM Aurélien Geron [e-mail]
wrote:

Hi @hamzza-K https://github.com/hamzza-K ,
I'm glad this fixed your issue. In case someone else has the same problem,
could you please indicate what exactly the problem was, and what fixed it?
Thanks!
Aurélien

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ageron/handson-ml/issues/416#issuecomment-489311093,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIDSJBWAQQ53OA6GADAYXKTPTVJ2VANCNFSM4HJ75Z4A
.