Keras: IOError while trying to load the mnist data

[memoiry]

Hi, I have just installed keras and test the mnist_cnn example provided in my ipython environment.

%run ~/desktop/code/keras/examples/mnist_cnn.py

it returns
`---------------------------------------------------------------------------
IOError Traceback (most recent call last)
/Users/xuguodong/desktop/code/keras/examples/mnist_cnn.py in ()
30
31 # the data, shuffled and split between train and test sets
---> 32 (X_train, y_train), (X_test, y_test) = mnist.load_data()
33
34 X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)

/Users/xuguodong/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/datasets/mnist.pyc in load_data(path)
15
16 if sys.version_info < (3,):
---> 17 data = cPickle.load(f)
18 else:
19 data = cPickle.load(f, encoding="bytes")

/Users/xuguodong/anaconda/lib/python2.7/gzip.pyc in readline(self, size)
462 bufs = []
463 while size != 0:
--> 464 c = self.read(readsize)
465 i = c.find('\n')
466

/Users/xuguodong/anaconda/lib/python2.7/gzip.pyc in read(self, size)
266 try:
267 while size > self.extrasize:
--> 268 self._read(readsize)
269 readsize = min(self.max_read_chunk, readsize * 2)
270 except EOFError:

/Users/xuguodong/anaconda/lib/python2.7/gzip.pyc in _read(self, size)
313 if buf == "":
314 uncompress = self.decompress.flush()
--> 315 self._read_eof()
316 self._add_read_data( uncompress )
317 raise EOFError, 'Reached EOF'

/Users/xuguodong/anaconda/lib/python2.7/gzip.pyc in _read_eof(self)
352 if crc32 != self.crc:
353 raise IOError("CRC check failed %s != %s" % (hex(crc32),
--> 354 hex(self.crc)))
355 elif isize != (self.size & 0xffffffffL):
356 raise IOError, "Incorrect length of data produced"

IOError: CRC check failed 0x54fca8bc != 0xfded8386L
`

it seems that problem is the ---> 32 (X_train, y_train), (X_test, y_test) = mnist.load_data() process

I have uninstalled the keras and install it again, but it seems make no sense, can anyone figure out there is any problem with the load_data method?

thx for help

[joelthchao]

I think your file is broken. Find where the mnist data download, usually in ~/.keras/ or /tmp/.keras/.
Remove it and try again.

[memoiry]

@joelthchao it works a little,
then it returns
`%run ~/desktop/code/keras/examples/mnist_cnn.py

Downloading data from https://s3.amazonaws.com/img-datasets/mnist.pkl.gz

Exception Traceback (most recent call last)
/Users/xuguodong/desktop/code/keras/examples/mnist_cnn.py in ()
30
31 # the data, shuffled and split between train and test sets
---> 32 (X_train, y_train), (X_test, y_test) = mnist.load_data()
33
34 X_train = X_train.reshape(X_train.shape[0], 1, img_rows, img_cols)

/Users/xuguodong/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/datasets/mnist.pyc in load_data(path)
7
8 def load_data(path="mnist.pkl.gz"):
----> 9 path = get_file(path, origin="https://s3.amazonaws.com/img-datasets/mnist.pkl.gz")
10
11 if path.endswith(".gz"):

/Users/xuguodong/anaconda/lib/python2.7/site-packages/Keras-1.0.1-py2.7.egg/keras/utils/data_utils.pyc in get_file(fname, origin, untar)
74 if os.path.exists(fpath):
75 os.remove(fpath)
---> 76 raise e
77 progbar = None
78

Exception: URL fetch failure on https://s3.amazonaws.com/img-datasets/mnist.pkl.gz: None -- [Errno 54] Connection reset by peer
`

[joelthchao]

@memoiry You should check if your internet connection works well.
Or you can manually download mnis.pkl.gz from here and place it to where you delete it.

[Jawwadjlf]

I faced the same error , CRC error on Mnist . I have downloaded the library again from this link [https://s3.amazonaws.com/img-datasets/mnist.pkl.gz] . and my problem has been solved.

[Tangzixia]

在windows中使用mnist.load_data()将文件下载到哪里了，找不到目录，无法删除

[joelthchao]

@Tangzixia Please check the code. Should be one of the two path:

os.path.expanduser(os.path.join('~', '.keras'))  # 'C:\\Users\\username\\.keras\\datasets'
os.path.join('/tmp', '.keras')  # '/tmp\\.keras\\datasets'

[kaqilah]

rm ~/.keras/datasets/* and restart the code will help solve the problem

[AbaloneVH]

@Tangzixia If your OS is Windows, then the default path is 'C:\Users\username.keras\datasets. The file 'mnist.npz' is the data you want.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[eltronix]

If you're on Mac, found a solution somewhere else:
https://github.com/ageron/handson-ml/issues/46

[KshetriDeepen]

from __future__ import print_function
from keras.callbacks import LambdaCallback
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.layers import LSTM
from keras.optimizers import RMSprop
from keras.utils.data_utils import get_file
import numpy as np
import random
import sys
import io

path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
with io.open(path, encoding='utf-8') as f:
text = f.read().lower()
print('corpus length:', len(text))

THIS IS THE ERRORraceback (most recent call last):
File "/Users/macbookrosegold/PycharmProjects/DataScience/script_generator.py", line 13, in
path = get_file('nietzsche.txt', origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
File "/Users/macbookrosegold/PycharmProjects/DataScience/venv/lib/python3.7/site-packages/keras/utils/data_utils.py", line 226, in get_file
raise Exception(error_msg.format(origin, e.errno, e.reason))
Exception: URL fetch failure on https://s3.amazonaws.com/text-datasets/nietzsche.txt: None -- [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)