Facenet: Failed to interpret det1.npy as a pickle while running ``align_dataset_mtcnn.py``

[TomHeaven]

I got the following error while following the guide to validate facenet on LFW：

Traceback (most recent call last):
  File "src/align/align_dataset_mtcnn.py", line 143, in <module>
    main(parse_arguments(sys.argv[1:]))
  File "src/align/align_dataset_mtcnn.py", line 55, in main
    pnet, rnet, onet = align.detect_face.create_mtcnn(sess, None)
  File "/Users/tomheaven/Documents/facenet/build/lib/src/align/detect_face.py", line 283, in create_mtcnn
    pnet.load(os.path.join(model_path, 'det1.npy'), sess)
  File "/Users/tomheaven/Documents/facenet/build/lib/src/align/detect_face.py", line 85, in load
    data_dict = np.load(data_path, encoding='latin1').item() #pylint: disable=no-member
  File "/Library/Python/2.7/site-packages/numpy/lib/npyio.py", line 429, in load
    "Failed to interpret file %s as a pickle" % repr(file))
IOError: Failed to interpret file '/Users/tomheaven/Documents/facenet/build/lib/src/align/det1.npy' as a pickle

I'm using python 2.7.10 on Mac OS X 10.12. I also load det1.npy manually with

import numpy as np
np.load('det1.npy')

but still failed.

[davidsandberg]

Hi,
Maybe someone else who is running thison Mac OS X can comment but it sounds like your. npy file has been corrupted. Did you try to load any of the det2 or det3?

[TomHeaven]

@davidsandberg I tried loading det2 and det3 and got those errors, too. I'm going to test it on ubuntu.

[TomHeaven]

I have found that the det1.npy can be loaded on ubuntu 16.04/python 2.7.12 with no problem. So I guess this is a compatibility issue of pickle. My suggestion is to use hdf5 instead of pickle to avoid similar issues.

[tintojames]

I was trying to load the same with python 3.5 but failed with almost same issue. This was because, the .npy file was pickled using python2 and was reading in python3. This can be solved using adding encoding parameter.

np.load('det1.npy',encoding='latin1')

[vantuan5644]

You need to modify a little bit in numpy lib.

Look inside your env's site-packages directory: ..._/site-packages/numpy/lib/npio.py_

In line 292: def load(file, mmap_mode=None, allow_pickle=False, fix_imports=True, ...)

Change allow_pickle attribute from False to True and everything gonna work again.

[mofanv]

You need to modify a little bit in numpy lib.

Look inside your env's site-packages directory: ..._/site-packages/numpy/lib/npio.py_

In line 292: def load(file, mmap_mode=None, allow_pickle=False, fix_imports=True, ...)

Change allow_pickle attribute from False to True and everything gonna work again.

Thanks! Works for me.

[Shmarkus]

The correct solution here would be to fix the src not the numpy lib (which most probably will be overwritten when you upgrade numpy version)
I'll post a pull request later on but for the impatient (facenet/src/align/detect_face.py, line 85):

data_dict = np.load(data_path, encoding='latin1', allow_pickle=True).item() #pylint: disable=no-member

[hana9090]

@Shmarkus & @vantuan5644
I enabled allow_pickle in both numpy and source of detect_face, but still, I got the same error!
is there anything to do here?

[Shmarkus]

Hi @hana9090 sorry for the late response.
First of all, do not modify the source.
Second thing is to understand, what setup (versions) are you running and how much customization have you done.

Also, please paste the exact error here as well!

[Shmarkus]

Hi, please try: 

np.load(path, encoding='latin1', allow_pickle=True)

[khangt1k25]

I am facing this issuse. In my case, my det1, det2,det3 work fine but det4, 5 ,6 failed to interpret. I have tried all of your solutions, but it didn't work. Do you have advice for me? Thanks