Insightface: About face2rec2.py

[tianxingyzxq]

I use face2rec2.py to generate the .rec and .idx files, but it seems like something goes wrong, the count does not match the sample number 37884,
time: 0.115676879883 count: 30000
time: 0.114284038544 count: 31000
time: 0.133539915085 count: 32000
time: 0.113406181335 count: 33000
time: 0.107145786285 count: 34000
time: 0.0922582149506 count: 35000
time: 0.0939009189606 count: 36000
time: 0.0914330482483 count: 37000
time: 0.0862410068512 count: 38000
time: 0.0344109535217 count: 39000
time: 0.0345239639282 count: 40000
time: 0.0348250865936 count: 41000
time: 0.0341598987579 count: 42000
time: 0.0342829227448 count: 43000
time: 0.0341610908508 count: 44000
my lst file is:
1 /home1/yx/insightface/datasets/20170530/photos/2017-05-30-0f8c29b0/cf163390.jpg 0
1 /home1/yx/insightface/datasets/20170530/photos/2017-05-30-0f8c29b0/45fd951a.jpg 0
1 /home1/yx/insightface/datasets/20170530/photos/2017-05-30-0f8c29b0/77b35b16.jpg 0
1 /home1/yx/insightface/datasets/20170530/photos/2017-05-30-0f8c29b0/4a58cdd2.jpg 0

1 /home1/yx/insightface/datasets/20170530/photos/2017-05-30-0f8c29b0/96fc4909.jpg 0

when i use the generated dateset to train, a error occur:
Traceback (most recent call last):
File "train_softmax.py", line 1039, in
main()
File "train_softmax.py", line 1036, in main
train_net(args)
File "train_softmax.py", line 1030, in train_net
epoch_end_callback = epoch_cb )
File "/home1/yx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/mxnet/module/base_module.py", line 482, in fit
next_data_batch = next(data_iter)
File "/home1/yx/insightface/src/data.py", line 980, in next
ret = self.cur_iter.next()
File "/home1/yx/insightface/src/data.py", line 830, in next
self.reset()
File "/home1/yx/insightface/src/data.py", line 704, in reset
self.triplet_reset()
File "/home1/yx/insightface/src/data.py", line 553, in triplet_reset
self.select_triplets()
File "/home1/yx/insightface/src/data.py", line 506, in select_triplets
label[i-ba][:] = header.label
File "/home1/yx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 437, in __setitem__
self._set_nd_basic_indexing(key, value)
File "/home1/yx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/mxnet/ndarray/ndarray.py", line 697, in _set_nd_basic_indexing
value = np.broadcast_to(value, shape)
File "/home1/yx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/numpy/lib/stride_tricks.py", line 173, in broadcast_to
return _broadcast_to(array, shape, subok=subok, readonly=True)
File "/home1/yx/anaconda2/envs/tensorflow/lib/python2.7/site-packages/numpy/lib/stride_tricks.py", line 128, in _broadcast_to
op_flags=[op_flag], itershape=shape, order='C').itviews[0]
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (2,) and requested shape (1,)

[tianxingyzxq]

@meanmee @nttstar

[aaaaaaaak]

请问一下你使用的lst文件是怎么生成的呢 里面包含哪些参数？

[tianxingyzxq]

lst文件如下：
1 t img_path_x1 t label_x
1 t img_path_x2 t label_x
1 t img_path_x3 t label_x
...
1 t img_path_y1 t label_y
label_x 从0 开始

[tianxingyzxq]

参考 About face2rec2.py #80 中写的 label一样的放在了一起

[tianxingyzxq]

查了一下应该是我的header.label写错了，现在的head.label是一个list，正确的应该是一个int类型的数据？？？

[nttstar]

idx=0的record, label是一个list, [identity_start, identity_end]. 图片的label都是一个int, identity的label是一个list, [image_idx_start, image_idx_end]

[qinxianyuzi]

read_list读取列表结束后，进行image_encode时在s = mx.recordio.pack(header, '')发生错误，错误为“s = label.tostring() + s，TypeError: can't concat str to bytes”。我打印出header为HEADER(flag=2, label=[32439.0, 32441.0], id=0, id2=0)，请问该如果解决呢？

[minhaj3]

@qinxianyuzi You are most probably getting this error because you are using python3; try python2.

[huismiling]

@qinxianyuzi Python3中‘’是str类型，b''表示为bytes。mx.recordio.pack(header, '')里面的最后一个参数改为b''就可以了，mx.recordio.pack(header, b'')

[YokkaBear]

@qinxianyuzi Python3中‘’是str类型，b''表示为bytes。mx.recordio.pack(header, '')里面的最后一个参数改为b''就可以了，mx.recordio.pack(header, b'')

Thank you, that's absolutely correct and solved my problem.

[xysong1201]

@qinxianyuzi Python3中‘’是str类型，b''表示为bytes。mx.recordio.pack(header, '')里面的最后一个参数改为b''就可以了，mx.recordio.pack(header, b'')

Thank you!! that works

[mengfu188]

@qinxianyuzi Python3中‘’是str类型，b''表示为bytes。mx.recordio.pack(header, '')里面的最后一个参数改为b''就可以了，mx.recordio.pack(header, b'')

mx.recordio.pack(header, ''.encode())

It can also run

[banhbaomocmeo]

@qinxianyuzi Python3中‘’是str类型，b''表示为bytes。mx.recordio.pack(header, '')里面的最后一个参数改为b''就可以了，mx.recordio.pack(header, b'')

thank for your help, i ran for 3m images then this error showed up :(