Incubator-mxnet: train.rec test.rec for cifar100

Created on 8 Oct 2015 · 3Comments · Source: apache/incubator-mxnet

where can i find train.rec and test.rec for cifar100

Source

ranti-iitg

Most helpful comment

The train.rec and test.rec is generated by original Alex's python format. We first dump images from the binary file, then pack them into rec file. Note you don't need to set resize parameter for im2rec in this case

A sample code I used before to dump images:

import os
import sys
import pickle
import csv
import numpy as np
from PIL import Image

if len(sys.argv) < 3:
    print "usage: convert cifar.pkl output_root_folder_path output_list_path"
    exit(-1)

cifar = pickle.load(file(sys.argv[1]))
root = sys.argv[2]
try:
    os.mkdir(root)
except:
    print "%s exists, ignore"

fo = csv.writer(open(sys.argv[3], "w"), lineterminator='\n', delimiter='\t')

data = cifar['data']
label = cifar['fine_labels']
path = cifar['filenames']

sz = data.shape[0]

for i in xrange(sz):
    if i % 1000 == 0:
        print i
    img = data[i]
    p = path[i]
    img = img.reshape((3, 32, 32))
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 0, 1)
    im = Image.fromarray(img)
    im.save(root + p)
    row = [i, label[i], p]
    fo.writerow(row)

antinucleon on 8 Oct 2015

👍3

All 3 comments

A sample code I used before to dump images:

import os
import sys
import pickle
import csv
import numpy as np
from PIL import Image

if len(sys.argv) < 3:
    print "usage: convert cifar.pkl output_root_folder_path output_list_path"
    exit(-1)

cifar = pickle.load(file(sys.argv[1]))
root = sys.argv[2]
try:
    os.mkdir(root)
except:
    print "%s exists, ignore"

fo = csv.writer(open(sys.argv[3], "w"), lineterminator='\n', delimiter='\t')

data = cifar['data']
label = cifar['fine_labels']
path = cifar['filenames']

sz = data.shape[0]

for i in xrange(sz):
    if i % 1000 == 0:
        print i
    img = data[i]
    p = path[i]
    img = img.reshape((3, 32, 32))
    img = np.swapaxes(img, 0, 2)
    img = np.swapaxes(img, 0, 1)
    im = Image.fromarray(img)
    im.save(root + p)
    row = [i, label[i], p]
    fo.writerow(row)

antinucleon on 8 Oct 2015

👍3

link to binary version of data http://www.cs.toronto.edu/~kriz/cifar-100-binary.tar.gz binary version of data, also can you share your trained model for cifar100 like cafe.

ranti-iitg on 8 Oct 2015

Do you know what networks are suitable for cifar100 to get to a high testing accuracy?@ranti-iitg @antinucleon