Tflearn: AlexNet 17 flowers - Evaluating not possible / Easier evaluating method?

[ArnoXf]

Hi!
Unfortunately I got stuck in the next problem. Now that the AlexNet example makes a good training process and I get about 97% accuracy after a while, I wanted to evaluate what was learned. Ideally I need a procedure, that is very resource-inexpensive, because I want to train on my GPU but apply what was learned on a less powerful machine. Loading the model shall take a longer time, but evaluating image by image should take very short there.

I was now trying your example described in #39 (https://github.com/tflearn/tflearn/blob/master/examples/basics/weights_persistence.py). My modified code therefore is very simple:

network = ...

# Training
model = tflearn.DNN(network, checkpoint_path='model_alexnet',
                max_checkpoints=1, tensorboard_verbose=0)
model.fit(X, Y, n_epoch=3, validation_set=0.1, shuffle=True,
      show_metric=True, batch_size=32, snapshot_step=400,
      snapshot_epoch=False, run_id='alexnet_oxflowers17')

# Load a model from auto-generated checkpoint
model.load("model_alexnet-800")

# Resume training
model.fit(X, Y, n_epoch=1,
      validation_set=(testX, testY),
      show_metric=True,
      snapshot_epoch=True,
      run_id='model_and_weights')`

But I get a lot (!) of messages like:

I tensorflow/core/common_runtime/bfc_allocator.cc:635] Bin (268435456): Total Chunks: 0, 
Chunks in use: 0 0B allocated for chunks. 0B client-requested for chunks. 0B in use in bin. 
0B client-requested in use in bin.

And in the end:

tensorflow.python.framework.errors.ResourceExhaustedError: OOM when allocating tensor with 
shape[12544,4096]

What am I doing wrong here? Or is there possibly a much simpler method to evaluate single images resource-inexpensively?

Thanks in advance,
Arno

[aymericdamien]

Actually, there is not evaluate method available now, the one available for DNN model (.evaluate()) has issue, but it should be fix soon.
The only way now would be to use model.predict(...) to predict all your data per batch, and loop over all your test set and calculate accuracy by yourself.

[ArnoXf]

Using model.predict() works for me to predict some images. If it's helpful for others, this is the code I am using:

from PIL import Image
import numpy as np

img = Image.open("17flowers/jpg/0/image_0006.jpg")
img = img.resize((224, 224), Image.ANTIALIAS)
img = np.asarray(img, dtype="float32")

img2 = Image.open("17flowers/jpg/1/image_0096.jpg")
img2 = img2.resize((224, 224), Image.ANTIALIAS)
img2 = np.asarray(img2, dtype="float32")

imgs = np.asarray([img, img2, img3])

So I create an array with two images for prediction. Then I am building my network and train it.

network = ...

model = tflearn.DNN(network, checkpoint_path='model_alexnet',
                max_checkpoints=1, tensorboard_verbose=0)
model.fit(X, Y, n_epoch=100, validation_set=0.1, shuffle=True,
      show_metric=True, batch_size=32, snapshot_step=400,
      snapshot_epoch=False, run_id='alexnet_oxflowers17')

And make a prediction of the two images I loaded in the beginning:

results = model.predict(imgs)
print(np.argmax(results[0]))
print(np.amax(results[0]))
print(np.argmax(results[1]))
print(np.amax(results[1]))

At the moment, the model predicts 1 of 2 or 1 of 3 images right, depending on the images loaded. I will have to train longer to get better results, my network was at an accuracy of ~70% when I got those prediction results.

Thanks for your help @aymericdamien !

[aymericdamien]

Nice! Note that this dataset is quite small, so it may not be very accurate.
I will add a .evaluate() method for DNN model class, so it will be easy to evaluate a model on any test set.

[TrinhNC]

tensorflow.python.framework.errors.ResourceExhaustedError: OOM when allocating tensor with shape[12544,4096]
How did you fix this problem? I encounter this while training AlexNet but I don't know how to fix it.

[aliyasineser]

I tried to train alexnet and I got ResourceExhaustedError also. Can anyone help?
GPU: GTX970M with 3GB memory