Tflearn: [utils.py] IndexError: indices are out-of-bounds

[shazz]

Hi,

I probably did something wrong but... I really don't find it.

During the fit call, I have this exception:

X (301, 2) Y (301, 2)
---------------------------------
Run id: JTHQIT
Log directory: /tmp/tflearn_logs/
---------------------------------
Training samples: 301
Validation samples: 0
--
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/tflearn/data_flow.py", line 186, in fill_feed_dict_queue
    data = self.retrieve_data(batch_ids)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/tflearn/data_flow.py", line 221, in retrieve_data
    utils.slice_array(self.feed_dict[key], batch_ids)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/tflearn/utils.py", line 187, in slice_array
    return X[start]
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/pandas/core/frame.py", line 2051, in __getitem__
    return self._getitem_array(key)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/pandas/core/frame.py", line 2096, in _getitem_array
    return self.take(indexer, axis=1, convert=True)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/pandas/core/generic.py", line 1669, in take
    convert=True, verify=True)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/pandas/core/internals.py", line 3932, in take
    indexer = maybe_convert_indices(indexer, n)
  File "/home/shazz/projects/anaconda/envs/tensorflow/lib/python3.5/site-packages/pandas/core/indexing.py", line 1872, in maybe_convert_indices
    raise IndexError("indices are out-of-bounds")
IndexError: indices are out-of-bounds

I really started from the titanic example, I just took a different dataset (weight, height => sex), that I clean using pandas, that's the only difference.

Code:

import tflearn
import data_importer

X, Y = data_importer.load_data(0)    
print("X", X.shape, "Y", Y.shape)

# Build neural network
net = tflearn.input_data(shape=[None, 2])
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 32)
net = tflearn.fully_connected(net, 2, activation='softmax')
net = tflearn.regression(net)

# Define model
model = tflearn.DNN(net)

# Start training (apply gradient descent algorithm)
model.fit(X, Y, n_epoch=10, batch_size=1, show_metric=True)

Data importer code and data are available here:
https://github.com/shazz/tflearn-experiments/tree/master/cdc

Any help welcome...... is a a bug in my code (probably) or in tflearn ?
Thanks !

[aymericdamien]

Hi, you need to convert X and Y to numpy array:

X, Y = data_importer.load_data(0)
import numpy as np
X = np.array(X)
Y = np.array(Y)

We should find a way to better track such error and return an appropriate exception

[shazz]

Merci Aymeric !

Works perfectly now !

Yep, maybe having some numpy type detection may help. Also, this exception doesn't stop the execution.
And I guess I should use:

pandas.DataFrame.values
 -> Numpy representation of NDFrame