Keras: to_categorical() throws "IndexError:"

[kickbox]

I have a matrix like this

> summary(as.factor(as.numeric(trainTransformed$waitCategory)))
   1    2    3    4    5    6    7    8    9 
1069   56   98   49   10    1    4    1    2 

y_train= matrix(as.numeric(trainTransformed$waitCategory))

As you can see there are only 9 classes. However whenever I try one-hot encoding I get the following error

> one_hot_labels <- to_categorical(y_train, num_classes =9)

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  IndexError: index 9 is out of bounds for axis 1 with size 9

Detailed traceback: 
  File "C:\PROGRA~1\Python35\lib\site-packages\tensorflow\contrib\keras\python\keras\utils\np_utils.py", line 41, in to_categorical
    categorical[np.arange(n), y] = 1

This error disappears when I use num_classes=10 instead of 9. However it creates a matrix with 10 columns, which is not what I want.

I think this issue is related to this

[dfalbel]

Keras expects an integer vector from 0 to num_classes. As it's stated in the docs:

y  Class vector to be converted into a matrix (integers from 0 to num_classes).

[kickbox]

@dfalbel Thank you. This was the problem.

[dfalbel]

Glad it helped! :)

[selwyn-mccracken]

@kickbox I hit the same issue - can you please clarify how you specified num_classes? I tried
y = to_categorical(iris$Species,num_classes=0:3)

but get

"TypeError: 'list' object cannot be interpreted as an integer"

I resorted to using model.matrix instead, which also works.
y = model.matrix(~Species-1,data = iris)

[kickbox]

If you look at my above code, the only change I had to do was y_train = y_train -1
before calling to_categorical

[vjp23]

I'm having this issue as well, and it feels to me as though it would make more sense for to_categorical to pad the vector with 0s when num_classes > max(y).

Let's say I'm one-hot-encoding a test set and a validation set, and they have a different number of classes (let's say a rare example only makes it into one set). I need both label vectors to be of the same length for my network, but with to_categorical this isn't possible, and I have to resort to doing it manuall, which is a bummer.

[jjallaire]

if you use num_classes I think that will allow you to fix the size of the label dimension (note above that a range was specified for num_classes but that yielded an error -- this should rather be specified as a single integer value)

[vjp23]

Ah, trying with a toy example it works with an arbitrary number of classes. I suppose there was some other issue with my class labels! Sorry for the trouble. :]

[nabsabraham]

Hi all,
I still have this problem - my image has 5 object classes in it but whether I set num_classes= 5/6/10/100 I still get an IndexError. The only time it seems to go away is when I call
to_categorical(masks)
but because it's an 8-bit image, I get 256 classes of objects which is not what I want. My masks array is uint8 so I'm not sure what I'm doing wrong. Any ideas folks?

[fadishaar84]

@nabsabraham, please, did you manage to solve the problem? I've tried the solution provided by @kickbox and did not work for me.