Autokeras: Low accuracy on mnist example

[Anindita-Pani]

I am trying to improve accuracy of the model and I have used the following code as per the example:

import autokeras as ak

from keras.datasets import mnist
from autokeras import ImageClassifier

if __name__ == '__main__':
    (x_train, y_train), (x_test, y_test) = mnist.load_data()
    x_train = x_train.reshape(x_train.shape + (1,))
    x_test = x_test.reshape(x_test.shape + (1,))

    clf = ImageClassifier(verbose=True, augment=False)
    print("Started fit.....")
    clf.fit(x_train, y_train, time_limit= 2 * 60 * 60)
    clf.final_fit(x_train, y_train, x_test, y_test, retrain=True)
    print("Completed fit....")
    y = clf.evaluate(x_test, y_test)
    print(y * 100)

    clf.load_searcher().load_best_model().produce_keras_model().save('autokeras_test_model_120.h5')
    print("saved model!")

The value of _y*100_ is 10.29

Also, I tried to save the best model and load it to check for its accuracy -

which gives accuracy 0.191

Is there a way to improve the accuracy?

[wircho]

I'm also interested in learning how to solve this issue. Pretty much any other model trained for a similar amount of time will produce better accuracy.

[mariolys07]

I had the same issue with a custom dataset. I haven't been able to reach 20% accuracy on any set.

[haifeng-jin]

@Anindita-Pani may I know which version of auto keras are u using? The previous version had a bug in the prediction and evaluation.

Our current local test results are over 98% or 99% on the latest version 0.2.14.
Thanks.

[haifeng-jin]

@mariolys07 Please follow #193 to continue the discussion on the custom datasets.
Thank you for your feedback!

[haifeng-jin]

@satyakesav Would you please try to run the examples/mnist.py with the latest version of autokeras and post the final accuracy here?

[Anindita-Pani]

@jhfjhfj1 I was not using the latest version. It gives score of 99% with the latest version(0.2.14)
Thanks !

[nivniv850]

Does anyone knows if the second code of @Anindita-Pani should work?
according to this thread, we need to add activation layer and training.