When trying to reproduce MNIST example using the latest master-branch version (which solves issue #1095), the fit method get's stuck and never finishes. Below the content of notebook I used.
Google colab notebook content:
%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__) # prints 2.2.0-rc4
!pip install -v "/content/drive/My Drive/autokeras-master.tar.gz"
.
.
.
.
#Successfully installed autokeras-1.0.3 colorama-0.4.3 keras-tuner-1.0.1 terminaltables-3.1.0
#Cleaning up...
#Removed build tracker '/tmp/pip-req-tracker-c06gu3n1'
tf.debugging.set_log_device_placement(True)
import autokeras as ak
from tensorflow.keras.datasets import mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
print(x_train.shape) # (60000, 28, 28)
print(y_train.shape) # (60000,)
print(y_train[:3]) # array([7, 2, 1], dtype=uint8)
clf = ak.ImageClassifier(max_trials=3)
clf.fit(x_train, y_train, epochs=10, verbose=2)
Output:
Executing op TensorSliceDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op TensorSliceDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op ZipDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op OptimizeDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op ModelDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op ReduceDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op TakeDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op SkipDataset in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op BatchDatasetV2 in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op BatchDatasetV2 in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op Fill in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarIsInitializedOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op LogicalNot in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op Assert in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op AssignVariableOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op Fill in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op VarIsInitializedOp in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op AssignVariableOp in device /job:localhost/replica:0/task:0/device:CPU:0
Executing op RandomUniform in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op Sub in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op Mul in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op Add in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op VarHandleOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op DestroyResourceOp in device /job:localhost/replica:0/task:0/device:GPU:0
Executing op DestroyResourceOp in device /job:localhost/replica:0/task:0/device:CPU:0
then the cell just hangs forever
Include the details about the versions of:
When using a prior version of both tensorflow (2.1.0) an autokeras (1.0.2) this error doesn't happen
I'm having the same issue using autokeras 1.0.3 (latest commit) on colab with the ImageClassifier.
When using a prior version of both tensorflow (2.1.0) an autokeras (1.0.2) this error doesn't happen
It might because of the modification of kerastuner. Please try to install a fairly latest version of kerastuner (note: the __version__ attribute of this version may not be correct):
KerasTuner: pip install git+git://github.com/keras-team/keras-tuner.git@d2d69cba21a0b482a85ce2a38893e2322e139c01
TensorFlow: 2.2.0 (or the default 2.2.0rc4 on current Google colab)
AutoKeras: 1.0.3 master branch
Thanks.
Thank you QQ for the tip. For now I will stick to TF 2.1.0 and AK 1.0.2, as is working so far.
I wanted to reported anyway so you guys are aware of the issue.
Using the latest version of KerasTuner fixed the problem, thanks!
this happens to me too on Google Colab and on local machine using this Keras Tuner:
KerasTuner: pip install git+git://github.com/keras-team/keras-tuner.git@d2d69cba21a0b482a85ce2a38893e2322e139c01 --no-dependencies
versions:
TensorFlow: default 2.2.0rc4 on current Google colab
AutoKeras: 1.0.3 master branch
We just release the real 1.0.3. You may try this https://colab.research.google.com/github/keras-team/autokeras/blob/master/docs/ipynb/image_classification.ipynb
Most helpful comment
Using the latest version of KerasTuner fixed the problem, thanks!