Serving: Crash when running bazel-bin/tensorflow_serving/example/inception_export --checkpoint_dir=inception-v3 --export_dir=inception-export from tutorial

Created on 18 Nov 2016 · 17Comments · Source: tensorflow/serving

Hi everybody, I am new to Tensorflow (and serving) and am currently going through the Tutorial for running the inception model in a Docker container. I have been starting the whole process several times now but always get a crash when running bazel-bin/tensorflow_serving/example/inception_export --checkpoint_dir=inception-v3 --export_dir=inception-export with the following error

Traceback (most recent call last):
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 167, in
tf.app.run()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 163, in main
export()
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/tf_serving/tensorflow_serving/example/inception_export.py", line 77, in export
logits, _ = inception_model.inference(images, NUM_CLASSES + 1)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/inception_model.py", line 87, in inference
scope=scope)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/inception_model.py", line 87, in inception_v3
scope='conv0')
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/scopes.py", line 155, in func_with_args
return func(args, *current_args)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/ops.py", line 234, in conv2d
outputs = batch_norm(conv, *batch_norm_params)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/scopes.py", line 155, in func_with_args
return func(args, *current_args)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/ops.py", line 111, in batch_norm
collections=moving_collections)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/scopes.py", line 155, in func_with_args
return func(args, **current_args)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/variables.py", line 289, in variable
trainable=trainable, collections=collections)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 1024, in get_variable
custom_getter=custom_getter)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 850, in get_variable
custom_getter=custom_getter)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 346, in get_variable
validate_shape=validate_shape)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 331, in _true_getter
caching_device=caching_device, validate_shape=validate_shape)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 677, in _get_single_variable
expected_shape=shape)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variables.py", line 224, in __init__
expected_shape=expected_shape)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variables.py", line 327, in _init_from_args
initial_value(), name="initial_value", dtype=dtype)
File "/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/org_tensorflow/tensorflow/python/ops/variable_scope.py", line 665, in
shape.as_list(), dtype=dtype, partition_info=partition_info)
TypeError: ones_initializer() got multiple values for keyword argument 'dtype'

Any suggestions what might have gone wrong there?

Many thanks for any ideas!
Tobias

Source

tobiaslorenz

Most helpful comment

It's now tf.summary.image , so replace line 224 of
inception/inception/image_processing.py to this and it should work.

Sorry for the confusion, as there's been a lot of API churn recently in
preparation of a release.

On Wed, Dec 14, 2016 at 10:09 PM, anuj2rock notifications@github.com
wrote:

@alextp https://github.com/alextp thanks for your reply. Yesterday I
reinstalled latest version of TF and pulled latest inception model.
I am trying to train inception from Scratch! So I have 2000 images of
2.5megapixel size, divided into 2 labels.
Step1:- I shraded my image data set of about 2000 images into TFRecord
files successfully, using ::
bazel-bin/inception/build_image_data --train_directory="/home/
airig/scratch/img_data_set/train_img" --validation_directory="/home/
airig/scratch/img_data_set/validation_img" --output_directory="/home/
airig/scratch/img_data_set/out_dir" --labels_file="/home/airig/
scratch/img_data_set/gro_labels.txt"
Step2:- I ran
bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16
--train_dir=/home/airig/scratch/img_data_set/train_dir
--data_dir=/home/airig/scratch/img_data_set/out_dir

but Now I am getting a new error:--

airig@airig-Inspiron-7559:~/scratch/tensorflow/models/inception$ bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so.8.0 locally
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "NegTrain" device_type: "CPU"') for unknown op: NegTrain
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "Skipgram" device_type: "CPU"') for unknown op: Skipgram
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
Traceback (most recent call last):
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
inception_train.train(dataset)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 216, in train
num_preprocess_threads=num_preprocess_threads)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 136, in distorted_inputs
num_readers=FLAGS.num_readers)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 491, in batch_inputs
image = image_preprocessing(image_buffer, bbox, train, thread_id)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 326, in image_preprocessing
image = distort_image(image, height, width, bbox, thread_id)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 224, in distort_image
tf.image_summary('image_with_bounding_boxes', image_with_box)
AttributeError: 'module' object has no attribute 'image_summary'

Can you please point out whats wrong here? something I missed?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/serving/issues/250#issuecomment-267247285,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAATxcjCOiwW58nZO2acdjmnRsSlaQNEks5rINl_gaJpZM4K2J9T
.

Alex

alextp on 15 Dec 2016

😄1 👍1

All 17 comments

https://github.com/tensorflow/tensorflow/issues/5742

kuprel on 22 Nov 2016

i meet the same issue.

civilman628 on 22 Nov 2016

update code to initializer=tf.ones_initializer() does not fix the issue, is that right? at lease it still report error on my side.

civilman628 on 23 Nov 2016

For me it fixed the error but the statement occurs twice in ops.py. I had
to change both to make it run.

Am 23.11.2016 00:31 schrieb "wang" [email protected]:

update code to initializer=tf.ones_initializer() does not fix the issue,
is that right? at lease it still report error on my side.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/serving/issues/250#issuecomment-262397720,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATW7BLW1hJFETUL0TSq4gqWTOLua7JYXks5rA3trgaJpZM4K2J9T
.

tobiaslorenz on 23 Nov 2016

Btw: many thanks for your help, kuprel!

Am 23.11.2016 06:09 schrieb "Tobias Lorenz" tobias.[email protected]:

For me it fixed the error but the statement occurs twice in ops.py. I had
to change both to make it run.

Am 23.11.2016 00:31 schrieb "wang" [email protected]:

update code to initializer=tf.ones_initializer() does not fix the issue,
is that right? at lease it still report error on my side.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/serving/issues/250#issuecomment-262397720,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATW7BLW1hJFETUL0TSq4gqWTOLua7JYXks5rA3trgaJpZM4K2J9T
.

tobiaslorenz on 23 Nov 2016

@tobiaslorenz do you meet #257 issue? i update 2 places for initializer=tf.ones_initializer() but get new error.

civilman628 on 23 Nov 2016

no, for me things worked out fine!

tobiaslorenz on 25 Nov 2016

😕1

How did you guys resolve the problem exactly?

sebastian-schlecht on 5 Dec 2016

@sebastian-schlecht change initializer=tf.ones_initializer to initializer=tf.ones_initializer() there are 2 lines in this file, so change twice.

civilman628 on 5 Dec 2016

👍2

Thanks @civilman628! For anyone directly stumbling on this issue while doing the tutorial the relevant file is:
/serving/bazel-bin/tensorflow_serving/example/inception_export.runfiles/inception_model/inception/slim/ops.py

sebastian-schlecht on 6 Dec 2016

👍2

@tobiaslorenz @kuprel @sebastian-schlecht @alextp @shlens @masanao-miyamoto Hi happy for you that these two line change worked!! but unfortunately for me this isn't working. And I'm stuck here for 12days now, can u pls help me work this out?

After two line changes in /slim/ops.py file
Line 94

initializer=tf.ones_initializer,

initializer=tf.ones_initializer(),
and Line 108

initializer=tf.ones_initializer,

initializer=tf.ones_initializer(),

I am getting this error >>

Traceback (most recent call last):
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
    inception_train.train(dataset)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 239, in train
    scope)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 108, in _tower_loss
    scope=scope)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_model.py", line 87, in inference
    scope=scope)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/inception_model.py", line 87, in inception_v3
    scope='conv0')
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 236, in conv2d
    outputs = batch_norm(conv, **batch_norm_params)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/done/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 110, in batch_norm
    initializer=tf.ones_initializer(),
TypeError: ones_initializer() takes at least 1 argument (0 given)

anuj2rock on 14 Dec 2016

@anuj2rock It looks like you might be using a version of tensorflow which doesn't have the recent ones_initializer change, so removing the parenthesis after ones_initializer should fix your error.

alextp on 14 Dec 2016

👍1

@alextp thanks for your reply. Yesterday I reinstalled latest version of TF and pulled latest inception model.
I am trying to train inception from Scratch! So I have 2000 images of 2.5megapixel size, divided into 2 labels.
Step1:- I shraded my image data set of about 2000 images into TFRecord files successfully, using ::
bazel-bin/inception/build_image_data --train_directory="/home/airig/scratch/img_data_set/train_img" --validation_directory="/home/airig/scratch/img_data_set/validation_img" --output_directory="/home/airig/scratch/img_data_set/out_dir" --labels_file="/home/airig/scratch/img_data_set/gro_labels.txt"
Step2:- I ran
bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir

but Now I am getting a new error:--

airig@airig-Inspiron-7559:~/scratch/tensorflow/models/inception$ bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so.8.0 locally
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "NegTrain" device_type: "CPU"') for unknown op: NegTrain
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "Skipgram" device_type: "CPU"') for unknown op: Skipgram
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
Traceback (most recent call last):
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
    inception_train.train(dataset)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 216, in train
    num_preprocess_threads=num_preprocess_threads)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 136, in distorted_inputs
    num_readers=FLAGS.num_readers)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 491, in batch_inputs
    image = image_preprocessing(image_buffer, bbox, train, thread_id)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 326, in image_preprocessing
    image = distort_image(image, height, width, bbox, thread_id)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 224, in distort_image
    tf.image_summary('image_with_bounding_boxes', image_with_box)
AttributeError: 'module' object has no attribute 'image_summary'

Can you please point out whats wrong here? something I missed?

anuj2rock on 15 Dec 2016

It's now tf.summary.image , so replace line 224 of
inception/inception/image_processing.py to this and it should work.

Sorry for the confusion, as there's been a lot of API churn recently in
preparation of a release.

On Wed, Dec 14, 2016 at 10:09 PM, anuj2rock notifications@github.com
wrote:

@alextp https://github.com/alextp thanks for your reply. Yesterday I
reinstalled latest version of TF and pulled latest inception model.
I am trying to train inception from Scratch! So I have 2000 images of
2.5megapixel size, divided into 2 labels.
Step1:- I shraded my image data set of about 2000 images into TFRecord
files successfully, using ::
bazel-bin/inception/build_image_data --train_directory="/home/
airig/scratch/img_data_set/train_img" --validation_directory="/home/
airig/scratch/img_data_set/validation_img" --output_directory="/home/
airig/scratch/img_data_set/out_dir" --labels_file="/home/airig/
scratch/img_data_set/gro_labels.txt"
Step2:- I ran
bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16
--train_dir=/home/airig/scratch/img_data_set/train_dir
--data_dir=/home/airig/scratch/img_data_set/out_dir

but Now I am getting a new error:--

airig@airig-Inspiron-7559:~/scratch/tensorflow/models/inception$ bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so.8.0 locally
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "NegTrain" device_type: "CPU"') for unknown op: NegTrain
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "Skipgram" device_type: "CPU"') for unknown op: Skipgram
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
Traceback (most recent call last):
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
inception_train.train(dataset)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 216, in train
num_preprocess_threads=num_preprocess_threads)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 136, in distorted_inputs
num_readers=FLAGS.num_readers)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 491, in batch_inputs
image = image_preprocessing(image_buffer, bbox, train, thread_id)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 326, in image_preprocessing
image = distort_image(image, height, width, bbox, thread_id)
File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/image_processing.py", line 224, in distort_image
tf.image_summary('image_with_bounding_boxes', image_with_box)
AttributeError: 'module' object has no attribute 'image_summary'

Can you please point out whats wrong here? something I missed?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/serving/issues/250#issuecomment-267247285,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAATxcjCOiwW58nZO2acdjmnRsSlaQNEks5rINl_gaJpZM4K2J9T
.

Alex

alextp on 15 Dec 2016

😄1 👍1

@alextp Cool that really helped!! But now problem is back to legendary "dtype error" as follows:---

/scratch/tensorflow/models/inception$ bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir

I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so.5 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so.8.0 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so.8.0 locally
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "NegTrain" device_type: "CPU"') for unknown op: NegTrain
E tensorflow/core/framework/op_kernel.cc:925] OpKernel ('op: "Skipgram" device_type: "CPU"') for unknown op: Skipgram
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.variable_op_scope(values, name, default_name) is deprecated, use tf.variable_scope(name, default_name, values)
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:tf.op_scope(values, name, default_name) is deprecated, use tf.name_scope(name, default_name, values)
WARNING:tensorflow:tf.variable_op_scope(values, name, default_name) is deprecated, use tf.variable_scope(name, default_name, values)
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
WARNING:tensorflow:VARIABLES collection name is deprecated, please use GLOBAL_VARIABLES instead; VARIABLES will be removed after 2017-03-02.
Traceback (most recent call last):
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
    inception_train.train(dataset)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 239, in train
    scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 108, in _tower_loss
    scope=scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_model.py", line 87, in inference
    scope=scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/inception_model.py", line 87, in inception_v3
    scope='conv0')
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 234, in conv2d
    outputs = batch_norm(conv, **batch_norm_params)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 111, in batch_norm
    collections=moving_collections)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/variables.py", line 289, in variable
    trainable=trainable, collections=collections)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1063, in get_variable
    custom_getter=custom_getter)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 889, in get_variable
    custom_getter=custom_getter)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 347, in get_variable
    validate_shape=validate_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 332, in _true_getter
    caching_device=caching_device, validate_shape=validate_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 683, in _get_single_variable
    validate_shape=validate_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 225, in __init__
    expected_shape=expected_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variables.py", line 322, in _init_from_args
    initial_value(), name="initial_value", dtype=dtype)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 672, in <lambda>
    shape.as_list(), dtype=dtype, partition_info=partition_info)
TypeError: ones_initializer() got multiple values for keyword argument 'dtype'

and contents of /inception/inception/ops.py are :--
line 94
initializer=tf.ones_initializer,
line 108
initializer=tf.ones_initializer,

when tried changing them to tf.ones_initializer() at both places , results in :-----
airig@airig-Inspiron-7559:~/scratch/tensorflow/models/inception$ bazel-bin/inception/imagenet_train --num_gpus=1 --batch_size=16 --train_dir=/home/airig/scratch/img_data_set/train_dir --data_dir=/home/airig/scratch/img_data_set/out_dir

Traceback (most recent call last):
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 41, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/imagenet_train.py", line 37, in main
    inception_train.train(dataset)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 239, in train
    scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_train.py", line 108, in _tower_loss
    scope=scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/inception_model.py", line 87, in inference
    scope=scope)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/inception_model.py", line 87, in inception_v3
    scope='conv0')
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 236, in conv2d
    outputs = batch_norm(conv, **batch_norm_params)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/ops.py", line 113, in batch_norm
    collections=moving_collections)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/scopes.py", line 155, in func_with_args
    return func(*args, **current_args)
  File "/home/airig/scratch/tensorflow/models/inception/bazel-bin/inception/imagenet_train.runfiles/inception/inception/slim/variables.py", line 289, in variable
    trainable=trainable, collections=collections)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 1063, in get_variable
    custom_getter=custom_getter)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 889, in get_variable
    custom_getter=custom_getter)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 347, in get_variable
    validate_shape=validate_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 332, in _true_getter
    caching_device=caching_device, validate_shape=validate_shape)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 626, in _get_single_variable
    dtype = dtypes.as_dtype(dtype)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/dtypes.py", line 589, in as_dtype
    raise TypeError("Cannot convert {} to a dtype. {}".format(type_value, e))
TypeError: Cannot convert <function _initializer at 0x7f3f06570a28> to a dtype. data type not understood

@tobiaslorenz @kuprel @sebastian-schlecht @alextp @shlens @masanao-miyamoto Please help!
What is the core issue here??
I am only latest pull of TF and inception both.

anuj2rock on 17 Dec 2016

@anuj2rock the latest version of tensorflow should work with the parenthesis after ones_initializer there.

alextp on 19 Dec 2016

Closing due to staleness. If this is still an issue, please file a new updated issue with current steps to reproduce the bug. If this is a question, please ask it on:

https://stackoverflow.com/questions/tagged/tensorflow-serving

Thanks!