Models: TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [-1, None]. Consider casting elements to a supported type.

[ZER-0-NE]

python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)":
b'unknown' 1.5.0

System information
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 7 x64
Python version: 3.5.2
Exact command to reproduce: python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnn_nas_coco.config

Describe the problem

I am trying to train my own dataset using faster_rcnn_nas_coco model. I get this unknown error. Other issues which I think were similar could not help: #11974

Source code / logs

Here is the traceback of the error I get:

C:\Users\User\DesktopAbhishek_Singh\models-master\research>python train.py --lo
gtostderr --train_dir=training/ --pipeline_config_path=training/faster_rcnn_nas_
coco.config
INFO:tensorflow:Scale of 0 disables regularizer.
INFO:tensorflow:Scale of 0 disables regularizer.
WARNING:tensorflow:From C:\Users\User\DesktopAbhishek_Singh\models-master\resea
rch\object_detection\trainer.py:228: create_global_step (from tensorflow.contrib
.framework.python.ops.variables) is deprecated and will be removed in a future v
ersion.
Instructions for updating:
Please switch to tf.train.create_global_step
INFO:tensorflow:Scale of 0 disables regularizer.
INFO:tensorflow:Scale of 0 disables regularizer.
2018-04-04 18:41:53.436438: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3
5\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructi
ons that this TensorFlow binary was not compiled to use: AVX AVX2
2018-04-04 18:41:53.904439: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3
5\tensorflow\core\common_runtime\gpu\gpu_device.cc:1105] Found device 0 with pro
perties:
name: Quadro K620 major: 5 minor: 0 memoryClockRate(GHz): 1.124
pciBusID: 0000:02:00.0
totalMemory: 2.00GiB freeMemory: 1.80GiB
2018-04-04 18:41:53.920039: I C:\tf_jenkins\workspace\rel-win\M\windows-gpu\PY\3
5\tensorflow\core\common_runtime\gpu\gpu_device.cc:1195] Creating TensorFlow dev
ice (/device:GPU:0) -> (device: 0, name: Quadro K620, pci bus id: 0000:02:00.0,
compute capability: 5.0)
INFO:tensorflow:A GPU is available on the machine, consider using NCHW data form
at for increased speed on GPU.
INFO:tensorflow:depth of additional conv before box predictor: 0
WARNING:tensorflow:From C:\Users\User\DesktopAbhishek_Singh\models-master\resea
rch\object_detection\core\box_predictor.py:396: calling reduce_mean (from tensor
flow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a
future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From C:\Users\User\DesktopAbhishek_Singh\models-master\resea
rch\object_detection\core\losses.py:316: softmax_cross_entropy_with_logits (from
tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future ver
sion.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

Traceback (most recent call last):
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\tensor_util.py", line 498, in make_tensor_proto
str_values = [compat.as_bytes(x) for x in proto_values]
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\tensor_util.py", line 498, in
str_values = [compat.as_bytes(x) for x in proto_values]
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\util\compat.py", line 65, in as_bytes
(bytes_or_text,))
TypeError: Expected binary or unicode string, got -1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "train.py", line 167, in
tf.app.run()
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\platform\app.py", line 124, in run
_sys.exit(main(argv))
File "train.py", line 163, in main
worker_job_name, is_chief, FLAGS.train_dir)
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\trainer.py", line 246, in train
clones = model_deploy.create_clones(deploy_config, model_fn, [input_queue])
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\slimdeploym
ent\model_deploy.py", line 193, in create_clones
outputs = model_fn(args, *kwargs)
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\trainer.py", line 181, in _create_losses
losses_dict = detection_model.loss(prediction_dict, true_image_shapes)
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\meta_architectures\faster_rcnn_meta_arch.py", line 1580, in loss
groundtruth_masks_list,
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\meta_architectures\faster_rcnn_meta_arch.py", line 1773, in _loss_box_class
ifier
weights=batch_cls_weights),
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\core\losses.py", line 73, in __call__
return self._compute_loss(prediction_tensor, target_tensor, **params)
File "C:\Users\User\DesktopAbhishek_Singh\models-master\research\object_detec
tion\core\losses.py", line 315, in _compute_loss
labels=tf.reshape(target_tensor, [-1, num_classes]),
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\ops\gen_array_ops.py", line 5184, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\op_def_library.py", line 513, in _apply_op_helper
raise err
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\ops.py", line 1022, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\constant_op.py", line 233, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\constant_op.py", line 212, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Program Files (x86)\Python\Python35\lib\site-packages\tensorflow\pyth
on\framework\tensor_util.py", line 502, in make_tensor_proto
"supported type." % (type(values), values))
TypeError: Failed to convert object of type to Tensor. Contents:
[-1, None]. Consider casting elements to a supported type.

Here is my config file:
model {
faster_rcnn {
num_classes: 12
image_resizer {
fixed_shape_resizer {
height: 300
width: 300
}
}
feature_extractor {
type: 'faster_rcnn_nas'
}
first_stage_anchor_generator {
grid_anchor_generator {
scales: [0.25, 0.5, 1.0, 2.0]
aspect_ratios: [0.5, 1.0, 2.0]
height_stride: 16
width_stride: 16
}
}
first_stage_box_predictor_conv_hyperparams {
op: CONV
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
truncated_normal_initializer {
stddev: 0.01
}
}
}
first_stage_nms_score_threshold: 0.0
first_stage_nms_iou_threshold: 0.7
first_stage_max_proposals: 300
first_stage_localization_loss_weight: 2.0
first_stage_objectness_loss_weight: 1.0
initial_crop_size: 17
maxpool_kernel_size: 1
maxpool_stride: 1
second_stage_box_predictor {
mask_rcnn_box_predictor {
use_dropout: false
dropout_keep_probability: 1.0
fc_hyperparams {
op: FC
regularizer {
l2_regularizer {
weight: 0.0
}
}
initializer {
variance_scaling_initializer {
factor: 1.0
uniform: true
mode: FAN_AVG
}
}
}
}
}
second_stage_post_processing {
batch_non_max_suppression {
score_threshold: 0.0
iou_threshold: 0.6
max_detections_per_class: 100
max_total_detections: 100
}
score_converter: SOFTMAX
}
second_stage_localization_loss_weight: 2.0
second_stage_classification_loss_weight: 1.0
second_stage_batch_size: 1
}
}

train_config: {
batch_size: 1
optimizer {
momentum_optimizer: {
learning_rate: {
manual_step_learning_rate {
initial_learning_rate: 0.0003
schedule {
step: 0
learning_rate: .0003
}
schedule {
step: 900000
learning_rate: .00003
}
schedule {
step: 1200000
learning_rate: .000003
}
}
}
momentum_optimizer_value: 0.9
}
use_moving_average: false
}
gradient_clipping_by_norm: 10.0
fine_tune_checkpoint: faster_rcnn_nas_coco_2018_01_28/model.ckpt"
from_detection_checkpoint: true
num_steps: 200000
data_augmentation_options {
random_horizontal_flip {
}
}
}

train_input_reader: {
tf_record_input_reader {
input_path: "data/train.record"
}
label_map_path: "data/CFPS.pbtxt"
}

eval_config: {
metrics_set: "pascal_voc_metrics"
num_examples: 8000
max_evals: 10}

eval_input_reader: {
tf_record_input_reader {
input_path: "data/test.record"
}
label_map_path: "training/CFPS.pbtxt"
shuffle: false
num_readers: 1
num_epochs: 1
}

[ZER-0-NE]

@k-w-w Any comments?

[k-w-w]

@ZER-0-NE Thanks for the ping. Assigning to the codeowners of the object detection model.

[ZER-0-NE]

Any comments? @derekjchow

[MaisaDaoud]

please pass the fulll code,

have a look here
tion\trainer.py", line 181, in _create_losses
losses_dict = detection_model.loss(prediction_dict, true_image_shapes)

--I think you have to use prediction_dict['XXX'],I had the same problem and solve it using the dictionary index.
--a second advice, print (prediction_dict) and print( true_image_shapes) and check if both tensors are of the same type

[ZER-0-NE]

--I think you have to use prediction_dict['XXX'],I had the same problem and solve it using the dictionary index.

I do not get this. My trainer.py script runs fine. @MaisaDaoud

[hubert0527]

My personal workaround on this issue, just replace the None with -1 and force the np dtype to be np.int32.

Add following code to the line before this line:
https://github.com/tensorflow/tensorflow/blob/42ca99b5aae03a8122ba0db94abfe1f3f5c257dc/tensorflow/python/framework/tensor_util.py#L377

if np_dt==None and len(np.array(values).shape)>0 and None in values:
        np_dt = np.int32
        values = [-1 if type(v)==type(None) else v for v in values]

[Aaronreb]

['XXX']

What do we put instead of ['XXX] ?

[Shruthi-Sampathkumar]

['XXX']

What do we put instead of ['XXX] ?

Did this solution work? @Aaronreb

[tensorflowbutler]

Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.