Models: InvalidArgumentError (see above for traceback): Invalid JPEG data

Created on 20 Jun 2017 · 14Comments · Source: tensorflow/models

System information
OS Platform and Distribution : Ubuntu Zesty 17.04:

Hello,
I am a new member in github and I hope I have followed your manual to ask my question. Please help me to handle this error in retraining inception module.
Successfully, I have converted my dicom images to jpeg with the following code (I used http://manpages.ubuntu.com/manpages/precise/man1/dcmj2pnm.1.html).

for file in ./*.dcm ; do
dcmj2pnm -d -v -im +ot +Jq 100 +Sxf 1.2 +Wi 1 $file $file.jpeg

echo "$file"

done

I have specified a particular set of sub-folders which contains only images from that category and put them in a directory whose name is MRII. But, when I want to retrain inception module I get the following error (To retrain my inception module, I use the approach in https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#0).

root@00aa0e1cd75b:/tf_files# python retrain.py \

--bottleneck_dir=bottlenecks \
--how_many_training_steps=500 \
--model_dir=inception \
--summaries_dir=training_summaries/basic \
--output_graph=retrained_graph.pb \
--output_labels=retrained_labels.txt \
--image_dir=MRII
2017-06-20 15:04:15.616625: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
Looking for images in 'Bold'
Looking for images in 'ASL'
Creating bottleneck at bottlenecks/Bold/1011_221_MR.MR.CCIR-00600_CCIR-00675_Benzinger_AV45.13.49.20150915.142526.1kyq4ky.dcm.jpeg.txt
2017-06-20 15:04:22.501680: W tensorflow/core/framework/op_def_util.cc:332] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
Not a JPEG file: starts with 0x50 0x35
Traceback (most recent call last):
File "retrain.py", line 1062, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "retrain.py", line 808, in main
jpeg_data_tensor, bottleneck_tensor)
File "retrain.py", line 440, in cache_bottlenecks
jpeg_data_tensor, bottleneck_tensor)
File "retrain.py", line 392, in get_or_create_bottleneck
create_bottleneck_file(bottleneck_path, image_lists, label_name, index, image_dir, category, sess, jpeg_data_tensor, bottleneck_tensor)
File "retrain.py", line 356, in create_bottleneck_file
bottleneck_values = run_bottleneck_on_image(sess, image_data, jpeg_data_tensor, bottleneck_tensor)
File "retrain.py", line 275, in run_bottleneck_on_image
{image_data_tensor: image_data})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid JPEG data, size 147471
[[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_DecodeJpeg/contents_0)]]

Caused by op u'DecodeJpeg', defined at:
File "retrain.py", line 1062, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "retrain.py", line 779, in main
create_inception_graph())
File "retrain.py", line 256, in create_inception_graph
RESIZED_INPUT_TENSOR_NAME]))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/importer.py", line 308, in import_graph_def
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Invalid JPEG data, size 147471
[[Node: DecodeJpeg = DecodeJpeg[acceptable_fraction=1, channels=3, dct_method="", fancy_upscaling=true, ratio=1, try_recover_truncated=false, _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_DecodeJpeg/contents_0)]]

How can I handle these errors??

support

Source

hosseinece

Most helpful comment

It looks that the image is not a JPEG, try to run and debug using this minimal code snipped

import tensorflow as tf
fn = 'yourimage.JPEG'
with tf.Graph().as_default():
    image_contents = tf.read_file(fn)
    image = tf.image.decode_jpeg(image_contents, channels=3)
    init_op = tf.initialize_all_tables()
    with tf.Session() as sess:
        sess.run(init_op)
        tmp = sess.run(image)

jematos92 on 20 Jun 2017

🚀4 👍4 ❤2

All 14 comments

It looks that the image is not a JPEG, try to run and debug using this minimal code snipped

import tensorflow as tf
fn = 'yourimage.JPEG'
with tf.Graph().as_default():
    image_contents = tf.read_file(fn)
    image = tf.image.decode_jpeg(image_contents, channels=3)
    init_op = tf.initialize_all_tables()
    with tf.Session() as sess:
        sess.run(init_op)
        tmp = sess.run(image)

jematos92 on 20 Jun 2017

🚀4 👍4 ❤2

Thanks!
Where should I use your code in poet steps? (I mean steps which are mentioned in https://codelabs.developers.google.com/codelabs/tensorflow-for-poets/#4)

hosseinece on 20 Jun 2017

Im not familiar with that codelab but you can just run that code snippet to verify that tf.image.decode_jpeg is actually working with your MRII images.

jematos92 on 20 Jun 2017

Thanks!
I have included this code in a file (decode_jpeg.py), then, I put this file and MRII directory in my docker's working directory. Here is my python code exactly:

decode_jpeg.py

import os, sys
import tensorflow as tf
fn = './MRII/ASL/*.JPEG'
with tf.Graph().as_default():
image_contents = tf.read_file(fn)
image = tf.image.decode_jpeg(image_contents, channels=3)
init_op = tf.initialize_all_tables()
with tf.Session() as sess:
sess.run(init_op)
tmp = sess.run(image)

Here is my new error:

root@73b086f2ee31:/tf_files# python decode_jpeg.py
WARNING:tensorflow:From decode_jpeg.py:7: initialize_all_tables (from tensorflow.python.ops.data_flow_ops) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use tf.tables_initializer instead.
2017-06-20 20:49:06.900449: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-20 20:49:06.909880: W tensorflow/core/framework/op_kernel.cc:1152] Not found: ./MRII/ASL/.JPEG
2017-06-20 20:49:06.910773: W tensorflow/core/framework/op_kernel.cc:1152] Not found: ./MRII/ASL/.JPEG
Traceback (most recent call last):
File "decode_jpeg.py", line 10, in
tmp = sess.run(image)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: ./MRII/ASL/*.JPEG
[[Node: ReadFile = ReadFile[_device="/job:localhost/replica:0/task:0/cpu:0"](ReadFile/filename)]]

Caused by op u'ReadFile', defined at:
File "decode_jpeg.py", line 5, in
image_contents = tf.read_file(fn)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 203, in read_file
result = _op_def_lib.apply_op("ReadFile", filename=filename, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 768, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2336, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1228, in __init__
self._traceback = _extract_stack()

NotFoundError (see above for traceback): ./MRII/ASL/*.JPEG
[[Node: ReadFile = ReadFile[_device="/job:localhost/replica:0/task:0/cpu:0"](ReadFile/filename)]]

Could you please let me know what the problem is? Thanks!

hosseinece on 20 Jun 2017

fn is cannot be found, just try with one image, something like:
fn = './MRII/ASL/type_your_image_name_here.JPEG'

jematos92 on 20 Jun 2017

Thanks!
I did that but this is the resulted error:

root@eac5e7360ae5:/tf_files# python decode_jpeg.py
WARNING:tensorflow:From decode_jpeg.py:7: initialize_all_tables (from tensorflow.python.ops.data_flow_ops) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use tf.tables_initializer instead.
2017-06-20 21:20:50.396507: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-20 21:20:50.417168: W tensorflow/core/framework/op_kernel.cc:1152] Not found: ./MRII/ASL/1011_221_MR.MR.CCIR-00600_CCIR-00675_Benzinger_AV45.15.1.20150915.142526.2nn3so.dcm.JPEG
2017-06-20 21:20:50.417929: W tensorflow/core/framework/op_kernel.cc:1152] Not found: ./MRII/ASL/1011_221_MR.MR.CCIR-00600_CCIR-00675_Benzinger_AV45.15.1.20150915.142526.2nn3so.dcm.JPEG
Traceback (most recent call last):
File "decode_jpeg.py", line 10, in
tmp = sess.run(image)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 778, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 982, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1032, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1052, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: ./MRII/ASL/1011_221_MR.MR.CCIR-00600_CCIR-00675_Benzinger_AV45.15.1.20150915.142526.2nn3so.dcm.JPEG
[[Node: ReadFile = ReadFile[_device="/job:localhost/replica:0/task:0/cpu:0"](ReadFile/filename)]]

NotFoundError (see above for traceback): ./MRII/ASL/1011_221_MR.MR.CCIR-00600_CCIR-00675_Benzinger_AV45.15.1.20150915.142526.2nn3so.dcm.JPEG
[[Node: ReadFile = ReadFile[_device="/job:localhost/replica:0/task:0/cpu:0"](ReadFile/filename)]]

hosseinece on 20 Jun 2017

This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there. Thanks!

ali01 on 30 Jun 2017

I meet the same problem, have you solved it?

tlatlbtle on 17 Jul 2017

Not yet!

Plz Let me know if you can solve it.

On Jul 17, 2017 at 6:44 AM,
I meet the same problem, have you solved it?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub (https://github.com/tensorflow/models/issues/1700#issuecomment-315732861), or mute the thread (https://github.com/notifications/unsubscribe-auth/AbxmR0z2yJ5w1eibk4rT3M48LiFzSi-nks5sO0kpgaJpZM4N_yCu).

hosseinece on 17 Jul 2017

@hosseinece You probably have a corrupted JPEG file due to failure in the download, I solved that by re-downloading the flowers folder directly from : http://download.tensorflow.org/example_images/flower_photos.tgz and extracting it into tf_files after deleting the old flower_photos folder.

Boussenna on 15 Oct 2017

🎉1

@hosseinece ，I meet a silimar question with you, May i ask if the question has been solved? thank you!

clovking on 23 Feb 2018

I met the the question.

killeolee on 4 May 2018

make sure that all data is in jpeg format. For unix, see https://unix.stackexchange.com/questions/166720/convert-all-non-jpg-images-to-jpgs
the message

Error during processing file

indicates that some files have errors. Inspect or remove those.