Scikit-image: "[Errno 36] File name too long:" when using imread on remote resource with long querystring

[jo-tham]

Description

When using skimage.io.imread with a remote resource, a long query string on the remote resource will cause a failure to read the remote resource, because the temporary file cannot be created.

e.g.

The following works fine

>>> im = imread('https://c1.staticflickr.com/9/8370/8429454143_1066b73c04_o.jpg?{}'.format(''.join(['s' for i in range(100)])))

while the one below fails

>>> im = imread('https://c1.staticflickr.com/9/8370/8429454143_1066b73c04_o.jpg?{}'.format(''.join(['s' for i in range(300)])))
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/skimage/io/util.py", line 28, in file_or_url_context
    with tempfile.NamedTemporaryFile(delete=False, suffix=ext) as f:
  File "/usr/lib/python3.5/tempfile.py", line 688, in NamedTemporaryFile
    (fd, name) = _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "/usr/lib/python3.5/tempfile.py", line 399, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
OSError: [Errno 36] File name too long: '/tmp/tmpmfnujlq6.jpg?ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.5/dist-packages/skimage/io/_io.py", line 61, in imread
    with file_or_url_context(fname) as fname:
  File "/usr/lib/python3.5/contextlib.py", line 59, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.5/dist-packages/skimage/io/util.py", line 34, in file_or_url_context
    os.remove(f.name)
UnboundLocalError: local variable 'f' referenced before assignment

Way to reproduce

[If reporting a bug, please include the following important information:]

• [x] Code example
• [x] Operating system and version

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.5 LTS"

• [x] Python version: Python 3.5.2
• [x] scikit-image version (run skimage.__version__): skimage.__version__
  '0.14.1'

Proposal

https://github.com/scikit-image/scikit-image/blob/d24634d86e4f90fe96377209d66ed114b9b601e4/skimage/io/util.py#L22

The querystring should be excluded from the temporary file name extension

[jni]

@jo-tham yikes, good catch! Do you know how to submit a pull request to make that change? I agree with your suggestion btw.

Are you trying to get your pics out of Flickr before they delete them? :joy:

btw, I just thought I'd check and indeed this works:

In [1]: 's' * 5
Out[1]: 'sssss'

[jo-tham]

Hi @jni - thanks for the quick reply (and the observation on string * int)

Hehe, I didn't know flickr is changing, that example was a coincidence, I just searched for some CC images.

I ran into the issue as I maintain services that use signed urls for s3 resources. I have not run into this in the live services, but I was trying different signing methods and this issue came up.

Yep, I can submit a PR first thing tomorrow (pacific time).