What I need help with / What I was wondering
Your question, or a clear description of what you're looking for help with.
When I download celeb with tfds in China I get the following error.
Dl Completed...: 0 url [00:00, ? url/s]
Dl Size...: 0 MiB [00:00, ? MiB/s][A
Extraction completed...: 0 file [00:00, ? file/s][A[A
Dl Completed...: 0%| | 0/1 [00:00Dl Size...: 0 MiB [00:00, ? MiB/s][A
Extraction completed...: 0 file [00:00, ? file/s][A[A
Dl Completed...: 0%| | 0/2 [00:00Dl Size...: 0 MiB [00:00, ? MiB/s][A
Extraction completed...: 0 file [00:00, ? file/s][A[A
Dl Completed...: 0%| | 0/3 [00:00Dl Size...: 0 MiB [00:00, ? MiB/s][A
Extraction completed...: 0 file [00:00, ? file/s][A[A
Dl Completed...: 0%| | 0/4 [00:00Dl Size...: 0 MiB [00:00, ? MiB/s][A
Extraction completed...: 0 file [00:00, ? file/s][A[A/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/home/xieyi/.local/lib/python3.6/site-packages/urllib3/connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)Dl Completed...: 0%| | 0/4 [00:01
Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.05s/ url]
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 50%|█████ | 2/4 [00:01<00:02, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 75%|███████▌ | 3/4 [00:01<00:01, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 75%|███████▌ | 3/4 [00:01<00:01, 1.05s/ url]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Dl Completed...: 100%|██████████| 4/4 [00:01<00:00, 1.28 url/s]
Dl Completed...: 100%|██████████| 4/4 [00:01<00:00, 1.28 url/s]Dl Size...: 0 MiB [00:01, ? MiB/s][A
Extraction completed...: 0 file [00:01, ? file/s][A[A
Extraction completed...: 0 file [00:01, ? file/s]Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 100%|██████████| 4/4 [00:01<00:00, 2.60 url/s]
Traceback (most recent call last):
File "create_celeba_datasets.py", line 18, in
main();
File "create_celeba_datasets.py", line 10, in main
celeba_builder.download_and_prepare();
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/api_utils.py", line 52, in disallow_positional_args_dec
return fn(args, *kwargs)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/dataset_builder.py", line 287, in download_and_prepare
download_config=download_config)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/dataset_builder.py", line 937, in _download_and_prepare
max_examples_per_split=download_config.max_examples_per_split,
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/dataset_builder.py", line 793, in _download_and_prepare
for split_generator in self._split_generators(dl_manager):
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/image/celeba.py", line 132, in _split_generators
"landmarks_celeba": LANDMARKS_DATA,
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/download/download_manager.py", line 359, in download_and_extract
return _map_promise(self._download_extract, url_or_urls)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/download/download_manager.py", line 395, in _map_promise
res = utils.map_nested(_wait_on_promise, all_promises)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/utils/py_utils.py", line 129, in map_nested
for k, v in data_struct.items()
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/utils/py_utils.py", line 129, in
for k, v in data_struct.items()
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/utils/py_utils.py", line 143, in map_nested
return function(data_struct)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/download/download_manager.py", line 379, in _wait_on_promise
return p.get()
File "/home/xieyi/.local/lib/python3.6/site-packages/promise/promise.py", line 510, in get
return self._target_settled_value(_raise=True)
File "/home/xieyi/.local/lib/python3.6/site-packages/promise/promise.py", line 514, in _target_settled_value
return self._target()._settled_value(_raise)
File "/home/xieyi/.local/lib/python3.6/site-packages/promise/promise.py", line 224, in _settled_value
reraise(type(raise_val), raise_val, self._traceback)
File "/home/xieyi/.local/lib/python3.6/site-packages/six.py", line 693, in reraise
raise value
File "/home/xieyi/.local/lib/python3.6/site-packages/promise/promise.py", line 85, in try_catch
return (handler(args, *kwargs), None)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/download/download_manager.py", line 246, in callback
resource, download_dir_path, checksum, dl_size)
File "/home/xieyi/.local/lib/python3.6/site-packages/tensorflow_datasets/core/download/download_manager.py", line 207, in _handle_download_result
raise NonMatchingChecksumError(resource.url, tmp_path)
tensorflow_datasets.core.download.download_manager.NonMatchingChecksumError: Artifact https://drive.google.com/uc?export=download&id=0B7EVK8r0v71pZjFTYXZWM3FlRnM, downloaded to /home/xieyi/tensorflow_datasets/downloads/ucexport_download_id_0B7EVK8r0v71pZjFTYXZWM3FlDDaXUAQO8EGH_a7VqGNLRtW52mva1LzDrb-V723OQN8.tmp.3d1c735fd8a9427b8cba7107a1e811a4/uc, has wrong checksum.
I dont know wether it is an internal error of tfds or a problem or a network problem.
What I've tried so far
A description of what you've tried so far to solve your problem.
I tried the following code on different machines in China.
celeba_builder = tfds.builder("celeb_a");
celeba_builder.download_and_prepare();
the problem occurs. I tried downloading mnist with tfds and succeeded. It seems an exclusive problem of celeba.
It would be nice if...
Could we have done anything to make things better (documentation, etc.)?
sorry I dont know how.
Environment information
(if applicable)
- Operating System: ubuntu 18.04
- Python version: 3.6.8
tensorflow-datasets/tfds-nightlyversion: 1.2.0tensorflow/tensorflow-gpu/tf-nightly/tf-nightly-gpuversion: 2.0.0-dev20190731
breadbread1984
All 18 comments
Have you tried with a VPN?
Qoyyuum
on 10 Sep 2019
yeah, HK's vps
breadbread1984
on 10 Sep 2019
yeah, HK's vps
Could you try a VPS outside of China?
Qoyyuum
on 11 Sep 2019
OK if that is the only solution.
breadbread1984
on 11 Sep 2019
I have exactly the same problem. Have you found any solution?
faajabbari
on 25 Feb 2020
no solution yet
breadbread1984
on 25 Feb 2020
For wrong checksums issues: See: https://github.com/tensorflow/datasets/issues/1482
Conchylicultor
on 25 Feb 2020
@Conchylicultor ?? the checksum doesnt add up cause the file simply didnt download so sums to 0
elfelround
on 15 Apr 2020
tried setting FORCE_REDOWNLOAD and register_checksums=True with no success
read somewhere it could be something to do with system trying to avoid same user downloading too much and to try using a vpn, anyone tried seeing if that worked?
Dl Completed...: 0 url [00:00, ? url/s]
Dl Size...: 0 MiB [00:00, ? MiB/s]
Dl Completed...: 0%| | 0/1 [00:00 Dl Size...: 0 MiB [00:00, ? MiB/s]
Dl Completed...: 0%| | 0/2 [00:00 Dl Size...: 0 MiB [00:00, ? MiB/s]
Dl Completed...: 0%| | 0/3 [00:00 Dl Size...: 0 MiB [00:00, ? MiB/s]
Dl Completed...: 0%| | 0/4 [00:00 Dl Size...: 0 MiB [00:00, ? MiB/s]
Extraction completed...: 0 file [00:00, ? file/s]/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Dl Completed...: 0%| | 0/4 [00:01
Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.03s/ url]
Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 25%|██▌ | 1/4 [00:01<00:03, 1.03s/ url]
Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 50%|█████ | 2/4 [00:01<00:01, 1.33 url/s]
Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 50%|█████ | 2/4 [00:01<00:01, 1.33 url/s]
Dl Size...: 0 MiB [00:01, ? MiB/s]
Dl Completed...: 75%|███████▌ | 3/4 [00:01<00:00, 1.33 url/s]
Dl Size...: 0 MiB [00:01, ? MiB/s]
Extraction completed...: 0 file [00:01, ? file/s]/usr/lib/python3/dist-packages/urllib3/connectionpool.py:860: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
Dl Completed...: 75%|███████▌ | 3/4 [00:02<00:00, 1.33 url/s]
Dl Size...: 0 MiB [00:02, ? MiB/s]
Dl Completed...: 100%|██████████| 4/4 [00:02<00:00, 1.32 url/s]
Dl Size...: 0 MiB [00:02, ? MiB/s]
Extraction completed...: 0 file [00:02, ? file/s]
Dl Size...: 0 MiB [00:02, ? MiB/s]
Dl Completed...: 100%|██████████| 4/4 [00:02<00:00, 1.49 url/s]
Traceback (most recent call last):
File "/usr/lib/python3.7/contextlib.py", line 130, in __exit__
self.gen.throw(type, value, traceback)
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/core/file_format_adapter.py", line 200, in incomplete_dir
yield tmp_dir
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/core/dataset_builder.py", line 334, in download_and_prepare
download_config=download_config)
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/core/dataset_builder.py", line 1030, in _download_and_prepare
max_examples_per_split=download_config.max_examples_per_split,
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/core/dataset_builder.py", line 883, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/core/dataset_builder.py", line 1055, in _prepare_split
total=split_info.num_examples, leave=False):
File "/home/elfelround/.local/lib/python3.7/site-packages/tqdm/std.py", line 1127, in __iter__
for obj in iterable:
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/image/celeba.py", line 193, in _generate_examples
for line in f.readlines()
File "/home/elfelround/.local/lib/python3.7/site-packages/tensorflow_datasets/image/celeba.py", line 194, in
if int(line.split()[1]) == file_id
ValueError: invalid literal for int() with base 10: 'html>