Gsutil: CHECKSUM file not found

[jdfoote]

When I run commands with gsutil the commands work but after it runs I keep getting an IO error saying that the following file isn't found:

'~/bin/google-cloud-sdk/platform/gsutil/gslib/CHECKSUM'

Indeed, the file does not appear to exist either in my local directory or in this repository.

[whchan]

This happens to me on macos too, tried both x86_64 and x86 SDK.

[houglum]

Interesting. The location of the CHECKSUM file varies based on installation method -- when installed as a package via pip, CHECKSUM resides under /gslib/CHECKSUM; when installed via tarball or with the gcloud SDK, it resides under /CHECKSUM or /platform/gsutil/CHECKSUM (i.e. not under gslib).

Is it possible you have gsutil installed both as a package and via the cloud SDK?

[jdfoote]

I think that's totally it. I forgot that I had tried to install via pip. Something went wrong (don't remember what now) and so I installed the cloud SDK.

I removed the cloud SDK, reinstalled with pip, and all seems to be working.

[houglum]

Cool, glad to hear that fixed it :)

While this is general Python package advice, I'd like to leave it for anyone else that stumbles upon this issue:
If there are multiple ways to install a Python program (via standalone zip/tarball, via pip, etc.), having multiple installations laying around usually will lead to problems (e.g. "I thought I was invoking /home/user/me/bin/foo, but I was actually invoking /usr/bin/foo"). If you must have multiple installations on your system, general best practice is to install the pip-installed version within its own virtualenv, separate from your main system environment and other installations of the program.

[voidcenter]

I had a similar issue when installing 179.0.0 with the SDK, it looks for CHECKSUM in gsutil/gslib/ while the CHECKSUM sits in gsutil/. copying over the CHECKSUM solves the problem.

[jefffriesen]

I had the same error as @voidcenter (and fix, thank you). 179.0.0 with the SDK on OSX.

I think maintainers should consider re-opening this ticket.

[houglum]

I just checked the Cloud SDK tarballs for Linux and OSX -- all of them include the CHECKSUM file under platform/gsutil. Is it possible you might be invoking another installation of gsutil, or have leftovers from such an installation in your Python's dist-packages or site-packages directories?

[jefffriesen]

This is my first time installing it. I followed these directions for Mac OS X (x86_64): https://cloud.google.com/sdk/downloads

Basically, just did this: ./google-cloud-sdk/install.sh

[houglum]

Interesting. What's the path to your gsutil executable? The two commands below should show paths like <cloud-sdk-root>/bin/gsutil and <cloud-sdk-root>/platform/gsutil/gsutil:

$ type gsutil
gsutil is hashed (/path/to/my/home/dir/google-cloud-sdk/bin/gsutil)

$ gsutil version -l | grep -e "sdk" -e "gsutil path"
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
gsutil path: /path/to/my/home/dir/google-cloud-sdk/platform/gsutil/gsutil

[jefffriesen]

type gsutil

gsutil is hashed (/Users/jeffers/google-cloud-sdk/bin/gsutil)

gsutil version -l | grep -e "sdk" -e "gsutil path"

using cloud sdk: True
pass cloud sdk credentials to gsutil: True
gsutil path: /Users/jeffers/google-cloud-sdk/platform/gsutil/gsutil

[houglum]

And this happens after the command has finished executing? If so, that sounds like it's probably the metrics_reporter code importing the gslib module incorrectly -- either from the wrong place or with the wrong environment details, such that the code in gslib/__init__.py gets confused and thinks it was installed in a different manner than it actually was. If it's the metrics reporter, you should be able to get rid of this error by preventing it from running via this command:

$ echo -n "DISABLED" > ~/.gsutil/analytics-uuid

[jefffriesen]

actually, that's what I get after the fix @voidcenter suggested

[tlm2021]

I seem to have hit the same issue, and @houglum's suggestion fixed it.

Installed by downloading the tarball and running ./install.sh, followed by ./bin/gcloud/init.sh. Was seeing the same error after the command finished executing.

 :: type gsutil          
gsutil is hashed (/Users/Travis/Dev/cloud/google-cloud-sdk/bin/gsutil)

 :: gsutil version -l | grep -e "sdk" -e "gsutil path"
using cloud sdk: True
pass cloud sdk credentials to gsutil: True
gsutil path: /Users/Travis/Dev/cloud/google-cloud-sdk/platform/gsutil/gsutil

Disabling analytics fixed it.

echo -n "DISABLED" > ~/.gsutil/analytics-uuid

This was on macOS 10.12.6.

[wminshew]

Linux 4.13.0-43-generic #48~16.04.1-Ubuntu SMP Thu May 17 12:56:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Had same error after fresh install. Downloaded .tgz, ran install.sh, opened new tab, ran gcloud init, tried gcloud storage.

λ gsutil mb -l us-central1 gs://my-awesome-bucket/
Creating gs://my-awesome-bucket/...
ServiceException: 409 Bucket my-awesome-bucket already exists.
Traceback (most recent call last):
File "", line 1, in
File "[X]/google-cloud-sdk/platform/gsutil/gslib/__init__.py", line 119, in
CHECKSUM_FILE, CHECKSUM = _GetFileContents('CHECKSUM')
File "[X}/google-cloud-sdk/platform/gsutil/gslib/__init__.py", line 107, in _GetFileContents
content = pkgutil.get_data('gslib', filename)
File "/usr/lib/python2.7/pkgutil.py", line 591, in get_data
return loader.get_data(resource_name)
File "/usr/lib/python2.7/pkgutil.py", line 255, in get_data
return open(pathname, "rb").read()
IOError: [Errno 2] No such file or directory: '[X]/google-cloud-sdk/platform/gsutil/gslib/CHECKSUM'

Copied CHECKSUM from '[X]/google-cloud-sdk/platform/gsutil' into '[X]/google-cloud-sdk/platform/gsutil/gslib'. Seems to have resolved the error.

[rowboater]

I also got this issue described above. No sign of the package in pip at all.

Copying the CHECKSUM as @wminshew describes above fixed it.

Whoever's the admin on this needs to reopen the ticket as this is a current outstanding issue.

[thomasmaclean]

What was your install method? The current version (published this week) includes the CHECKSUM file in both expected locations (/ and /gslib/):

https://files.pythonhosted.org/packages/6e/6b/4130ff59fb14452c6991e8a68e7d558761c89db02d5bf1afcc8d25a40206/gsutil-4.54.tar.gz

How were you installing and what version did you have that lacked a CHECKSUM file?

[nickmarx12345678]

can confirm I just encountered this on install via:
https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-319.0.0-darwin-x86_64.tar.gz, following the instructions for mac os x here: https://cloud.google.com/storage/docs/gsutil_install

[Ruyven]

Can confirm that the same problem still happens almost 3 years after the original issue report, after following the official instructions on macOS.

Google Cloud SDK version: 346.0.0-darwin-x86_64
gsutil version: 4.63
Python version: 2.7.16 (the one that came preinstalled)

Tried this solution but it didn't solve the problem, I could only silence the warning by copying the CHECKSUM file from platform/gsutil to platform/gsutil/gslib.

Any chance to get this issue reopened since the problem still exists?