Dvc: gs: catch and wrap exceptions

Created on 1 Oct 2019  路  7Comments  路  Source: iterative/dvc

$ dvc -V
0.60.1+39104a

$ dvc get https://github.com/vkoriukina/pretrained-models.pytorch/ --rev dvc_support model_weights/resnet50-19c8e357.pth
Adding '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmpccebo54zdvc-repo/.dvc/tmp' to '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmpccebo54zdvc-repo/.dvc/.gitignore'.
ERROR: unexpected error - 403 GET https://www.googleapis.com/storage/v1/b/vkoriukina_bucket/o?projection=noAcl&prefix=9e%2F9c86b324d80e65229fab49b8d9a8e8: [email protected] does not have storage.objects.list access to vkoriukina_bucket.

I do not have permissons to GCS bucket. The second error is legit. While the first one contains not user friendly temporary paths like ../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmpccebo54zdvc-repo/.dvc/tmp

bug c1-quick-fix p1-important research ui

Most helpful comment

After I fixed the permission issue:

$ dvc get https://github.com/vkoriukina/pretrained-models.pytorch/ --rev dvc_support model_weights/resnet50-19c8e357.pth
Adding '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmp1crq1jj_dvc-repo/.dvc/tmp' to '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmp1crq1jj_dvc-repo/.dvc/.gitignore'.
$ ll resnet50-19c8e357.pth
-rw-r--r--  1 dmitry  staff  102502400 Sep  3 16:59 resnet50-19c8e357.pth

The paths are very confusing.

All 7 comments

First step research where that exc is comming from. Then wrap the exception into something more helpful. Not sure about first Adding message just yet.

After I fixed the permission issue:

$ dvc get https://github.com/vkoriukina/pretrained-models.pytorch/ --rev dvc_support model_weights/resnet50-19c8e357.pth
Adding '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmp1crq1jj_dvc-repo/.dvc/tmp' to '../../../../private/var/folders/hq/h82mgrhx37nc89_1rcg6gm280000gn/T/tmp1crq1jj_dvc-repo/.dvc/.gitignore'.
$ ll resnet50-19c8e357.pth
-rw-r--r--  1 dmitry  staff  102502400 Sep  3 16:59 resnet50-19c8e357.pth

The paths are very confusing.

A point on error messages. There are other HTTP error codes/messages, which we can encounter. We now only wrap 404 Not Found, for everything else we should at least show a status line - some people might not immediately recognize that 503 means Temporary Unavailable.

We might wrap 403 with our own message, but simply saying 403 Forbidden might be enough .though.

On confusing paths. We should remove that message for tmp repos entirely.

@efiop I will take this issue after exams :)

@algomaster99 Maybe someone else would want to take it earlier, you could pick something new then :slightly_smiling_face:

The first message no longer shows up, as we've removed it in another patch. @Suor The original 403 issue has a legit error in it, so it is useful as is. Would be nice to catch gs exceptions and re-raise them as a cause in DvcException though, but that is for the next iteration. Repurposing the ticket...

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ghost picture ghost  路  3Comments

anotherbugmaster picture anotherbugmaster  路  3Comments

prihoda picture prihoda  路  3Comments

ghost picture ghost  路  3Comments

gregfriedland picture gregfriedland  路  3Comments