Dvc: Issue with dvc push to AWS s3 remote

[jonhilgart22]

Please provide information about your setup
DVC version(i.e. dvc --version), Platform and method of installation (pip, homebrew, pkg Mac, exe (Windows), DEB(Linux), RPM(Linux))

DVC: 0.62.1
Mac: Mojave 10.13
Install with pip

issue with dvc push

[efiop]

Hi @jonhilgart22 ! Could you show dvc version(note, it is not the same as dvc --version) output please?

[jonhilgart22]

[efiop]

@jonhilgart22 Thanks! And could you show dvc push -v output, please?

[jonhilgart22]

[efiop]

@jonhilgart22 And pip freeze | grep tqdm please?

[jonhilgart22]

[jonhilgart22]

Weird - I just tried dvc push again and it worked.

[efiop]

@jonhilgart22 Hm, just like that? O_o You've done nothing else and it started working?

[jonhilgart22]

yep! strange

[efiop]

@jonhilgart22 Maybe you've updated python or something recently? We've seen some flukes like that when virtualenv breaks after updating a python.

[jonhilgart22]

Not sure but thanks for the help!

[jonhilgart22]

Alright, I'm running into this issue again.

I've installed DVC inside of a pipenv.

[jonhilgart22]

[efiop]

Hi @casperdcl ! Seems to be related to tqdm. Maybe you have some ideas? :slightly_smiling_face:

[casperdcl]

@jonhilgart22 can you run tqdm --version in the pipenv too?

[jonhilgart22]

tqdm=4.36.1

[casperdcl]

I wonder if this is some sort of edge case with = in the filenames or something similar. The best way to debug this would be if @jonhilgart22 could patch tqdm/std.py and post the error output.

diff --git a/tqdm/std.py b/tqdm/std.py
index 8c73c8f..9d9938b 100644
--- a/tqdm/std.py
+++ b/tqdm/std.py
@@ -466,7 +466,10 @@ class tqdm(Comparable):
                 bar_format = "{l_bar}{bar}{r_bar}"

             full_bar = FormatReplace()
-            nobar = bar_format.format(bar=full_bar, **format_dict)
+            try:
+                nobar = bar_format.format(bar=full_bar, **format_dict)
+            except:
+                raise KeyError(bar_format + "<-" + str(format_dict))
             if not full_bar.format_called:
                 # no {bar}, we can just format and return
                 return nobar

or
pip install --force-reinstall git+https://github.com/tqdm/tqdm.git@bar_format_bug#egg=tqdm

[jonhilgart22]

after

pip install --force-reinstall git+https://github.com/tqdm/tqdm.git@bar_format_bug#egg=tqdm

I see this.

[casperdcl]

well I just typed that all out rather than asking for text to copy-paste :)

problem was ncols_desc==0.

[efiop]

@casperdcl Great investigation! Thank you! :tada:

@jonhilgart22 Btw, may I ask why are you taking a photo of your screen instead of copy-pasting? I'm honestly really curious :slightly_smiling_face:

[casperdcl]

yes I felt it was urgent. Thankfully didn't get distracted trying to tweak an OCR program to work on the image!

Also thankfully not an upstream @tqdm/tqdm bug.

Also surprised by how critical this bug is yet how hard it would've been to trace without the last response (https://github.com/iterative/dvc/issues/2598#issuecomment-543319161) and without knowing a lot about tqdm internals.

[efiop]

@jonhilgart22 0.63.3 is out, please feel free to give it a try :) Thanks for the feedback!

[jonhilgart22]

Thanks for the quick response!