Tqdm: Nested loops not displayed correctly

[mrclary]

• [x] I have marked all applicable categories:
  
  
  
  • [ ] exception-raising bug
  
  
  • [x] visual output bug
  
  
  • [ ] documentation request (i.e. "X is missing from the documentation." If instead I want to ask "how to use X?" I understand [StackOverflow#tqdm] is more appropriate)
  
  
  • [ ] new feature request
  
  
• [x] I have visited the [source website], and in particular
  read the [known issues]
• [x] I have searched through the [issue tracker] for duplicates
• [x] I have mentioned version numbers, operating system and
  environment, where applicable:

tqdm = 4.34.0
Python = 3.7.6 (default, Jan 8 2020, 13:42:34)
Platform = [Clang 4.0.1 (tags/RELEASE_401/final)] darwin

tqdm spams output lines when using nested loops.
This issue has been reported before in the context of QtConsole, but it seems to manifest even for Python consoles for all tqdm versions > 4.33.0.

Version 4.34.0:

(c2w_37_test) >> python tqdm_test.py 
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.51it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.91it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.82it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.93it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 43.59it/s]
2nd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00,  8.95it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 41.82it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.02it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.44it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.12it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.28it/s]
2nd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00,  8.74it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 42.95it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.18it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.66it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.17it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 43.77it/s]
2nd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00,  8.83it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 45.07it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.38it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 46.15it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 40.51it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 43.83it/s]
2nd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00,  8.75it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.27it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 43.05it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 42.18it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 41.62it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 44.21it/s]
2nd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00,  8.57it/s]
1st loop: 100%|█████████████████████████████████████| 5/5 [00:02<00:00,  1.75it/s]
(c2w_37_test) >>

Even version 4.33.0 has the undesirable effect of putting the command prompt after the first loop line when completed, rather than after all loop lines.
Version 4.33.0:

(c2w_37_test) >> python tqdm_test.py 
1st loop: 100%|█████████████████████████████████████| 5/5 [00:02<00:00,  1.74it/s]
(c2w_37_test) >> ███████████████████████████████████| 5/5 [00:00<00:00,  8.48it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 40.88it/s]

tqdm_test.py

from tqdm import tqdm
import time

for i in tqdm(range(5), desc='1st loop'):
    for j in tqdm(range(5), desc='2nd loop'):
        for k in tqdm(range(5), desc='3rd loop'):
            time.sleep(0.02)

[casperdcl]

You need an extra print('\n' * 2) (or tqdm.write('\n')) after your outermost for loop since there were two nested loops which the outermost one couldn't have been aware of.

We should add that to the readme...

[Hiestaa]

I seem to experience the same issue even with only 2 nested loops.

Version

• tqdm = 4.45.0
• Python = 3.7.4
• Platform = macOS 10.14.6 (18G3020) Kernel Version: Darwin 18.7.0

Sample code

from tqdm import tqdm, trange
import time

for x in trange(10, desc="loop1"):
    for y in trange(10, desc="loop2"):
        time.sleep(0.1)

Output

loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.69it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.73it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.74it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.75it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.77it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.77it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.65it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.75it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.65it/s]
loop2: 100%|██████████████████████████████████████████| 10/10 [00:01<00:00,  9.67it/s]
loop1: 100%|██████████████████████████████████████████| 10/10 [00:10<00:00,  1.03s/it]

The progress bar for loop2 when closed after the first completion of all iterations always overrides the progress bar for loop1 and moves both progress bars one line further down.

I am not sure to understand the suggestion to add an extra call to print or tqdm.write after the outermost for loop - I tried to add print('\n') or tqdm.write('') pretty much everywhere but all this does is adding undesired new lines to the output. I also tried manual positioning with position=0 and position=1 for first and second loops respectively but I didn't notice any change.

[mrclary]

@Hiestaa , the behavior you see with two loops is the same behavior I demonstrated with three loops; I did not mean to imply that it only happened with three loops.

The suggestion for adding an extra print statement does not refer to this issue, but to the secondary issue that I mentioned: the incorrect placement of the cursor after all loops have completed.

(c2w_37_test) >> python tqdm_test.py 
1st loop: 100%|█████████████████████████████████████| 5/5 [00:02<00:00,  1.74it/s]
(c2w_37_test) >> ███████████████████████████████████| 5/5 [00:00<00:00,  8.48it/s]
3rd loop: 100%|█████████████████████████████████████| 5/5 [00:00<00:00, 40.88it/s]

[casperdcl]

from tqdm import tqdm, trange
import time

# leave all bars
for x in trange(10, desc="loop1"):
    for y in trange(10, desc="loop2"):
        time.sleep(0.1)

# don't leave inner bars
for x in trange(10, desc="loop1"):
    for y in trange(10, desc="loop2", leave=False):
        time.sleep(0.1)

# auto-detect and don't leave inner bars
for x in trange(10, desc="loop1", leave=None):
    for y in trange(10, desc="loop2", leave=None):
        time.sleep(0.1)

# concurrent
multithreaded_function_with_concurrent_bars(jobs=10, progress_leave=True)
print("\n" * 9)

If you have an issue despite NOT doing concurrent things then you probably need to upgrade tqdm.

[casperdcl]

closing in favour of #811 & #879