Tqdm: built-in support for callback/hook based updates

[jlumpe]

For external library functions whose internals you do not have access to, frequently the only way to use them with tqdm is to pass a callback that updates the progress bar. This is a common enough use case that there is an example of it in the README.

I think a method should be added directly to the tqdm class to support this. The method would take an optional function to be wrapped and return a callback function. An example would be something like this:

with tqdm(total=100) as pbar:
    result = some_library_func(number_of_things=100, cb=pbar.callback())

    # Equivalent to:
    result = some_library_func(number_of_things=100, cb=lambda *args, **kw: pbar.update())

Where the callback function calls pbar.update(), but accepts and ignores any arguments the library function passes to it. In the case where the callback is some function that needs to return a value and is called may times, the method would simply wrap this function:

import numpy as np
import pandas as pd

df = pd.DataFrame(np.random.rand(10000, 100))

# (I know tqdm already has Pandas integration)
with tqdm(total=df.shape[0]) as pbar:
    result = df.apply(pbar.callback(some_slow_func))

This enables very easy use with existing functions or lambda functions, without having to define a new wrapper function each time. Extensions could include taking the amount to update by from the return value of the function.

I can definitely contribute this myself, I just wanted to see if there is support for it before starting.

[chengs]

for pandas
maybe it's better to use

for idx, row in tqdm(df.iterrows()):
    # do something

but, I do think it is a good idea for other use cases.

[casperdcl]

Not sure about the first case (merits of aliasing callback(*a, **k) to update()).

1. lambda notation does this quite nicely and clearly
2. you'd usually want to use *a, **k to update descriptions/postfixes, which would require you to write a custom callback anyway

About providing a decorator wrapper t.callback(f) where

class tqdm(...):
  ...
  def callback(self, f):
    def inner(*a, **k):
      res = f(*a, **k)
      self.update()
      return res
    return inner

Well, I don't know. Maybe if there's enough support for it...

[CentralLT]

@casperdcl your still working on this issue?
@jlumpe is this still a pressing matter?

[casperdcl]

EDIT: nvm srry different issue

[casperdcl]

tqdm.update itself can be used as a callback. You can also define your own callback wrappers, e.g. see TqdmUpTo from https://github.com/tqdm/tqdm/#hooks-and-callbacks

It's up to users to write their own wrappers.