Fastai: learn.tta() fails on a learner imported with load_learner()

[bpisaacoff]

Please confirm you have the latest versions of fastai, fastcore, fastscript, and nbdev prior to reporting a bug (delete one): YES

Describe the bug
learn.tta() fails on a learner which is imported with load_learner()

To Reproduce
Colab notebook reproducing the behaivor at https://colab.research.google.com/drive/1l1tnOtboOwOhsYjwi8K8cUEPIPpDJMaU?usp=sharing
Steps to reproduce the behavior:

1. Train a learner on a dataloader w/ augmentations.
2. Create a test dataloader tst_dl=learn.dls.test_dl(get_image_files(path))
3. Run preds, targs = learn.tta(dl=tst_dl)
   This runs without error. (However targs returns None which is not expected)

The error arises from exporting the learner and then loading. After step 1 above

1. Export the learner learn.export('/content/learnfile.pkl')
2. Load the learner back in in_learn=load_learner('/content/learnfile.pkl')
3. Create a test dataloader tst_dl=in_learn.dls.test_dl(get_image_files(path))
4. Running .tta() now produces an error. preds, targs = in_learn.tta(dl=tst_dl)

Expected behavior
.tta() should work on the loaded learner exactly as it did on the learner which was just trained.

Error with full stack trace

epoch   train_loss  valid_loss  accuracy    time
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-3d9acc24d6be> in <module>()
----> 1 preds, targs = in_learn.tta(dl=tst_dl)

13 frames
/usr/local/lib/python3.6/dist-packages/fastai/learner.py in tta(self, ds_idx, dl, n, item_tfms, batch_tfms, beta, use_max)
    562     if item_tfms is not None or batch_tfms is not None: dl = dl.new(after_item=item_tfms, after_batch=batch_tfms)
    563     try:
--> 564         self(_before_epoch)
    565         with dl.dataset.set_split_idx(0), self.no_mbar():
    566             if hasattr(self,'progress'): self.progress.mbar = master_bar(list(range(n)))

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in __call__(self, event_name)
    131     def ordered_cbs(self, event): return [cb for cb in sort_by_run(self.cbs) if hasattr(cb, event)]
    132 
--> 133     def __call__(self, event_name): L(event_name).map(self._call_one)
    134 
    135     def _call_one(self, event_name):

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in map(self, f, *args, **kwargs)
    381              else f.format if isinstance(f,str)
    382              else f.__getitem__)
--> 383         return self._new(map(g, self))
    384 
    385     def filter(self, f, negate=False, **kwargs):

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in _new(self, items, *args, **kwargs)
    331     @property
    332     def _xtra(self): return None
--> 333     def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
    334     def __getitem__(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)
    335     def copy(self): return self._new(self.items.copy())

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in __call__(cls, x, *args, **kwargs)
     45             return x
     46 
---> 47         res = super().__call__(*((x,) + args), **kwargs)
     48         res._newchk = 0
     49         return res

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in __init__(self, items, use_list, match, *rest)
    322         if items is None: items = []
    323         if (use_list is not None) or not _is_array(items):
--> 324             items = list(items) if use_list else _listify(items)
    325         if match is not None:
    326             if is_coll(match): match = len(match)

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in _listify(o)
    235     if isinstance(o, list): return o
    236     if isinstance(o, str) or _is_array(o): return [o]
--> 237     if is_iter(o): return list(o)
    238     return [o]
    239 

/usr/local/lib/python3.6/dist-packages/fastcore/foundation.py in __call__(self, *args, **kwargs)
    298             if isinstance(v,_Arg): kwargs[k] = args.pop(v.i)
    299         fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
--> 300         return self.fn(*fargs, **kwargs)
    301 
    302 # Cell

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in _call_one(self, event_name)
    135     def _call_one(self, event_name):
    136         assert hasattr(event, event_name), event_name
--> 137         [cb(event_name) for cb in sort_by_run(self.cbs)]
    138 
    139     def _bn_bias_state(self, with_bias): return norm_bias_params(self.model, with_bias).map(self.opt.state)

/usr/local/lib/python3.6/dist-packages/fastai/learner.py in <listcomp>(.0)
    135     def _call_one(self, event_name):
    136         assert hasattr(event, event_name), event_name
--> 137         [cb(event_name) for cb in sort_by_run(self.cbs)]
    138 
    139     def _bn_bias_state(self, with_bias): return norm_bias_params(self.model, with_bias).map(self.opt.state)

/usr/local/lib/python3.6/dist-packages/fastai/callback/core.py in __call__(self, event_name)
     42                (self.run_valid and not getattr(self, 'training', False)))
     43         res = None
---> 44         if self.run and _run: res = getattr(self, event_name, noop)()
     45         if event_name=='after_fit': self.run=True #Reset self.run to True at each end of fit
     46         return res

/usr/local/lib/python3.6/dist-packages/fastai/callback/progress.py in before_epoch(self)
     21 
     22     def before_epoch(self):
---> 23         if getattr(self, 'mbar', False): self.mbar.update(self.epoch)
     24 
     25     def before_train(self):    self._launch_pbar()

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in update(self, val)
     92             yield o
     93 
---> 94     def update(self, val): self.main_bar.update(val)
     95 
     96 # Cell

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in update(self, val)
     57         elif val <= self.first_its or val >= self.last_v + self.wait_for or val >= self.total:
     58             cur_t = time.time()
---> 59             avg_t = (cur_t - self.start_t) / val
     60             self.wait_for = max(int(self.update_every / (avg_t+1e-8)),1)
     61             self.pred_t = avg_t * self.total

AttributeError: 'NBProgressBar' object has no attribute 'start_t'

Additional context
Running on Google colab

[KushajveerSingh]

The error is not due to exporting the learner. I verified it. There is something wrong in learn.tta(). If you run learn.tta() before exporting then the code will still fail with the same error.

UPDATE: The error occurs when we run learn.tta multiple times. When I first ran learn.tta it worked as it should. But when I ran it again learn.tta I got the above error.

learn.epoch = 0 solves the issue. I think the reason is in MasterBar.update start_t is only defined when val=0 i.e. epoch=0.

[jph00]

I'm looking into this. Just FYI on this:

However targs returns None which is not expected

test_dl has no targs, so this is expected.

[jph00]

Many thanks for the clear issue and repro! :) Should be fixed now - was just missing an initializer for learn.epoch.