Pytorch-lightning: Trainer.test() does not allow to save logs

[Varal7]

🐛 Bug

When we use trainer.test() on a fresh trainer, the call to self.log_metrics(log_metrics, {}, step) fails to produce logs.

To Reproduce

Steps to reproduce the behavior:

Code sample

model = ...  
test_dl = ... 
logger = ...
trainer = pl.Trainer(
   logger=logger
)
trainer.test(model, test_dataloaders=test_dl)

Expected behavior

Metrics are logged.

[Varal7]

Because of the way aggregate_metrics is implemented, logs are only produced when we are leaving the current step. Because there is only one step in testing, the logs are stored in self._metrics_to_agg = [metrics] but never logged.

[edenlightning]

@ justusschock any ideas?

[Varal7]

Btw, I was able to log my logs by adding self.logger.log_metrics(logs) at the end of test_epoch_end but I believe this should be the default.

[edenlightning]

@Varal7 would you be able to send a PR with the fix?

[williamFalcon]

results = trainer.test()

results is a dict with the results