Pytorch-lightning: trainer.evaluate() without trainer.fit()

[festeh]

Describe the bug
I'd like to only evaluate fitted model with Trainer instance by calling its evaluate method

To Reproduce
Steps to reproduce the behavior:

model = MyLightningModule()
trainer = Trainer()
trainer.evaluate(model, model.val_dataloader(), max_batches=100)

Gets TypeError: 'NoneType' object is not callable when it tries to call trainer.get_val_dataloaders which is does not have, because fit wasn't called before
Expected behavior
Passing the model and dataloader into evaluate should be enough to evaluate model's performance.

[awaelchli]

Perhaps evaluate is not meant to be used like that. Maybe use test instead?
Define the test methods and then call trainer.test(): see here and here

You can then limit the size of your dataset inside the test_dataloader method.

Or do I misundestand?

[williamFalcon]

Evaluate should really be a private method. not meant to be used externally

[festeh]

Got it, thanks!

[sshleifer]

Can I get metrics named 'val_*' without training?
Trainer.test(test_dataloaders...) will produce metrics called test_ with my current code. The easiest way I can think of is to make an empty train dataset?