What does model.eval() do in pytorch?

Gulzar

Feb 1

0 views1

When should I use .eval()? I understand it is supposed to allow me to "evaluate my model". How do I turn it back off for training?

Example training code using .eval().

deep-learning

pytorch

python machine-learning

1 answer

Accepted answer · original discussion

tarasio

Feb 1

Permalink

model.eval() is a kind of switch for some specific layers/parts of the model that behave differently during training and inference (evaluating) time. For example, Dropouts Layers, BatchNorm Layers etc. You need to turn them off during model evaluation, and .eval() will do it for you. In addition, the common practice for evaluating/validation is using torch.no_grad() in pair with model.eval() to turn off gradients computation:

# evaluate model:
model.eval()

with torch.no_grad():
    ...
    out_data = model(data)
    ...

BUT, don't forget to turn back to training mode after eval step:

# training step
...
model.train()
...

Your perspective belongs here.

3 question comments

Use comments to ask for clarification. Post a solution as an answer.

BerrielPermalink

Feb 1

Does this answer your question? What does model.train() do in pytorch?

Charlie ParkerPermalink

May 12

is there a flag to detect if the model is in eval mode? e.g. mdl.is_eval()?

Charlie ParkerPermalink

Dec 19

it simple changes the self.training via self.training = training recursively for all modules by doing self.train(False). In fact that is what self.train does, changes the flag to true recursively for all modules. see code: github.com/pytorch/pytorch/blob/…