Proper way to do gradient clipping?

I’m sorry but I don’t understand the question. Optimizer never calls backward() itself, unless you give it a callable argument (see torch.optim docs for more details on that). BTW you might want to use torch.nn.utils.clip_grad_norm now.

Read full topic