【求助】强化学习backword()一直报错

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 64]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!<br><br>已经没有inplace操作了，为啥还会一直报这个错呢<br>

你看看 你的代码里面是不是有 w+=1的这种操作，这种会被认定成inplace操作，然后报错。。。。一个很坑的问题。。。

<br>【 在 IWhisper#856 的大作中提到: 】<br><font class="f006">: 你看看 你的代码里面是不是有 w+=1的这种操作，这种会被认定成inplace操作，然后报错。。。。一个很坑的问题。。。 </font><br>检查好几遍了，没有这个问题啊<img src="/img/ubb/em/9.gif" alt="em9" style="display:inline;border-style:none">

你把报错的附近的几行发上来看一眼

<br>【 在 IWhisper#856 的大作中提到: 】<br><font class="f006">: 你把报错的附近的几行发上来看一眼 </font><br>q_actor_loss = agent.critic_eval(states, mu).flatten().clone()<br>actor_loss = -T.mean(q_actor_loss).clone()<br>agent.actor_optim.zero_grad()<br>actor_loss.backward(retain_graph=True)<br>agent.actor_optim.step()

backward<br>【 在 IWhisper#697 的大作中提到: 】<br><font class="f006">:&nbsp;&nbsp;</font><br><font class="f006">: q_actor_loss = agent.critic_eval(states, mu).flatten().clone() </font><br><font class="f006">: actor_loss = -T.mean(q_actor_loss).clone() </font><br><font class="f006">: ............ </font>

<br>【 在 IWhisper#666 的大作中提到: 】<br><font class="f006">: backward </font><br>只有backward()报新的错：RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.<br><img src="/img/ubb/em/15.gif" alt="em15" style="display:inline;border-style:none">

只看这里没用，backward报的错，但是其他地方可能漏了inplace<br>【 在 IWhisper#697 的大作中提到: 】<br><font class="f006">:&nbsp;&nbsp;</font><br><font class="f006">: q_actor_loss = agent.critic_eval(states, mu).flatten().clone() </font><br><font class="f006">: actor_loss = -T.mean(q_actor_loss).clone() </font><br><font class="f006">: ............ </font>

这部分看起来莫得，你可能需要pdb进到forward里面去看

已经解决啦，换了一个torch版本<img src="/img/ubb/em/3.gif" alt="em3" style="display:inline;border-style:none">