How do I correctly apply action masking during DDPG training in Python?

I'm implementing the Deep Deterministic Policy Gradient (DDPG) algorithm in PyTorch, and I'm facing issues with applying an action mask during the training process. Currently, I apply an action mask in the simulation step to ensure only valid actions are selected. However, I'm uncertain whether I should also apply a mask during the training step. Specifically, I need help with: Generating next_actions: Should I apply the action mask when generating next_actions using self.actor_target(next_states) for computing the Q-value target? next_actions = self.actor_target(next_states) # Should an action mask be applied here? Calculating policy_loss: When calculating the policy loss using the main actor (self.actor), should I apply the action mask to self.actor(states) before passing it to the critic? policy_loss = -self.critic(states, self.actor(states)).mean() # Should an action mask be applied here? I've tried both approaches (with and without masking), but the results are inconsistent, and I'm not sure which approach aligns with best practices in DDPG implementation. What is the recommended practice for handling action masks during DDPG training in this context?