Deep Q-learning is most suitable for problems with a large or infinite number of potential states. This is because the more states there are, the more information the Q-learning algorithm has to work with, and the more accurate its predictions about future states and rewards will be, with a large number of potential states the algorithm is able to learn from more data. In problems with fewer potential states, the algorithm may not have enough data to work with in order to make accurate predictions about future states and rewards.

Hence a large number of potential states is that the algorithm is able to generalize better. In problems with fewer potential states, the algorithm may not be able to learn the underlying structure of the problem as well, and thus may not be able to generalize as well to new states.