Two books that really helped me to understand RL/deep RL are:

Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton

Deep Reinforcement Learning: Fundamentals, Research and Applications by Hao Dong, Zihan Ding, Shanghang Zhang

Before, I started getting familiar with RL through online courses and papers, but after even coding and research, I read Sutton's book again and found it really great. It deserves to be the reference for RL. Start with it, and you can see how it improves your vision of RL.