It seems to me that Generalized Policy Iteration (GPI) and Actor-Critic are the same, and Q-learning methods are a separate family of algorithms. I think both GPI and Actor-Critic describe the iterative process of policy evaluation (critic) and policy improvement (actor), while Q-learning is only bootstrapping using the Bellman optimality equation.

To elaborate on my understanding: Policy evaluation (critic) is done via Monte-Carlo or temporal difference methods, including function approximation if necessary, and policy improvement (actor) can be done by being greedy (tabular case) or using the policy gradient theorem (large state space). Q-learning is not doing any of these. It's just trying to estimate the $Q^\ast$ using the Bellman optimality equation by iteratively fitting the Bellman optimality equation for Q value.

I would appreciate it if anyone can confirm whether my understanding is correct or give a more systematic/precise summary of the taxonomy of online RL algorithms.