Below is an example (p.89) from "RL and Optimal Control" book by D.Bertsekas on the construction of a a case study where the rollout algorithm is worse than the base heuristic on which the rollout algorithm is constructed. It is not clear why sequential improvement is violated. First of all, the first sentence is not clear to me: is it the rollout algorithm or the base heuristic that accidentally produces an optimal control sequence? It is not clear to me why sequential improvement is violated in the provided example. I will be happy for clear explanation.

Full article content could not be extracted automatically. Read the original below.