Correctly evaluating (different) time series models

Currently, I am going through Hyndman's book to learn more about time series (I use the 2nd version with the forecast package). I have trouble understanding the "correct" way of evaluating multiple (let's say 5 different) time series models. I can think of 3 different ways, and I am not sure which is the correct one: Split data into train/test => Fit all models on the training set and evaluate the models on the test set. Even though this would give us an indication of which model is the best, it will estimate the test error too small since we made a model choice based on it. Use time series cross validation as described in that part . This helps us finding the best model based on the lowest CV error. But after retraining the "best" model on the whole training data we don't have a test set left. Split the data into train/test => Fit all models on train and use time series CV on train => After finding the best model, retrain on the best model on train data set and evaluate it on the test data. In my opinion, the last approach would be the correct one if our goal is to find the best model and also get a reasonable error on a test set. Is my intuition correct, or am I missing something here? I don't know if I think too complicated? I have a second question that is closely related to this one: Is it possible to use the tsCV function from forecast on a combination of forecast models/hybrid model (e.g. the average forecast of an ARIMA, ETS, other models). In other words, the same a…