The idea of batchnorm is to normalize the input to a range that is suitable for subsequent computation (say tanh activation). The most naive idea is to normalize each sample by subtracting the batch mean and then dividing it by the batch's standard derivation. So the overall normalized batch will have zero mean and unit variance.

However, how do we know that zero mean and unit variance is the best for subsequent operations? We don't.

That is why it also introduces beta and gamma to readjust the mean to beta and standard derivation gamma. But these parameters will be learned instead of preset. gamma is initialized to 1 to begin with. But if you are recovering a checkpoint or transferring a model, you may want to initialize it to the value from the trained model.

For practical purposes, I believe that should be automatically taken care of when you reload a checkpoint with tensorflow and so you should not need to worry about it.