It is common to use mutual information as feature selection method. However, I fail to see why this is the case, since the mutual information $I(X, Y)$ depends on both entropies $H(X)$ and $H(Y)$ via the formula:

$$ I(X, Y) = H(X) + H(Y) - H(X,Y)$$

meaning that comparing $I(X_i, Y)$ and $(X_j, Y)$ as a measure for selecting between $X_i$ and $X_j$ is not straightforward since the measure can be bloated by the marginal entropies. It is like selecting between $X_i$ and $X_j$ based on the covariance with $Y$ instead of correlation.

The only way I can think that such a comparison is allowed is due to the equivalent formula:

$$I(X, Y) = H(Y) - H(Y|X)$$

As the first term $H(Y)$ is the same for all $X_i$ then the ramking shouldn't depend on the "scale" of $H(Y)$. Is that correct or am I missing something?