Couple things:

  1. If your outcome is sale price -- a continuous variable -- why are you entertaining Logistic regression as a means of estimation?

  2. Correlation is a problem for inference, but I don't really see it as a problem for prediction. If all you want is to predict stuff, correlation isn't a big deal. Conversely, uncorrelated predictors may hide hidden relationships since correlation measures the strength of a linear relationship. The Wikipedia article on correlation shows several cases where data are uncorrelated and yet a pattern exists.

Here is a little example. I generate 1000 training examples from

$$ y \sim \mathcal{N}(X\beta, \Sigma) $$

and test on 10,000 examples. I do this in two cases: where the features are highly correlated (largest correlation is >0.8 in absolute value) and the other where they are uncorrelated. Here is the histogram of the testing errors. Can you tell me which plot -- red or blue -- comes from the correlated features?

enter image description here

  1. Correlation matrix is not a great way to determine which features to select for prediction since features may be uncorrelated with the outcome individually but jointly could help with prediction. Consider the XOR example. You have 2 binomial signals (1s or 0s). The outcome is 1 if only one of the signals is 1, else 0. If you looked at either signal in isolation with the outcome, the two would appear to be uncorrelated. But jointly, the outcome is determined. In addition, looking at the correlation matrix gives the false impression that you can select individual categories from a categorical variable. For example, BMW might be a good predictor of net income, but if the BMW variable comes from a variable for which type of car a person owns, you can't just select the BMW variable and disregard the others.

  2. Consider using something like LASSO or Ridge regression if you have lots of variables.

code for the plot:

library(tidyverse)
library(rethinking)

K = 3
beta = rnorm(K, 2, 4)

S = diag(K) - diag(K)
while( max(abs(S))<0.9){
  S = rlkjcorr(1, K)
}

correlated_rmse = replicate(1000,{
  

  
  
  X = MASS::mvrnorm(n = 1000, mu = rep(0,K), Sigma = S)
  y = 2.0 + rnorm(X%*%beta, 1)
  d = as_tibble(X)
  d$y = y
  
  model = lm(y~., data = d)
  
  Xtest = MASS::mvrnorm(n = 10000, mu = rep(0,K), Sigma = S)
  ytest = 2.0 + rnorm(Xtest%*%beta, 1)
  
  dtest = as_tibble(Xtest)
  predictions = predict(model, newdata = dtest)
  
  Metrics::rmse(ytest, predictions)
  
})


uncorrelated_rmse = replicate(1000,{
  
  S = diag(K) 
  
  X = MASS::mvrnorm(n = 1000, mu = rep(0,K), Sigma = S)
  y = 2.0 + rnorm(X%*%beta, 1)
  d = as_tibble(X)
  d$y = y
  
  model = lm(y~., data = d)
  
  Xtest = MASS::mvrnorm(n = 10000, mu = rep(0,K), Sigma = S)
  ytest = 2.0 + rnorm(Xtest%*%beta, 1)
  
  dtest = as_tibble(Xtest)
  predictions = predict(model, newdata = dtest)
  
  Metrics::rmse(ytest, predictions)
  
})


p1 = hist(uncorrelated_rmse, main = '')
p2 = hist(correlated_rmse, main = '')

plot(p1, col = rgb(0,0,1,0.25), main = '', xlab = '')
plot(p2, col = rgb(1,0,0,0.25), add = T, main = '', xlab = '')