I think what you’re looking for is the median of the pooled data.

In a classification problem, the most naive way to find class probabilities is to use your training data. In MNIST, you have a 10% chance of each number. If you’re a doctor specializing in disease X than affects 7% of people, say that everyone has a 7% chance of having disease X.

The point of regression is to tighten up that estimate for a subject with some other values measured. Perhaps the doctor knows that 7% of people have the disease but that 70% of people who work around asbestos have the disease. If a patient works around asbestos, then 7% probably is the wrong probability to assume. Maybe 70% is wrong, but it’s probably more along the lines of the right answer than 7%. (Or maybe not. Maybe there’s some other covariate like smoking status or genetics that should be included in the model.)

When you do a regression with a numeric response variable, the naive way to guess the conditional parameter of interest (often the conditional mean, given covariates like asbestos), is to guess the estimate from the pooled distribution of all training data. This is why r-squared compares to the sum of deviations from the mean of the response variable, as that would be the most naive sensible way to guess. If your regression predicts median instead of mean, then your first guess would be the median of the pooled data. Ditto for other quantile regressions (e.g. 0.25 quantile).