My direct answer to the question is that p-values clearly do not depend on the alternative hypothesis, as the alternative is not present in the calculations, but at the same time p-values are dependent on the alternative hypothesis insofar as a one-tailed p-value is different from a two-tailed p-value and we typically use the alternative hypothesis to specify the number of tails.

The difficulties here might be the result of some standard shortcomings of statistical description and definition and so I will address those as I unpack the direct answer a bit.

‘Hypothesis’

What is a hypothesis for the purposes of a significance test? It’s what I call a ‘statistical hypothesis’ rather than a hypothesis regarding the real world. To make that distinction clear, consider the hypothesis that the sun rises in the east because of the way that the Earth rotates around its north-south axis. That hypothesis is not one that can be plugged into a t-test, for example. Now consider the hypothesis that the mean number of bubbles in a typical pint of Guinness stout brewed with yeast A is equal to the number of bubbles in a typical pint Guinness brewed with yeast B. That hypothesis can be treated a statistical hypothesis that can be evaluated using a t-test because counts of bubbles from multiple pints can be converted into an observed value of the test statistic, t.

A statistical hypothesis is nothing more than a point or a region within the parameter space of the statistical model chosen for the analysis. For the t-test, the null hypothesis is present in the calculation of the t-value (although it is frequently omitted from textbook formulas, with dire consequences!). Let’s use $\bar{x}_A$ to be the mean of the first group of bubble counts and $\bar{x}_B$ to be the mean of the second, $SED$ to be the standard error of the difference between those means, and $\delta_0$ to be the null hypothesis. The calculation of the observed t-value is then $$t=\frac{(\bar{x}_A-\bar{x}_A)-\delta_0}{SED}$$ Yes, in this case $\delta_0$ is zero and so it can be left out of the formula without changing the numerical result, but it should never be omitted for two reasons: explicitness helps to reduce confusion, and the null hypothesis is not always zero (the dreaded ‘nill-null’) and so it cannot always be omitted!

The p-value is determined by finding the extremity of the observed t-value compared to the distribution of Student’s t, and so that equation demonstrates immediately that the p-value depends on the null hypothesis. The absence of the alternative hypothesis similarly demonstrates the irrelevance of the alternative to the p-value. (Yes, it’s relevance is still to come, read on.)

In the previous paragraph I rely on the undefined ‘extremity’ to do a lot of work and so I need to unpack it a bit. I will say initially that it is the statistical model that defines what is meant by extreme, and it does that by providing a theoretical sampling distribution of the test statistic against which the observed value of test statistic can be calibrated. If the observed test statistic value falls near the centre of that distribution then it is not extreme, but if it falls towards one or other edge of the distribution then it is extreme in some proportion to the nearness to the edge. (I’m deliberately ignoring the complications of multimodal distributions, and of two-tailed p-value because the evidential interpretation of a neo-Fisherian p-value is assisted by the routine use of one-tailed p-values.)

One way to express the extremeness of an observed test statistic is as the integral of the sampling distribution from the observed value out to the end. That integral is a probability and hence the usual definition of a p-value as a probability. However, because some of the most damaging misconceptions about p-values relate to (or depend on) its probabilistic nature, it can be helpful to think of a p-value as a fractional ranking of extremeness rather than a probability of observing something more extreme. I’ll use a simple permutations test to illustrate that idea.

To perform a permutations test you delineate all possible arrangements of the data (with the statistical model being no more than the assumption of data exchangeability under the null hypothesis), and order those arrangements according to their corresponding test statistic values. (The test statistic is commonly the means, but can be the medians or any other interesting statistic calculated from each data arrangement.) Those ordered values of test statistic define the test statistic sampling distribution under the null hypothesis according to the model.

To get the p-value you simply determine the numerical rank of the test statistic value for the observed data arrangement within that distribution, and divide that rank by the total number of possible arrangements to get the p-value. In other words, the p-value is a fractional ranking and encodes how strange it would be to obtain the observed data arrangement according to the model if the null hypothesis is true.

Alternative hypothesis

There are two different alternative hypotheses that need to be considered. The first is the specific effect size that serves as the alternative hypothesis during the planning stage of a hypothesis test (e.g. the effect size to be plugged into the calculation of statistical power that can be used for sample size determination). That pre-data alternative has no effect on the data actually observed and hence no effect on the observed p-value.

The other alternative hypothesis is the one that the original question refers to. It is the complement of the null hypothesis and, given that a p-value depends on the null hypothesis, one might reasonably consider the p-value also depends on the alternative. However, I prefer to think that it is the null hypothesis that is doing the work, partly because the null appears (should appear!) in the formulation of a test statistic, but also because the alternative hypothesis is usually a range (or ranges) of parameter values in the statistical model whereas the null is usually a single point.