As others have have already mentioned: if you are in a continuous setting and you want a power-maximising test for some simple null against some simple alternative then the Neyman-Pearson lemma tells us that no randomization is needed. I suspect that this also holds in the composite setting, with a mixture/prior over the alternative.

But I think these answers miss why this is the case. And the answer has to do with the linearity of the power-objective; in particular its non-concavity.

Neyman-Pearson: a linear optimization problem

As is standard, let us model a randomized test as a map $\phi : \Omega \to [0, 1]$, from some sample space $\Omega$ to $[0, 1]$. Here, its value in $[0, 1]$ indicates the probability with which we will subsequently reject. For a null distribution $\mathbb{P}$ and alternative distribution $\mathbb{Q}$, we can formulate the Neyman-Pearson lemma as finding the $\phi$ such that it maximizes:

$$ \max_{\phi : \Omega \to [0, 1]}\ \mathbb{E}^{\mathbb{Q}}[\phi], \\ \text{s.t.}\ \mathbb{E}^{\mathbb{0}}[\phi] \leq \alpha. $$

Here, the expected value is equivalent to the probability that $\phi$ leads to a rejection, including randomization.

Now, note that this is actually a linear optimization problem with one constraint! So by complementary slackness we expect that there is typically only one $\omega \in \Omega$ for which $\phi(\omega) \in (0, 1)$. This also shows why randomization doesn't pop-up in the continuous setting, because there it doesn't matter what happens on a single element $\omega$ in our sample space; the probability we will hit this element is zero anyway.

Explanation: Neyman-Pearson as a knapsack problem

Referring to complementary slackness may not be satisfying for those unfamiliar with linear programming. So let me work you through this with an intuitive example, which simultaneously also shows why thresholding the likelihood ratio is what the Neyman-Pearson lemma tells us to do.

Suppose that our sample space $\Omega$ contains $N$ elements, and for simplicity let us just label these discrete points as $\Omega = \{1, 2, 3, ..., N\}$. Then we can encode the distributions $\mathbb{P}$ and $\mathbb{Q}$ as $N$-vectors that sum to 1: $p = (p_1, \dots, p_N$, $q = (q_1, \dots, q_N)$. Moreover, $\phi$ can be viewed as an $N$-vector $\phi = (\phi_1, \dots, \phi_N)$ with elements $\phi_i \in [0, 1]$, $i = 1, \dots, N$. The optimization problem can then be formulated as:

$$ \max_{\phi}\ q'\phi, \\ \text{s.t.}\ p'\phi \leq \alpha, $$ where I use $q'$ and $p'$ to denote the transpose.

We can then view this as a kind of continuous knapsack problem, where we have $N$ items that we want to bring with us (maybe liquids or so, in the continuous setting). Then $\phi_i$ captures how much of product $i$ we take with us (0: nothing, 1: all of it), $q_i$ captures how much we can sell it for at the destination, $p_i$ captures how much space it takes up, and $\alpha$ is our total capacity.

If you think about this problem for a moment, you'll quickly realize that you want to bring the items that have the highest value-to-space-ratio $q_i / p_i$ (the highest likelihood ratio!). So we can sort all the items by these ratios. For simplicity, say $q_1/p_1 > q_2/p_2 > \dots > q_N / p_N$. And then we simply bring the first $M$, say, items for which we have capacity: $p_1 + \dots + p_M \leq \alpha$. In terms of testing, this means we reject on the sample points on which the likelihood ratios are highest.

For the $M+1$th item we may run into an issue: if we bring it, then we shoot over our capacity, but if we do not bring it then there is some capacity left. So what we do, we bring only part of this item: $\phi_{M+1} \in (0, 1)$. This is exactly the role of randomization in the Neyman-Pearson lemma. If we were to reject when we observe sample point $M+1$, then we would shoot over our Type 1 error budget. If, instead, we do not reject at all at sample point $M+1$, then we have some Type 1 error budget left!

If we were to remove randomization, then the Neyman-Pearson lemma actually breaks down (as mentioned in the comment by @Stan). In our knapsack problem, this no-randomization-restriction means we must either fully take an item with us, or not at all. This complicates the solution: we may have to remove one 'large' item (high $p_i$) to be able to fit in two smaller items. This means that we may perhaps not reject when $M$ happens, but do reject when $M+1$ or $M+2$ happens.

Conclusion

So to conclude, randomization appears in the Neyman-Pearson lemma when there is a sample point for which rejecting at this sample point means that we shoot over our Type I error budget, but not-rejecting at this sample point means that we have some slack in our Type I error budget.

Randomized testing without randomization?

In a recent paper, I study the following simple idea: what if we do not want a binary reject/not-reject decision, but simply report the value of the test $\phi$ in $[0, 1]$ as our 'evidence'. So a kind of randomized testing, but without randomization.

This has some fascinating consequences, if I may say so myself:

1. This is equivalent to using an e-value as a measure of evidence. It shows that e-values are just continuously-interpreted randomized tests on a different scale: $e = \phi / \alpha$.
2. We can use the value of a test $\phi$ on $[0, 1]$ as a subsequent significance level. That is, suppose that some experiment reports a level $\alpha$ test with outcome $\phi_1 = 0.6$, then we may decide to initialise another experiment with a test $\phi_2$ at significance level $0.6$, and the overall (unconditional) Type I error of $\phi_2$ remains $\alpha$. I find this truly mind-blowing. You can even do this within an experiment, reporting a value $\phi_i$ on each observation, and then doing a new test $\phi_{i+1}$ at significance level $\phi_i$ for the next observation. You can view this as a generalization of randomization, where randomization would be doing a subsequent test at level $\phi_i$ but with uninformative data (external randomization), which leads us to simply reject with probability $\phi_i$.
3. To truly use this, we must consider other power targets, as maximizing $\mathbb{E}^{\mathbb{Q}}[\phi]$ will inevitably lead to tests that don't really take value in $(0, 1)$, as discussed in my answer above. But this is not a problem, as we are no longer necessarily interested in maximizing the probability that our test hits 1. Instead, we may gamble for a lot of evidence or play it safe for a moderate amount of evidence. Whatever is relevant to the experiment at hand. As soon as we move away from the traditional power target to concave power targets, $\mathbb{E}^{\mathbb{Q}}[U(\phi)]$, for some strictly concave utility $U$, we have that $\phi > 0$ ($\mathbb{Q}$-almost surely).