Does hypothesis testing help make a decision in case of an A/B test?

I developed a text-generation pipeline based on recent advancements in Large-Language Models (LLMs). Users can type a topic, and my complex pipeline generates an article. I measure user satisfaction by asking how satisfied they are on a 5-point ordinal scale under each article (C-SAT). I have implemented a pipeline variation that uses cheaper, dumber LLMs in some places. I performed an A/B test to determine the difference between the current version of the pipeline and the cheaper one. Let's say the average C-SAT is 3.9 vs. 3.8, so the cheaper version has the C-SAT score lower by 0.1. Now, I have to decide whether to introduce the new version of the pipeline to reduce costs and take the risk of reducing the average C-SAT. I want to know if the decrease in the C-SAT is significant enough to give up on cutting costs. Q1: Does hypothesis testing make sense in this case? Q2: If so, then what could be a population? The number of future articles is now known. Moreover, one of the versions will not be continued. Does it mean I can’t apply the test? A result of a test would be the evidence against the null hypothesis. Let’s say my null hypothesis is “population distributions of both A and B samples have equal mean." From the perspective of the original problem ("if the decrease in the C-SAT is significant enough to give up on cutting costs"), such H0 is an intermediate problem. Q3: How do I know that finding an answer to such an intermediate problem helps me find an answer to my ori…