I am quite new in this method and I need to calculate a priori power analysis for my research.

Since there is no previous study to build models on, I collected pilot data to test the experiment and calculate the power. I have several mixed effects models to test and to register on OSF, I need to report all of them and choose the highest one.

The problem is, for some models, I get normal power for 60~70 participants. But for few models, no matter what I tried, power remains really low, and increasing number of participant to even 1000 doesn't fix it. Sometimes power even decreases at some point.

example:

Model 1 — Logistic Regression: regulation ~ metacomp_rate * mw_prop + (1 | participant_n) Power at N=60: 3-8% Interaction coefficient: -1.04 (SE = 3.73, z = -0.28, p = 0.78)

Model 2 — Logistic Regression: regulation ~ metacomp_rate * frequency_prob + (1 | participant_n) Power at N=60: 75% Interaction coefficient: 1.20 (SE = 1.08, z = 1.11, p = 0.26)

Model 3 — Linear Mixed Model: metacomp_rate ~ comprehension * mw_prop + (1 | participant_n) Power at N=60: 75% Interaction coefficient: -0.426 (SE = 0.286)

VARIABLE DESCRIPTIONS:

metacomp_rate: discrete levels {0, 0.25, 0.50, 0.75, 1.0}
mw_prop: discrete levels, {0, 0.50, 1}
frequency_prob: {0, 1, 2} (number of thought probes in text)
regulation: binary {0, 1}
comprehension: discrete levels {0, 0.25, 0.50, 0.75, 1.0}

I understand that power is low due to low effect and high SE in model 1. I increased the effect as well, but power got even lower.

I am not sure how to move forward in this context. What are the others steps I could take?

Another question, even if we use coefficients from pilot data, is it still not the "observed power" issue?

Further details:

Our study is fully within-subject. Participants (participant_n) will read 9 expository texts. During reading, thought probes (e.g. "are you mind wandering?"). Frequency of these thought probes are 0/1/2 per text, randomly assigned. Total 9 thought probes for all participants. All participants will predict their performance (metacomprehension- metacomp_rate, answer 4 comprehension questions (correct answers' proportion) and regulation ("would you like to reread?’).