tl;dr: What is the difference between a regression model that allows its dummy variables to be correlated and one that doesn't do so? For example, how does that affect Standard Error of the regression coefficients (coefs)?
In my reproducible SEM (structural equation model) modelA below, imagine I remove the Group_HI ~~ 0*Group_MT (denoting correlation bet. two dummies is 0) altogether leading to modelB. What is the difference between a model that uses Group_HI ~~ 0*Group_MT (modelA) and one that doesn't include anything to describe the relation between Group_HI and Group_MT (modelB)?
Specifically, it seems modelB produces Standard Errors for coefs that are closer to those given by a MANOVA:
summary(lm(cbind(MP,SE)~Group,data=d))
Response MP :
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.6667 0.6150 7.588 8.19e-10 ***
GroupHI 3.1833 0.8136 3.913 0.000281 *** ## This SE
GroupMT 6.7451 0.8438 7.994 1.95e-10 *** ## This SE
Response SE :
Estimate Std. Error t value Pr(>|t|)
(Intercept) 4.2000 0.3378 12.432 < 2e-16 ***
GroupHI 2.1500 0.4469 4.811 1.48e-05 *** ## This SE
GroupMT 3.7412 0.4635 8.071 1.49e-10 *** ## This SE
But modelA produces Standard Errors for coefs that are closer to those given by:
stacked_DVs <- pivot_longer(d, c(MP,SE), names_to = "DV")
coef(summary(lm(value~DV*Group-1,data = stacked_DVs)))
Estimate Std. Error t value Pr(>|t|)
DVMP 4.666667 0.4961705 9.405370 2.364383e-15
DVSE 4.200000 0.4961705 8.464833 2.559422e-13
GroupHI 3.183333 0.6563718 4.849893 4.648941e-06 ## This SE
GroupMT 6.745098 0.6807403 9.908475 1.912183e-16 ## This SE
Reproducible data and R code:
library(tidyverse); library(fastDummies) ; library(lavaan)
d <- read_csv("https://raw.githubusercontent.com/rnorouzian/v/main/memory.csv")
d <- dummy_cols(d, "Group")[-5]
modelA <- "MP ~ Group_HI
SE ~ Group_HI
MP ~ Group_MT
SE ~ Group_MT
Group_HI ~~ 0*Group_MT #Question: what is the effect of removing this line on SE of coefs? (See `modelB`)
MP ~~ MP
SE ~~ SE
MP ~~ SE"
summary(sem(modelA, data = d, meanstructure = TRUE))
Regressions:
Estimate Std.Err z-value P(>|z|)
MP ~
Group_HI 3.183 0.659 4.830 0.000 ## This SE
SE ~
Group_HI 2.150 0.362 5.938 0.000
MP ~
Group_MT 6.745 0.684 9.868 0.000 ## This SE
SE ~
Group_MT 3.741 0.375 9.963 0.000
modelB <- "MP ~ Group_HI
SE ~ Group_HI
MP ~ Group_MT
SE ~ Group_MT
### Group_HI ~~ 0*Group_MT #This line removed.
MP ~~ MP
SE ~~ SE
MP ~~ SE"
summary(sem(modelB, data = d, meanstructure = TRUE))
Regressions:
Estimate Std.Err z-value P(>|z|)
MP ~
Group_HI 3.183 0.790 4.031 0.000 ## This SE
SE ~
Group_HI 2.150 0.434 4.956 0.000 ## This SE
MP ~
Group_MT 6.745 0.819 8.235 0.000 ## This SE
SE ~
Group_MT 3.741 0.450 8.315 0.000 ## This SE