Cross Validated
2026-11-07 23:01 UTC
By Significant page
AI-113-20261107-social-media-fa899685
How should I model two short autocorrelated time series when pooling observation periods destroys their Pearson correlation?
I am an undergraduate mathematics student and have only recently started studying statistical time-series analysis. I encountered the following problem while comparing two observational time series related to solar emission, and I am trying to understand how to analyse their relationship without drawing misleading conclusions from Pearson's correlation coefficient. I have two quantities, $X$ and $Y$ , which are expected to be related. The original measurements of $X$ are irregularly distributed in time, with multiple observations on some days and none on others. I calculated the median of the available measurements for each observed day, $$ X_d=\operatorname{median}\{p_i:t_i\in d\}, $$ and matched the resulting values with an independently obtained daily reference index $N_d$ . According to my processing notes, the reference variable used for the correlation was $Y_d=\ln N_d$ . Only dates for which both quantities were available were included. Missing days were not interpolated. I then calculated the ordinary sample Pearson correlation coefficient within separate observation periods and again after combining them. The results were unexpected. In the first period, with 11 paired daily observations, I obtained $r\approx0.91$ . In the second period, with 13 observations, I obtained $r\approx0.89$ . However, when I combined all 24 pairs and recalculated the correlation, it dropped to approximately $r=0.11$ . I examined the mean levels and found that the mean of $X$ increased by…
I am an undergraduate mathematics student and have only recently started studying statistical time-series analysis. I encountered the following problem while comparing two observational time series related to solar emission, and I am trying to understand how to analyse their relationship without drawing misleading conclusions from Pearson's correlation coefficient. I have two quantities, $X$ and $Y$ , which are expected to be related. The original measurements of $X$ are irregularly distributed in time, with multiple observations on some days and none on others. I calculated the median of the available measurements for each observed day, $$ X_d=\operatorname{median}\{p_i:t_i\in d\}, $$ and matched the resulting values with an independently obtained daily reference index $N_d$ . According to my processing notes, the reference variable used for the correlation was $Y_d=\ln N_d$ . Only dates for which both quantities were available were included. Missing days were not interpolated. I then calculated the ordinary sample Pearson correlation coefficient within separate observation periods and again after combining them. The results were unexpected. In the first period, with 11 paired daily observations, I obtained $r\approx0.91$ . In the second period, with 13 observations, I obtained $r\approx0.89$ . However, when I combined all 24 pairs and recalculated the correlation, it dropped to approximately $r=0.11$ . I examined the mean levels and found that the mean of $X$ increased by…
Full article content could not be extracted automatically. Read the original below.
Source:
Cross Validated
· stats.stackexchange.com