I have several groups of features and their ICCs (calculated across participants, same participants in each group). One feature group (A) has e.g. 20 feautures (and therefore 20 ICCs), the other feature group (B) has 2000 features (and 2000 ICCs).

Further, ICCs show some dependencies within group (some features are similar, therefore ICCs are similar). Dependencies within group are idiosyncratic for each group. There are also more than 2 feature groups, but for this example it should be enough).

I want to come to a simple conclusion, that feature group A is more reliable than group B, or otherwise, or no difference.

Obviously, group B has more high-ICC features, but also more low-ICC features, because it comprises a huge amount of features in general.

So I could draw different kinds of conclusions:

Are there some mathematical / statistical models to compare such cases in an elegant way? Or are there other interesting kinds of comparisons between groups go be done (besides central tendency, top-N comparisons, etc).

One possibility (if that helps), is to do feature selection based on exploration data (ICCs calculated on 80% of participant), and validation data (remaining participants).