Skip to main content ↑ ↓
Topic: Multiple class x features p-value corrections (Read 5113 times) previous topic - next topic

Multiple class x features p-value corrections

Hi All,
This is really a stats question but reflects a situation I commonly come up against in interpreting xcms results, and I'm sure many forum members do too.

Scenario: I generate an xcmsSet with 100 samples, with 5 replicates over 20 classes generating 200 features.  One class is a control class, and the other 19 are different treatments.  I want to determine which features in each of the treatment classes are significantly different compared to the same features in the control class. What I ultimately want to do is produce a separate chart or table for each class where only features significantly different to the control class are listed. The scripting for this is straight-forward; how to apply the underlying stats is not.

A 2- class problem would be easy; I would do multiple t-tests on each feature followed by a Bonferroni or FDR adjustment on the list of pairwise p-values to determine significance for each feature.  However, for the multiple-class scenario, how should p-value adjustment be carried out?

What I've been doing to date is an ANOVA on each feature, and if the ANOVA p value is < 0.05 performing a post-hoc test using the TukeyHSD procedure to produce class-pairwise adjusted p values to detemine which treatment classes are significantly differerent from the control class.  In this case, there is no p-value adjustment for the ANOVA even though I am conducting multiple ANOVAs. I'm worried that although the Tukey's test corrects for family-wise error between classes, I am making no allowance for error-rate correction between features. 

In my approach, should I adjust the ANOVA p values with a multiple-testing correction for features (e.g. Bonferroni, FDR, etc) first, and then only if the adjusted ANOVA p value is < 0.05 go onto a post-hoc Tukey test? Or is there a better rcommended approach?



thanks
Tony