Shrinkage improves estimation of microbial associations under different normalization methods

[cb2527]

Hi there,

Going through the NetCoMi documentation I've found that the main recommendation made by Badri et al. (a key paper cited) - that shrinkage improves association estimations - doesn't seem to be integrated into NetCoMi. It recommends applying corpcor as an association measure after clr as a result.

I'm asking as I'm working with a dataset where n<<p and I have the problem noted in this paper of a high number of near 1 (likely random) associations have been found if I use clr and Pearson. I think the issue is predominantly in the OTUs present in my first sample (inoculum) and quickly outcompeted by the time the bioreactor is operational (samples 2-4) (see heat map, ordered by most abundant in the final (steady-state suspended solids) sample.

Is there a suitable association measure to reduce the number of near 1 correlations for n<<p situations? I am conscious that with only 4 samples the reliability of my findings is quite low, but this would really be helpful as a preliminary analysis.

[Ayala-Ruan-CesarM]

Hello There!

Although I don't have a specific answer to your particular problem I am kinda in thee same situation as yourself. With a low "n", I found that simulate replicates could help. Here I left you with a script that could help you simulate replicates (https://github.com/Ayala-Ruan-CesarM/Useful_Code/blob/main/replicates_simulation.py) under three different distributions (normal, poisson, negative binomial) and variation factor. You could try artificially increasing your original "n" in your inoculum and re-run the association. A good point of reference would be to ensure there's not too much dissimilarity between your original sample and the simulated replicates. To assess such changes, the script has built-in graphs.
Hope it helps