Skip to content

Latest commit

 

History

History
212 lines (174 loc) · 8.23 KB

software.md

File metadata and controls

212 lines (174 loc) · 8.23 KB

Software

The easiest way to use software on the VSC, is to use the pre-installed software, that is installed as an Easy Build module. Alternatively, you could create your own conda/mamba environment and use it in a python script or a jupyter notebook. More advanced ways to user/run software are: - using singularity containers

  • running nextflow workflow

CPU vs GPU

Not all software are relying on GPU.

  • CPU software: Fiji, QuPath
  • GPU software: Cellpose, Omnipose, Napari and its plugins, Ilastik, ...

Installed software

How to use a Easy Build module in command line (add a ! to use this command in a jupyter notebook)

  • See loaded modules
ml
module list
  • See available modules
module avail
  • Search for a module X
module av |& grep -i X
  • Search for a module X and get details
module spider X
  • Load a module X
module load x
  • Unload a module X
module unload x
  • Swap a module Y to X
module swap Y  X
  • Remove all loaded modules except the cluster one
module purge

⚠️ Do not load modules built with intel and foss toolchains

🗎 https://docs.vscentrum.be/software/software_stack.html#using-the-module-system

How to load module in jupyter notebook

Select the appropriate version of the jupyter notebook and load the easy build module accordingly

image

i.e. 6.4.0 GCC core 11.3.0 IPython 8.5.0 is compatible with foss-2022a or intel 2022a modules (see compatibility table below)

FOSS GCC CUDA OpenMPI OpenBLAS FFTW ScaLAPACK
2022b 12.2.0 12.0.0 4.1.4 0.3.21 3.3.10 2.2.0-fb
2022a 11.3.0 11.7.0 4.1.4 0.3.20 3.3.10 2.2.0-fb
2021b 11.2.0 - 4.1.1 0.3.18 3.3.10 2.1.0-fb
2021a 10.3.0 - 4.1.1 0.3.15 3.3.9 2.1.0-fb
2020b 10.2.0 - 4.0.5 0.3.12 3.3.8 2.1.0
2020a 9.3.0 - 4.0.3 0.3.9 3.3.8 2.1.9
2019b 8.3.0 - 3.1.4 0.3.7 3.3.8 2.0.2

For more information about modules: https://hprc.tamu.edu/kb/Software/GNU-Compiler-Collection/

Compatibility python version - GCCcore on Tier-1 (Hortense)

Table for jupyter notebook

IPython version Python GCCcore
7.25.0-GCCcore-10.3.0 Python 3.9.5 GCCcore-10.3.0
6.4.0-GCCcore-11.2.0 Python 3.9.6 GCCcore-11.2.0
8.5.0-GCCcore-11.3.0 Python 3.10.4 GCCcore-11.3.0
7.0.3 GCCcore 12.2.0 Python 3.10.8 GCCcore 12.2.0
7.0.2 GCCcore 12.3.0 Python 3.11.3 GCCcore 12.3.0

How to see what is loaded with the module: ml show IPython/8.5.0-GCCcore-11.3.0

Modules for bioimage analysis in python

  1. Napari: Napari/0.4.15-foss-2021b or Napari/0.4.18-foss-2022a
  2. Cellpose: Cellpose/2.2.2-foss-2022a or Cellpose/2.2.2-foss-2022a-CUDA-11.7.0
  3. Omnipose: Omnipose/0.4.4-foss-2022a-CUDA-11.7.0 or Omnipose/0.4.4-foss-2022a
  4. stardist: stardist/0.8.3-foss-2021b-CUDA-11.4.1 or stardist/0.8.3-foss-2021b
  5. AICSImageIO : AICSImageIO/4.14.0-foss-2022a
  6. devbio-napari : devbio-napari/0.10.1-foss-2022a-CUDA-11.7.0
  7. n2v : n2v/0.3.2-foss-2022a-CUDA-11.7.0
  8. Monai: MONAI/1.0.1-foss-2022a
  9. QuPath: QuPath/0.5.0-GCCcore-12.3.0-Java-17

Module for spatial omics in python

  1. Scanpy: scanpy/1.9.1-foss-2021b
  2. Seurat: Seurat/4.3.0-foss-2021b-R-4.1.2
  3. Squidpy: Squidpy/1.2.2-foss-2021b
  4. Giotto: Giotto-Suite/3.0.1-foss-2022a-R-4.2.1

Module for bio-image analysis tools

  1. Fiji: Fiji/2.9.0-Java-1.8
  2. Cellprofiler: CellProfiler/4.2.4-foss-2021a

General libraries

  1. Scikit-learn: scikit-learn/1.1.2-foss-2022a or scikit-learn/1.0.1-foss-2021b
  2. Scikit-image: scikit-image/0.19.3-foss-2022a or scikit-image/0.19.1-foss-2021b
  3. scipy: SciPy-bundle/2022.05-foss-2022a, SciPy-bundle/2021.10-foss-2021b
  4. seaborn: Seaborn/0.11.2-foss-2021b
  5. tifftile: Scikit-image/0.19.1-foss-2021b
  6. tensorflow: TensorFlow/2.7.1-foss-2021b-CUDA-11.4.1 or TensorFlow/2.7.1-foss-2021b
  7. R : R/4.2.1-foss-2022a or R/4.0.0-foss-2020a
  8. R studio: RStudio-Server/1.3.959-foss-2020a-Java-11-R-4.0.0
  9. Jupyter notebook: JupyterLab/3.1.6-GCCcore-11.2.0
  10. Matplotlib: matplotlib/3.5.2-foss-2022a or matplotlib/3.4.3-intel-2021b
  11. Bioconductor: R-bundle-Bioconductor/3.14-foss-2021b-R-4.1.2

How to create your own conda or mamba environment

Install miniconda

cd $VSC_DATA 
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -p $VSC_DATA/miniconda3

-Make conda available to the path:

export PATH="${VSC_DATA}/miniconda3/bin:${PATH}"

Install mambaforge

mamba is a reimplementation of the conda package manager in C++. It allows parallel downloading of repository data and package files using multi-threading and use libsolv for much faster dependency solving

cd $VSC_DATA 
wget "https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-$(uname)-$(uname -m).sh"
bash Mambaforge-$(uname)-$(uname -m).sh
export PATH="${VSC_DATA}/mambaforge/bin:${PATH}"

Create the conda environment

  • Create the conda environment from the yaml file
conda env create -f cellpose-omnipose-gpu.yml

or

mamba env create -f cellpose-omnipose-gpu.yml
  • Make it available for the jupyter notebook
source activate cellpose-omnipose-gpu
export PATH="${VSC_DATA}/miniconda3/bin:${PATH}
conda install ipykernel
python -m ipykernel install  --prefix=${VSC_HOME}/.local/ --name 'cellpose'

In this example, the conda envirmnemnt will be accessible under the name cellpose

Use the conda enviroment with a jupyter notebook

⚠️ The ondemands interface doesn't wok well on Firefox, it's best to use Chrome⚠️

  • After connecting to JupyterLab image
  • Create a new notebook File > New > Notebook image
  • Select the newly create conda enviroment (e.g. here, cellpose) image
  • Start to write the notebook