Skip to content

Commit 2d7173e

Browse files
MitchobMitchob
committed
updating flavor delection page
1 parent b7fa595 commit 2d7173e

2 files changed

Lines changed: 123 additions & 88 deletions

File tree

‎_data/sidebars/main.yml‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,17 +40,23 @@ subitems:
4040
- title: Using BioShell
4141
url: /using-bioshell
4242
subitems:
43-
- title: Choosing a flavour
43+
- title: Choosing an environment size
4444
url: /flavours
4545
subitems:
46-
- title: Flavour types
47-
url: /flavours#flavour-types
48-
- title: Quick sizing guide
49-
url: /flavours#sizing-guide
50-
- title: Worked examples
51-
url: /flavours#worked-examples
52-
- title: How to choose
53-
url: /flavours#how-to-choose
46+
- title: A familiar starting point
47+
url: /flavours#familiar-starting-point
48+
- title: Small — light preprocessing
49+
url: /flavours#small
50+
- title: Medium — scRNA-seq and notebooks
51+
url: /flavours#medium
52+
- title: Large — variant calling and HPC-scale
53+
url: /flavours#large
54+
- title: Not sure? Start small.
55+
url: /flavours#start-small
56+
- title: Quick reference
57+
url: /flavours#quick-reference
58+
- title: Further reading
59+
url: /flavours#further-reading
5460
- title: Tools and reference data
5561
url: /tools
5662
subitems:

‎pages/flavours.md‎

Lines changed: 108 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -1,110 +1,139 @@
11
---
2-
title: Choosing a flavour
3-
description: How to select the right combination of vCPUs and memory for your BioShell environment.
2+
title: Choosing the right environment size
3+
description: How to choose the right number of CPUs, memory, and storage for your BioShell environment.
44
---
55

6-
A flavour is the combination of virtual CPUs (vCPUs) and memory (RAM) allocated to your
7-
BioShell environment. BioShell is a shared national resource, so you are encouraged to
8-
request a flavour that closely matches your actual needs — this keeps capacity available for
9-
everyone.
6+
> **What is a flavour?**
7+
> A flavour is the combination of virtual CPUs and memory allocated to your BioShell
8+
> environment, essentially the "spec" of your cloud computer. Different flavours suit
9+
> different workloads, just as you might choose a lightweight laptop for email but a
10+
> workstation for video editing. You pick a flavour when you request access and can request
11+
> a change later if your needs grow.
1012
11-
Estimating your requirements can be hard at the start of a project. Use the guidance below
12-
to make a reasonable starting choice. You can always request a larger flavour later if your
13-
analysis grows.
13+
Cloud systems are shared research resources. As a general principle, you are encouraged to
14+
request resources that closely match your actual needs. This supports fair access for all
15+
users and preserves capacity for everyone.
16+
17+
Estimating requirements can be challenging, particularly at the start of a project you may
18+
not yet know which software tools you will use or how demanding they will be. The guidance
19+
below is designed to help you make a reasonable first choice and adjust from there.
20+
21+
---
22+
23+
## A familiar starting point {#familiar-starting-point}
24+
25+
A good way to think about environment sizing is to start from what you already know: your
26+
laptop.
27+
28+
A standard modern laptop typically has 4–8 CPU cores and 8–16 GB of memory. A BioShell
29+
environment of equivalent size will run anything your laptop can handle, and often faster,
30+
because the environment does not share resources with a browser, email client, or other
31+
background applications.
32+
33+
If you are new to BioShell, or unsure of your requirements, starting with a
34+
**laptop-equivalent size** (4 CPUs / 8–16 GB RAM) is a reasonable default. You can always
35+
request a larger environment if you find you need it.
1436

1537
---
1638

17-
## Flavour types at a glance {#flavour-types}
18-
19-
| Type | Characteristics | Best for |
20-
|------|----------------|----------|
21-
| **t3** | Low memory relative to CPUs | Testing, small jobs, getting started |
22-
| **m3** | Balanced CPU and memory | Most general-purpose workloads |
23-
| **c3** | CPU-optimised (same memory ratio as m3) | Compute-heavy tasks such as alignment and assembly |
24-
| **r3** | High memory per CPU | Memory-intensive analysis such as variant calling or large data processing |
25-
26-
### Available flavours
27-
28-
| vCPUs | RAM (GB) | Nectar flavours | Nirin flavours |
29-
|-------|----------|----------------|----------------|
30-
| 1 | 1 | t3.xsmall | c3.1c1m5d |
31-
| 1 | 2 | m3.xsmall, c3.xsmall | c3.1c2m10d |
32-
| 1 | 4 | r3.xsmall | — |
33-
| 2 | 4 | m3.small, c3.small | c3.2c4m20d, c3.2c4m10d |
34-
| 2 | 8 | r3.small | — |
35-
| 4 | 8 | m3.medium, c3.medium | c3.4c8m20d, c3.4c8m10d |
36-
| 4 | 16 | r3.medium | — |
37-
| 8 | 16 | m3.large, c3.large | c3.8c16m20d, c3.8c16m10d |
38-
| 8 | 32 | r3.large | — |
39-
| 16 | 32 | m3.xlarge, c3.xlarge | — |
40-
| 16 | 64 | r3.xlarge | — |
41-
| 32 | 64 | m3.xxlarge, c3.xxlarge | — |
42-
| 32 | 128 | r3.xxlarge | — |
43-
| 64 | 128 | c3.3xlarge | — |
39+
## Suggested sizes by workload {#workload-sizes}
40+
41+
### Small: light preprocessing and exploration {#small}
42+
43+
**Who this suits:** users running quality control, adapter trimming, short read filtering, or
44+
exploring tools and datasets interactively in JupyterLab or RStudio with small to moderate
45+
datasets.
46+
47+
| | |
48+
|---|---|
49+
| **CPUs** | 2–4 |
50+
| **Memory** | 4–8 GB |
51+
| **Storage** | Up to 100 GB |
52+
53+
**Example:** John is starting a research project analysing drought-resistant genes from 20
54+
crop samples (~140 GB raw data). His pipeline covers quality control, trimming, alignment,
55+
annotation, and phylogenetic tree construction. Because he is working on a subset of genes
56+
rather than whole genomes, each individual job is small. A standard laptop-equivalent
57+
environment handles this comfortably.
4458

4559
---
4660

47-
## Quick sizing guide {#sizing-guide}
61+
### Medium: single-cell RNA-seq analysis {#medium}
62+
63+
**Who this suits:** users running interactive multi-step R or Python analysis workflows,
64+
particularly those involving large in-memory data objects.
4865

49-
| Task type | Suggested resources |
50-
|-----------|-------------------|
51-
| Light preprocessing (QC, trimming, filtering) | 1–4 vCPUs, 2–8 GB RAM |
52-
| Alignment and assembly (e.g. `bwa`, `STAR`, `SPAdes`) | 8–16 vCPUs, 16–32 GB RAM |
53-
| Memory-intensive analysis (variant calling, genome-wide statistics) | 16–32+ vCPUs, 32–128 GB RAM |
54-
| Interactive analysis and visualisation (JupyterLab, RStudio) | 2–4 vCPUs, 8–16 GB RAM |
66+
| | |
67+
|---|---|
68+
| **CPUs** | 4–8 |
69+
| **Memory** | 32–64 GB |
70+
| **Storage** | Variable, depends on sample count |
71+
72+
**Example:** A researcher running the
73+
[SIH scRNAvigator notebooks](https://github.com/Sydney-Informatics-Hub/scrna-analysis) in
74+
RStudio. The workflow covers quality control, doublet detection, dataset integration, cell
75+
annotation, differential gene expression, and pathway enrichment analysis. Integration and
76+
doublet detection steps load large data objects into memory simultaneously, making this
77+
workflow **memory-bound rather than CPU-bound**, RAM matters more than core count.
78+
79+
The scRNAvigator documentation recommends at least 32 GB of memory for local use. A 32 GB
80+
environment is suitable for small to moderate cohorts; larger datasets may require 64 GB or
81+
more. If you are unsure, start at 32 GB and scale up if jobs fail or run very slowly.
5582

5683
---
5784

58-
## Worked examples {#worked-examples}
85+
### Large: whole exome variant calling and high-throughput workflows {#large}
5986

60-
### Light processing
87+
**Who this suits:** users running GATK best-practice workflows, genome-wide analyses, large
88+
alignments, or multiple samples in parallel.
6189

62-
John is starting a project analysing drought-resistant genes from 20 crop samples (~7 GB
63-
raw sequencing data per sample). His pipeline includes quality control with `FastQC`,
64-
adapter trimming, alignment to a reference genome, annotation, and phylogenetic tree
65-
construction.
90+
| | |
91+
|---|---|
92+
| **CPUs** | 8–16 |
93+
| **Memory** | 32–64 GB |
94+
| **Storage** | Up to 1 TB |
6695

67-
The most resource-intensive steps require moderate CPU and memory. Because John is analysing
68-
a subset of genes rather than whole genomes, each run is relatively small:
96+
**Example:** Georgie is running `GATK4` variant calling across 15 human exomes. Each sample
97+
produces approximately 15 GB of BAM files, plus similarly sized temporary intermediates.
98+
Total storage for the project is approximately 870 GB, round up to 1 TB to allow room for
99+
workflow outputs and reruns. `GATK4` benefits from both high memory and multiple CPU cores,
100+
so a balanced large environment is appropriate here.
69101

70-
- 2–4 vCPUs
71-
- Up to 10 GB RAM
102+
For larger cohorts (~30 exomes or more), or when running multiple jobs in parallel, consider
103+
requesting a proportionally larger environment or splitting the workload across multiple
104+
instances.
72105

73-
A balanced `m3` flavour is a good starting point:
106+
---
74107

75-
- **Nectar:** `m3.medium` (4 vCPUs / 8 GB RAM)
76-
- **Nirin:** `c3.4c8m10d` or `c3.4c8m20d` (4 vCPUs / 8 GB RAM)
108+
## Not sure where to start? Start small. {#start-small}
77109

78-
### Memory-intensive processing
110+
If you are uncertain, the best approach is to begin with a smaller environment and scale up
111+
based on what you observe:
79112

80-
Georgie is running `GATK4` best-practice variant calling on 15 human exome samples. Each
81-
sample produces BAM files of ~15 GB, with similar-sized temporary files during processing.
82-
Total storage is approximately 1 TB. `GATK4` tools benefit from both high memory and
83-
multiple CPU cores:
113+
- If jobs run slowly and CPU usage is consistently near 100%, you likely need more CPUs.
114+
- If jobs fail with memory errors, or RAM usage is consistently near the limit, increase
115+
memory.
116+
- If both are highly utilised, move to a larger balanced configuration.
84117

85-
- 8 vCPUs
86-
- 32 GB RAM
118+
Scaling incrementally avoids over-allocating shared resources and makes it easier to
119+
identify bottlenecks.
87120

88-
Recommended flavours for ~15 exomes:
121+
---
89122

90-
- **Nectar:** `r3.large` (8 vCPUs / 32 GB RAM)
91-
- **Nirin:** `c3.8c16m20d` or `c3.8c16m10d` (8 vCPUs / 16 GB RAM)
123+
## Quick reference {#quick-reference}
92124

93-
If processing ~30 exomes or running multiple jobs in parallel, consider `r3.xlarge`
94-
(16 vCPUs / 64 GB RAM) on Nectar, or multiple 8 vCPU Nirin instances.
125+
| Workload | CPUs | Memory | Storage |
126+
|----------|------|--------|---------|
127+
| Light — QC, trimming, small datasets | 2–4 | 4–8 GB | Up to 100 GB |
128+
| Moderate — alignment, assembly, interactive notebooks | 4–8 | 8–32 GB | 100–500 GB |
129+
| Heavy — variant calling, large-scale analysis, multi-sample workflows | 8–16 | 32–64 GB | 500 GB – 1 TB |
130+
| Very large — genome-wide, high memory workflows | 16–32 | 64–128 GB | 1 TB+ |
95131

96132
---
97133

98-
## How to choose if you are unsure {#how-to-choose}
134+
## Further reading {#further-reading}
99135

100-
1. **Check software documentation** — most bioinformatics tools publish minimum and
101-
recommended system requirements.
102-
2. **Start small and scale up** — begin with a smaller flavour. If jobs run slowly with
103-
CPU usage consistently near 100%, you need more vCPUs. If jobs fail with memory errors,
104-
you need more RAM.
105-
3. **Review previous runs** — if you have run similar analyses before, check your peak CPU
106-
and RAM usage from those logs to guide your estimate.
136+
For the full list of available environment sizes on each platform, see:
107137

108-
> **Note:** For interactive work in JupyterLab or RStudio, 2–4 vCPUs and 8–16 GB RAM is
109-
> sufficient for most datasets. Increase memory if you are loading large files directly
110-
> into your session.
138+
- [Nectar flavours](https://support.ehelp.edu.au/support/solutions/articles/6000205341-nectar-flavors)
139+
- [Nirin flavours and charge rates](https://opus.nci.org.au/spaces/Help/pages/152207479/Nirin+Flavors+and+Charge+Rates)

0 commit comments

Comments
 (0)