Skip to content

Commit c8b6724

Browse files
MitchobMitchob
committed
adding training instructions
1 parent a23aeef commit c8b6724

2 files changed

Lines changed: 331 additions & 17 deletions

File tree

‎_data/sidebars/main.yml‎

Lines changed: 38 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,23 @@ subitems:
2121
url: /access#pathways
2222
- title: Access duration and renewal
2323
url: /access#duration
24+
- title: Choosing an environment size
25+
url: /flavours
26+
subitems:
27+
- title: A familiar starting point
28+
url: /flavours#familiar-starting-point
29+
- title: Small — light preprocessing
30+
url: /flavours#small
31+
- title: Medium — scRNA-seq and notebooks
32+
url: /flavours#medium
33+
- title: Large — variant calling and HPC-scale
34+
url: /flavours#large
35+
- title: Not sure? Start small.
36+
url: /flavours#start-small
37+
- title: Quick reference
38+
url: /flavours#quick-reference
39+
- title: Further reading
40+
url: /flavours#further-reading
2441
- title: Connecting to BioShell
2542
url: /connecting
2643
subitems:
@@ -40,23 +57,6 @@ subitems:
4057
- title: Using BioShell
4158
url: /using-bioshell
4259
subitems:
43-
- title: Choosing an environment size
44-
url: /flavours
45-
subitems:
46-
- title: A familiar starting point
47-
url: /flavours#familiar-starting-point
48-
- title: Small — light preprocessing
49-
url: /flavours#small
50-
- title: Medium — scRNA-seq and notebooks
51-
url: /flavours#medium
52-
- title: Large — variant calling and HPC-scale
53-
url: /flavours#large
54-
- title: Not sure? Start small.
55-
url: /flavours#start-small
56-
- title: Quick reference
57-
url: /flavours#quick-reference
58-
- title: Further reading
59-
url: /flavours#further-reading
6060
- title: Tools and reference data
6161
url: /tools
6262
subitems:
@@ -109,6 +109,27 @@ subitems:
109109
url: /community#research
110110
- title: Get involved
111111
url: /community#get-involved
112+
- title: For trainers
113+
url: /training-development
114+
subitems:
115+
- title: VM setup and access
116+
url: /training-development#vm-setup
117+
- title: Disk budget
118+
url: /training-development#disk-budget
119+
- title: Using CVMFS resources
120+
url: /training-development#cvmfs
121+
- title: Developing training materials
122+
url: /training-development#developing-materials
123+
- title: Trainee directory layout
124+
url: /training-development#directory-layout
125+
- title: Non-CVMFS resources
126+
url: /training-development#non-cvmfs
127+
- title: How trainees are provisioned
128+
url: /training-development#provisioning
129+
- title: Cloud-init check
130+
url: /training-development#cloud-init
131+
- title: Pre-snapshot checklist
132+
url: /training-development#checklist
112133
- title: About
113134
url: /about
114135
subitems:

‎pages/training-development.md‎

Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
---
2+
title: Training development instructions
3+
description: How to set up and prepare a BioShell training environment for the BioCommons Training Cooperative, including VM configuration, CVMFS use, trainee directory layout, and pre-snapshot requirements.
4+
---
5+
6+
> **Draft:** These instructions are under active development. Contact Mitchell for
7+
> clarification on any steps or requirements — feedback is welcome.
8+
9+
This guide is for **training developers** (`tdevNN` users) setting up BioShell training
10+
environments on virtual machines. You will configure tools, build training materials, and
11+
prepare a template that is automatically applied to trainee accounts when VMs are provisioned.
12+
13+
---
14+
15+
## 1. VM setup and access {#vm-setup}
16+
17+
### Launch a VM instance
18+
19+
VM instances are provisioned by Giorgia or Mitchell. Machines are named with the prefix `D`
20+
(for dev) followed by a number, for example `D1`.
21+
22+
### Log in via SSH
23+
24+
A username and IP address will be provided by the training team. Your username follows the
25+
format `tdevNN` where `NN` matches your VM's prefix number.
26+
27+
```bash
28+
ssh tdevNN@<IP_Address>
29+
```
30+
31+
---
32+
33+
## 2. Disk budget {#disk-budget}
34+
35+
The base VM template is approximately **12 GB** on a **30 GB** disk, leaving roughly **16 GB**
36+
of usable space. This must cover:
37+
38+
- Training materials saved in the template directory (`/etc/skel/`)
39+
- CVMFS-cached container layers and reference data (written to the CVMFS cache on first use)
40+
- Trainee working files generated during the session
41+
42+
> **Critical:** If the disk fills up during user creation, the `useradd` step will fail and
43+
> the trainee account will not be created. Keep `/etc/skel/` as lean as possible.
44+
45+
Check disk availability with:
46+
47+
```bash
48+
df -h
49+
```
50+
51+
Example output:
52+
53+
```
54+
Filesystem Size Used Avail Use% Mounted on
55+
tmpfs 197M 1.1M 196M 1% /run
56+
/dev/vda2 30G 12G 17G 43% /
57+
tmpfs 984M 0 984M 0% /dev/shm
58+
tmpfs 5.0M 0 5.0M 0% /run/lock
59+
tmpfs 197M 16K 197M 1% /run/user/1000
60+
```
61+
62+
To recover disk space:
63+
64+
```bash
65+
sudo apt clean # Clear apt cache (~640 MB saving)
66+
sudo rm -rf /root/.cache/go-build # Clear Go build cache (~490 MB)
67+
sudo rm -rf /tmp/* /var/tmp/* # Clear temp files
68+
sudo journalctl --vacuum-size=50M # Trim system logs
69+
```
70+
71+
---
72+
73+
## 3. Using CVMFS resources {#cvmfs}
74+
75+
CVMFS provides read-only, on-demand access to reference data and Singularity containers while
76+
consuming very little cached local disk space. Always prioritise CVMFS resources over local
77+
copies — they do not count against the disk budget.
78+
79+
For documentation on using data in the CVMFS repository, see the
80+
[CVMFS reference data guide](https://docs.google.com/document/d/11YKFZjxzpjKPvyqK3SuAX7zUdgB24ziuGtgdvPcYrBs/edit?tab=t.0#heading=h.fbpu5xkpq96d).
81+
82+
### Singularity containers via shelley-bio
83+
84+
Use `shelley-bio` to find and install tools from CVMFS-hosted Singularity images:
85+
86+
```bash
87+
# Search for available tools
88+
shelley-bio find <toolname>
89+
90+
# List available versions
91+
shelley-bio versions <toolname>
92+
93+
# Build a module-loadable tool via its CVMFS container
94+
shelley-bio build <toolname>
95+
```
96+
97+
Always check whether a required tool or reference is available on CVMFS before considering
98+
alternatives. Where a CVMFS path exists, point configs and scripts directly at it — do not
99+
copy resources into the trainee directory.
100+
101+
---
102+
103+
## 4. Developing training materials {#developing-materials}
104+
105+
Set up your home directory (`/home/tdevNN`) exactly as you want trainees to experience it.
106+
Build and test your workflows there, using CVMFS containers and references throughout. The
107+
goal is a working, self-contained training environment that a trainee could follow from
108+
start to finish.
109+
110+
> **Data size:** When the Australian BioCommons dataset repository is ready it will be the
111+
> main access point for training data — like CVMFS, data stored there will not count against
112+
> VM size. Until then, keep data small. Use the minimum input data needed to demonstrate the
113+
> workflow. If your ideal dataset is too large to copy to every trainee home directory, see
114+
> [Section 6](#non-cvmfs) for how to pull it at boot time instead.
115+
116+
Once everything works end-to-end, your home directory becomes the template.
117+
118+
---
119+
120+
## 5. Trainee directory layout {#directory-layout}
121+
122+
Organise your home directory to reflect what trainees should see on login:
123+
124+
```
125+
~/
126+
├── README.md # Introduction and step-by-step instructions
127+
├── data/ # Minimal test input data if not available on CVMFS
128+
├── containers/ # Containers not available on CVMFS (see Section 6)
129+
├── references/ # References not available on CVMFS (see Section 6)
130+
├── configs/ # Workflow configuration files
131+
├── scripts/ # Example and helper scripts
132+
└── results/ # Empty output directory for trainee use
133+
```
134+
135+
Not all directories are needed for every module — include only what is relevant.
136+
137+
### Check your size before copying
138+
139+
```bash
140+
du -sh /home/<username>/
141+
```
142+
143+
Once satisfied, copy your home directory to `/etc/skel/`:
144+
145+
```bash
146+
sudo cp -r /home/<username>/. /etc/skel/
147+
```
148+
149+
Replace `<username>` with your actual username (e.g. `tdev01`).
150+
151+
Review and clean up the template directory:
152+
153+
```bash
154+
ls -a /etc/skel/
155+
```
156+
157+
Remove any unwanted files. The contents of `/etc/skel/` act as the template and will be
158+
applied automatically to every trainee's home directory when their account is created.
159+
160+
Once you are confident the template is correct, delete the contents of your own home
161+
directory to free space.
162+
163+
> **Tip:** Ask to have a snapshot taken at key checkpoints if you want a backup before
164+
> making significant changes.
165+
166+
---
167+
168+
## 6. If resources are not available on CVMFS {#non-cvmfs}
169+
170+
If a required container or reference is not available on CVMFS and is too large to include
171+
directly in the trainee directory, it must be fetched at boot time via the VM provisioning
172+
script. Prepare the required scripts and share with Giorgia or Mitchell.
173+
174+
### Adding a pull step to the provisioning script
175+
176+
Add downloads before the `useradd` block, targeting the relevant folder within `/etc/skel/`:
177+
178+
```bash
179+
# Pull a Singularity container into the trainee containers directory
180+
mkdir -p /home/$USERNAME/containers
181+
singularity pull /home/$USERNAME/containers/<tool>.sif <registry-path>
182+
183+
# Pull a reference file into the trainee references directory
184+
mkdir -p /home/$USERNAME/references
185+
wget -q -O /home/$USERNAME/references/<file> <url>
186+
```
187+
188+
Resources pulled this way will appear in the correct location within the trainee's home
189+
directory, consistent with the layout in [Section 5](#directory-layout).
190+
191+
> **Note:** Boot-time pulls add to your disk budget. Factor their size into your 30 GB total.
192+
193+
---
194+
195+
## 7. How trainees are provisioned {#provisioning}
196+
197+
When a VM is built from your snapshot (done by Giorgia or Mitchell), the provisioning script
198+
runs automatically:
199+
200+
- The trainee's username will be `userNN` (e.g. `user1`)
201+
- Their password is deterministically generated from their username, unique per VM
202+
- The contents of `/etc/skel/` at snapshot time become the trainee's home directory
203+
- After the user is created, `/etc/skel/` is cleared to reduce VM image size
204+
205+
---
206+
207+
## 8. Cloud-init datasource check (pre-snapshot requirement) {#cloud-init}
208+
209+
Running `apt upgrade` during VM setup can silently alter cloud-init configuration, causing all
210+
VMs built from the snapshot to inherit the wrong hostname. The following must be checked and
211+
corrected before every snapshot.
212+
213+
### Step 1 — Check for the offending apt file
214+
215+
```bash
216+
ls /etc/cloud/cloud.cfg.d/
217+
```
218+
219+
A working VM should contain only `05_logging.cfg` and `README`. If `90_dpkg.cfg` is present,
220+
remove it:
221+
222+
```bash
223+
sudo rm /etc/cloud/cloud.cfg.d/90_dpkg.cfg
224+
```
225+
226+
### Step 2 — Verify the datasource order
227+
228+
```bash
229+
cat /etc/cloud/cloud.cfg | grep -A5 "datasource_list"
230+
```
231+
232+
The output should show only `ConfigDrive` and `OpenStack`:
233+
234+
```yaml
235+
datasource_list:
236+
- ConfigDrive
237+
- OpenStack
238+
```
239+
240+
### Step 3 — Clean cloud-init state
241+
242+
```bash
243+
sudo cloud-init clean --logs
244+
```
245+
246+
> **Important:** `cloud-init clean` deletes `/etc/cloud/ds-identify.cfg`. Recreate it
247+
> immediately:
248+
249+
```bash
250+
sudo tee /etc/cloud/ds-identify.cfg << 'EOF'
251+
policy: enabled
252+
EOF
253+
```
254+
255+
Verify it was created correctly:
256+
257+
```bash
258+
cat /etc/cloud/ds-identify.cfg
259+
```
260+
261+
Expected output: `policy: enabled`
262+
263+
### Step 4 — Confirm the datasource
264+
265+
```bash
266+
sudo cloud-init init
267+
sudo cloud-init status --long
268+
```
269+
270+
Check the `detail` line in the output — it should read `DataSourceOpenStackLocal` or
271+
`DataSourceOpenStack`:
272+
273+
```
274+
detail: DataSourceOpenStackLocal [net,ver=2]
275+
```
276+
277+
If it reads `DataSourceNoCloud`, recheck Steps 1–4 and contact Mitchell or Giorgia if the
278+
problem persists.
279+
280+
---
281+
282+
## 9. Pre-snapshot checklist {#checklist}
283+
284+
- [ ] Workflow tested end-to-end from a trainee's perspective
285+
- [ ] All tools use CVMFS Singularity containers via `shelley-bio` where available
286+
- [ ] All references point to CVMFS paths where available
287+
- [ ] Input data is as small as practical
288+
- [ ] `du -sh /home/<username>/` confirms the directory size is acceptable
289+
- [ ] Any non-CVMFS resources that are too large to bundle are pulled via the provisioning script
290+
- [ ] `sudo cp -r /home/<username>/. /etc/skel/` completed successfully
291+
- [ ] `/etc/skel/` contents verified before snapshotting
292+
- [ ] Total estimated disk usage (base ~12 GB + skel + boot-time pulls) comfortably under 30 GB
293+
- [ ] `/etc/cloud/ds-identify.cfg` is present and reads `policy: enabled`

0 commit comments

Comments
 (0)