|
| 1 | +# Managing pangenome data |
| 2 | + |
| 3 | +`MetaPanG` profiles against `PanGBank` collections. Two commands let you see what |
| 4 | +is available and manage the files kept locally: `metapang list` to browse the |
| 5 | +catalog, and `metapang cache` to inspect, pre-download, and remove cached data. |
| 6 | + |
| 7 | +## `metapang list` |
| 8 | + |
| 9 | +Browse the `PanGBank` catalog and check what this `MetaPanG` release supports. |
| 10 | + |
| 11 | +With no argument it lists every collection and release: |
| 12 | + |
| 13 | +``` |
| 14 | +metapang list |
| 15 | +``` |
| 16 | + |
| 17 | +| Column | Meaning | |
| 18 | +| --- | --- | |
| 19 | +| `collection` | The collection name. | |
| 20 | +| `version` | A release of that collection. | |
| 21 | +| `pangenomes` | Number of species pangenomes in the release. | |
| 22 | +| `supported` | Whether this `MetaPanG` release can profile against it (see [Collection compatibility](#collection-compatibility)). | |
| 23 | +| `discarded` | How many species are discarded by default in that release. | |
| 24 | +| `link` | The web page for the release. | |
| 25 | + |
| 26 | +Passing a `collection[@version]` shows that release in detail, including the |
| 27 | +species discarded by default and their reasons, plus a link to browse every |
| 28 | +species on the web: |
| 29 | + |
| 30 | +``` |
| 31 | +metapang list GTDB_refseq@2.0.0 |
| 32 | +``` |
| 33 | + |
| 34 | +The version defaults to `latest` when omitted. |
| 35 | + |
| 36 | +## `metapang cache` |
| 37 | + |
| 38 | +The files needed for a run (the collection index, and the graph and pangenome of |
| 39 | +each species) are downloaded from `PanGBank` on demand and cached locally, so each |
| 40 | +resource is retrieved only once and reused by later runs. |
| 41 | + |
| 42 | +### Where the cache lives |
| 43 | + |
| 44 | +By default the cache is `.metapang-cache` in the working directory. It can be |
| 45 | +relocated, in increasing order of precedence: |
| 46 | + |
| 47 | +| Method | Scope | |
| 48 | +| --- | --- | |
| 49 | +| `cache_directory` in a config file (see [Configuration](configuration.md)) | Persistent | |
| 50 | +| `METAPANG_PANGBANK_CACHE_DIRECTORY` environment variable | Per shell | |
| 51 | +| `metapang cache --path DIR ...` | Per invocation | |
| 52 | + |
| 53 | +``` |
| 54 | +metapang cache --path /data/metapang-cache list |
| 55 | +``` |
| 56 | + |
| 57 | +`metapang cache path` prints the resolved directory. |
| 58 | + |
| 59 | +### Inspecting the cache |
| 60 | + |
| 61 | +`metapang cache list` shows the cached collections and their contents, one row per |
| 62 | +item, with its kind (`index`, `pangenome`, or `dbg`), completeness status, and |
| 63 | +size, followed by the cache total. |
| 64 | + |
| 65 | +``` |
| 66 | +metapang cache list |
| 67 | +``` |
| 68 | + |
| 69 | +### Pre-downloading with `fetch` |
| 70 | + |
| 71 | +`metapang cache fetch` populates the cache ahead of time, so a later `profile` run |
| 72 | +has everything it needs offline. |
| 73 | + |
| 74 | +``` |
| 75 | +metapang cache fetch TARGET [OPTIONS] |
| 76 | +``` |
| 77 | + |
| 78 | +`TARGET` is `collection[@version][:species[,species,...]]`. |
| 79 | + |
| 80 | +- Without `:species`, only the release bank index is fetched. This is what species |
| 81 | + detection needs. |
| 82 | +- With `:species`, each named species graph (and its pangenome) is fetched as well. |
| 83 | + This is what read mapping needs. |
| 84 | + |
| 85 | +| Option | Default | Description | |
| 86 | +| --- | --- | --- | |
| 87 | +| `--index` / `--no-index` | on | Fetch the release bank index. | |
| 88 | +| `--dbg` / `--no-dbg` | on | Fetch the de Bruijn graph of each named species. | |
| 89 | +| `--pangenome` / `--no-pangenome` | on | Fetch the `.h5` pangenome of each named species. | |
| 90 | +| `-f`, `--force` | off | Re-download even if already cached. | |
| 91 | + |
| 92 | +``` |
| 93 | +# prepare for detection: the index only |
| 94 | +metapang cache fetch GTDB_refseq@2.0.0 |
| 95 | +
|
| 96 | +# prepare to profile two species end to end: index plus their graphs |
| 97 | +metapang cache fetch GTDB_refseq@2.0.0:s__Klebsiella_pneumoniae,s__Abiotrophia_defectiva |
| 98 | +``` |
| 99 | + |
| 100 | +An unsupported `collection@version` is rejected up front, and unknown species |
| 101 | +names are reported before anything is downloaded. Already-cached items are kept |
| 102 | +and reported as present unless `--force` is given. |
| 103 | + |
| 104 | +### Clearing the cache |
| 105 | + |
| 106 | +`metapang cache clear` removes the whole cache, or a subset given as |
| 107 | +`collection[@version][:name]`. |
| 108 | + |
| 109 | +``` |
| 110 | +# everything (asks for confirmation) |
| 111 | +metapang cache clear |
| 112 | +
|
| 113 | +# one collection release |
| 114 | +metapang cache clear GTDB_refseq@2.0.0 |
| 115 | +
|
| 116 | +# a single species within a release |
| 117 | +metapang cache clear GTDB_refseq@2.0.0:s__Abiotrophia_defectiva |
| 118 | +``` |
| 119 | + |
| 120 | +Add `-y` to skip the confirmation prompt. |
| 121 | + |
| 122 | +## Collection compatibility |
| 123 | + |
| 124 | +Each `MetaPanG` release ships a compatibility policy that decides which |
| 125 | +`PanGBank` collections it can profile against. Run [`metapang list`](#metapang-list) |
| 126 | +to see, per release, whether it is supported and how many species are discarded. |
| 127 | + |
| 128 | +**Supported versions.** A collection version this release was not validated |
| 129 | +against is rejected up front, before any work is done: |
| 130 | + |
| 131 | +``` |
| 132 | +metapang profile -q reads.fastq.gz -b GTDB_refseq@1.0.0 |
| 133 | +# Error: 'GTDB_refseq@1.0.0' is not compatible with this MetaPanG version |
| 134 | +# (supported versions: 2.0.0). |
| 135 | +``` |
| 136 | + |
| 137 | +This is a hard check and cannot be overridden; upgrade or downgrade `MetaPanG` |
| 138 | +to match the collection. |
| 139 | + |
| 140 | +**Discarded species.** Within a supported version, some species may be |
| 141 | +discarded by default (for example, unreliable pangenomes). |
| 142 | + |
| 143 | +When any are excluded the `profile` run log lists them: |
| 144 | + |
| 145 | +``` |
| 146 | +Excluding 1 discarded species from detection (use --include-discarded to include them): |
| 147 | + - s__Some_species: low quality pangenome |
| 148 | +``` |
| 149 | + |
| 150 | +Unlike the version check, discards are overridable with `--include-discarded`: |
| 151 | + |
| 152 | +``` |
| 153 | +# include every discarded species |
| 154 | +metapang profile -q reads.fastq.gz -b GTDB_refseq@2.0.0 --include-discarded |
| 155 | +
|
| 156 | +# include only specific ones (comma-separated, note the '=') |
| 157 | +metapang profile -q reads.fastq.gz -b GTDB_refseq@2.0.0 \ |
| 158 | + --include-discarded=s__Some_species,s__Other_species |
| 159 | +``` |
| 160 | + |
| 161 | +Naming a discarded species explicitly through the `:pangenomes` suffix of `-b` |
| 162 | +also profiles it, since that skips detection entirely. |
0 commit comments