2022.11 (#3223)

* 2022.11

* adding RS210 to the documentation

* fix text from CHANGELOG.md

* addressing @justinshaffer comment [skip ci]

Author: antgonza

CHANGELOG.md

@@ -1,5 +1,17 @@
 # Qiita changelog
 
+Version 2022.11
+---------------
+
+* Adding new Woltka v0.1.4 and WoLr2 - WoLr1 and RS210 (updated version of rep200) are available but not default.
+* Update qp-woltka so it runs faster, contribution with @wasade. Add newest improvements for woltka processing so it runs faster.
+* Fix qp-qiime2, "filter features" bug that expects a "feature metadata" value from the dropdown.
+* Avoid having [multiple git version](https://github.com/qiita-spots/qiita/issues/3216) commands running in the system - this in theory should fix some of the slow downs.
+* Fix sample_name in per preparation sample information files.
+* Now the sequencing artifacts (like per-sample-FASTQ, FASTA, etc) accept log files.
+* SortMeRNA via qp-meta will generate processing logs and will be added to their artifacts - note that this will only be available in newer runs AKA you can delete the outputs of past runs and rerun it.
+* Added new "Admin Sample Validation" to validate a list of sample names against what's available in a study, contributed by @sarayupai.
+
 Version 2022.09
 ---------------
 

qiita_core/__init__.py

@@ -6,4 +6,4 @@
 # The full license is in the file LICENSE, distributed with this software.
 # -----------------------------------------------------------------------------
 
-__version__ = "2022.09"
+__version__ = "2022.11"

qiita_db/__init__.py

@@ -27,7 +27,7 @@
 from . import user
 from . import processing_job
 
-__version__ = "2022.09"
+__version__ = "2022.11"
 
 __all__ = ["analysis", "artifact",  "archive", "base", "commands",
            "environment_manager", "exceptions", "investigation", "logger",

qiita_pet/__init__.py

@@ -6,4 +6,4 @@
 # The full license is in the file LICENSE, distributed with this software.
 # -----------------------------------------------------------------------------
 
-__version__ = "2022.09"
+__version__ = "2022.11"

qiita_pet/handlers/api_proxy/__init__.py

@@ -38,7 +38,7 @@
 from .user import (user_jobs_get_req)
 from .util import check_access, check_fp
 
-__version__ = "2022.09"
+__version__ = "2022.11"
 
 __all__ = ['prep_template_summary_get_req', 'data_types_get_req',
            'study_get_req', 'sample_template_filepaths_get_req',

qiita_pet/support_files/doc/source/dev/index.rst

@@ -8,6 +8,7 @@ The following is a full list of the available developer tutorials
     plugins
     rest
     resource_allocation
+    workflows
 
 To request documentation on any developer use-cases not addressed here,
 please add an issue `here <https://github.com/biocore/qiita/issues>`__.

qiita_pet/support_files/doc/source/dev/workflows.rst

@@ -0,0 +1,40 @@
+.. _workflows:
+
+.. index :: workflows
+
+Qiita Workflows
+===============
+
+The Qiita Workflows allow developers to define full software pipelines of how a
+give data type can be processed. A given Qiita Workflow will also define which
+commands a user can see in the GUI and allow to add that "default" workflow to
+a preparation via a single click on the GUI.
+
+How to add a workflow?
+----------------------
+
+#. Add the workflow
+
+   - `INSERT INTO qiita.default_workflow (name)...;`
+
+
+#. Add the data types that can use that workflow
+
+   - `INSERT INTO qiita.default_workflow_data_type (data_type_id, default_workflow_id) ...;`
+
+
+#. Add nodes (commands) via the default_parameter_set of the commands (you are selecting
+   the command and their parameters at once). Note that order is not important as we are going to link (organize) them in the next point.
+
+   - `INSERT INTO qiita.default_workflow_node (default_parameter_set_id, default_workflow_id) ...;`
+
+
+#. Link the nodes (via edges)
+
+   - `INSERT INTO qiita.default_workflow_edge (parent_id, child_id) ...;`
+
+
+#. Link the specific output (as commands can have multiple) of the parent nodes/commands to their
+   children (inputs)
+
+   - `INSERT INTO qiita.default_workflow_edge_connections (default_workflow_edge_id, parent_output_id, child_input_id) ...;`

qiita_pet/support_files/doc/source/processingdata/processing-recommendations.rst

@@ -45,7 +45,7 @@ Shotgun sequencing
 ------------------
 
 Qiita currently has one active shotgun metagenomics data analysis pipeline: a per sample
-bowtie2 alignment step with Woltka classification using either the WoLr1 or Rep200 databases.
+bowtie2 alignment step with Woltka classification using either the WoLr1, WoLr2 (default) or RS210 databases.
 Below you will find more information about each of these options.
 
 .. note::
@@ -67,16 +67,16 @@ subsequent meta-analyses. Currently, the `fastp` command is set to autodetect ad
 wetlab processing and we provide the following host references for your convenience:
 
 - auto-detect adapters and artifacts + phix filtering: This is a `deblur artifacts <https://github.com/biocore/deblur/blob/master/deblur/support_files/artifacts.fa>`_ reference, mainly for debugging and testing
-- auto-detect adapters and cheetah + phix filtering
-- auto-detect adapters and cow + phix filtering
-- auto-detect adapters and hamster + phix filtering
-- auto-detect adapters and horse + phix filtering
+- auto-detect adapters and `cheetah <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/003/709/585/GCF_003709585.1_Aci_jub_2/GCF_003709585.1_Aci_jub_2_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `cow <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/205/GCA_000003205.6_Btau_5.0.1/GCA_000003205.6_Btau_5.0.1_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `hamster <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/017/639/785/GCF_017639785.1_BCM_Maur_2.0/GCF_017639785.1_BCM_Maur_2.0_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `horse <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/002/305/GCA_000002305.1_EquCab2.0/GCA_000002305.1_EquCab2.0_genomic.fna.gz>`_ + phix filtering
 - auto-detect adapters and merge_genomes + phix filtering : is the combined genomes of a cheetah, cow, hamster, horse, human, mouse, pig, rabbit, and rat
-- auto-detect adapters and mouse + phix filtering
-- auto-detect adapters and pig + phix filtering
-- auto-detect adapters and rabbit + phix filtering
-- auto-detect adapters and rat + phix filtering
-- auto-detect adapters only filtering [not recommended]
+- auto-detect adapters and `mouse <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/635/GCF_000001635.27_GRCm39/GCF_000001635.27_GRCm39_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `pig <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/003/025/GCF_000003025.6_Sscrofa11.1/GCF_000003025.6_Sscrofa11.1_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `rabbit <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/003/625/GCF_000003625.3_OryCun2.0/GCF_000003625.3_OryCun2.0_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters and `rat <https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/895/GCA_000001895.4_Rnor_6.0/GCA_000001895.4_Rnor_6.0_genomic.fna.gz>`_ + phix filtering
+- auto-detect adapters only filtering
 
 Note that the command produces up to 6 output artifacts based on the aligner and database selected:
 
@@ -89,7 +89,7 @@ Note that the command produces up to 6 output artifacts based on the aligner and
 
 .. note::
    Woltka provides easy transformations for the "per gene Prediction table" to generate functional
-   profiles, `more information <https://github.com/qiyunzhu/woltka/blob/master/doc/wol.md#comparison>`
+   profiles, `more information <https://github.com/qiyunzhu/woltka/blob/master/doc/wol.md#comparison>`_.
 
 Aligners
 ^^^^^^^^
@@ -127,8 +127,6 @@ Reference databases
 
 Note that some of these are legacy option but not available for new processing.
 
-
-
 #. WoLr2 (“Web of Life” release 2): A significant upgrade from WoLr1. The genome
    pool is an even representation of microbial diversity, sampled from
    non-redundant bacterial and archaeal genomes from NCBI (RefSeq and GenBank,
@@ -154,6 +152,21 @@ Note that some of these are legacy option but not available for new processing.
       - Genera: 6,811
       - Species: 12,258
 
+#. RS210: Collection of reference microbial genomes sampled from the NCBI RefSeq
+   genome database, as of 2022-01-01. This time point corresponds to RefSeq
+   release 210.
+
+    - Genomes: 29,648
+    - Nucleotides: 926,894
+    - Basepairs: 111,767,286,504 (includes linkers)
+    - Numbers of taxonomic units:
+
+      - Archaea: 606
+      - Bacteria: 21,047
+      - Fungi: 409
+      - Protozoa: 93
+      - Viral: 7,493
+
 #. WoLr1 ("Web of Life" release 1): An even representation of microbial diversity, selected using an prototype
    selection algorithm based on the MinHash distance matrix among all non-redundant bacterial and archaeal genomes
    from NCBI (RefSeq and GenBank, complete and draft), plus several genome quality control criteria. A

qiita_ware/__init__.py

@@ -6,4 +6,4 @@
 # The full license is in the file LICENSE, distributed with this software.
 # -----------------------------------------------------------------------------
 
-__version__ = "2022.09"
+__version__ = "2022.11"

setup.py

@@ -10,7 +10,7 @@
 from setuptools import setup
 from glob import glob
 
-__version__ = "2022.09"
+__version__ = "2022.11"
 
 
 classes = """