Release 3.3.1 (#872)

* copy changes from bgruening * this file should not be here since years (#845) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * this file should not be here since years * Add Arabidopsis TAIR10 (A_thaliana_Jun_2009) (#853) Using output from: faCount A_thaliana_Jun_2009.fa #seq len A C G T N cpg Chr1 30427671 9709674 5435374 5421151 9697113 164359 697370 Chr2 19698289 6315641 3542973 3520766 6316348 2561 457572 Chr3 23459830 7484757 4258333 4262704 7448059 5977 559031 Chr4 18585056 5940546 3371349 3356091 5914038 3032 439585 Chr5 26975502 8621974 4832253 4858759 8652238 10278 630299 ChrC 154478 48546 28496 27570 49866 0 4639 ChrM 366924 102464 82661 81609 100190 0 13697 total 119667750 38223602 21551439 21528650 38177852 186207 2802193 hpc $ python Python 2.7.11 (default, Jul 25 2019, 12:10:26) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 119667750-186207 119481543 * Fix python version in Azure tests (#860) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * Fix python version * Update azure-pipelines.yml * fixed typo (#864) * Develop (#827) * Merged into the wrong branch without noticing :( (#814) * use better conda link (#799) * Estimated filtering fix (#813) * oops * fix testing and set a max number of filtered reads * apparently a bunch of things were getting skipped * fix wrappers * update computeMatrix wrapper * Decrease memory needs (#817) * Use an iterator to not blow memory up * Update a bit more * The GC bias stuff is all deprecated, I'm not fixing that old code * Cache resulting counts rather than just decreasing the bin size (#818) * Cache resulting counts rather than just decreasing the bin size * sanity check * Implement #815 * [skip ci] update change log * Implement #816 (#825) * Implement #816 * expose option * Add a test using pseudocounts and skipZeroOverZero * syntax * Fix tests * Make --skipZeroOverZero a galaxy macro and add to bigwigCompare * [ci skip] a bit of formatting * Fix #822 (#826) * fixes linting issues (#837) * Delete #test.bg# (#859) File is removed upon clean. * fixed typo * Update test images, skip testing if the wrong matplotlib version is used (#865) * Update test images, skip testing if the wrong matplotlib version is used * Update test-template.yml * linting * can't conda activate on azure * now the heatmap is correct and the profile is wrong * lint * only one test should fail now * Fix #844 * Should fix one test at least * fix last tests * fix #838 (#843) * fix #838 * fixes * Update CHANGES.txt * Close #868 #867 and #851 (#869) * Fix #868 * Fix #867 * Default ALL the things! * Fix #866 (#871) * release 3.3.1 * try github actions * each action is a file * OK, that's inflexible * OK, the action.yml thing is a mess * syntax * ok, try this * uses * spacing * ok * do anchors work? * boo, so duplicative! * oops * maybe this will work for pypi * ensure dist is empty * nev * rename * bump version number
deeptools · Sep 10, 2019 · df8536a · df8536a
1 parent 76dbf59
commit df8536a
Show file tree

Hide file tree

Showing 47 changed files with 275 additions and 118 deletions.
diff --git a/.azure-pipelines/test-template.yml b/.azure-pipelines/test-template.yml
@@ -1,9 +1,9 @@
 steps:
-- bash: conda create -n foo -q --yes -c conda-forge -c bioconda numpy scipy matplotlib==2.1.2 nose flake8 plotly==2.0.12 pysam pyBigWig py2bit deeptoolsintervals cython
+- bash: conda create -n foo -q --yes -c conda-forge -c bioconda python=$(python.version) numpy scipy matplotlib==3.1.1 nose flake8 plotly pysam pyBigWig py2bit deeptoolsintervals
   displayName: Installing dependencies
 - bash: |
     source activate foo
-    python setup.py install
+    python -m pip install . --no-deps --ignore-installed -vvv
   displayName: Installing deeptools
 - bash: |
     source activate foo

diff --git a/.github/workflows/pypi.yml b/.github/workflows/pypi.yml
@@ -0,0 +1,37 @@
+name: pypi
+on: [push]
+jobs:
+  pypi:
+    name: upload to pypi
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup conda
+      if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
+      run: |
+        curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o miniconda.sh
+        bash miniconda.sh -b -p $HOME/miniconda
+        export PATH="$HOME/miniconda/bin:$PATH"
+        hash -r
+        conda config --set always_yes yes --set changeps1 no
+    - name: create env
+      if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        conda create -n foo -q --yes -c conda-forge -c bioconda python=3.7 twine
+    - name: sdist
+      if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        rm -f dist/*
+        python setup.py sdist
+    - name: upload
+      if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
+      env:
+        TWINE_USERNAME: "__token__"
+        TWINE_PASSWORD: ${{ secrets.pypi_password }}
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        twine upload dist/*
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -0,0 +1,90 @@
+name: Test
+on: [push]
+jobs:
+  build-linux:
+    name: Test on Linux
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup conda
+      run: |
+        curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o miniconda.sh
+        bash miniconda.sh -b -p $HOME/miniconda
+        export PATH="$HOME/miniconda/bin:$PATH"
+        hash -r
+        conda config --set always_yes yes --set changeps1 no
+    - name: create env
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        conda create -n foo -q --yes -c conda-forge -c bioconda python=3.7 numpy scipy matplotlib==3.1.1 nose flake8 plotly pysam pyBigWig py2bit deeptoolsintervals
+    - name: install deeptools
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        python -m pip install . --no-deps --ignore-installed -vvv
+    - name: PEP8
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        flake8 . --exclude=.venv,.build,build --ignore=E501,F403,E402,F999,F405,E722,W504,W605
+    - name: Test deepTools
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        nosetests --with-doctest -sv deeptools
+  build-osx:
+    name: Test on OSX
+    runs-on: macOS-latest
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup conda
+      run: |
+        curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o miniconda.sh
+        bash miniconda.sh -b -p $HOME/miniconda
+        export PATH="$HOME/miniconda/bin:$PATH"
+        hash -r
+        conda config --set always_yes yes --set changeps1 no
+    - name: create env
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        conda create -n foo -q --yes -c conda-forge -c bioconda python=3.7 numpy scipy matplotlib==3.1.1 nose flake8 plotly pysam pyBigWig py2bit deeptoolsintervals
+    - name: install deeptools
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        python -m pip install . --no-deps --ignore-installed -vvv
+    - name: Test deepTools
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        nosetests --with-doctest -sv deeptools
+  planemo:
+    name: First planemo chunk
+    runs-on: ubuntu-latest
+    needs: build-linux
+    strategy:
+      matrix:
+        chunk: [1, 2, 3]
+    steps:
+    - uses: actions/checkout@v1
+    - name: Setup conda
+      run: |
+        curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o miniconda.sh
+        bash miniconda.sh -b -p $HOME/miniconda
+        export PATH="$HOME/miniconda/bin:$PATH"
+        hash -r
+        conda config --set always_yes yes --set changeps1 no
+    - name: create env
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        conda create -n foo -q --yes -c conda-forge -c bioconda python=3.7 numpy scipy matplotlib==3.1.1 nose flake8 plotly pysam pyBigWig py2bit deeptoolsintervals planemo samtools
+    - name: install deeptools
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        python -m pip install . --no-deps --ignore-installed -vvv
+    - name: planemo
+      run: |
+        export PATH=$HOME/miniconda/bin:$PATH
+        source activate foo
+        ./.planemo.sh ${{ matrix.chunk }}
diff --git a/CHANGES.txt b/CHANGES.txt
@@ -1,3 +1,12 @@
+3.3.1
+
+ * Fixed `--plotNumbers` not working in `plotCorrelation`. This was issue #838.
+ * Fixed compatibility with matplotlib 3 and restrict to at least that version.
+ * The Y-axis labels should once again appear in both plotHeatmap and plotProfile (issue #844). This was related to the previous point.
+ * Testing is no longer performed with python 2.7, which will reach end of life in a couple months.
+ * Various documentation updates (issues #868, #867 and #851).
+ * Increased support for BED files with track header lines (issue #866).
+
 3.3.0
 
  * `plotCoverage` now has a `--BED` option, to restrict plots and output to apply to a specific set of regions given by a BED or GTF file or files (issue #829).

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -8,8 +8,6 @@ jobs:
     vmImage: 'ubuntu-16.04'
   strategy:
     matrix:
-      Python27:
-        python.version: '2.7'
       Python37:
         python.version: '3.7'
     maxParallel: 4
@@ -24,8 +22,6 @@ jobs:
     vmImage: 'macOS-10.13'
   strategy:
     matrix:
-      Python27:
-        python.version: '2.7'
       Python37:
         python.version: '3.7'
     maxParallel: 1
@@ -54,7 +50,7 @@ jobs:
   steps:
   - bash: echo "##vso[task.prependpath]$CONDA/bin"
     displayName: Add conda to PATH
-  - bash: conda create -n foo -q --yes -c conda-forge -c bioconda numpy scipy matplotlib==2.1.2 nose flake8 plotly==2.0.12 pysam pyBigWig py2bit deeptoolsintervals cython planemo setuptools
+  - bash: conda create -n foo -q --yes -c conda-forge -c bioconda python=$(python.version) numpy scipy matplotlib==3.1.1 nose flake8 plotly pysam pyBigWig py2bit deeptoolsintervals planemo samtools
     displayName: Installing dependencies
   - bash: |
       source activate foo

diff --git a/deeptools/_version.py b/deeptools/_version.py
@@ -2,4 +2,4 @@
 # This file is originally generated from Git information by running 'setup.py
 # version'. Distribution tarballs contain a pre-generated copy of this file.
 
-__version__ = '3.3.0'
+__version__ = '3.3.1'
diff --git a/deeptools/alignmentSieve.py b/deeptools/alignmentSieve.py
@@ -30,7 +30,7 @@ def parseArguments():
     general.add_argument('--numberOfProcessors', '-p',
                          help='Number of processors to use. Type "max/2" to '
                          'use half the maximum number of processors or "max" '
-                         'to use all available processors.',
+                         'to use all available processors. (Default: %(default)s)',
                          metavar="INT",
                          type=parserCommon.numberOfProcessors,
                          default=1,
@@ -80,7 +80,7 @@ def parseArguments():
 
     filtering.add_argument('--filterRNAstrand',
                            help='Selects RNA-seq reads (single-end or paired-end) in '
-                                'the given strand.',
+                                'the given strand. (Default: %(default)s)',
                            choices=['forward', 'reverse'],
                            default=None)
 
@@ -129,15 +129,15 @@ def parseArguments():
                            help='The minimum fragment length needed for read/pair '
                            'inclusion. This option is primarily useful '
                            'in ATACseq experiments, for filtering mono- or '
-                           'di-nucleosome fragments.',
+                           'di-nucleosome fragments. (Default: %(default)s)',
                            metavar='INT',
                            default=0,
                            type=int,
                            required=False)
 
     filtering.add_argument('--maxFragmentLength',
                            help='The maximum fragment length needed for read/pair '
-                           'inclusion.',
+                           'inclusion. A value of 0 indicates no limit. (Default: %(default)s)',
                            metavar='INT',
                            default=0,
                            type=int,

diff --git a/deeptools/bamCompare.py b/deeptools/bamCompare.py
@@ -85,7 +85,7 @@ def getOptionalArgs():
                           'If a method is specified, then it will be used to compensate '
                           'for sequencing depth differences between the samples. '
                           'As an alternative, this can be set to None and an option from '
-                          '--normalizeUsing <method> can be used.',
+                          '--normalizeUsing <method> can be used. (Default: %(default)s)',
                           choices=['readCount', 'SES', 'None'],
                           default='readCount')
 
@@ -97,14 +97,14 @@ def getOptionalArgs():
                           'If you do not have a good sequencing depth for '
                           'your samples consider increasing the sampling '
                           'regions\' size to minimize the probability '
-                          'that zero-coverage regions are used.',
+                          'that zero-coverage regions are used. (Default: %(default)s)',
                           default=1000,
                           type=int)
 
     optional.add_argument('--numberOfSamples', '-n',
                           help='*Only relevant when SES is chosen for the '
                           'scaleFactorsMethod.* Number of samplings taken '
-                          'from the genome to compute the scaling factors.',
+                          'from the genome to compute the scaling factors. (Default: %(default)s)',
                           default=1e5,
                           type=int)
 
@@ -125,7 +125,7 @@ def getOptionalArgs():
                           'values are interpreted as negative fold changes. '
                           'Instead of performing a computation using both files, the scaled signal can '
                           'alternatively be output for the first or second file using '
-                          'the \'--operation first\' or \'--operation second\'',
+                          'the \'--operation first\' or \'--operation second\'. (Default: %(default)s)',
                           default='log2',
                           choices=['log2', 'ratio', 'subtract', 'add', 'mean',
                                    'reciprocal_ratio', 'first', 'second'],
@@ -137,7 +137,7 @@ def getOptionalArgs():
                           'You can specify different values as pseudocounts for '
                           'the numerator and the denominator by providing two '
                           'values (the first value is used as the numerator '
-                          'pseudocount and the second the denominator pseudocount).',
+                          'pseudocount and the second the denominator pseudocount). (Default: %(default)s)',
                           default=[1],
                           type=float,
                           nargs='+',

diff --git a/deeptools/bamCoverage.py b/deeptools/bamCoverage.py
@@ -67,7 +67,7 @@ def get_optional_args():
 
     optional.add_argument('--scaleFactor',
                           help='The computed scaling factor (or 1, if not applicable) will '
-                          'be multiplied by this.',
+                          'be multiplied by this. (Default: %(default)s)',
                           default=1.0,
                           type=float,
                           required=False)

diff --git a/deeptools/bamPEFragmentSize.py b/deeptools/bamPEFragmentSize.py
@@ -51,7 +51,7 @@ def parse_arguments():
 
     parser.add_argument('--numberOfProcessors', '-p',
                         help='Number of processors to use. The default is '
-                        'to use 1.',
+                        'to use 1. (Default: %(default)s)',
                         metavar="INT",
                         type=int,
                         default=1,
@@ -65,18 +65,18 @@ def parse_arguments():
                         nargs='+')
     parser.add_argument('--plotTitle', '-T',
                         help='Title of the plot, to be printed on top of '
-                        'the generated image. Leave blank for no title.',
+                        'the generated image. Leave blank for no title. (Default: %(default)s)',
                         default='')
     parser.add_argument('--maxFragmentLength',
-                        help='The maximum fragment length in the histogram. A value of 0 (the default) indicates to use twice the mean fragment length',
+                        help='The maximum fragment length in the histogram. A value of 0 (the default) indicates to use twice the mean fragment length. (Default: %(default)s)',
                         default=0,
                         type=int)
     parser.add_argument('--logScale',
                         help='Plot on the log scale',
                         action='store_true')
     parser.add_argument('--binSize', '-bs',
                         metavar='INT',
-                        help='Length in bases of the window used to sample the genome. (default 1000)',
+                        help='Length in bases of the window used to sample the genome. (Default: %(default)s)',
                         default=1000,
                         type=int)
     parser.add_argument('--distanceBetweenBins', '-n',
@@ -87,7 +87,7 @@ def parse_arguments():
                         'for high coverage samples, while smaller values are useful for '
                         'lower coverage samples. Note that if you specify a value that '
                         'results in too few (<1000) reads sampled, the value will be '
-                        'decreased. (default 1000000)',
+                        'decreased. (Default: %(default)s)',
                         default=1000000,
                         type=int)
     parser.add_argument('--blackListFileName', '-bl',

diff --git a/deeptools/bigwigCompare.py b/deeptools/bigwigCompare.py
@@ -54,7 +54,7 @@ def parse_arguments(args=None):
                         'You can specify different values as pseudocounts for '
                         'the numerator and the denominator by providing two '
                         'values (the first value is used as the numerator '
-                        'pseudocount and the second the denominator pseudocount).',
+                        'pseudocount and the second the denominator pseudocount). (Default: %(default)s)',
                         default=1,
                         nargs='+',
                         action=parserCommon.requiredLength(1, 2),
@@ -73,11 +73,10 @@ def parse_arguments(args=None):
                         'the negative of the inverse of the ratio '
                         'if the ratio is less than 0. The resulting '
                         'values are interpreted as negative fold changes. '
-                        '*NOTE*: Only with --operation subtract can --normalizeUsing RPGC or '
-                        '--normalizeUsing RPKM be used. Instead of performing a '
+                        'Instead of performing a '
                         'computation using both files, the scaled signal can '
                         'alternatively be output for the first or second file using '
-                        'the \'--operation first\' or \'--operation second\'',
+                        'the \'--operation first\' or \'--operation second\' (Default: %(default)s)',
                         default='log2',
                         choices=['log2', 'ratio', 'subtract', 'add', 'mean',
                                  'reciprocal_ratio', 'first', 'second'],

diff --git a/deeptools/computeGCBias.py b/deeptools/computeGCBias.py
@@ -94,7 +94,7 @@ def getRequiredArgs():
 
     optional.add_argument('--sampleSize',
                           default=5e7,
-                          help='Number of sampling points to be considered.',
+                          help='Number of sampling points to be considered. (Default: %(default)s)',
                           type=int)
 
     optional.add_argument('--extraSampling',
@@ -130,7 +130,7 @@ def getRequiredArgs():
                       'standard fragment size for Illumina machines. However, '
                       'if the depth of sequencing is low, a larger bin size '
                       'will be required, otherwise many bins will not '
-                      'overlap with any read')
+                      'overlap with any read (Default: %(default)s)')
 
     return parser