Merge branch 'master'

Author: abartesaghi

config/pyp_config.toml

@@ -4,6 +4,57 @@ Changelog
 
 **Legend**: :fa:`star,text-success` New feature, :fa:`plus-square,text-primary` Improvement, :fa:`bug,text-danger` Bug fix
 
+v0.6.3 (3/01/2024)
+******************
+
+   :fa:`plus-square,text-primary` Allow import of clean particles obtained after 3D classification into pre-processing block.
+
+   :fa:`plus-square,text-primary` Stop saving unnecessary metadata files during constrained refinement.
+
+   :fa:`plus-square,text-primary` Implement particle list picker that was missing from some import blocks.
+
+   :fa:`plus-square,text-primary` Implement parameter groups in UI to better handle conditional parameters.
+
+   :fa:`plus-square,text-primary` Add links to download tomograms and metadata for ArtiaX plugin.
+
+   :fa:`plus-square,text-primary` Provide more granular information when determining handedness of tilt-series.
+
+   :fa:`plus-square,text-primary` Allow users to control the timeout for deleting the scratch folder of zombie jobs.
+
+   :fa:`plus-square,text-primary` Add new parameter to control size of patches during patch-tracking to prevent tiltxcorr errors.
+
+   :fa:`plus-square,text-primary` Upgrade program versions to MotionCor3 1.1.1 and AreTomo2 1.1.2.
+
+   :fa:`plus-square,text-primary` Allow use of environment variables when specifying the local scratch directory.
+
+   :fa:`bug,text-danger` Hide the export tab from particle filtering blocks for tomography projects.
+
+   :fa:`bug,text-danger` Fix bug that ocurred when skipping frame alignment during movie processing.
+
+   :fa:`bug,text-danger` Fix bug in function used to export sessions to .star format.
+
+   :fa:`bug,text-danger` Fix bug in tomography sessions that ocurred when using size-based particle picking.
+
+   :fa:`bug,text-danger` Fix bug when exporting metadata in star format that saved the files to the incorrect folder.
+
+   :fa:`bug,text-danger` Fix bug when setting number of patches when running AreTomo2.
+
+   :fa:`bug,text-danger` Fix inconsistencies in the determination of parameter changes between consecutive runs.
+
+   :fa:`bug,text-danger` Stop trying to launch external programs for sub-tomogram averaging after particle extraction.
+
+   :fa:`bug,text-danger` Fix issue with missing metadata entries during tilt-series re-processing.
+
+   :fa:`bug,text-danger` Correctly discard particles that are too close to gold fiducials.
+
+   :fa:`bug,text-danger` Fix issue with management of virion selection thresholds that affected geometric particle picking.
+
+   :fa:`bug,text-danger` Fix bug when creating montages that ocurred when particle radius was equal to half the box size.
+
+   :fa:`bug,text-danger` Fix bug when re-running pre-processing after virion selection.
+
+   :fa:`bug,text-danger` Fix bug with links used to download maps for older iterations.
+
 v0.6.2 (2/01/2024)
 ******************
 

docs/changelog.rst

@@ -7,13 +7,17 @@ Visualization in ChimeraX/ArtiaX
 Step 1: Download the necessary files
 ------------------------------------
 
-Select the refinement block where you will extract the particle locations from. Click on the ``Edit`` menu for that block and select the ``Show filesystem location`` to find out the location of the project in the filesystem.
+You will need to download the following files to your local computer:
 
-From this location, you will need to download the following files to your local computer:
+- Tomogram reconstruction (``*.rec``)
+- Refined map (``*.mrc``)
+- Corresponding particle orientations (``*.star``)
 
-- Tomogram reconstruction (``mrc/tile_series_name.rec``)
-- Refined map (``frealign/maps/block_name_r{class_number}_{iteration_number}_crop.mrc``)
-- Corresponding particle orientations (``frealign/artiax/tile_series_name_K1.star``)
+You can get these files as follows:
+
+- Go to the Pre-processing block, click on the **Tilt-series** tab, select a tilt-series and go to the **Reconstruction** tab. Download the ``.rec`` file by clicking on the gray/green badge
+- Go to the Particle refinement block, select the **Reconstruction** tab and select the ``Cropped Map`` option from the dropdown menu
+- In the same Particle refinement block, go the **Metadata** tab, type the name of the tilt-series and click ``Search``. Download the ``.star`` file by clicking on the gray/green badge
 
 Step 2: Load data into ChimeraX/ArtiaX
 --------------------------------------

docs/guide/chimerax_artiax.rst

@@ -223,12 +223,12 @@ Step 4: Reference-based refinement
 
       - Set the location of the ``Initial model`` by clicking on the icon :fa:`search, text-primary`, navigating to the folder where you downloaded the data for the tutorial, selecting the file  `EMPIAR-10025_init_ref.mrc`, and clicking :badge:`Choose File,badge-primary`
 
-      - Set ``Last iteration`` to 5
-
       - Set ``Max resolution (A)`` to 8:7:6
 
       - Check ``Use signed correlation``
 
+      - Set ``Last iteration`` to 5
+
       - Check ``Show advanced options``
 
       - Check ``Use alignment priors``
@@ -346,14 +346,14 @@ Step 7: Particle refinement
 
       - Select the ``Input parameter file`` by clicking on the icon :fa:`search, text-primary` and selecting the file `sp-fine-refinement-*_r01_02_clean.par.bz2`
 
-      - Set ``Last iteration`` to 6
-
-      - Set ``Search mode`` to local
-
       - Set ``Max resolution (A)`` to 6:4:3
 
       - Check ``Use signed correlation``
 
+      - Set ``Last iteration`` to 6
+
+      - Set ``Search mode`` to local
+
   * Click :badge:`Save,badge-primary`, :badge:`Run,badge-primary`, and :badge:`Start Run for 1 block,badge-primary` to launch the job
 
   * Click inside the :badge:`Particle refinement,badge-secondary` block to inspect the results:
@@ -461,12 +461,12 @@ Step 11: Movie frame refinement
 
       - Select the ``Input parameter`` by clicking on the icon :fa:`search, text-primary` and selecting the file `sp-coarse-refinement-*_r01_07.par.bz2`
 
+      - Set ``Max resolution (A)`` to 3
+
       - Set ``Last iteration`` to 3
 
       - Check ``Skip refinement``
 
-      - Set ``Max resolution (A)`` to 3
-
       - Go to the **Constrained refinement** tab
 
     .. tabbed:: Constrained refinement
@@ -475,8 +475,6 @@ Step 11: Movie frame refinement
 
       - Check ``Movie frame refinement``
 
-      - Check ``Show advanced options``
-
       - Check ``Regularize translations``
 
       - Set ``Spatial sigma`` to 15

docs/tutorials/spa_empiar_10025.rst

@@ -134,26 +134,26 @@ Step 3: Pre-processing
 
         - Set ``Max resolution`` to 5.0
 
-        - Click on the **Virion/spike detection** tab
-
-      .. tabbed:: Virion/spike detection
-
-        - Set ``Virion detection method`` to auto
-
-        - Set ``Virion radius (A)`` to 500.0
-
         - Click on the **Tomogram reconstruction** tab
 
       .. tabbed:: Tomogram reconstruction
-        
-        - Click ``Show advanced options``
 
         - Set ``Binning factor for reconstruction`` to 8
 
+        - Click ``Show advanced options``
+
         - Set ``Thickness of reconstruction (unbinned voxels)`` to 2048
 
+        - Click on the **Virion/spike detection** tab
+
+      .. tabbed:: Virion/spike detection
+
+        - Set ``Virion radius (A)`` to 500.0
+
+        - Set ``Virion detection method`` to auto
+
         - Click on the **Resources** tab
-      
+
       .. tabbed:: Resources
 
         - Set ``Threads per task`` to 7
@@ -305,14 +305,14 @@ Step 6: Reference-based refinement
 
         - Specify the location of the ``Initial model`` by clicking on the icon :fa:`search, text-primary`, navigating to the folder where you downloaded the data for the tutorial, and selecting the file `EMPIAR-10164_init_ref.mrc`
 
-        - Click :fa:`search,text-primary` in ``Input parameter file`` to select the initial parameters text file ``tomo-preprocessing-*_original_volumes.txt`` from :badge:`Pre-processing,badge-secondary`
-
-        - Check ``Skip refinement``
+        - Click :fa:`search,text-primary` in ``Alignments from sub-volume averaging`` to select the initial parameters text file ``tomo-preprocessing-*_original_volumes.txt`` from :badge:`Pre-processing,badge-secondary`
 
         - Set ``Max resolution (A)`` to 8.0
 
         - Check ``Use signed correlation``
 
+        - Check ``Skip refinement``
+
         - Click on the **Constrained refinement** tab
 
       .. tabbed:: Constrained refinement
@@ -325,20 +325,20 @@ Step 6: Reference-based refinement
 
         - Check ``Refine particle alignments``
 
-        - Set ``Particle Phi range (degrees)`` and ``Particle Theta range (degrees)`` to 10
+        - Set ``Phi range (degrees)`` and ``Theta range (degrees)`` to 10
 
-        - Set ``Particle translation range (voxels)`` to 50
+        - Set ``Translation range (voxels)`` to 50
 
-        - Click on the **Resources** tab
+        - Click on the **Reconstruction** tab
 
       .. tabbed:: Reconstruction
 
-        - Check ``Show advanced options``
-
         - Set ``Max tilt-angle`` to 50
 
         - Set ``Min tilt-angle`` to -50
 
+        - Click on the **Resources** tab
+
       .. tabbed:: Resources
 
         - Set ``Walltime per task`` to 9:00:00
@@ -367,10 +367,10 @@ Step 7. Fully constrained refinement
 
       .. tabbed:: Refinement
 
-        - Set ``Last iteration`` to 5
-
         - Set ``Max resolution (A)`` to 8:10:8:6
 
+        - Set ``Last iteration`` to 5
+
         - Click on the **Constrained refinement** tab
 
       .. tabbed:: Constrained refinement
@@ -383,7 +383,7 @@ Step 7. Fully constrained refinement
 
         - Set ``Optimizer - Exhaustive search points`` to 0
 
-        - Set ``Particle Phi range``, ``Particle Psi range`` and ``Particle Theta range`` to 20.0
+        - Set ``Phi range``, ``Psi range`` and ``Theta range`` to 20.0
 
         - Click on the **Exposure weighting** tab
 
@@ -491,12 +491,12 @@ Step 10. Region-based local refinement (before masking)
 
         - Select the location of the ``Initial parameter file`` by clicking on the icon :fa:`search,text-primary` and selecting the file `tomo-fine-refinement-*_r01_02.par.bz2` (select the file ``tomo-fine-refinement-*_r01_02_clean.par.bz2`` if bad particles were permanently removed in the previous step)
 
+        - Set ``Max resolution (A)`` to 6:5
+
         - Set ``Last iteration`` to 3
 
         - Check ``Skip refinement``
 
-        - Set ``Max resolution (A)`` to 6:5
-
         - Click on the **Constrained refinement** tab
 
       .. tabbed:: Constrained refinement
@@ -505,9 +505,7 @@ Step 10. Region-based local refinement (before masking)
 
         - Set ``Number of regions`` to 8,8,2
 
-        - Check ``Show advanced options``
-
-        - Set ``Particle translation range (voxels)`` to 20.0
+        - Set ``Translation range (voxels)`` to 20.0
 
     * Click :badge:`Save,badge-primary`, :badge:`Run,badge-primary`, and :badge:`Start Run for 1 block,badge-primary` to run the job
 
@@ -559,12 +557,14 @@ Step 12: Region-based constrained refinement
 
       .. tabbed:: Refinement
 
+        - Set ``Max resolution (A)`` to 6:5:5:4:3.5
+
+        - Uncheck ``Resume refinement``
+
         - Set ``First iteration`` to 4
 
         - Set ``Last iteration`` to 6
 
-        - Set ``Max resolution (A)`` to 6:5:5:4:3.5
-
         - Specify the location of the ``Shape mask`` produced in Step 11 by clicking on the icon :fa:`search, text-primary`, navigating to the location of the :badge:`Masking,badge-secondary` block by copying the path we saved above, and selecting the file `frealign/maps/mask.mrc`
 
     * Click :badge:`Save,badge-primary`, :badge:`Run,badge-primary`, and :badge:`Start Run for 1 block,badge-primary` to run the job
@@ -588,24 +588,22 @@ Step 13: Particle-based CTF refinement
 
       .. tabbed:: Refinement
 
-        - Set ``Last iteration`` to 7
-
         - Set ``Max resolution (A)`` to 3.1
 
+        - Set ``Last iteration`` to 7
+
         - Click on the **Constrained refinement** tab
 
       .. tabbed:: Constrained refinement
 
+        - Set ``Last exposure for refinement`` to 10
+
         - Uncheck ``Refine tilt-geometry``
 
         - Uncheck ``Refine particle alignments``
 
         - Check ``Refine CTF per-particle``
 
-        - Check ``Show advanced options``
-
-        - Set ``Last exposure for refinement`` to 10
-
     * Click :badge:`Save,badge-primary`, :badge:`Run,badge-primary`, and :badge:`Start Run for 1 block,badge-primary`
 
     * Click inside the :badge:`Particle refinement,badge-secondary` block to inspect the results
@@ -643,14 +641,12 @@ Step 14: Movie frame refinement
 
       .. tabbed:: Constrained refinement
 
+        - Set ``Last exposure for refinement`` to 4
+
         - Uncheck ``Refine CTF per-particle``
 
         - Check ``Movie frame refinement``
 
-        - Check ``Show advanced options``
-
-        - Set ``Last exposure for refinement`` to 4
-
         - Check ``Regularize translations``
 
         - Set ``Spatial sigma`` to 200.0
@@ -673,7 +669,7 @@ Step 15: Refinement after movie frame refinement
     * Click on the menu icon :fa:`bars, text-primary` from the :badge:`Movie refinement,badge-secondary` block and choose the :fa:`edit, text-primary` Edit option.
 
     * Go to the **Refinement** tab:
-        
+
       .. tabbed:: Refinement
 
         - Set ``Max resolution (A)`` to 3.3
@@ -682,25 +678,27 @@ Step 15: Refinement after movie frame refinement
 
       .. tabbed:: Constrained refinement
 
+        - Check ``Show advanced options``
+
+        - Set ``Min number of projections for refinement`` to 2
+
         - Check ``Refine tilt-geometry``
 
-        - Check ``Refine particle alignments``
+        - Set ``Translation range (pixels)`` to 10.0
 
-        - Uncheck ``Movie frame refinement``
-        
-        - Set ``Micrograph translation range (pixels)`` to 10.0
+        - Set ``Tilt-angle range (degrees)`` and ``Tilt-axis range (degrees)`` to 1.0
 
-        - Set ``Micrograph tilt-angle range (degrees)`` and ``Micrograph tilt-axis range (degrees)`` to 1.0
+        - Check ``Refine particle alignments``
 
-        - Set ``Particle Phi range (degrees)`` to 1.0
+        - Set ``Phi range (degrees)`` to 1.0
 
-        - Set ``Particle Psi range (degrees)`` to 1.0
+        - Set ``Psi range (degrees)`` to 1.0
 
-        - Set ``Particle Theta range (degrees)`` to 1.0
+        - Set ``Theta range (degrees)`` to 1.0
 
-        - Set ``Particle translation range (voxels)`` to 10.0
-        
-        - Set ``Min number of projections for refinement`` to 2
+        - Set ``Translation range (voxels)`` to 10.0
+
+        - Uncheck ``Movie frame refinement``
 
     * Click :badge:`Save,badge-primary`, :badge:`Run,badge-primary`, and :badge:`Start Run for 1 block,badge-primary`