Releases: SDT-org/SDT2
SDT2 Pre-release build
Feature complete version of SDT 2.
Some newly implemented features like the stats panel are still in development
For demo purposes only.
SDT 2 Beta 4
Updates
Complete UI overhaul
The SDT2 user interface has been completely redesigned with ease of use and intuitive controls that follow current UI design guidelines for major operating systems. Users have control over color options; responsive controls that follow a cohesive design across operating systems.
Multi-file tabs
Multiple analysis can now be viewed through the use of tabs above the plot. Switching between tabs does not stop a current analysis, but only one tab may run an analysis at a time. A new analysis can be started by clicking the + tab to the right of the current tab.
Clustering
This release introduces agglomerative hierarchical clustering options from the SciPy cluster hierarchy package.
Users may now choose from 7 methods, 4 strictly distance based
- Single Linkage (Nearest Neighbor)
- Complete Linkage (Farthest Neighbor)
- Average Linkage (UPGMA)
- Weighted Average Linkage (WPGMA)
And 3 using multidimensional scaling MDS - Median Linkage (UPGMC)
- Centroid Linkage (WPGMC)
- Ward's Linkage
Clustermap
The Clustermap plot provides a new way to visualize sequence data in SDT 2, allowing you to set specific clustering thresholds and clustering methods to explore relationships within datasets. SDT 2 provides several agglomerative, linkage-based clustering methods through the SciPy package library.
To create a hierarchical clustering, a precomputed pairwise distance matrix is used to perform Single, Complete, and Average linkage methods. For the Centroid, Median and Ward's linkage methods, the dimensionality of the dataset must first be reduced to 2D Euclidean distances through multidimensional scaling (MDS) via the sci-kit learn package prior to clustering. The dendrogram is visualized using the D3 library to reorder the heatmap.
Heatmaps
We have switched the heatmap component from Plotly to two separate components using the D3 library for JavaScript. Plotly's heatmap implementation renders primarily to Canvas, which limited our image exports to raster formats only (JPG, PNG, BMP). To enable high-quality vector exports, we implemented dual renderers with D3 (the library Plotly itself is built upon). The Canvas renderer provides smooth, performant interaction for users navigating large heatmap datasets. When users choose to export a heatmap in SVG format, SDT2 seamlessly transfers the current Canvas view settings to the SVG renderer, which generates a vector-based image of the identical visualization. While the SVG renderer would be too performance-intensive for the primary interactive graph, particularly for use with large datasets, it's ideal for creating resolution-independent exports for publication and printing.
Expanded data types
SDT2 distribution plots can now create plots representing percent identity scores, length, and GC content of a dataset.
Histogram
The histogram has been re-written to improve user experience offering adjustable bin sizes, axis options, horizontal and vertical orientation, and plot titles.
Violin
SDT2 now offers highly customizable violin plots to show the distribution of percent identities, lengths or GC content.
Windows installer
We have created a user-friendly installer using NSIS. Upon downloading of the installer, users can follow prompts for a simple install.
Save to SDT file
Data that has been analyzed can be saved to a shareable .sdt file format that will also preserve plot settings.
2.0.0.beta3.20241014
Discrete Colorscale: We have implemented an RGB 3-color bar with
programmable cutoffs, similar to the original SDT. The cutoff values set the
percent identity boundaries at which the different color scales start and end.
This option is selectable at the very bottom of the "Colorscale" dropdown in
the heatmap plot.
Error Reporting: Users should now be prompted by a dialog box that
displays the error associated with the crash along with the option to email or
post directly to the SDT2 GitHub repository. Please note that some antivirus
software may not allow the email option unless an exception is issued.
Multiprocessing: SDT2 is capable of utilizing multi-core computing to
dramatically speed up computation times. Each new core effectively doubles
the memory requirements of an analysis. Macs and typical Windows systems
default to automatic management of the page file ("swap" on non-Windows
systems,) which allows for disk space to be used in place of memory should a
user exceed system memory. However, this should be approached with
caution. If configured incorrectly, the application—and possibly the system—
could crash. For systems with an inadequate or missing page file, we've
added error recovery during alignment that allows the user to restart
alignment with different settings. We have changed the recommended cores
dropdown to instead display the recommendation directly under the slider.
The estimates should conservatively remain below available system memory
as we have implemented fairly accurate system memory calculations, which
account for available system memory and an estimate of page file use. It is
possible, with a high enough disk allocation, to align fairly large sequences
(>160k), but sequences of this length are almost always better run on a
single core, regardless of memory estimates.
Graph Clustering: The export option offers a rudimentary graph clustering
output, and we are still debating its inclusion in the final build. It converts
the similarity output matrix to binary and then clusters by connected
components. Threshold 1 sets the initial clustering, where all sequences
that share that identity will be connected and considered as a distinct
cluster. Threshold 2 performs the same but within each established cluster.
This was originally intended to be a genus/species level clustering by
programmable input.
