merge for v1.8 update

CHANGELOG.md

@@ -1,53 +1,99 @@
 # Changelog
 
-## [1.7.1]
-- `glycorender` version bump from `0.2.3` to `0.2.5`  (1933574)
-- upgraded `nbdev2` to `nbdev3` for the documentation (+ removed now unnecessary files) (eb3f727)
-- improved start-up time of the package (i.e., time at first import in a session) (10a39f0)
+## [1.8.0]
+- fixed `Quarto` accessing of `pyproject.toml` attributes for doc building (cd9b62f)
+
+### glycan_data
+#### loader
+##### Added ✨
+- Added new N- and O-glycomics dataset from https://pubmed.ncbi.nlm.nih.gov/41460292/ to `glycomics_data_loader` (`mouse_taysachs_N_PMID41460292` and `mouse_taysachs_O_PMID41460292`) (52c6cf9)
+- Added new N-glycomics dataset from https://pubmed.ncbi.nlm.nih.gov/39877544/ to `glycomics_data_loader` (`human_serum_N_PMID39877544`) (57f6260)
+- Added new N-glycomics dataset from https://pubmed.ncbi.nlm.nih.gov/37639587/ to `glycomics_data_loader` (`human_neutrophils_N_PMID37639587`) (5d81cc3)
+- Added new N- and O-glycomics dataset from https://www.biorxiv.org/content/10.1101/2024.11.28.625934v1 to `glycomics_data_loader` (`human_macrophages_N_2024-11-28-625934` and `human_macrophages_O_2024-11-28-625934`) (4813910, 1688897)
+- Added new N-, O-, and GSL-glycomics dataset from https://pubmed.ncbi.nlm.nih.gov/36788594/ to `glycomics_data_loader` (`human_leukemia_N_PMID36788594`, `human_leukemia_O_PMID36788594`, and `human_leukemia_GSL_PMID36788594`) (5510e55)
+- Added new N-glycomics datasets from https://pubmed.ncbi.nlm.nih.gov/39947398/ to `glycomics_data_loader` (`human_colorectal_N_PMID39947398` and `human_pbmc_cancer_N_PMID39947398`) (fdd2340, 144051a)
 
-### motif
-#### draw
 ##### Changed 🔄
-- Generic substituents will now be properly formatted in `GlycoDraw` (89eb687)
-- Unknown base monosaccharides in `GlycoDraw` now correctly default to blank hexagons (89eb687)
-- Make sure `GlycoDraw` can draw !-containing sequences (e.g., `Internal_LewisA`) even with `restrict_vocab=True` (1933574)
+- Specified wildcards in `glycomics_human_colorectal_O_PMC9254241` (e71550d)
 
 ##### Fixed 🐛
-- Make sure `reducing_end_label` is perfectly y-centered in `GlycoDraw` (7e9e980)
-- Fixed setting utf-8 as default encoding in `annotate_figure` (1933574)
+- Made sure that incomplete API access in `get_molecular_properties` does not lead to outright failure (52c6cf9)
+- `glycomics_data_loader` and other `LazyLoader` instances are now robust against duplicate column names with the `.1`, `.2` suffix (they will be stripped now) (44e8473, 1cdb270)
 
 ##### Deprecated ⚠️
 
-#### processing
+### motif
+#### annotate
 ##### Added ✨
-- Added `LacdiNAc` to the `common_names` support in Universal Input (d1140d1)
-- Added `max_specify_glycan` function to infer sequence ambiguities/uncertainties as best as possible (e2cf92a)
+- `get_k_saccharides` and `annotate_dataset` can now dynamically create enrichment motifs of the type `Sia(a2-3)Gal` or `Terminal_Sia(a2-3/6)` if multiple sialic acid types are present in input data (522b7cf)
+
+##### Fixed 🐛
+- Made sure curly bracket sequence content ("floaty bits") are correctly counted in `count_unique_subgraphs_of_size_k` (522b7cf)
+- Make sure all narrow linkage wildcards, even if not present in `linkages`, are being correctly parsed in `count_unique_subgraphs_of_size_k` (5220912)
+
+#### graph
+##### Changed 🔄
+- Added `_prefilter_labels` for more cheap checks to avoid graph operations and thus make `compare_glycans` and `subgraph_isomorphism` considerably faster (b865229)
+- Made `glycan_to_graph` function much faster (up to 10x) (750cdb1)
+- Made `graph_to_string_int` function ~40% faster (750cdb1)
+
+##### Deprecated ⚠️
+- Deprecated `evaluate_adjacency`; will be handled in-line in `glycan_to_graph` (750cdb1)
+- Deprecated `canonicalize_glycan_graph`; will be handled in-line in `graph_to_string_int` (750cdb1)
+- Deprecated `neighbor_is_branchpoint`; no longer in use (e020ffb)
+
+#### draw
+##### Changed 🔄
+- `HexN`, `dHexNAc`, and `HexA` shapes now get drawn in fewer objects/more efficiently (10da7c5)
 
 ##### Fixed 🐛
-- `canonicalize_iupac` is now more robust when handling variant modification dialects in IUPAC-condensed (i.e., not mistaking them for CSDB-linear), such as `Galβ1-3(6SGlcNAcβ1-6)GalNAcol` (046ea12)
-- `min_process_glycans` and `get_lib` now correctly handle glycans with floating modifications, such as `{6S}{Neu5Ac(a2-3)}Gal(b1-4)GlcNAc(b1-6)[Gal(b1-3)]GalNAc` (68f1e1b)
+- Fixed displaying beta-linkages instead of alpha-linkages in `annotate_figure` (e71550d)
+
+##### Deprecated ⚠️
+- Deprecated `scale_in_range`; has been in-lined instead (855a9f8)
+- Deprecated `process_repeat`; has been in-lined instead (855a9f8)
 
 #### analysis
 ##### Changed 🔄
-- `characterize_monosaccharide` is now much faster (0de71c5)
+- `get_volcano` can now also deal with input dataframes that have the `Glycan` column be the index instead (e71550d)
+- Equivalence p-values in `get_differential_expression` now also use the same sample-size adjusted alpha as regular p-values (3884125)
+- Specifying `return_plot=True` in `get_heatmap` will now also return the column names and the transformed dataframe, next to the plot object (3b72129)
+- Improved default plot styling for outputs from functions (855a9f8)
 
 ##### Fixed 🐛
-- Fixed temporary file handling in `annotate_volcano=True` in `get_volcano` (1933574)
+- CLR-transformation for paired data in `preprocess_data` now correctly uses the shared geometric mean as reference, to preserve within-pair differences (3884125)
+- Fixed equivalence p-values in `get_differential_expression` if `sets=True` (3884125)
+- CLR-transformed motif-level quantification in `preprocess_data` and `get_pca` used the glycan-level geometric mean as a reference, rather than the motif-level geometric mean, which is now fixed (c71c385)
+- `get_roc` now saves the figures for all classes, not just the last, in a set-up of `filepath` + multi-group comparison (855a9f8)
+- User-provided `random_state` values/generators are now correctly propagated through to `multi_feature_scoring` (855a9f8)
 
-#### annotate
+#### tokenization
 ##### Added ✨
-- Added new `get_minimal_ksaccharide_ambiguity` function to find the minimal needed narrow linkage wildcard to encompass all variants in dataset (8a0bbce)
+- `mz_to_composition` now has a new keyword argument `deprioritized`, which is a set of disfavored monosaccharides/modifications that will only be used if no composition can be found otherwise (i.e., less harsh than full exclusion via `filter_out`). This keyword argument is now also exposed in `mz_to_structures` (316f962)
 
+#### tokenization
 ##### Changed 🔄
-- `feature_set` options `exhaustive` and the `terminal` variants now fully lean into narrow linkage wildcards for dynamically generated wildcards (e.g., `a2-3/6`), instead of the broader `a2-?` versions, which are scoped based on the provided data (8a0bbce)
-- `get_terminal_structures` can now be used for any `size` value, not only 1 and 2 (ef353fb)
-- `annotate_dataset` will now internally use `get_terminal_structures` for the `terminal3` feature-set keyword (ef353fb)
+- `canonicalize_iupac` now is even more robust regarding typo correction (acf05e1)
 
-##### Fixed 🐛
-- Fixed topologically incorrect disaccharides in `get_terminal_structures` output (ef353fb)
+### network
+#### biosynthesis
+##### Added ✨
+- Added `build_network_from_glycans` handler to do a BFS-search to get the bulk biosynthetic network going (b865229)
+- Added `hierarchical` option (now the new default) to the keyword argument options in `plot_format` in `plot_network`, for a more organized network display (8d03348)
+- `extend_network` now has the new `auto_steps` keyword argument, which (if `to_extend` is a target composition), will calculate the minimum number of steps, cross-check it against the provided maximum as `steps`, and then iteratively extend the most favorable leaf nodes toward the target composition (f8f2fa9)
+
+##### Changed 🔄
+- `construct_network` is now more than twice as fast (a1c810c, b865229)
+- Dynamic wildcard construction in `get_differential_biosynthesis` now also creates the most parsimonious narrow wildcards, similar to `annotate` (e71550d)
+- Renamed the `Feature` column in `get_differential_biosynthesis` to `Glycan` (e71550d)
+- `extend_network` now accepts compositions in any format in the `to_extend` keyword argument, using Universal Input (18f7ba5)
+- `extend_network` now early-exits if the composition provided in `to_extend` already exists within the network, outputting the existing matching structures in the network (18f7ba5)
+- `monolink_to_enzyme` is now comma-separated instead of tab-separated and is more complete (10dc46e)
 
-### ml
-#### models
-- When using `prep_model` with `trained=True` on `SweetNet`-type models, the function now auto-corrects the `num_classes` value, if a wrong output dimension is provided (i.e., if it clashes with the trained model) (ccf2d34)
 ##### Fixed 🐛
-- Fixed warning message in `train_ml_model` about not specifying `feature_calc` (0de71c5)
+- Fixed reaction hover label in `plot_network` (8d03348)
+- Fixed a bug in `add_high_man_removal` which set the edge labels with a `lambda` function instead of a string (f2b5f99)
+
+##### Deprecated ⚠️
+- Deprecated `find_shared_virtuals`, `adjacencyMatrix_to_network`, `get_virtual_nodes`, `get_neighbors`, `create_adjacency_matrix`; now all handled in-line (a1c810c, b865229)
+- Deprecated `find_path`, `find_shortest_path`, `deorphanize_nodes`, `shells_to_edges`, which is all now handled by the new `build_network_from_glycans` (b865229)

README.md

@@ -192,13 +192,13 @@ from glycowork.motif.annotate import annotate_dataset
 out = annotate_dataset(glycans, feature_set = ['known', 'terminal', 'exhaustive'], condense=True)
 ```
 
-|  | Internal_LewisX | Internal_LewisA | H_antigen_type2 | Chitobiose | Trimannosylcore | Terminal_LacNAc_type1 | Internal_LacNAc_type2 | Terminal_LacNAc_type2 | Terminal_LacdiNAc_type2 | core_fucose | core_fucose(a1-3) | Fuc | Gal | GalNAc | GalNAcOS | GlcNAc | Man | Neu5Ac | Xyl | Fuc(a1-3/4/6)GlcNAc | Fuc(a1-4)GlcNAc | Man(a1-3/6)Man | Fuc(a1-3)GlcNAc | Man(b1-4)GlcNAc | Fuc(a1-2)Gal | Gal(b1-3/4)GlcNAc | Gal(b1-4)GlcNAc | GlcNAc(b1-4)GlcNAc | GlcNAc(b1-2)Man | Neu5Ac(a2-3)Gal | Man(a1-6)Man | Fuc(a1-6)GlcNAc | Gal(b1-3)GlcNAc | Xyl(b1-2)Man | Man(a1-3)Man | Terminal_Fuc(a1-2) | Terminal_Fuc(a1-6) | Terminal_Man(a1-6) | Terminal_Fuc(a1-4) | Terminal_GlcNAc(b1-2) | Terminal_Xyl(b1-2) | Terminal_Gal(b1-3/4) | Terminal_Fuc(a1-2/3/4/6) | Terminal_Man(a1-3/6) | Terminal_Fuc(a1-3) | Terminal_Gal(b1-3) | Terminal_Man(a1-3) | Terminal_Neu5Ac(a2-3) | Terminal_Gal(b1-4) |
+|  | Internal_LewisX | Internal_LewisA | H_antigen_type2 | Chitobiose | Trimannosylcore | Terminal_LacNAc_type1 | Internal_LacNAc_type2 | Terminal_LacNAc_type2 | Terminal_LacdiNAc_type2 | core_fucose | core_fucose(a1-3) | Fuc | Gal | GalNAc | GalNAcOS | GlcNAc | Man | Neu5Ac | Xyl | Man(b1-4)GlcNAc | GlcNAc(b1-2)Man | Fuc(a1-6)GlcNAc | Fuc(a1-4)GlcNAc | Man(a1-6)Man | Fuc(a1-3)GlcNAc | Neu5Ac(a2-3)Gal | Gal(b1-3)GlcNAc | Xyl(b1-2)Man | GlcNAc(b1-4)GlcNAc | Gal(b1-4)GlcNAc | Fuc(a1-2)Gal | Man(a1-3/6)Man | Man(a1-3)Man | Fuc(a1-3/4/6)GlcNAc | Gal(b1-3/4)GlcNAc | Terminal_Neu5Ac(a2-3) | Terminal_Gal(b1-3) | Terminal_GlcNAc(b1-2) | Terminal_Fuc(a1-3) | Terminal_Fuc(a1-2) | Terminal_Man(a1-6) | Terminal_Man(a1-3) | Terminal_Man(a1-3/6) | Terminal_Fuc(a1-2/3/4/6) | Terminal_Gal(b1-4) | Terminal_Fuc(a1-4) | Terminal_Xyl(b1-2) | Terminal_Gal(b1-3/4) | Terminal_Fuc(a1-6) |
 |----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|
-| Neu5Ac(a2-3)Gal(b1-4)\[Fuc(a1-3)\]GlcNAc(b1-2)Man(a1-3)\[Gal(b1-3)\[Fuc(a1-4)\]GlcNAc(b1-2)Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-6)\]GlcNAc | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 3 | 2 | 0 | 0 | 4 | 3 | 1 | 0 | 3 | 1 | 2 | 1 | 1 | 0 | 2 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 0 | 1 | 0 | 1 | 1 | 1 | 2 | 0 | 2 | 3 | 2 | 1 | 1 | 1 | 1 | 1 |
-| Man(a1-3)\[Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 0 | 0 | 0 | 0 | 2 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 1 | 0 | 0 |
-| Man(a1-3)\[Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 0 | 0 | 0 | 0 | 2 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 0 | 1 | 0 | 0 |
-| GlcNAc(b1-2)Man(a1-3)\[GlcNAc(b1-2)Man(a1-6)\]\[Xyl(b1-2)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-3)\]GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 4 | 3 | 0 | 1 | 1 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 1 | 2 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 2 | 1 | 0 | 1 | 2 | 1 | 0 | 1 | 0 | 0 |
-| Fuc(a1-2)Gal(b1-4)GlcNAc(b1-2)Man(a1-6)\[Gal(b1-4)GlcNAc(b1-2)Man(a1-3)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-6)\]GlcNAc | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 2 | 2 | 0 | 0 | 4 | 3 | 0 | 0 | 1 | 0 | 2 | 0 | 1 | 1 | 2 | 2 | 1 | 2 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 1 | 0 | 2 | 0 | 2 | 2 | 2 | 0 | 0 | 1 | 0 | 2 |
+| Neu5Ac(a2-3)Gal(b1-4)\[Fuc(a1-3)\]GlcNAc(b1-2)Man(a1-3)\[Gal(b1-3)\[Fuc(a1-4)\]GlcNAc(b1-2)Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-6)\]GlcNAc | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 0 | 1 | 0 | 3 | 2 | 0 | 0 | 4 | 3 | 1 | 0 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 2 | 1 | 3 | 2 | 1 | 1 | 2 | 1 | 0 | 1 | 1 | 2 | 3 | 1 | 1 | 0 | 2 | 1 |
+| Man(a1-3)\[Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
+| Man(a1-3)\[Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 3 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 2 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 0 | 0 |
+| GlcNAc(b1-2)Man(a1-3)\[GlcNAc(b1-2)Man(a1-6)\]\[Xyl(b1-2)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-3)\]GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 4 | 3 | 0 | 1 | 1 | 2 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 0 | 0 | 2 | 1 | 1 | 0 | 0 | 0 | 2 | 1 | 0 | 1 | 1 | 2 | 1 | 0 | 0 | 1 | 0 | 0 |
+| Fuc(a1-2)Gal(b1-4)GlcNAc(b1-2)Man(a1-6)\[Gal(b1-4)GlcNAc(b1-2)Man(a1-3)\]Man(b1-4)GlcNAc(b1-4)\[Fuc(a1-6)\]GlcNAc | 0 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 2 | 2 | 0 | 0 | 4 | 3 | 0 | 0 | 1 | 2 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 2 | 1 | 2 | 1 | 1 | 2 | 0 | 0 | 2 | 0 | 1 | 1 | 1 | 2 | 2 | 2 | 0 | 0 | 2 | 1 |
 | GalNAcOS(b1-4)GlcNAc(b1-2)Man(a1-3)\[GalNAc(b1-4)GlcNAc(b1-2)Man(a1-6)\]Man(b1-4)GlcNAc(b1-4)GlcNAc | 0 | 0 | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 4 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
 
 ``` python