From 25e3154bf145ffc22167dbaa5c4862f5d6dbf3c2 Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Wed, 19 Feb 2025 13:20:51 +0000 Subject: [PATCH 01/12] [skip ci] :memo: Improve Documentation - Update CONTRIBUTING.rst --- CONTRIBUTING.rst | 56 +++++++++++++++++++++--------------------------- 1 file changed, 25 insertions(+), 31 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 65fe4d801..52b649a6e 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -1,11 +1,10 @@ .. highlight:: shell -============ +============= Contributing -============ +============= -Contributions are welcome, and they are greatly appreciated! Every little bit -helps, and credit will always be given. +Contributions are welcome and greatly appreciated! Every little bit helps, and credit will always be given. You can contribute in many ways: @@ -26,21 +25,17 @@ If you are reporting a bug, please include: Fix Bugs ~~~~~~~~ -Look through the GitHub issues for bugs. Anything tagged with "bug" and "help -wanted" is open to whoever wants to implement it. +Look through the GitHub issues for bugs. Anything tagged with "bug" and "help wanted" is open to whoever wants to implement it. Implement Features ~~~~~~~~~~~~~~~~~~ -Look through the GitHub issues for features. Anything tagged with "enhancement" -and "help wanted" is open to whoever wants to implement it. +Look through the GitHub issues for features. Anything tagged with "enhancement" and "help wanted" is open to whoever wants to implement it. Write Documentation ~~~~~~~~~~~~~~~~~~~ -TIA Toolbox could always use more documentation, whether as part of the -official TIA Toolbox docs, in docstrings, or even on the web in blog posts, -articles, and such. +TIA Toolbox could always use more documentation, whether as part of the official TIA Toolbox docs, in docstrings, or even on the web in blog posts, articles, and such. Submit Feedback ~~~~~~~~~~~~~~~ @@ -50,9 +45,8 @@ The best way to send feedback is to file an issue at https://github.com/TissueIm If you are proposing a feature: * Explain in detail how it would work. -* Keep the scope as narrow as possible, to make it easier to implement. -* Remember that this is a volunteer-driven project, and that contributions - are welcome :) +* Keep the scope as narrow as possible to make it easier to implement. +* Remember that this is a volunteer-driven project, and contributions are welcome :) Get Started! ------------ @@ -64,7 +58,7 @@ Ready to contribute? Here's how to set up ``tiatoolbox`` for local development. $ git clone git@github.com:your_name_here/tiatoolbox.git -3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: +3. Install your local copy into a virtual environment. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development:: $ mkvirtualenv tiatoolbox $ cd tiatoolbox/ @@ -76,13 +70,18 @@ Ready to contribute? Here's how to set up ``tiatoolbox`` for local development. Now you can make your changes locally. -5. When you're done making changes, check that your changes pass flake8 and the - tests:: +5. When you're done making changes, check that your changes pass pre-commit and the tests:: - $ flake8 tiatoolbox tests + $ pre-commit run --all-files $ python setup.py test or pytest - To get flake8, just pip install them into your virtualenv. + To get `pre-commit `_, just pip install it into your virtual environment using:: + + $ pip install pre-commit + + To set up the git hook for pre-commit, run the following command after installing pre-commit:: + + $ pre-commit install 6. Commit your changes and push your branch to GitHub:: @@ -98,20 +97,15 @@ Pull Request Guidelines Before you submit a pull request, check that it meets these guidelines: 1. The pull request should include tests. -2. If the pull request adds functionality, the docs should be updated. Put - your new functionality into a function with a docstring, and add the - feature to the list in README.rst. -3. The pull request should work for Python 3.8, 3.9 and 3.10, and for PyPy. Check - https://travis-ci.com/tialab/tiatoolbox/pull_requests - and make sure that the tests pass for all supported Python versions. +2. If the pull request adds functionality, the docs should be updated. Put your new functionality into a function with a docstring, and add the feature to the list in README.rst. +3. The pull request should work for Python 3.9, 3.10, 3.11, and 3.12, and for PyPy. Check https://github.com/TissueImageAnalytics/tiatoolbox/actions/workflows/python-package.yml and make sure that the tests pass for all supported Python versions. Tips ---- To run a subset of tests:: -$ pytest tests.test_tiatoolbox - + $ pytest tests.test_tiatoolbox Deploying --------- @@ -120,8 +114,8 @@ A reminder for the maintainers on how to deploy. Make sure all your changes are committed (including an entry in HISTORY.rst). Then run:: -$ poetry version patch # use: "poetry version --help" for other options -$ git push -$ git push --tags + $ poetry version patch # use: "poetry version --help" for other options + $ git push + $ git push --tags -Travis will then deploy to PyPI if tests pass. +GitHub Actions will then deploy to PyPI if tests pass. From 9e783520269faea6e589a2bd42b5046a749e8b96 Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Wed, 19 Feb 2025 13:52:10 +0000 Subject: [PATCH 02/12] [skip ci] :memo: Update examples/README.md --- examples/README.md | 95 ++++++++++++++++++++-------------------------- 1 file changed, 42 insertions(+), 53 deletions(-) diff --git a/examples/README.md b/examples/README.md index 376b61c11..f625b0b8c 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,113 +1,102 @@ # About the Example Notebooks -In this directory, you will find some example use cases of the TIAToolbox functionalities in the form of Jupyter Notebooks. All of these notebooks are designed and maintained to run on Colab platforms (unless otherwise stated) but you can run them on your own system as well. In the first cell of each example notebook, there are links to open the notebook in Google Colab and GitHub. Simply clicking on one of these links opens that notebook on the selected platform. You can right-click on "Open in Colab" and select "Open in new tab" if the left click does not work for you. +In this directory, you will find example use cases of TIAToolbox functionalities in the form of Jupyter Notebooks. These notebooks are designed to run on Google Colab (unless otherwise stated), but you can also run them on your local system. Each example notebook includes links to open the notebook in Google Colab and GitHub. Simply click on one of these links to open the notebook on the selected platform. If the left click does not work, you can right-click on "Open in Colab" and select "Open in new tab". -## Local Machine +## Running Notebooks Locally -To run the notebook on other platforms, such as your own computer, set up your Python environment, as explained in the [installation guide](../docs/installation.rst). +To run the notebooks on your local machine, set up your Python environment as explained in the [installation guide](../docs/installation.rst). -## [Google Colab](https://colab.research.google.com/) +## Running Notebooks on [Google Colab](https://colab.research.google.com/) -Each notebook contains all the information needed to run the example remotely on Colab. All that you need is a local computer at your fingertips, with a standard browser connected to the Internet. The local computer does not even need a programming language to be installed—all the crucial resources are provided at the remote site, free of charge, thanks to Google Colaboratory. Check that "colab" appears in the address bar. Familiarize yourself with the drop-down menus near the top of the window. You can edit the notebook during the session, for example substituting your own image files for the image files used in this demo. Experiment by changing the parameters of functions. It is not possible for an ordinary user to permanently change this version of the notebook on GitHub or Colab, so you cannot inadvertently mess it up. Use the notebook's File Menu if you wish to save your own (changed) notebook. +Each notebook contains all the information needed to run the example remotely on Colab. All you need is a local computer with a standard browser connected to the Internet. The local computer does not need to have any programming languages installed—all necessary resources are provided at the remote site, free of charge, thanks to Google Colaboratory. Ensure that "colab" appears in the address bar. Familiarize yourself with the drop-down menus near the top of the window. You can edit the notebook during the session, for example, by substituting your own image files for those used in the demo. Experiment by changing the parameters of functions. You cannot permanently change this version of the notebook on GitHub or Colab, so you cannot inadvertently mess it up. Use the notebook's File Menu if you wish to save your own (changed) version. -### GPU or CPU runtime +### GPU or CPU Runtime -Processes in the notebooks can be accelerated by using a GPU. Therefore, whether you are running the notebook on your system or Colab, you need to check and specify if you are using GPU or CPU hardware acceleration. In Colab, you need to make sure that the runtime type is set to GPU in the *"Runtime→Change runtime type→Hardware accelerator"*. If you are *not* using GPU, consider changing the `ON_GPU` flag to `False` value, otherwise, some errors will be raised when running the following cells. +Processes in the notebooks can be accelerated by using a GPU. Whether you are running the notebook on your system or Colab, you need to check and specify if you are using GPU or CPU hardware acceleration. In Colab, ensure that the runtime type is set to GPU in the *"Runtime → Change runtime type → Hardware accelerator"*. If you are *not* using GPU, consider changing the `device` variable to `cpu`, otherwise, some errors may occur when running the following cells. -> **IMPORTANT**: If you are using Colab and install tiatoolbox, please note that you need to restart the runtime after tiatoolbox installation before proceeding through (menu) *"Runtime→Restart runtime"* . This is needed to load the latest versions of prerequisite packages installed with TIAToolbox. Doing so, you should be able to run all the remaining cells altogether (*"Runtime→Run after"* from the next cell) or one by one. +> **IMPORTANT**: If you are using Colab and install TIAToolbox, please note that you need to restart the runtime after installation before proceeding (menu: *"Runtime → Restart runtime"*). This is necessary to load the latest versions of prerequisite packages installed with TIAToolbox. After restarting, you should be able to run all the remaining cells together (*"Runtime → Run after"* from the next cell) or one by one. -## Structure of the examples directory +## Structure of the Examples Directory -We explain here the structure of the examples directory and briefly describe our notebooks. (Except for technical minutiae, the words _folder_ and _directory_ are interchangeable.) The examples directory includes general notebooks explaining different functionalities/modules incorporated in TIAToolbox. Most of these notebooks are written with less advanced users in mind—some familiarity with Python is assumed—but the capabilities they demonstrate would also be useful to more advanced users. +The examples directory includes general notebooks explaining different functionalities/modules incorporated in TIAToolbox. Most of these notebooks are written with less advanced users in mind—some familiarity with Python is assumed—but the capabilities they demonstrate are also useful to more advanced users. -The example directory contains two subdirectories called `full-pipelines` and `inference-pipelines` that include examples of using TIAToolbox in training of neural networks or inference of WSIs for high-level CPath applications, such as patient survival prediction and MSI status prediction from H&E whole slide images. +The examples directory contains two subdirectories: `full-pipelines` and `inference-pipelines`, which include examples of using TIAToolbox for training neural networks or inference of WSIs for high-level computational pathology applications, such as patient survival prediction and MSI status prediction from H&E whole slide images. -## A) Examples of TIAToolbox functionalities +## A) Examples of TIAToolbox Functionalities -We now give a list of our Jupyter notebooks, giving a brief description of the TIAToolbox functionalities that each of the notebook provides. +Below is a list of our Jupyter notebooks, with brief descriptions of the TIAToolbox functionalities each notebook provides. -### 1- Reading Whole Slide Images ([01-wsi-reading](./01-wsi-reading.ipynb)) +### 1. Reading Whole Slide Images ([01-wsi-reading](./01-wsi-reading.ipynb)) -This notebook shows how to use TIAToolbox to read different kinds of WSIs. TIAToolbox provides a uniform interface to various WSI formats. To see what formats are dealt with, click [here](https://tia-toolbox.readthedocs.io/en/latest/usage.html?highlight=wsiread#tiatoolbox.wsicore.wsireader.WSIReader) and then search for _format_. In this notebook, you will learn some well-known techniques for WSI masking and patch extraction. +This notebook shows how to use TIAToolbox to read different kinds of WSIs. TIAToolbox provides a uniform interface to various WSI formats. Learn some well-known techniques for WSI masking and patch extraction. [![image](../docs/images/wsi-reading.png)](./01-wsi-reading.ipynb) -### 2- Stain normalization of histology images ([02-stain-normalization](./02-stain-normalization.ipynb)) +### 2. Stain Normalization of Histology Images ([02-stain-normalization](./02-stain-normalization.ipynb)) -Stain normalization is a common pre-processing step in computational pathology, whose objective is to reduce, as much as possible, colour variation that has no clinical significance. This variation may be caused by using different scanners, different staining protocols and practices, staining agents that have been left on the laboratory shelf for different lengths of time, different settings when using the scanner, etc. It has been shown in many studies that stain normalization can make an algorithm more robust against such differences. TIAToolbox makes a few different stain-normalization algorithms available to the use. The implemented stain normalization methods in TIAToolbox are: - -- Reinhard stain normalization -- Ruifork -- Macenko -- Vahadane. - -Alternatively, if you prefer, you can use your own stain matrix for stain normalization. In the images below, the object of an algorithm is to change to source image to make its colours similar to those in the target image. +Stain normalization is a common pre-processing step in computational pathology to reduce color variation that has no clinical significance. TIAToolbox offers several stain-normalization algorithms, including Reinhard, Ruifork, Macenko, and Vahadane. [![image](../docs/images/stain-norm-example.png)](./02-stain-normalization.ipynb)) -### 3- Extracting tissue mask (tissue region) from whole slide images ([03-tissue-masking](./03-tissue-masking.ipynb)) +### 3. Extracting Tissue Mask from Whole Slide Images ([03-tissue-masking](./03-tissue-masking.ipynb)) -WSIs often show large blank (glass) background areas that contain no information. Therefore, it is essential to detect the informative (tissue) region in the WSI before taking any action (like patch extraction and classification). We call this step, "tissue masking" which is the focus of this example notebook. This notebook shows how to extract tissue regions from a WSI with the help of TIAToolbox and a single line of Python code. +This notebook shows how to extract tissue regions from a WSI using TIAToolbox with a single line of Python code. [![image](../docs/images/tissue-mask.png)](./03-tissue-masking.ipynb) -### 4- Extracting patches from whole slide images ([04-patch-extraction](./04-patch-extraction.ipynb)) +### 4. Extracting Patches from Whole Slide Images ([04-patch-extraction](./04-patch-extraction.ipynb)) -This notebook uses TIAToolbox to extract patches from a large histology image. Tiatoolbox can extract patches based on point annotations or using a fixed-size sliding window. The patch extraction module of TIAToolbox supports mask-based patch extraction which means you can extract (overlapping, if you prefer) patches from a certain region of WSI (for example a region consisting of a particular type of tissue). +Learn how to use TIAToolbox to extract patches from large histology images based on point annotations or using a fixed-size sliding window. [![image](../docs/images/patch-extraction.png)](./04-patch-extraction.ipynb) -### 5- Patch prediction in whole slide images ([05-patch-prediction](./05-patch-prediction.ipynb)) +### 5. Patch Prediction in Whole Slide Images ([05-patch-prediction](./05-patch-prediction.ipynb)) -In this notebook, we use TIAToolbox for patch-level prediction, using a range of deep learning models. TIAToolbox can be used to make predictions on pre-extracted image patches or on larger image tiles / whole-slide images (WSIs), where image patches are extracted on the fly. There are various state-of-the-art deep learning models implemented in TIAToolbox, pretrained on datasets related to different cancer types. With just two lines of Python code, any of these models can be used out of the box to predict the type of patches in a WSI. For example, in colorectal cancer, TIAToolbox can classify whole slide image regions into nine different categories (Empty glass, Lymphocytes, Normal colon mucosa, Debris, Smooth muscle, Cancer-associated stroma, Adipose, Mucus, Colorectal adenocarcinoma epithelium). +Use TIAToolbox for patch-level prediction with a range of deep learning models. Predict the type of patches in a WSI with just two lines of Python code. [![image](../docs/images/patch-prediction.png)](./05-patch-prediction.ipynb) -### 6- Semantic segmentation of whole slide images ([06-semantic-segmentation](./06-semantic-segmentation.ipynb)) +### 6. Semantic Segmentation of Whole Slide Images ([06-semantic-segmentation](./06-semantic-segmentation.ipynb)) -_Semantic segmentation_ groups together similar parts of an image that belong to the same class, as in the image immediately above and in the image below. Semantic segmentation of tissue regions plays an important role in developing algorithms for cancer diagnosis and prognosis, as it can help measure tissue attributes in an objective and reproducible fashion. In this notebook, we use pretrained models to automatically segment different tissue region types in a set of input images or WSIs. We first use a pretrained model to semantically annotate images of breast cancer, needing only two lines of codes to do so. After that, we explain how to use a pretrained TIAToolbox model inference pipeline to do prediction on a set of WSIs. +Use pretrained models to automatically segment different tissue region types in WSIs. [![image](../docs/images/sematic-segment.png)](./06-semantic-segmentation.ipynb) -### 7- Advanced model techniques ([07-advanced-modeling](./07-advanced-modeling.ipynb)) +### 7. Advanced Model Techniques ([07-advanced-modeling](./07-advanced-modeling.ipynb)) -This notebook is aimed at advanced users of TIAToolbox, familiar with object-oriented programming concepts in Python and the TIAToolbox models framework. We demonstrate the use of TIAToolbox models with your current workflow and how you can integrate your solutions into the TIAToolbox model framework. By doing so, you will be able to utilize extensively tested TIAToolbox tools in your experiments and speed up your computational pathology research. +This notebook is aimed at advanced users familiar with object-oriented programming concepts in Python and the TIAToolbox models framework. [![image](../docs/images/advanced-techniques.png)](./07-advanced-modeling.ipynb) -### 8- Nucleus instance segmentation in whole slide images using the HoVer-Net model ([08-nucleus-instance-segmentation](./08-nucleus-instance-segmentation.ipynb)) +### 8. Nucleus Instance Segmentation in Whole Slide Images Using the HoVer-Net Model ([08-nucleus-instance-segmentation](./08-nucleus-instance-segmentation.ipynb)) -Each WSI can contain up to a million nuclei of various types. These can analysed systematically and used for predicting clinical outcomes. Nucleus segmentation and classification must be carried out before using nuclear features in downstream analysis. In this example, we will demonstrate the use of the TIAToolbox implementation of the [HoVer-Net model](https://www.sciencedirect.com/science/article/pii/S1361841519301045) to solve the problem of nucleus instance segmentation and classification. +Demonstrate the use of the TIAToolbox implementation of the HoVer-Net model for nucleus instance segmentation and classification. [![image](../docs/images/hovernet.png)](./08-nucleus-instance-segmentation.ipynb) -### 9- Multi-task segmentation in whole slide images using the HoVer-Net+ model ([09-multi-task-segmentation](./09-multi-task-segmentation.ipynb)) +### 9. Multi-task Segmentation in Whole Slide Images Using the HoVer-Net+ Model ([09-multi-task-segmentation](./09-multi-task-segmentation.ipynb)) -Each WSI consists of a multitude of different tissue types, each containing many nuclei of varying types. In computational pathology, it is often important to generate tissue specific morphological features for downstream analyses. It can therefore be beneficial to perform multiple tasks such as semantic segmentation of tissue regions and nuclear instance segmentation/classification simultaneously in order to exploit useful information learnt from each task to further advance both tasks. If inflammatory cells are more common in connective tissue, and epithelial cells are more common in the epithelium, then it is logical that performing these tasks simultaneously may be beneficial to each task. In this example, we will demonstrate the use of the TIAToolbox implementation of the [HoVer-Net+ model](https://arxiv.org/pdf/2108.13904.pdf) to solve the problem of nucleus instance segmentation/classification and the semantic segmentation of intra-epithelial layers. +Demonstrate the use of the TIAToolbox implementation of the HoVer-Net+ model for nucleus instance segmentation/classification and semantic segmentation of intra-epithelial layers. [![image](../docs/images/hovernetplus.png)](./09-multi-task-segmentation.ipynb) -### 10- Image Alignment ([10-wsi_registration](./10-wsi-registration.ipynb)) - -This notebook presents an example to show how to use TIAToolbox for registration of an image pair using [Deep Feature Based Registration](https://arxiv.org/pdf/2202.09971.pdf) (DFBR) [1], followed by non-rigid alignment using [SimpleITK](https://simpleitk.readthedocs.io/en/master/registrationOverview.html). The registration tool in the TIAToolbox also comprises a pre-alignment step, a pre-requisite to DFBR. In particular, we will introduce the use of our registration tool `wsi_registration`. - -In this example, the affine transformation is computed using thumbnails of the fixed and moving images. The estimated transformation is then used to extract corresponding tiles from both fixed and moving images at a higher magnification level. The non-rigid deformation between the two tiles is then dealt with using the SimpleITK. +### 10. Image Alignment ([10-wsi_registration](./10-wsi-registration.ipynb)) -[1] Awan, Ruqayya, et al. "Deep Feature based Cross-slide Registration." arXiv preprint arXiv:2202.09971 (2022). +Show how to use TIAToolbox for registration of an image pair using Deep Feature Based Registration (DFBR) followed by non-rigid alignment using SimpleITK. [![image](../docs/images/wsi-registration.png)](./10-wsi-registration.ipynb) -### 11- Feature Extraction using Foundation Models ([11-import-foundation-models](./11-import-foundation-models.ipynb)) +### 11. Feature Extraction Using Foundation Models ([11-import-foundation-models](./11-import-foundation-models.ipynb)) -This Jupyter notebook in TIAToolbox explains how to extract features from whole slide images (WSI) using pre-trained models from the `timm` library. It guides users through selecting appropriate model architectures, visualizing the extracted features using `UMAP` feature embedding, and verifying the model's performance by checking if different tissue types are correctly identified and separated in the feature map. +Explain how to extract features from WSIs using pre-trained models from the `timm` library. [![image](../docs/images/feature_extraction.png)](./11-import-foundation-models.ipynb) -## B) Examples of high-level analysis (pipelines) using TIAToolbox +## B) Examples of High-Level Analysis (Pipelines) Using TIAToolbox -List of Jupyter notebooks that demonstrate how you can use TIAToolbox to simplify high-level analysis in computational pathology. +List of Jupyter notebooks demonstrating how to use TIAToolbox for high-level analysis in computational pathology. -### 1- Prediction of molecular pathways and key mutations in colorectal cancer from whole slide images ([idars](./inference-pipelines/idars.ipynb)) +### 1. Prediction of Molecular Pathways and Key Mutations in Colorectal Cancer from Whole Slide Images (idars) Prediction of molecular pathways and key mutations directly from Haematoxylin and Eosin stained histology images can help bypass additional genetic (e.g., polymerase chain reaction or PCR) or immunohistochemistry (IHC) testing, with a view to saving both money and time. In this notebook, we use TIAToolbox's pretrained models to reproduce the inference results obtained by the IDaRS pipeline due to Bilal et al. In TIAToolbox, we include models that are capable of predicting the following in whole slide images: @@ -120,11 +109,11 @@ Prediction of molecular pathways and key mutations directly from Haematoxylin an [![image](../docs/images/idars.png)](./inference-pipelines/idars.ipynb) -### 2- Prediction of HER2 status in breast cancer from H&E stained whole slide images +### 2. Prediction of HER2 Status in Breast Cancer from H&E Stained Whole Slide Images -This notebook demonstrates how the functionalities available in TIAToolbox can be used to reproduce the ["SlideGraph+ method" ("SlideGraph+: Whole Slide Image-Level Graphs to Predict HER2Status in Breast Cancer" by Lu et al. (2021))](https://arxiv.org/abs/2110.06042) to predict HER2 status of breast cancer samples from H&E stained whole slide images. As a brief overview, this method involves several steps generating a graph that represents a WSI. The graph is then fed into a special convolutional graph network, called SlideGraph, to predict whether that WSI is HER2 negative or positive. +Demonstrate how to use TIAToolbox to reproduce the SlideGraph+ method to predict HER2 status of breast cancer samples from H&E stained WSIs. -- Example notebook on training the SlideGraph model is available here: [slide-graph for training](./full-pipelines/slide-graph.ipynb) -- Example notebook on using SlideGraph model for WSI inference is available here: [slide-graph for inference](./inference-pipelines/slide-graph.ipynb) +- Example notebook on training the SlideGraph model: slide-graph for training +- Example notebook on using SlideGraph model for WSI inference: slide-graph for inference [![image](../docs/images/her2-prediction-example.png)](./inference-pipelines/slide-graph.ipynb) From d3f8b688b924e43e1c8908c9e85fc220ea328519 Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Wed, 19 Feb 2025 15:30:03 +0000 Subject: [PATCH 03/12] [skip ci] :memo: Update examples/README.md --- examples/README.md | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/examples/README.md b/examples/README.md index f625b0b8c..b39cc9e00 100644 --- a/examples/README.md +++ b/examples/README.md @@ -28,31 +28,38 @@ Below is a list of our Jupyter notebooks, with brief descriptions of the TIATool ### 1. Reading Whole Slide Images ([01-wsi-reading](./01-wsi-reading.ipynb)) -This notebook shows how to use TIAToolbox to read different kinds of WSIs. TIAToolbox provides a uniform interface to various WSI formats. Learn some well-known techniques for WSI masking and patch extraction. +This notebook shows how to use TIAToolbox to read different kinds of WSIs. TIAToolbox provides a uniform interface to various WSI formats. To see what formats are dealt with, click [here](https://tia-toolbox.readthedocs.io/en/latest/usage.html?highlight=wsiread#tiatoolbox.wsicore.wsireader.WSIReader) and then search for _format_. In this notebook, you will learn some well-known techniques for WSI masking and patch extraction. [![image](../docs/images/wsi-reading.png)](./01-wsi-reading.ipynb) ### 2. Stain Normalization of Histology Images ([02-stain-normalization](./02-stain-normalization.ipynb)) -Stain normalization is a common pre-processing step in computational pathology to reduce color variation that has no clinical significance. TIAToolbox offers several stain-normalization algorithms, including Reinhard, Ruifork, Macenko, and Vahadane. +Stain normalization is a common pre-processing step in computational pathology to reduce color variation that has no clinical significance. This variation may be caused by using different scanners, different staining protocols and practices, staining agents that have been left on the laboratory shelf for different lengths of time, different settings when using the scanner, etc. It has been shown in many studies that stain normalization can make an algorithm more robust against such differences. TIAToolbox offers several stain-normalization algorithms, including: + +- Reinhard stain normalization +- Ruifork +- Macenko +- Vahadane. + +Alternatively, if you prefer, you can use your own stain matrix for stain normalization. In the images below, the object of an algorithm is to change to source image to make its colours similar to those in the target image. [![image](../docs/images/stain-norm-example.png)](./02-stain-normalization.ipynb)) ### 3. Extracting Tissue Mask from Whole Slide Images ([03-tissue-masking](./03-tissue-masking.ipynb)) -This notebook shows how to extract tissue regions from a WSI using TIAToolbox with a single line of Python code. +We call this step, "tissue masking" which is the focus of this example notebook. This notebook shows how to extract tissue regions from a WSI using TIAToolbox with a single line of Python code. WSIs often show large blank (glass) background areas that contain no information. Therefore, it is essential to detect the informative (tissue) region in the WSI before taking any action (like patch extraction and classification). [![image](../docs/images/tissue-mask.png)](./03-tissue-masking.ipynb) ### 4. Extracting Patches from Whole Slide Images ([04-patch-extraction](./04-patch-extraction.ipynb)) -Learn how to use TIAToolbox to extract patches from large histology images based on point annotations or using a fixed-size sliding window. +Learn how to use TIAToolbox to extract patches from large histology images based on point annotations or using a fixed-size sliding window. The patch extraction module of TIAToolbox supports mask-based patch extraction which means you can extract (overlapping, if you prefer) patches from a certain region of WSI (for example a region consisting of a particular type of tissue). [![image](../docs/images/patch-extraction.png)](./04-patch-extraction.ipynb) ### 5. Patch Prediction in Whole Slide Images ([05-patch-prediction](./05-patch-prediction.ipynb)) -Use TIAToolbox for patch-level prediction with a range of deep learning models. Predict the type of patches in a WSI with just two lines of Python code. +Use TIAToolbox for patch-level prediction with a range of deep learning models. Predict the type of patches in a WSI with just two lines of Python code. TIAToolbox can be used to make predictions on pre-extracted image patches or on larger image tiles / whole-slide images (WSIs), where image patches are extracted on the fly. For example, in colorectal cancer, TIAToolbox can classify whole slide image regions into nine different categories (Empty glass, Lymphocytes, Normal colon mucosa, Debris, Smooth muscle, Cancer-associated stroma, Adipose, Mucus, Colorectal adenocarcinoma epithelium). [![image](../docs/images/patch-prediction.png)](./05-patch-prediction.ipynb) @@ -82,13 +89,15 @@ Demonstrate the use of the TIAToolbox implementation of the HoVer-Net+ model for ### 10. Image Alignment ([10-wsi_registration](./10-wsi-registration.ipynb)) -Show how to use TIAToolbox for registration of an image pair using Deep Feature Based Registration (DFBR) followed by non-rigid alignment using SimpleITK. +Shows how to use TIAToolbox for registration of an image pair using [Deep Feature Based Registration (DFBR)](https://arxiv.org/pdf/2202.09971.pdf) followed by non-rigid alignment using [SimpleITK](https://simpleitk.readthedocs.io/en/master/registrationOverview.html). The registration tool in the TIAToolbox also comprises a pre-alignment step, a pre-requisite to DFBR. In this example, the affine transformation is computed using thumbnails of the fixed and moving images. The estimated transformation is then used to extract corresponding tiles from both fixed and moving images at a higher magnification level. The non-rigid deformation between the two tiles is then dealt with using the SimpleITK. + +[1] Awan, Ruqayya, et al. "Deep Feature based Cross-slide Registration." arXiv preprint arXiv:2202.09971 (2022). [![image](../docs/images/wsi-registration.png)](./10-wsi-registration.ipynb) ### 11. Feature Extraction Using Foundation Models ([11-import-foundation-models](./11-import-foundation-models.ipynb)) -Explain how to extract features from WSIs using pre-trained models from the `timm` library. +Explains how to extract features from WSIs using pre-trained models from the `timm` library. The notebook guides users through selecting appropriate model architectures, visualizing the extracted features using `UMAP` feature embedding, and verifying the model's performance by checking if different tissue types are correctly identified and separated in the feature map. [![image](../docs/images/feature_extraction.png)](./11-import-foundation-models.ipynb) @@ -111,9 +120,9 @@ Prediction of molecular pathways and key mutations directly from Haematoxylin an ### 2. Prediction of HER2 Status in Breast Cancer from H&E Stained Whole Slide Images -Demonstrate how to use TIAToolbox to reproduce the SlideGraph+ method to predict HER2 status of breast cancer samples from H&E stained WSIs. +Demonstrates how to use TIAToolbox to reproduce the ["SlideGraph+ method" ("SlideGraph+: Whole Slide Image-Level Graphs to Predict HER2Status in Breast Cancer" by Lu et al. (2021))](https://arxiv.org/abs/2110.06042) to predict HER2 status of breast cancer samples from H&E stained WSIs. As a brief overview, this method involves several steps generating a graph that represents a WSI. The graph is then fed into a special convolutional graph network, called SlideGraph, to predict whether that WSI is HER2 negative or positive. -- Example notebook on training the SlideGraph model: slide-graph for training -- Example notebook on using SlideGraph model for WSI inference: slide-graph for inference +- Example notebook on training the SlideGraph model: [slide-graph for training](./full-pipelines/slide-graph.ipynb) +- Example notebook on using SlideGraph model for WSI inference: [slide-graph for inference](./inference-pipelines/slide-graph.ipynb) [![image](../docs/images/her2-prediction-example.png)](./inference-pipelines/slide-graph.ipynb) From 034342db4230622a997d22d56bfa54224be21279 Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Wed, 19 Feb 2025 15:35:56 +0000 Subject: [PATCH 04/12] [skip ci] :memo: Update examples/README.md --- examples/README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/examples/README.md b/examples/README.md index b39cc9e00..7bfd10097 100644 --- a/examples/README.md +++ b/examples/README.md @@ -47,7 +47,7 @@ Alternatively, if you prefer, you can use your own stain matrix for stain normal ### 3. Extracting Tissue Mask from Whole Slide Images ([03-tissue-masking](./03-tissue-masking.ipynb)) -We call this step, "tissue masking" which is the focus of this example notebook. This notebook shows how to extract tissue regions from a WSI using TIAToolbox with a single line of Python code. WSIs often show large blank (glass) background areas that contain no information. Therefore, it is essential to detect the informative (tissue) region in the WSI before taking any action (like patch extraction and classification). +WSIs often show large blank (glass) background areas that contain no information. Therefore, it is essential to detect the informative (tissue) region in the WSI before taking any action (like patch extraction and classification). We call this step, "tissue masking" which is the focus of this example notebook. This notebook shows how to extract tissue regions from a WSI using TIAToolbox with a single line of Python code. [![image](../docs/images/tissue-mask.png)](./03-tissue-masking.ipynb) @@ -65,25 +65,25 @@ Use TIAToolbox for patch-level prediction with a range of deep learning models. ### 6. Semantic Segmentation of Whole Slide Images ([06-semantic-segmentation](./06-semantic-segmentation.ipynb)) -Use pretrained models to automatically segment different tissue region types in WSIs. +Use pretrained models to automatically segment different tissue region types in WSIs. _Semantic segmentation_ groups together similar parts of an image that belong to the same class, as in the image immediately above and in the image below. Semantic segmentation of tissue regions plays an important role in developing algorithms for cancer diagnosis and prognosis, as it can help measure tissue attributes in an objective and reproducible fashion. [![image](../docs/images/sematic-segment.png)](./06-semantic-segmentation.ipynb) ### 7. Advanced Model Techniques ([07-advanced-modeling](./07-advanced-modeling.ipynb)) -This notebook is aimed at advanced users familiar with object-oriented programming concepts in Python and the TIAToolbox models framework. +This notebook is aimed at advanced users familiar with object-oriented programming concepts in Python and the TIAToolbox models framework. We demonstrate the use of TIAToolbox models with your current workflow and how you can integrate your solutions into the TIAToolbox model framework. By doing so, you will be able to utilize extensively tested TIAToolbox tools in your experiments and speed up your computational pathology research. [![image](../docs/images/advanced-techniques.png)](./07-advanced-modeling.ipynb) ### 8. Nucleus Instance Segmentation in Whole Slide Images Using the HoVer-Net Model ([08-nucleus-instance-segmentation](./08-nucleus-instance-segmentation.ipynb)) -Demonstrate the use of the TIAToolbox implementation of the HoVer-Net model for nucleus instance segmentation and classification. +Demonstrates the use of the TIAToolbox implementation of the [HoVer-Net model](https://www.sciencedirect.com/science/article/pii/S1361841519301045) model for nucleus instance segmentation and classification. Each WSI can contain up to a million nuclei of various types. These can analysed systematically and used for predicting clinical outcomes. Nucleus segmentation and classification must be carried out before using nuclear features in downstream analysis. [![image](../docs/images/hovernet.png)](./08-nucleus-instance-segmentation.ipynb) ### 9. Multi-task Segmentation in Whole Slide Images Using the HoVer-Net+ Model ([09-multi-task-segmentation](./09-multi-task-segmentation.ipynb)) -Demonstrate the use of the TIAToolbox implementation of the HoVer-Net+ model for nucleus instance segmentation/classification and semantic segmentation of intra-epithelial layers. +Demonstrates the use of the TIAToolbox implementation of the [HoVer-Net+ model](https://arxiv.org/pdf/2108.13904.pdf) model for nucleus instance segmentation/classification and semantic segmentation of intra-epithelial layers. Each WSI consists of a multitude of different tissue types, each containing many nuclei of varying types. In computational pathology, it is often important to generate tissue specific morphological features for downstream analyses. It can therefore be beneficial to perform multiple tasks such as semantic segmentation of tissue regions and nuclear instance segmentation/classification simultaneously in order to exploit useful information learnt from each task to further advance both tasks. [![image](../docs/images/hovernetplus.png)](./09-multi-task-segmentation.ipynb) From 80ea8afdf466aa8de22eedaf83cb45000c2a32f7 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 19 Feb 2025 15:36:20 +0000 Subject: [PATCH 05/12] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- examples/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/README.md b/examples/README.md index 7bfd10097..9bffedc36 100644 --- a/examples/README.md +++ b/examples/README.md @@ -65,7 +65,7 @@ Use TIAToolbox for patch-level prediction with a range of deep learning models. ### 6. Semantic Segmentation of Whole Slide Images ([06-semantic-segmentation](./06-semantic-segmentation.ipynb)) -Use pretrained models to automatically segment different tissue region types in WSIs. _Semantic segmentation_ groups together similar parts of an image that belong to the same class, as in the image immediately above and in the image below. Semantic segmentation of tissue regions plays an important role in developing algorithms for cancer diagnosis and prognosis, as it can help measure tissue attributes in an objective and reproducible fashion. +Use pretrained models to automatically segment different tissue region types in WSIs. _Semantic segmentation_ groups together similar parts of an image that belong to the same class, as in the image immediately above and in the image below. Semantic segmentation of tissue regions plays an important role in developing algorithms for cancer diagnosis and prognosis, as it can help measure tissue attributes in an objective and reproducible fashion. [![image](../docs/images/sematic-segment.png)](./06-semantic-segmentation.ipynb) From 7e6b7009cd8a970ddd144f71037ada2741daa5d4 Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Wed, 19 Feb 2025 15:48:24 +0000 Subject: [PATCH 06/12] [skip ci] :bug: Fix hyperlinks to Jupyter Notebooks. --- docs/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index 91fd2df51..a16b690a8 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -2016,7 +2016,7 @@ def all_but_ipynb(dir_path, contents): # Replace the target string file_data = file_data.replace(".rst", ".html") file_data = file_data.replace(".ipynb", ".html") -file_data = file_data.replace("](./", "](./jnb/") +file_data = file_data.replace("](./", "](./_notebooks/jnb/") file_data = file_data.replace("../docs/", "./") # Write the file out again From 16b998f7c5cd953725b54379a23c905f22c3edcb Mon Sep 17 00:00:00 2001 From: Shan E Ahmed Raza <13048456+shaneahmed@users.noreply.github.com> Date: Thu, 20 Feb 2025 12:24:25 +0000 Subject: [PATCH 07/12] [skip ci] :memo: Update installation.rst --- docs/installation.rst | 95 ++++++++++++++++++++----------------------- 1 file changed, 43 insertions(+), 52 deletions(-) diff --git a/docs/installation.rst b/docs/installation.rst index 28c9911da..4739b2c86 100644 --- a/docs/installation.rst +++ b/docs/installation.rst @@ -7,28 +7,26 @@ Installation Prerequisites ============= -The prerequisites for tiatoolbox installation are OpenSlide binaries and OpenJpeg version 2.3.0 or above. -Please follow the instructions below to install prerequisite software according to the platform you are using. + +The prerequisites for TIAToolbox installation are OpenSlide binaries and OpenJPEG version 2.3.0 or above. Please follow the instructions below to install the prerequisite software according to your platform. Linux (Ubuntu) -------------- -On Linux the prerequisite software can be installed using the command + +On Linux, the prerequisite software can be installed using the following command: .. code-block:: console $ apt-get -y install libopenjp2-7-dev libopenjp2-tools openslide-tools -The same command is used when working on the Colab or Kaggle platforms. -When working on Google Colab, we remove the packages ``datascience`` and ``albumentations`` because they conflict -and produce an error message. +The same command is used when working on the Colab or Kaggle platforms. When working on Google Colab, we remove the packages ``datascience`` and ``albumentations`` because they conflict and produce an error message. Windows (10+) -------------------- -1. Download OpenSlide binaries from `this page `_. Extract the folder and add ``bin`` and ``lib`` subdirectories to -Windows `system path `_. If you are using a conda environment you can also copy ``bin`` and ``lib`` subdirectories to ``[Anaconda Installation Path]/envs/[tiatoolbox-environment]/Library/``. +------------- -2. Install OpenJPEG. The easiest way is to install OpenJpeg is through conda -using +1. Download OpenSlide binaries from `this page `_. Extract the folder and add the ``bin`` and ``lib`` subdirectories to the Windows `system path `_. If you are using a conda environment, you can also copy the ``bin`` and ``lib`` subdirectories to ``[Anaconda Installation Path]/envs/[tiatoolbox-environment]/Library/``. + +2. Install OpenJPEG. The easiest way to install OpenJPEG is through conda: .. code-block:: console @@ -37,7 +35,7 @@ using macOS ----- -On macOS there are two popular package managers, `homebrew`_ and `macports`_. +On macOS, there are two popular package managers, `homebrew`_ and `macports`_. .. _homebrew: https://brew.sh/ .. _macports: https://www.macports.org/ @@ -56,26 +54,26 @@ MacPorts $ port install openjpeg openslide - Installing Stable Release ========================= -Please note that TIAToolbox is tested for python version 3.9, 3.10, 3.11 and 3.12. +Please note that TIAToolbox is tested for Python versions 3.9, 3.10, 3.11, and 3.12. Recommended ----------- + To install TIAToolbox, run this command in your terminal after you have installed the prerequisite software: .. code-block:: console $ pip install tiatoolbox -This is the preferred method to install TIA Toolbox, as it will always install the most recent stable release. +This is the preferred method to install TIAToolbox, as it will always install the most recent stable release. Upgrade ------- -To upgrade an existing version of tiatoolbox to the latest stable release, run this command in your terminal: +To upgrade an existing version of TIAToolbox to the latest stable release, run this command in your terminal: .. code-block:: console @@ -84,21 +82,19 @@ To upgrade an existing version of tiatoolbox to the latest stable release, run t Without Dependencies -------------------- -If you already have setup a Python environment with all the pre-requisite software and dependencies installed and you would like to keep the existing versions of these dependencies, run this command in your terminal: +If you already have a Python environment set up with all the prerequisite software and dependencies installed and you would like to keep the existing versions of these dependencies, run this command in your terminal: .. code-block:: console $ pip install --no-deps tiatoolbox -If you don't have `pip`_ installed, this `Python installation guide`_ can guide -you through the process. +If you don't have `pip`_ installed, this `Python installation guide`_ can guide you through the process. .. _pip: https://pip.pypa.io .. _Python installation guide: http://docs.python-guide.org/en/latest/starting/installation/ - -Alternative Method(s) -===================== +Alternative Methods +=================== Using Anaconda -------------- @@ -115,13 +111,12 @@ or $ mamba install tiatoolbox +Please note that conda-forge installation support is limited on Windows as OpenSlide binaries are not supported on official conda channels. An alternate way to install using conda on Windows could be to install it in `WSL2 with CUDA support `_. In some cases, WSL2 runs faster on Python code, and therefore we **recommend** this option. -Please note that conda-forge installation support is limited on Windows as openslide binaries are not supported on official conda channels. An alternate way to install using conda on Windows could be to install it in `WSL2 with CUDA support `_. In some cases, WSL2 runs faster on Python codes and therefore we **recommend** this option. - -From sources +From Sources ------------ -The sources for TIA Toolbox can be downloaded from the `Github repo`_. +The sources for TIAToolbox can be downloaded from the `GitHub repo`_. You can either clone the public repository: @@ -141,69 +136,65 @@ Once you have a copy of the source, you can install it with: $ python setup.py install - -.. _Github repo: https://github.com/TissueImageAnalytics/tiatoolbox.git +.. _GitHub repo: https://github.com/TissueImageAnalytics/tiatoolbox.git .. _tarball: https://github.com/TissueImageAnalytics/tiatoolbox/tarball/master Using Docker ------------ -To run TIA toolbox in an isolated environment, use our `Docker image `_ . We host different Dockerfiles in our github repository `tiatoolbox-docker `_. Please report any issues related to the docker image in the repository `tiatoolbox-docker `_. +To run TIAToolbox in an isolated environment, use our `Docker image `_. We host different Dockerfiles in our GitHub repository `tiatoolbox-docker `_. Please report any issues related to the Docker image in the repository `tiatoolbox-docker `_. -After `installing Docker `_ (or Docker Desktop), you can use our TIA toolbox image in 3 different ways. +After `installing Docker `_ (or Docker Desktop), you can use our TIAToolbox image in three different ways. Use the Pre-Built Docker Image ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -TIAToolbox provides pre-built docker containers which can be downloaded using the instructions below: -1. Pull the Image From Github Container Registry -"""""""""""""""""""""""""""""""""""""""""""""""""""" +TIAToolbox provides pre-built Docker containers which can be downloaded using the instructions below: + +1. Pull the Image From GitHub Container Registry: + .. code-block:: console $ docker pull ghcr.io/tissueimageanalytics/tiatoolbox:latest -2. Use the Pre-Built Docker Image as a Base Image in a Dockerfile -""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -.. code-block:: console +2. Use the Pre-Built Docker Image as a Base Image in a Dockerfile: - $ FROM ghcr.io/tissueimageanalytics/tiatoolbox:latest +.. code-block:: dockerfile -Build the image locally + FROM ghcr.io/tissueimageanalytics/tiatoolbox:latest + +Build the Image Locally ^^^^^^^^^^^^^^^^^^^^^^^ -1. Navigate to the Dockerfile that you want to use, -based on the Python version and Operating System that you prefer -2. Build the -Docker image +1. Navigate to the Dockerfile that you want to use, based on the Python version and Operating System that you prefer. + +2. Build the Docker image: .. code-block:: console $ docker build -t . -3. Check that the image -has been created +3. Check that the image has been created: .. code-block:: console $ docker images -4. Deploy the image -as a Docker container +4. Deploy the image as a Docker container: .. code-block:: console $ docker run -it --rm --name -5. Connect to the -running container +5. Connect to the running container: .. code-block:: console $ docker exec -it bash -To add your own script and run it through the Docker container, first copy your script into the docker environment and then execute it. +To add your own script and run it through the Docker container, first copy your script into the Docker environment and then execute it: -.. code-block:: console +.. code-block:: dockerfile - $ COPY /path/to/