[Question] Does PaDiM support uint16?

[vmiller987]

Hello all!

I have a dataset that we received as uint16. Originally, I had converted these to 8bit and I was able to produce models with results. This was part of my initial Anomalib research and trying parts out. However, my conversion was a hard coded situation and wouldn't be easily reproducible in a production environment. I am now trying to make it work with the original uint16 tiff files. However, the predictions that are being returned are identical.

I was asked to work on this with Yolo as well, and stumbled into Yolo doesn't support 16 bit images. Which leads me this, does PaDiM support uint16?

Predictions Example

Number of Batchs: 8
Batch 1:
Predicted Score: tensor([[0.9185]])
Batch 2:
Predicted Score: tensor([[0.9185]])
Batch 3:
Predicted Score: tensor([[0.9185]])
Batch 4:
Predicted Score: tensor([[0.9185]])
Batch 5:
Predicted Score: tensor([[0.9185]])
Batch 6:
Predicted Score: tensor([[0.9185]])
Batch 7:
Predicted Score: tensor([[0.9185]])
Batch 8:
Predicted Score: tensor([[0.9185]])

[alexriedel1]

The images are read in this function:

anomalib/src/anomalib/data/utils/image.py

Line 291 in 12c1e84

def read_image(path: str | Path, as_tensor: bool = False) -> torch.Tensor | np.ndarray:

anomalib uses Pillow to read image files. Pillow doesn't supports 16 bit images, so you may need to write your custom read_image function. The CNNs that run under Padim should have no problems working with 16 bit images. However its unclear if there is any effect on the accuracy using 16 bit.

[vmiller987]

Pillow can read uint16. But I do see it states limited support.

https://pillow.readthedocs.io/en/stable/handbook/concepts.html#concept-modes

We generally use SimpleITK for our image handling. I shall work on read_image and look at making a SimpleITK version out of it. Thank you for pointing this out and giving my a direction to investigate.

[vmiller987]

Converting to SimpleITK solves my issue. I am able to train and get varying predictions again! Work wants me to try and make 16 bit work, so that is my next step as the models aren't as clean as the 8 bit models.

Thanks again for the help!

[vmiller987]

In case anyone else stumbles into this, here's my code. Roast me.

def read_image(path: str | Path, as_tensor: bool = False) -> torch.Tensor | np.ndarray:
    """Read RGB image from disk.

    Args:
        path (str | Path): Path to image file
        as_tensor (bool): If ``True``, return torch.Tensor. Defaults to ``False``

    Returns:
        torch.Tensor | np.ndarray: Image as tensor or array, normalized to [0,1]

    Examples:
        >>> image = read_image("image.jpg")
        >>> type(image)
        <class 'numpy.ndarray'>

        >>> image = read_image("image.jpg", as_tensor=True)
        >>> type(image)
        <class 'torch.Tensor'>
    """
    image = sitk.ReadImage(str(path))
    image_array = sitk.GetArrayFromImage(image)

    if image_array.ndim == 2:
        image_array = np.stack([image_array] * 3, axis=-1)  # (H, W) -> (H, W, 3)
    elif image_array.ndim == 3:
        if image_array.shape[0] <= 3:
            # Likely (C, H, W), transpose to (H, W, C)
            image_array = np.transpose(image_array, (1, 2, 0))
        if image_array.shape[2] == 1:
            image_array = np.repeat(image_array, 3, axis=2)  # (H, W, 1) -> (H, W, 3)

    if image_array.dtype == np.uint16:
        image_array = image_array.astype(np.float32) / 65535.0
    elif image_array.dtype == np.uint8:
        image_array = image_array.astype(np.float32) / 255.0

    if as_tensor:
        image_array = np.transpose(image_array, (2, 0, 1))  # (H, W, C) -> (C, H, W)
        return torch.from_numpy(image_array).to(torch.float32)

    return image_array

    # Original Implementation
    # image = Image.open(path).convert("RGB")
    # return (
    #     to_dtype(to_image(image), torch.float32, scale=True)
    #     if as_tensor
    #     else np.array(image) / 255.0
    # )