How to calculate the metrics? such as Pixel Accuracy(PA),Intersection over Union (IoU) and Average Precision (AP) et.al.

[zhousai-zs]

Thank you for your great work!

I wanna know how to calculate the metrics, but I couldn't find them in the code. Please let me know. Thank you.

[robustsam]

Please refer to Section 3.3 in supplementary material to understand how to calculate different metrics. Thank you.

[zhousai-zs]

Thanks for your reply, Is there a corresponding code about calc metrics? And, how to generate boundingbox in dataset.py as prompt input?

[robustsam]

The code for calculating IOU, PA and Dice is shown as below. For the AP part, please refer to the official COCO evaluation code . Thank you!

def compute_pixel_accuracy(gt_mask, pred_mask):
    """
    Calculate pixel accuracy (PA) between a predicted mask and a ground truth mask.
    """

    gt_mask = gt_mask[0][0]
    pred_mask = pred_mask[0][0]
    assert gt_mask.shape == pred_mask.shape, "Shape mismatch between ground truth and predicted masks."

    gt_flat = gt_mask.ravel()
    pred_flat = pred_mask.ravel()

    correct_pixels = np.sum(gt_flat == pred_flat)
    total_pixels = gt_flat.size
    pixel_accuracy = correct_pixels / total_pixels

    return pixel_accuracy

def compute_iou(predicted_mask, ground_truth_mask):
    """
    Computes the IoU between a predicted mask and a ground truth mask.
    """
    intersection = np.logical_and(predicted_mask, ground_truth_mask).sum()
    union = np.logical_or(predicted_mask, ground_truth_mask).sum()
    iou = intersection / union if union > 0 else 0
    return iou

def compute_dice_similarity_coefficient(gt_mask, pred_mask):
    """
    Calculate Dice coefficient between a predicted mask and a ground truth mask.
    """
    if gt_mask.shape != pred_mask.shape:
        raise ValueError("Input masks must have the same shape")

    intersection = np.sum(np.logical_and(gt_mask, pred_mask)) ### TP
    union = np.sum(np.logical_or(gt_mask, pred_mask)) # Number of total pixels in masks (no matter is gt or pred)

    dsc = (2.0 * intersection) / (union + intersection) ### DSC = (2 * |A ∩ B|) / (|A| + |B|)

    return dsc

[zhousai-zs]

Hello, I wanna know how to use box prompt for testing on dataset? Is there any code for this part? How to make changes on dataset.py?

[robustsam]

Hello, I wanna know how to use box prompt for testing on dataset? Is there any code for this part? How to make changes on dataset.py?

Hi, please refer to the code below to get bounding box of specific mask (you need to convert it into numpy array first):

def get_bounding_box(binary_mask):
    # Ensure the binary mask is 2D
    if binary_mask.ndim == 3:
        binary_mask = binary_mask[:, :, 0] > 0.5  # Take the first channel if 3D
    else:
        binary_mask = binary_mask > 0.5  # Threshold for a 2D binary mask

    # Find the coordinates of the non-zero pixels
    nonzero_pixels = np.argwhere(binary_mask)

    # Check if there are no non-zero pixels (no object in the mask)
    if len(nonzero_pixels) == 0:
        return None
    
    # Get the min and max bbox coordinates
    min_y, min_x = np.min(nonzero_pixels, axis=0)
    max_y, max_x = np.max(nonzero_pixels, axis=0)

    return (min_x, min_y, max_x, max_y)