Multi-Point threshold for Anomaly Detection

Note: BASE REPO

https://github.com/rashidrao-pk/advis_distrimuse_unito

Dataset source files and public model checkpoints are listed in the collapsible download section below.

1. Start With Environment

Install Python using Conda

conda create -n AD python==3.9.18 -y
conda activate AD
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

git clone https://github.com/rashidrao-pk/AD_MultiPointThreshold
conda activate AD
pip install -r requirements.txt

---

2. Download Datasets and Model Checkpoints

The project uses external datasets and pretrained checkpoints. They are not stored directly in this repository because of file size limitations.

2.1 Install download tools

Install the Kaggle and Hugging Face CLIs:

pip install -U kaggle huggingface_hub

# or using conda
# conda install -c conda-forge huggingface_hub

verify

hf --help

For Kaggle downloads, make sure your Kaggle API token is configured. Download kaggle.json from your Kaggle account settings and place it in:

mkdir -p ~/.kaggle
cp kaggle.json ~/.kaggle/kaggle.json
chmod 600 ~/.kaggle/kaggle.json

2.2 Download datasets from Kaggle

Supported public Kaggle datasets:

Dataset	Kaggle URL	Suggested Local Folder
Robotics Hazards	https://www.kaggle.com/datasets/rashidrao/robotics-hazards	data/Robotics_Hazards
Cobots Synthetic / DistriMuSe UniGra	https://www.kaggle.com/datasets/rashidrao/cobots-synthetic/	data/Cobots_Synthetic

Download and unzip:

mkdir -p data

kaggle datasets download -d rashidrao/robotics-hazards \
  -p data/Robotics_Hazards \
  --unzip

kaggle datasets download -d rashidrao/cobots-synthetic \
  -p data/Cobots_Synthetic \
  --unzip

Recommended folder structure:

data/
├── Robotics_Hazards/
├── Cobots_Synthetic/
└── MVtec/

Set the dataset root path:

export DATA_ROOT=$(pwd)/data

2.3 Download pretrained checkpoints from Hugging Face

Pretrained model checkpoints are available here:

Checkpoint Repository	Dataset / Use Case
https://huggingface.co/rashidrao/AD_Cobots_Synthetic	Cobots Synthetic / DistriMuSe UniGra
https://huggingface.co/rashidrao/AD_Robotics_Hazards	Robotics Hazards
https://huggingface.co/rashidrao/AD_MVTec	MVTec AD

Download all checkpoint repositories:

mkdir -p checkpoints

hf download rashidrao/AD_Cobots_Synthetic \
  --local-dir checkpoints/AD_Cobots_Synthetic

hf download rashidrao/AD_Robotics_Hazards \
  --local-dir checkpoints/AD_Robotics_Hazards

hf download rashidrao/AD_MVTec \
  --local-dir checkpoints/AD_MVTec

or

pip install -U huggingface_hub

Download Cobots Synthetic Checkpoints

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="rashidrao/AD_Cobots_Synthetic",
    local_dir="checkpoints/AD_Cobots_Synthetic"
)

Download Robotics Hazards Checkpoints

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="rashidrao/AD_Robotics_Hazards",
    local_dir="checkpoints/AD_Robotics_Hazards"
)

Download MVTec Checkpoints

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="rashidrao/AD_MVTec",
    local_dir="checkpoints/AD_MVTec"
)

Expected checkpoint structure:

checkpoints/
├── AD_Cobots_Synthetic/
│   ├── model_PLeft_64.pt
│   ├── model_PRight_64.pt
│   ├── model_RoboArm_64.pt
│   └── model_ConvBelt_64.pt
├── AD_Robotics_Hazards/
└── AD_MVTec/

2.4 Verify downloaded checkpoints

List available models:

python utils/scripts/model_loader.py --list

Inspect a checkpoint and test a forward pass:

python utils/scripts/model_loader.py \
  --model_path checkpoints/AD_Cobots_Synthetic/model_RoboArm_64.pt \
  --show_summary \
  --test_forward \
  --device auto

2.5 Quick inference with pretrained checkpoints

Example with the Cobots Synthetic / DistriMuSe UniGra RoboArm model:

python utils/scripts/inference.py \
  --dataset Cobots_Synthetic \
  --safety_area RoboArm \
  --checkpoints checkpoints/AD_Cobots_Synthetic \
  --static_mask_paths masks/PLeft.png masks/PRight.png masks/RoboArm.png masks/ConvBelt.png \
  --threshold_dir results/thresholds

Example with all safety areas:

python utils/scripts/inference.py \
  --dataset Cobots_Synthetic \
  --safety_area ALL \
  --checkpoints checkpoints/AD_Cobots_Synthetic \
  --static_mask_paths masks/PLeft.png masks/PRight.png masks/RoboArm.png masks/ConvBelt.png \
  --threshold_dir results/thresholds

2.6 Notes on thresholds

Thresholds are dataset-specific and camera/setup-specific. If you change the dataset, camera view, preprocessing, or safety-area masks, recalibrate thresholds before reporting final results.

python utils/scripts/calibrate_threshold.py \
  --safety_area RoboArm \
  --checkpoints checkpoints/AD_Cobots_Synthetic \
  --threshold_strategy percentile \
  --threshold_percentile 99.0

3. Run Scripts

Optional: Train models Again?

Train a VAE-GAN model on one (PLeft, PRight, RoboArm, ConvBelt) or all safety areas.

# Single area (default settings)
python utils/scripts/train.py --safety_area RoboArm

# All areas sequentially
python utils/scripts/train.py --safety_area ALL

4. Inference

Run anomaly detection from multiple input sources including following input sources;

# Pre-cropped frames, evaluate against annotations
python utils/scripts/inference.py --dataset MVtec --object hazelnut
python utils/scripts/inference.py --dataset Robotics_Hazards
python utils/scripts/inference.py --dataset Cobots_Synthetic

---

5. Configurations

Check Machine Specific configuration setup here

---

6. Training

Check Model Trainings here

---

7. Verify and Inspect Model Checkpoints

Check model checkpoints here

---

7. Inference

Check inference here

---

10. Troubleshooting

Got errors? check Troubleshooting Guides here

---

11. Citation & References

• Base Repository: https://github.com/rashidrao-pk/advis_distrimuse_unito
• DistriMuSe Dataset: https://zenodo.org/records/18742241
• Synthetic Dataset Generator: https://github.com/valerialabugr/SimIndus-Dataset
• MVTec AD Dataset: https://www.mvtec.com/company/research/datasets/ad

---

12. Version History

• v1.0 (Current): Multi-dataset support, model inspection utilities
• Dataset switching: MVtec, Robotics_Hazards, Cobots_Synthetic
• Enhanced configuration management
• Model loader and inspection tools
• Comprehensive documentation

---

👥 Contributing

We welcome contributions! Check out our Contributing Guide to get started.

Thank you to all our contributors!