Fix for running panoptic segmentation on video in the Detectron2 Tutorial.ipynb

[Reka-Berci-Hajnovics]

📚 Documentation Issue

In the Detectron2 Tutorial.ipynb Google Colab notebook the segmentation on a video is broken.
Detectron2 Tutorial.ipynb

This issue can be fixed by installing another youtube-dl and setting the locale to UTF-8:

# Install dependencies, download the video, and crop 5 seconds for processing
import locale
locale.getpreferredencoding = lambda: "UTF-8"

#ERROR: Unable to extract uploader id
#!pip install youtube-dl
#!youtube-dl https://www.youtube.com/watch?v=ll8TgCZ0plk -f 22 -o video.mp4
!pip install https://github.com/yt-dlp/yt-dlp/archive/master.tar.gz
!yt-dlp https://www.youtube.com/watch?v=ll8TgCZ0plk -f 269 -o video.mp4
!ffmpeg -i video.mp4 -t 00:00:06 -c:v copy video-clip.mp4

The next code block returns an error as well, because of the relative paths used in the demo.py script:

# Run frame-by-frame inference demo on this video (takes 3-4 minutes) with the "demo.py" tool we provided in the repo.
#!git clone https://github.com/facebookresearch/detectron2
# Note: this is currently BROKEN due to missing codec. See https://github.com/facebookresearch/detectron2/issues/2901 for workaround.
#%run detectron2/demo/demo.py --config-file detectron2/configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml --video-input video-clip.mp4 --confidence-threshold 0.6 --output video-output.mkv \
#  --opts MODEL.WEIGHTS detectron2://COCO-PanopticSegmentation/panoptic_fpn_R_101_3x/139514519/model_final_cafdb1.pkl

#Fix:
#Set the path to the parent 'detectron2' folder
sys.path.insert(0, os.path.abspath('detectron2'))
#Open the detectron2 folder in the left side panel of Google Colab. In the demo folder you can find the demo.py script.
#Now you can modify the demo.py script. Comment or delete the import of VisualizationDemo, and add the following import:
#from predictor import VisualizationDemo
#When the pull request will be accepted, the manual modification of demo.py script will no longer be needed.

%run detectron2/demo/demo.py --config-file detectron2/configs/COCO-PanopticSegmentation/panoptic_fpn_R_101_3x.yaml --video-input video-clip.mp4 --confidence-threshold 0.6 --output video-output.mkv \
  --opts MODEL.WEIGHTS detectron2://COCO-PanopticSegmentation/panoptic_fpn_R_101_3x/139514519/model_final_cafdb1.pkl