TorchGeo on EC2 with CUDA

[roybenhayun]

after hours of trying various configurations to set up an CUDA enabled EC2 instance for running a TorchGeo model, it keeps coming to a point that even after installing Nvidia drivers, native CUDA Toolkit and Conda cudatoolkit, - torch.cuda.is_available() keeps returning False.
here's a link illustrating the issue: https://saturncloud.io/blog/how-to-fix-pytorch-cuda-not-available-with-anaconda/

any advice on how to get CUDA available on Windows? or even better a known working setup?

ideally, Windows would be great, Ubuntu could be good, but really anything out of the box will be much appreciated :->

[roybenhayun]

short update: after using PIP for installing also torch and torchvision dependencies with pip install torch torchvision --index-url https://download.pytorch.org/whl/cu113 progressed a bit and now torch.cuda.is_available() does return True.

However, when trying to run with the CUDA device getting either RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED , or, RuntimeError: CUDA error: an illegal memory access was encountered.

(FWIW, I know it's not a pure TorchGeo issue, but hoping that someone from the TrochGeo community may have a suggestion)

so just a short update. the question remains - anything out of the box will be much appreciated :->

[roybenhayun]

sure. from curiosity, which cloud\OS platform are you using?
maybe I'll have to change anyway...

[adamjstewart]

I either use macOS (laptop) or Linux (research cluster).

[calebrob6]

If you pip install torchgeo you should get a PyTorch version that is CUDA-enabled as long as you have semi-updated drivers (as torchgeo depends on PyTorch). I usually use this conda file to get an environment:

name: geospatial
channels:
  - conda-forge
dependencies:
  - gdal
  - python<3.11  # don't use the latest version of python in case wheels aren't made for everything
  - pip
  - pip:
    - torchgeo[datasets,docs,style,tests]>=0.4.0
    - jupyter
    - tensorboard

I'd probably do pip install -U git+https://github.com/microsoft/torchgeo.git later in the environment to get the important stuff on main.

[roybenhayun]

Hi @calebrob6 , many thanks!
I reverted my work trying to get a first working baseline with CUDA using the conda file you gave + the excellent training example you did :->

after installing and running the example, getting lightning.fabric.utilities.exceptions.MisconfigurationException: No supported gpu backend found!. it looked like torch was installed without CUDA built so added a check in code:

print("CUDA available:", torch.cuda.is_available())
print("Torch built with CUDA:", torch.backends.cuda.is_built())
print("cuDNN Version:", torch.backends.cudnn.version())
print("cuDNN Enabled:", torch.backends.cudnn.enabled)
print("cuDNN available:", torch.backends.cudnn.is_available())
	
output:
CUDA available: False
Torch built with CUDA: False
cuDNN Version: None
cuDNN Enabled: True
cuDNN available: False

so tried again from scratch, from the drivers to CUDA Toolkit, to CuDNN and then to Conda from the official site + your specific comment on python < 3.11 which seems critical.

it's running now as I reply, hopefully, in a few epochs will know if the training example works on CUDA on EC2 :->

just to share some context, the training-example is indeed a reference starting point for an actual project. really hoping to get it to work and get great results using TrochGeo as wished, hoped and desired :->
without CUDA the cycle will be too slow.

will keep updating on the progress of this one, and if it works will post specific instructions on the setup.

cheers!

[roybenhayun]

@calebrob6 thanks for your help. now we have a working setup of CUDA on EC2.

[roybenhayun]

managed to run several epochs successfully :->

here are working instructions with EC2 Windows Server 2019 AMI on AWS P2.xlarge instance:

CUDA setup

need to setup NVidia drivers, CUDA Toolkit and CuDNN. see https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#install-windows

NOTE: some EC2 AMIs e.g., the one mentioned above, already come with the NVidia drvers installed.

NVIDIA Drivers

see 2.1.1. Installing NVIDIA Graphic Drivers
check if drivers are already installed

• A: Right-click on your desktop and select NVIDIA Control Panel. From the NVIDIA Control Panel menu, select Help > System Information. The driver version is listed at the top of the Details window.
• from cmd control /name Microsoft.DeviceManager

CUDA Toolkit

see 2.1.2. Installing the CUDA Toolkit for Windows

• install CUDA 11.8 from https://developer.nvidia.com/cuda-11-8-0-download-archive
• check version with nvcc -V

CuDNN

see 2.2. Downloading CuDNN for Windows

• download CuDNN from https://developer.nvidia.com/rdp/cudnn-download e.g., on this setup, Download cuDNN v8.9.5 (September 12th, 2023), for CUDA 11.x
• manually copy binaries to paths as specified in the install guide

Conda environment setup

some installations are with Conda, some with PIP

create Conda torch env

• use the interactive selection in https://pytorch.org/get-started/locally/
• NODE: in Caleb's environment python<3.11 => installed python 3.10.12
  for example, on current machine with CUDA 11.8:
  conda install python=3.10.12 pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

Install torchgeo with PIP

1. install TorchGeo with Pip: pip install torchgeo
2. install Lightning with Pip: pip install pytorch-lightning
3. install TensorBoard with Pip: pip install -U tensorboard

checks

• check with "pip list" the torch was installed with CUDA built
• run the runtime prints from the previous post
• run an example... if there is an issue with the setup, all kind of problems may pop... some may even have bogus error messages...

HTH

[calebrob6]

Glad to hear it works!

One thing to note, after the next torchgeo release you'll need to pip install lightning instead of pip install pytorch-lightning as the pytorch lightning package redid stuff. (@adamjstewart to double check)