Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce Python Dependencies #74

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
+103 −381
Open

Reduce Python Dependencies #74

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
48e5074
Reimplement the plssvm_target_platforms.py utility script to reduce t…
breyerml Oct 15, 2024
7c02de9
Further reduce Python dependencies.
breyerml Nov 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,12 +263,6 @@ else ()
message(FATAL_ERROR "Can't find Python3. Please manually specify PLSSVM_TARGET_PLATFORMS (e.g. -DPLSSVM_TARGET_PLATFORMS=\"cpu;nvidia:sm_70,sm_86;amd:gfx906;intel:skl\"!")
endif ()

include(${CMAKE_CURRENT_SOURCE_DIR}/cmake/check_python_libs.cmake)
set(PLSSVM_TARGET_PLATFORMS_PYTHON_SCRIPT_REQUIRED_LIBS cpuinfo GPUtil pyamdgpuinfo pylspci)
message(STATUS "Checking required Python3 libraries (${PLSSVM_TARGET_PLATFORMS_PYTHON_SCRIPT_REQUIRED_LIBS}) to automatically determine the PLSSVM_TARGET_PLATFORMS.")
set(PLSSVM_TARGET_PLATFORMS_PYTHON_SCRIPT_REQUIRED_LIBS_ERROR_MESSAGE "or manually define PLSSVM_TARGET_PLATFORMS (e.g. -DPLSSVM_TARGET_PLATFORMS=\"cpu;nvidia:sm_70,sm_86;amd:gfx906;intel:skl\"!")
check_python_libs(${PLSSVM_TARGET_PLATFORMS_PYTHON_SCRIPT_REQUIRED_LIBS} ${PLSSVM_TARGET_PLATFORMS_PYTHON_SCRIPT_REQUIRED_LIBS_ERROR_MESSAGE})

# run our `plssvm_target_platforms.py` script to determine the PLSSVM_TARGET_PLATFORMS string
execute_process(
COMMAND ${Python3_EXECUTABLE} "${CMAKE_CURRENT_SOURCE_DIR}/utility_scripts/plssvm_target_platforms.py" "--quiet"
Expand Down
22 changes: 8 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,13 +178,9 @@ Valid targets are:
- `intel`: compile for Intel GPUs; **at least one** architectural specification is necessary, e.g., `intel:skl`

At least one of the above targets must be present. If the option `PLSSVM_TARGET_PLATFORMS` is not present, the targets
are automatically determined using the Python3 `utility_scripts/plssvm_target_platforms.py` script (required Python3 dependencies:
[`argparse`](https://docs.python.org/3/library/argparse.html), [`py-cpuinfo`](https://pypi.org/project/py-cpuinfo/),
[`GPUtil`](https://pypi.org/project/GPUtil/), [`pyamdgpuinfo`](https://pypi.org/project/pyamdgpuinfo/), and
[`pylspci`](https://pypi.org/project/pylspci/)).
are automatically determined using the Python3 `utility_scripts/plssvm_target_platforms.py` script.

Note that when using DPC++ only a single architectural specification for `cpu`, `nvidia` or `amd` is allowed and that
automatically retrieving AMD GPU information on Windows is currently not supported due to `pyamdgpuinfo` limitations.
Note that when using DPC++ only a single architectural specification for `cpu`, `nvidia` or `amd` is allowed.


```bash
Expand All @@ -194,8 +190,9 @@ python3 utility_scripts/plssvm_target_platforms.py --help
usage: plssvm_target_platforms.py [-h] [--quiet]

optional arguments:
-h, --help show this help message and exit
--quiet only output the final PLSSVM_TARGET_PLATFORMS string
-h, --help show this help message and exit
--quiet only output the final PLSSVM_TARGET_PLATFORMS string
--gpus_only only output gpu architectures to the final PLSSVM_TARGET_PLATFORMS string
```

Example invocation:
Expand All @@ -204,10 +201,9 @@ Example invocation:
python3 utility_scripts/plssvm_target_platforms.py
```
```
Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz: {'avx512': True, 'avx2': True, 'avx': True, 'sse4_2': True}
supported CPU SIMD flags: {'avx512': True, 'avx2': True, 'avx': True, 'sse4_2': True}

Found 1 NVIDIA GPU(s):
1x NVIDIA GeForce RTX 3080: sm_86
Found 1 NVIDIA GPU(s): [sm_86]

Possible -DPLSSVM_TARGET_PLATFORMS entries:
cpu:avx512;nvidia:sm_86
Expand All @@ -221,13 +217,11 @@ or with the `--quiet` flag provided:
python3 utility_scripts/plssvm_target_platforms.py --quiet
```
```
cpu:avx512;intel:dg1
cpu:avx512;nvidia:sm_86
```

If the architectural information for the requested GPU could not be retrieved, one option would be to have a look at:

- for NVIDIA GPUs: [Your GPU Compute Capability](https://developer.nvidia.com/cuda-gpus)
- for AMD GPUs: [clang AMDGPU backend usage](https://llvm.org/docs/AMDGPUUsage.html)
- for Intel GPUs and CPUs: [Ahead of Time Compilation](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-dpcpp-cpp-compiler-dev-guide-and-reference/top/compilation/ahead-of-time-compilation.html) and [Intel graphics processor table](https://dgpu-docs.intel.com/devices/hardware-table.html)


Expand Down
16 changes: 1 addition & 15 deletions install/python_requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
### optional and required python packages

## for the data set generation
# LIBSVM
argparse
## for the data set generation (LIBSVM file format)
scikit-learn
humanize

Expand All @@ -17,23 +15,11 @@ pandas
# for plotting the generated data set (only available if #features <= 3)
matplotlib


## for the automatic PLSSVM target platform determination
# argparse
py-cpuinfo
GPUtil
pyamdgpuinfo; platform_system != "Windows"
pylspci


## for the performance tracker parser
# argparse
# matplotlib
pyyaml
pint


## for the performance analysis
# argparse
# scikit-learn
wrapt-timeout-decorator
Loading
Loading