Skip to content

Latest commit

 

History

History
114 lines (67 loc) · 2.93 KB

how_to_build.md

File metadata and controls

114 lines (67 loc) · 2.93 KB

Build from Source Code

This guide shows how to build an Intel® Optimization for Horovod* PyPI package from source and install it.

Prepare

Install GPU Driver

An Intel GPU driver is needed to build with GPU support. Refer to Install Intel GPU Driver.

Install oneAPI Base Toolkit

Need to install components of Intel® oneAPI Base Toolkit:

  • Intel® oneAPI DPC++ Compiler
  • Intel® oneAPI Math Kernel Library (oneMKL)
  • Intel® oneAPI Collective Communications Library (oneCCL)
  • Intel® oneAPI MPI Library (IntelMPI)
$ wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/e6ff8e9c-ee28-47fb-abd7-5c524c983e1c/l_BaseKit_p_2024.2.1.100_offline.sh
# 3 components are necessary: DPC++/C++ Compiler with DPC++ Libiary, oneMKL and oneCCL(IntelMPI will be installed automatically as oneCCL's dependency).
$ sudo sh l_BaseKit_p_2024.2.1.100_offline.sh

For any more details, please follow the procedure in https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit.html.

Setup environment variables

# DPC++ Compiler/oneMKL
source /path to basekit/intel/oneapi/compiler/latest/env/vars.sh
source /path to basekit/intel/oneapi/mkl/latest/env/vars.sh
# oneCCL (and Intel® oneAPI MPI Library as its dependency)
source /path to basekit/intel/oneapi/mpi/latest/env/vars.sh
source /path to basekit/intel/oneapi/ccl/latest/env/vars.sh

Build

Intel® Optimization for Horovod* depends on TensorFlow* to build from source.

$ pip install tensorflow==2.15.1

Download source code

$ git clone https://github.com/intel/intel-optimization-for-horovod
$ cd intel-optimization-for-horovod
# The repo defaults to the `main` branch. You can also check out a release branch:
$ git checkout <branch_name>

$ git submodule init && git submodule update

Build wheel and install

  • set environment variables
$ source /path/to/intel/oneapi/compiler/latest/env/vars.sh
$ source /path/to/intel/oneapi/mkl/latest/env/vars.sh # for runtime only
$ source /path/to/intel/oneapi/mpi/latest/env/vars.sh
$ source /path/to/intel/oneapi/ccl/latest/env/vars.sh

Build from source

If you want to change code locally and build from source:

  • Option 1: build and install from source code package
# will get a package located at dist/intel_optimization_for_horovod-*.tar.gz
$ python setup.py sdist

$ CC=icx CXX=icpx pip install --no-cache-dir dist/*.tar.gz
  • Option 2: build python wheels and install
$ CC=icx CXX=icpx python setup.py bdist_wheel

$ pip install dist/*.whl

Runtime

Intel® Optimization for Horovod* depends on Intel® Extension for Tensorflow* at runtime to run on intel GPU.

  • To run TensorFlow* applications
$ pip install --upgrade intel-extension-for-tensorflow[xpu]

You can then follow the steps in Contributing guide to run some tests to confirm it works as expected.