Skip to content

Commit

Permalink
Moving everything from github.iu.edu to github.com
Browse files Browse the repository at this point in the history
  • Loading branch information
Daniel Kogler committed Feb 4, 2016
1 parent f6f7ee7 commit a7c92d5
Show file tree
Hide file tree
Showing 2,181 changed files with 447,569 additions and 1 deletion.
22 changes: 22 additions & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
High Performance ParalleX was created by the Indiana University Center for
Research in Extreme Scale Technologies (CREST) in Bloomington, Indiana.

Authors:
Abhishek Kulkarni <adkulkar [at] indiana.edu>
Benjamin Martin <benjmart [at] indiana.edu>
Bo Zhang <zhang416 [at] indiana.edu>
Daniel Kogler <dkogler [at] indiana.edu>>
Ezra Kissel <kissel [at] indiana.edu>
Jackson DeBuhr <jdebuhr [at] indiana.edu>
Jayashree Candadai <jayaajay [at] indiana.edu>
Jesun Firoz <jsfiroz [at] indiana.edu>
Kevin Bohan <pbohan [at] indiana.edu>
Kevin Huck <khuck [at] cs.uoregon.edu>
Luke Dalessandro <ldalessa [at] indiana.edu>
Marcin Zalewski <zalewski [at] indiana.com>
Martin Swany <swany [at] iu.edu>
Matt Anderson <andersmw [at] indiana.edu>
Nilesh Mahajan <nnmahaja [at] indiana.edu>
Ryan Newton <rrnewton [at] gmail.com>
Thejaka Amila Kanewala <thejkane [at] iu.edu>
Udayanga Wickramasinghe <uswickra [at] umail.iu.edu>
128 changes: 128 additions & 0 deletions INSTALL
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
# Installation Instructions

## Configuring

HPX-5 is designed to use standard autotools configuration techniques. HPX-5
releases come bootstrapped, *there is no need to bootstrap in this case.* If you
have a development release you will need to follow the bootstrapping
documentation at the end of this guide.

Building HPX-5 without a network (the default) allows you to run
HPX-applications on a single node. For distributed execution either MPI or
Photon networking should be enabled, which is detailed in next section.

The HPX-5 configuration infrastructure supports the use of ‘pkg-config’ to look
for installed packages. In particular, if your compilation environment requires
explicit paths to MPI, you should compile with --enable-mpi --with-mpi={ompi,
mvapich2,etc}.

See ‘./configure --help’ for further details.

## Configuring with MPI

The `--enable-mpi` configuration flag will add support for `mpirun` and the ISIR
network. By default, HPX–5 will try and see if `mpi.h` and `libmpi.so` are
available with no additional flags (e.g., for `CC=mpicc` or on Cray `CC=cc`).

If you want to specify which MPI should be used or the build system cannot find
MPI, you may specify the .pc file you may need to use the `--with-mpi=pkg`
option. The pkg parameter can be either the prefix for the .pc file (ompi,
mpich, etc.) or a complete path to the .pc file.

```
$ ./configure --prefix=/path/to/install --enable-mpi
```

## Configuring with Photon

The Photon network library is included in the HPX–5 release. To configure HPX–5
with Photon use the option `--enable-photon`. HPX–5 can also use a
system-installed or pkg-config version of Photon. This can be controlled using
`--with-photon=system` or `--with-photon=pkg`, respectively. Note that HPX–5
does not provide its own distributed job launcher, so it is necessary to use
either the `--enable-pmi` or `--enable-mpi` option in addition to
`--enable-photon` in order to build support for `mpirun` or `aprun`.

```
$ ./configure --prefix=/path/to/install --enable-mpi --enable-photon
```

## To complete the build and install use:
make
make install

## Running the test-suite

To run the included testsuite set TESTS_CMD=<command> during configuration.

```
$ ./configure TESTS_CMD="mpirun -np 2 --map-by node:PE=16" --enable-mpi
$ make check
```

## CRAY/uGNI

HPX-5 supports either `PrgEnv-intel` or `PrgEnv-gnu`.

HPX-5 can be built to support fully `-static` linking, however we recommend that
you enable dynamic linking by setting `CRAYPE_LINK_TYPE=dynamic` in your
environment.

HPX-5 requires the explicit use of the `cc` wrapper on Cray playtforms.

In addition, the --with-hugetlbfs option causes the HPX–5 heap
to be mapped with huge pages, which is necessary for larger heaps on some
Cray Gemini machines. The hugepages modules provide the environment necessary
for compilation. Huge page support should work where available, but we have
seen no performance benefits to using it at this point.

```
$ ./configure --prefix=/path/to/install/ CC=cc
```
We recommend that users of the Cray Intel module also `module load gcc`, as the
intel tools try to provide compatibility with the gnu toolchain that it sees in
the path, and the default gnu toolchain in the CLE is ancient. See
https://www.nersc.gov/users/software/compilers/intel-fortran-c-and-c/intel-bug-reports/c-11-header-files-appear-missing-on-edison/
for more details.

### MPI Details

If you have MPICH loaded, you need to specify the pkg-config package manually.

```
‘$ ./configure --prefix=/path/to/install --enable-mpi --with-mpi=mpich
CC=cc’
```

### Photon Details

Photon requires an explicit `--enable-ugni --disable-libfabric` configuration argument.

```
$ ./configure --prefix=/path/to/install/ CC=cc --enable-photon
--enable-pmi PHOTON_CARGS="--enable-ugni --disable-libfabric"
```

## Bootstrapping

HPX-5 provides a bootstrap script in its build. In the HPX-5 directory run the
bootstrap script.

```
$ ./bootstrap
```

Bootstrap is a bash script that generate the initialization required to create
a configure script when using GNU autotools. In addition, it will download and
unpack some dependencies to the `contrib` folder.

To bootstrap you would need the latest version of autotools. There is a script
to install the latest tools in scripts folder.

GNU Make 3.81+ Makefile/build processor
GNU Autoconf 2.6.9 Configuration script builder
GNU Automake 1.15 Aclocal macro generator
GNU M4 1.4.17 Macro processor for configuration
Libtool 2.4.6 Shared library manager
Pkg-config 0.26+ Returns information about installed libraries

33 changes: 33 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@

Copyright (c) 2013-2016, Trustees of Indiana University
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.

* Neither the name of Indiana University nor the names of its
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

33 changes: 33 additions & 0 deletions Makefile.am
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
###################
# Makefile for HPX #
####################

EXTRA_DIST = \
README.md README.apex \
LICENSE INSTALL AUTHORS

AUTOMAKE_OPTIONS = foreign
ACLOCAL_AMFLAGS = -I config -I contrib/hwloc-1.11.0/config

pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = hpx.pc

SUBDIRS = contrib libsync libhpx include examples
DIST_SUBDIRS = contrib libsync libhpx include examples libapex docs tests

if HAVE_APEX
SUBDIRS += libapex
endif

if ENABLE_DOCS
SUBDIRS += docs
endif

if ENABLE_TESTS
SUBDIRS += tests
endif

bin_SCRIPTS = scripts/hpx-config

install-exec-hook:
cd $(DESTDIR)$(libdir); find $(DESTDIR)$(libdir) -type f -name \*.la -delete
156 changes: 156 additions & 0 deletions README.apex
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
--------------------------------------------------------------------------------
General Instructions for building HPX-5 with APEX measurement/adaptation support
--------------------------------------------------------------------------------

To include APEX support in HPX-5, add the APEX pkgconfig path to the
PKG_CONFIG_PATH environment variable before running ./configure.
For example:

export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/path/to/apex/installation/lib/pkgconfig

Then, add the "--with-apex" option to the configure. For example:

./configure --enable-photon --enable-mpi --with-apex={=system,PKG}

For instructions on downloading and building APEX, please see:
https://github.com/khuck/xpress-apex

To build HPX-5 with APEX on Edison, do the following:

-----------------------------------------------------
1) load the right modules
-----------------------------------------------------

module swap PrgEnv-intel PrgEnv-gnu
module load gcc/4.9.2
module load cmake/3.0.0
module load boost
module load automake/1.14
module load autoconf
module unload cray-shmem
module unload darshan
module load papi

-----------------------------------------------------
2) build APEX
-----------------------------------------------------

APEX is built with CMake. The minimum CMake settings needed
for APEX are:

-DCMAKE_INSTALL_PREFIX=<some path to an installation location>

1. Get the code:
wget https://github.com/khuck/xpress-apex/archive/v0.1.tar.gz
tar -xvzf v0.1.tar.gz
cd xpress-apex-v0.1

2. Enter the repo directory, make a build directory:

cd xpress-apex-0.1
mkdir build
cd build

3. configure using CMake:

cmake -DCMAKE_INSTALL_PREFIX=<installation-path> ..

4. build with make:

make
make test
make doc
make install

Other CMake setting, depending on you needs can be set:
https://github.com/khuck/xpress-apex/blob/master/README.rst

-----------------------------------------------------
3) Build HPX
-----------------------------------------------------

# get HPX from gitlab
git checkout [email protected]:extreme/hpx.git

# set environment variables for HPX for photon support
export CRAYPE_LINK_TYPE=dynamic
export HPX_PHOTON_BACKEND=ugni

#set Pkgconfig for APEX
XPRESS_APEX=${HOME}/src/xpress-apex/install/lib/pkgconfig
if [ ${PKG_CONFIG_PATH} ] ; then
export PKG_CONFIG_PATH=$XPRESS_APEX:${PKG_CONFIG_PATH}
else
export PKG_CONFIG_PATH=$XPRESS_APEX
fi

# set the path to hpx-autotools, wherever they are
export PATH=/path/to/install/bin:$PATH
# bootstrap to make the configure script with APEX options
./bootstrap
# run configure
./configure --prefix=path-to-hpx-install --enable-mpi CC=cc PHOTON_CARGS="--enable-ugni" --enable-photon --with-apex
make && make install

-----------------------------------------------------
4) Build LULESH
-----------------------------------------------------

# (get hpx-apps from gitlab)
cd hpx-apps/lulesh/parcels
$ export PKG_CONFIG_PATH=/path-to-hpx-install/lib/pkgconfig:$PKG_CONFIG_PATH
$ export LD_LIBRARY_PATH=/path-to-hpx-install/lib:$LD_LIBRARY_PATH

Edit the lulesh-hpx.c

diff --git a/lulesh/parcels/lulesh-hpx.c b/lulesh/parcels/lulesh-hpx.c
index 9cd2fc5..3313a46 100644
--- a/lulesh/parcels/lulesh-hpx.c
+++ b/lulesh/parcels/lulesh-hpx.c
@@ -18,6 +18,10 @@
#include "lulesh-hpx.h"
#include "trace.h"

+#if defined(HAVE_APEX)
+#include "apex.h"
+#endif
+
#ifdef DIAGNOSTICS
uint64_t remote_comms = 0;
uint64_t total_comms = 0;
@@ -476,6 +480,10 @@ int main(int argc, char **argv) {
printf(" Number of domains: %d nx: %d maxcycles: %d core-major ordering: %d\n",nDoms,nx,maxcycles,core_major);
}

+#ifdef HAVE_APEX
+ apex_setup_power_cap_throttling();
+#endif
+
e = hpx_run(&_main, &nDoms, &nx, &maxcycles);
hpx_finalize();
return e;

./bootstrap
./configure CC=cc CFLAGS="-O3"
make

# Set runtime environment variables for APEX before the aprun call.

export APEX_PROFILE_OUTPUT=0
export APEX_SCREEN_OUTPUT=1
export APEX_POLICY=1
export APEX_CONCURRENCY=1
export APEX_THROTTLING=1
export APEX_ENERGY_THROTTLING=1
export APEX_THROTTLING_MAX_THREADS=48
export APEX_THROTTLING_MIN_THREADS=12
export APEX_THROTTLING_MAX_WATTS=200.0
export APEX_THROTTLING_MIN_WATTS=180.0

aprun -n 16 -N 1 -j2 -d 48 ./luleshparcels -n 343 -x 48 -i 100 --hpx-threads=48

# To see the concurrency charts (contact [email protected] for a python script
# to aggregate them into one script):

module load gnuplot
gnuplot-persist concurrency.*.gnuplot
Loading

0 comments on commit a7c92d5

Please sign in to comment.