Merge 1.5.1 Release into master

Author: sean-smith

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,6 @@
+*Issue #, if available:*
+
+*Description of changes:*
+
+
+By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

.kitchen.cloud.yml

@@ -16,6 +16,13 @@ driver_config:
 provisioner:
   name: chef_zero
   require_chef_omnibus: 12.19.36
+  retry_on_exit_code:
+    - 35 # 35 is the exit code signaling that the node is rebooting
+  max_retries: 1
+  wait_for_retry: 120
+  client_rb:
+    exit_status: :enabled # Opt-in to the standardized exit codes
+    client_fork: false  # Forked instances don't return the real exit code
 
 platforms:
   - name: amazon-linux-latest

.kitchen.yml

@@ -13,13 +13,13 @@ platforms:
 suites:
   - name: default
     run_list:
-      - recipe[cfncluster::_update_packages]
+      - recipe[cfncluster::_default_pre]
       - recipe[cfncluster::default]
     attributes:
 
   - name: default_gpu
     run_list:
-      - recipe[cfncluster::_update_packages]
+      - recipe[cfncluster::_default_pre]
       - recipe[cfncluster::default]
     attributes:
       cfncluster:

CHANGELOG.md

@@ -3,6 +3,20 @@ cfncluster-cookbook CHANGELOG
 
 This file is used to list changes made in each version of the cfncluster cookbook.
 
+1.5.1
+-----
+
+Major new features/updates: 
+
+  - Added parameter to specify custom cfncluster-node package
+
+Bug fixes/minor improvements:
+
+  - Fixed poise-python dependecy issue
+  - Poll on EBS Volume attachment status
+  - Added more info on failure of pre and post install files
+  - Fixed SLURM cron job to publish pending metric
+
 1.4.1
 -----
 

CODE_OF_CONDUCT.md

@@ -0,0 +1,4 @@
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+[email protected] with any additional questions or comments.

CONTRIBUTING.md

@@ -0,0 +1,61 @@
+# Contributing Guidelines
+
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 
+documentation, we greatly value feedback and contributions from our community.
+
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 
+information to effectively respond to your bug report or contribution.
+
+
+## Reporting Bugs/Feature Requests
+
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
+
+When filing an issue, please check [existing open](https://github.com/awslabs/cfncluster-cookbook/issues), or [recently closed](https://github.com/awslabs/cfncluster-cookbook/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
+
+* A reproducible test case or series of steps
+* The version of our code being used
+* Any modifications you've made relevant to the bug
+* Anything unusual about your environment or deployment
+
+
+## Contributing via Pull Requests
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
+
+1. You are working against the latest source on the *develop* branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+
+## Finding contributions to work on
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/cfncluster-cookbook/labels/help%20wanted) issues is a great place to start. 
+
+
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+[email protected] with any additional questions or comments.
+
+
+## Security issue notifications
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+
+
+## Licensing
+
+See the [LICENSE](https://github.com/awslabs/cfncluster-cookbook/blob/develop/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

amis/build_ami.sh

@@ -0,0 +1,84 @@
+#!/bin/bash -x
+
+# This script creates a set of CfnCluster AMIs.
+#
+# The following variables must be exported in the environment:
+# AWS_FLAVOR_ID=<instance-type>
+# AWS_VPC_ID=<us-east-1-vpc-id>
+# AWS_SUBNET_ID=<us-east-1-subnet-id>
+# NVIDIA_ENABLED=<no|yes>
+#
+# NOTE: The VPC and the Subnet must be in the us-east-1 region, because the packer templates refer to
+# AMI IDs from this region. Moreover, the CentOs AMIs are private to the CfnCluster AWS account.
+#
+# Usage: build_ami.sh <os> <region> [private|public] [<build-date>]
+#   os: the os to build (supported values: all|centos6|centos7|alinux|ubuntu1404|ubuntu1604)
+#   region: region to build (supported values: all|us-east-1|...)
+#   private|public: specifies AMIs visibility (optional, default is private)
+#   build-date: timestamp to append to the AMIs names (optional)
+
+set -e
+
+os=$1
+region=$2
+public=$3
+build_date=$4
+
+available_os="centos6 centos7 alinux ubuntu1404 ubuntu1604"
+available_regions="eu-west-1,eu-west-2,eu-west-3,ap-southeast-1,ap-southeast-2,eu-central-1,ap-northeast-1,ap-northeast-2,ap-northeast-3,us-west-2,sa-east-1,us-west-1,us-east-2,ap-south-1,ca-central-1"
+cwd="$(dirname $0)"
+export VENDOR_PATH="${cwd}/../../vendor/cookbooks"
+
+if [ "x${os}" == "x" ]; then
+  echo "Must provide OS to build."
+  echo "Options: all ${available_os}"
+  exit 1
+fi
+
+if [ "x${region}" == "x" ]; then
+  echo "Must provide AWS region to build for."
+  echo "Options: us-east-1 all"
+  exit 1
+fi
+
+if [ "${public}" == "public" ]; then
+  export AMI_PERMS="all"
+fi
+
+# NOTE: the AMI is always built in us-east-1 and then copied to the specified regions
+if [ "${region}" == "all" ]; then
+  export BUILD_FOR=${available_regions}
+else
+  export BUILD_FOR=${region}
+fi
+
+RC=0
+
+rm -rf "${VENDOR_PATH}" || RC=1
+berks vendor "${VENDOR_PATH}" --berksfile "${cwd}/../Berksfile" || RC=1
+if [ "x${build_date}" == "x" ]; then
+  export BUILD_DATE=$(date +%Y%m%d%H%M)
+else
+  export BUILD_DATE=${build_date}
+fi
+
+
+case ${os} in
+  all)
+    for x in ${available_os}; do
+      packer build -machine-readable -var-file="${cwd}/packer_variables.json" "${cwd}/packer_${x}.json"
+      RC=$?
+    done
+    ;;
+  centos6|centos7|alinux|ubuntu1404|ubuntu1604)
+    packer build -machine-readable -var-file="${cwd}/packer_variables.json" "${cwd}/packer_${os}.json"
+    RC=$?
+    ;;
+  *)
+    echo "Unknown OS: ${os}"
+    RC=1
+    ;;
+esac
+
+echo "RC: ${RC}"
+exit ${RC}