Merge 1.5.1 Release into master

Author: sean-smith

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,6 @@
+*Issue #, if available:*
+
+*Description of changes:*
+
+
+By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

CHANGELOG.rst

@@ -2,6 +2,26 @@
 CHANGELOG
 =========
 
+1.5.1
+=====
+* change:``cfncluster``: Added "ec2:DescribeVolumes" permissions to
+CfnClusterInstancePolicy
+* change:``cfncluster``: Removed YAML CloudFormation template, it can be
+generated by the https://github.com/awslabs/aws-cfn-template-flip tool
+
+* updates:``cfncluster``: Add support for eu-west-3 region
+
+* feature:``cfncluster-cookbook``: Added parameter to specify custom
+cfncluster-node package
+
+* bugfix:``cfncluster``: Fix --template-url command line parameter
+* bugfix:``cfncluster-cookbook``: Poll on EBS Volume attachment status
+* bugfix:``cfncluster-cookbook``: Fixed SLURM cron job to publish pending
+metric
+* bugfix:``cfncluster-node``: Fixed Torque behaviour when scaling up from an
+empty cluster
+
+
 1.4.2
 =====
 * bugfix:``cfncluster``: Fix crash when base directory for config file
@@ -83,22 +103,22 @@ by incompatible dependencies.
 =====
 Offiical release of the CfnCluster 1.x CLI, templates and AMIs. Available in all regions except BJS, with
 support for Amazon Linux, CentOS 6 & 7 and Ubuntu 14.04 LTS. All AMIs are built via packer from the CfnCluster
-Cookbook project (https://github.com/awslabs/cfncluster-cookbook). 
+Cookbook project (https://github.com/awslabs/cfncluster-cookbook).
 
 1.0.0-beta
 ==========
 
 This is a major update for CfnCluster. Boostrapping of the instances has moved from shell scripts into Chef
 receipes. Through the use of Chef, there is now wider base OS support, covering Amazon Linux, CentOS 6 & 7
-and also Ubuntu. All AMIs are now created using the same receipes. All previously capabilites exisit and the 
+and also Ubuntu. All AMIs are now created using the same receipes. All previously capabilites exisit and the
 changes should be non-instrusive.
 
 
 0.0.22
 ======
 * updates:``ami``: Pulled latest CentOS6 errata
 * feature:``cfncluster``: Support for specifiying MasterServer and ComputeFleet root volume size
-* bugfix:``cfncluster-node``: Fix for SGE parallel job detection 
+* bugfix:``cfncluster-node``: Fix for SGE parallel job detection
 * bugfix:``ami``: Removed ZFS packages
 * bugfix:``cfncluster-node``: Fix torque node additon with pbs_server restart
 * updates:``ami``: Updated Chef client to 12.4.1 + berkshelf
@@ -116,7 +136,7 @@ changes should be non-instrusive.
 0.0.20
 ======
 * feature:``cfncluster``: Support for D2 instances
-* updates:``ami``: Pulled latest CentOS6 errata 
+* updates:``ami``: Pulled latest CentOS6 errata
 * updates:``ami``: Pulled latest cfncluster-node package
 * updates:``ami``: Pulled latest ec2-udev-rules package
 * updates:``ami``: Pulled latest NVIDIA driver 346.47

CODE_OF_CONDUCT.md

@@ -0,0 +1,4 @@
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+[email protected] with any additional questions or comments.

CONTRIBUTING.md

@@ -0,0 +1,61 @@
+# Contributing Guidelines
+
+Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional 
+documentation, we greatly value feedback and contributions from our community.
+
+Please read through this document before submitting any issues or pull requests to ensure we have all the necessary 
+information to effectively respond to your bug report or contribution.
+
+
+## Reporting Bugs/Feature Requests
+
+We welcome you to use the GitHub issue tracker to report bugs or suggest features.
+
+When filing an issue, please check [existing open](https://github.com/awslabs/cfncluster/issues), or [recently closed](https://github.com/awslabs/cfncluster/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
+reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
+
+* A reproducible test case or series of steps
+* The version of our code being used
+* Any modifications you've made relevant to the bug
+* Anything unusual about your environment or deployment
+
+
+## Contributing via Pull Requests
+Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:
+
+1. You are working against the latest source on the *develop* branch.
+2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
+3. You open an issue to discuss any significant work - we would hate for your time to be wasted.
+
+To send us a pull request, please:
+
+1. Fork the repository.
+2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change.
+3. Ensure local tests pass.
+4. Commit to your fork using clear commit messages.
+5. Send us a pull request, answering any default questions in the pull request interface.
+6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation.
+
+GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and 
+[creating a pull request](https://help.github.com/articles/creating-a-pull-request/).
+
+
+## Finding contributions to work on
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/cfncluster/labels/help%20wanted) issues is a great place to start. 
+
+
+## Code of Conduct
+This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). 
+For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact 
+[email protected] with any additional questions or comments.
+
+
+## Security issue notifications
+If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue.
+
+
+## Licensing
+
+See the [LICENSE](https://github.com/awslabs/cfncluster/blob/develop/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+
+We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

README.rst

@@ -27,3 +27,16 @@ Issues
 Please open a GitHub issue for any feedback or issues:
 https://github.com/awslabs/cfncluster.  There is also an active AWS
 HPC forum which may be helpful:https://forums.aws.amazon.com/forum.jspa?forumID=192.
+
+CfnCluster 1.5 IAM Change
+=========================
+Between CfnCluster 1.4.2 and 1.5.0 we made a change to the CfnClusterInstancePolicy that adds “ec2:DescribeVolumes” permissions. If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://cfncluster.readthedocs.io/en/latest/iam.html
+
+CfnCluster 1.2 and Earlier
+==========================
+
+For various security (on our side) and maintenance reasons, CfnCluster
+1.2 and earlier have been deprecated.  AWS-side resources necessary to
+create a cluster with CfnCluster 1.2 or earlier are no longer
+available.  Existing clusters will continue to operate, but new
+clusters can not be created.

amis.txt

@@ -1,80 +1,85 @@
+# alinux
+ap-northeast-1: ami-0dde39e2d5b865e22
+ap-northeast-2: ami-00ff07cc9e3be7301
+ap-northeast-3: ami-00683abdf6e60ebcd
+ap-south-1: ami-0aa10b91c994a481d
+ap-southeast-1: ami-05698f82a71d681a5
+ap-southeast-2: ami-01da8b5a3a0212da1
+ca-central-1: ami-0845890c227805aeb
+eu-central-1: ami-54cffcbf
+eu-west-1: ami-08cb4b06d393675b0
+eu-west-2: ami-0cb0bfe1b8e796cd7
+eu-west-3: ami-0d277eafd4f569bc7
+sa-east-1: ami-056a9e6c13067d81f
+us-east-1: ami-0f4de43b53d2f8210
+us-east-2: ami-00c76d9774b0e366b
+us-west-1: ami-0cf41903fbdc094cc
+us-west-2: ami-03eee80fee70a432d
 # centos6
-ap-northeast-1: ami-2467fc42
-ap-northeast-2: ami-2d7ddd43
-ap-south-1: ami-5270273d
-ap-southeast-1: ami-eb681a97
-ap-southeast-2: ami-966d90f4
-ca-central-1: ami-75a32611
-eu-central-1: ami-c78a17a8
-eu-west-1: ami-a4e578dd
-eu-west-2: ami-fa7b609e
-eu-west-3: ami-38f14745
-sa-east-1: ami-4290d22e
-us-east-1: ami-7e785d04
-us-east-2: ami-a27e54c7
-us-west-1: ami-833d3ee3
-us-west-2: ami-22c5735a
+ap-northeast-1: ami-04d400f3c012a86ae
+ap-northeast-2: ami-068f5c3e07b48f123
+ap-northeast-3: ami-094d5e2e854fb015e
+ap-south-1: ami-0c8a6bf281179c3ba
+ap-southeast-1: ami-09555649a6bee7b69
+ap-southeast-2: ami-0610ffef47d7427d9
+ca-central-1: ami-0a36ee35b01f0e7c5
+eu-central-1: ami-89b38062
+eu-west-1: ami-00900f4e20bdb9f6d
+eu-west-2: ami-092aee04074dfd1f4
+eu-west-3: ami-042907d2cd70ecb40
+sa-east-1: ami-09d0f784c22548994
+us-east-1: ami-090b5303b54f5b35a
+us-east-2: ami-036cfbf5fc84c861e
+us-west-1: ami-0cc517b5b49ed6862
+us-west-2: ami-0e1ae8c0ca706fe64
 # centos7
-ap-northeast-1: ami-4465fe22
-ap-northeast-2: ami-2c7ddd42
-ap-south-1: ami-e271268d
-ap-southeast-1: ami-1065176c
-ap-southeast-2: ami-85738ee7
-ca-central-1: ami-aca722c8
-eu-central-1: ami-7c851813
-eu-west-1: ami-46e17c3f
-eu-west-2: ami-f879629c
-eu-west-3: ami-2bfe4856
-sa-east-1: ami-ae93d1c2
-us-east-1: ami-86795cfc
-us-east-2: ami-9d7852f8
-us-west-1: ami-533e3d33
-us-west-2: ami-37c4724f
-# alinux
-ap-northeast-1: ami-1366fd75
-ap-northeast-2: ami-127cdc7c
-ap-south-1: ami-3f732450
-ap-southeast-1: ami-306b194c
-ap-southeast-2: ami-3c6e935e
-ca-central-1: ami-cca421a8
-eu-central-1: ami-d98a17b6
-eu-west-1: ami-7fe77a06
-eu-west-2: ami-157a6171
-eu-west-3: ami-7bfe4806
-sa-east-1: ami-7892d014
-us-east-1: ami-3e456044
-us-east-2: ami-3a7e545f
-us-west-1: ami-7d3f3c1d
-us-west-2: ami-59db6d21
+ap-northeast-1: ami-0906ec2a1fce4fe7a
+ap-northeast-2: ami-098685fc947384e15
+ap-northeast-3: ami-07d36e7c754056e76
+ap-south-1: ami-0fce8c70d048719f4
+ap-southeast-1: ami-030771765a1101265
+ap-southeast-2: ami-0930016c10f90c7f0
+ca-central-1: ami-05652bb89f388a076
+eu-central-1: ami-dbc0f330
+eu-west-1: ami-0433c843433f06547
+eu-west-2: ami-07080ddf1e2406ca2
+eu-west-3: ami-062bd0a49558d0ba0
+sa-east-1: ami-005a0e54c37bc2a7b
+us-east-1: ami-01a681672ca8e9de1
+us-east-2: ami-07af67d83c0ae927c
+us-west-1: ami-00f4867c7e867784b
+us-west-2: ami-03ac1fbbf78947521
 # ubuntu1404
-ap-northeast-1: ami-b967fcdf
-ap-northeast-2: ami-6e7ede00
-ap-south-1: ami-2374234c
-ap-southeast-1: ami-1c6a1860
-ap-southeast-2: ami-6b699409
-ca-central-1: ami-bba124df
-eu-central-1: ami-ff8b1690
-eu-west-1: ami-a5e578dc
-eu-west-2: ami-7d7a6119
-eu-west-3: ami-3bf14746
-sa-east-1: ami-e190d28d
-us-east-1: ami-3a456040
-us-east-2: ami-ef78528a
-us-west-1: ami-f33f3c93
-us-west-2: ami-31c47249
+ap-northeast-1: ami-04a90ea564e67f222
+ap-northeast-2: ami-0255a27fa98c0528f
+ap-northeast-3: ami-0cca427720a003215
+ap-south-1: ami-014f89aad9256d002
+ap-southeast-1: ami-0ce7dce420b05943b
+ap-southeast-2: ami-0ea64377e3b3b90df
+ca-central-1: ami-0cd24474b7cea7208
+eu-central-1: ami-7ab18291
+eu-west-1: ami-0e1dae876983c1547
+eu-west-2: ami-0ea3003a9feecc513
+eu-west-3: ami-0605f0c49b21107f6
+sa-east-1: ami-09657aae9df227834
+us-east-1: ami-063f3448775e0f9eb
+us-east-2: ami-0fc2efe14121a0f34
+us-west-1: ami-0c9cdda5cdea561c3
+us-west-2: ami-0bf2e2047ba8e44f2
 # ubuntu1604
-ap-northeast-1: ami-3667fc50
-ap-northeast-2: ami-137cdc7d
-ap-south-1: ami-e071268f
-ap-southeast-1: ami-5e681a22
-ap-southeast-2: ami-076d9065
-ca-central-1: ami-b9a124dd
-eu-central-1: ami-8b8b16e4
-eu-west-1: ami-6de57814
-eu-west-2: ami-b17a61d5
-eu-west-3: ami-41f1473c
-sa-east-1: ami-6393d10f
-us-east-1: ami-567e5b2c
-us-east-2: ami-b17852d4
-us-west-1: ami-0c3e3d6c
-us-west-2: ami-1bc77163
+ap-northeast-1: ami-0b98c32fcc2c04bc2
+ap-northeast-2: ami-0dc1b8292d6a0ab0b
+ap-northeast-3: ami-0681a1857c07f6497
+ap-south-1: ami-05247ccd8ba04d440
+ap-southeast-1: ami-0e2b3651fe0a5734f
+ap-southeast-2: ami-0ee926ceddd54fc92
+ca-central-1: ami-01d71e35696000851
+eu-central-1: ami-dec0f335
+eu-west-1: ami-075d0841a0892ca5c
+eu-west-2: ami-06fa81f4e8255415a
+eu-west-3: ami-0fbfe76a8613c0d41
+sa-east-1: ami-033feb8d267b3a76e
+us-east-1: ami-003db261b3fbed1e3
+us-east-2: ami-05d8e0d6e52ecd2ed
+us-west-1: ami-0cd5cc8977ef144a5
+us-west-2: ami-0d3a3ff36f97c7ba4