Merge branch 'develop' into master

Author: demartinofra

.travis.yml

@@ -27,7 +27,7 @@ script:
   - if [[ $TRAVIS_PYTHON_VERSION != 2.6 ]]; then
     pip install 'boto3==1.7.84' --force-reinstall;
     pip install moto
-    python cli/tests/cfncluster-unittest.py;
+    python cli/tests/pcluster-unittest.py;
     pip install boto3 --upgrade;
     fi
   - sh tests/test.sh

CHANGELOG.rst

@@ -2,6 +2,13 @@
 CHANGELOG
 =========
 
+2.0.0
+=====
+* Rename CfnCluster to AWS ParallelCluster
+* Support multiple EBS Volumes
+* Add AWS Batch as a supported scheduler
+* Support Custom AMI's
+
 1.6.1
 =====
 * Fix a bug in `cfncluster configure` introduced in 1.6.0
@@ -28,7 +35,7 @@ CHANGELOG
 1.5.2
 =====
 * feature:``cfncluster``: Added ClusterUser as a stack output. This makes it easier to get the username of the head node.
-* feature:``cfncluster``: Added `cfncluster ssh cluster_name`, this allows you to easily ssh into your clusters. It allows arbitrary command execution and extra ssh flags to be provided after the command. See https://cfncluster.readthedocs.io/en/latest/commands.html#ssh
+* feature:``cfncluster``: Added `cfncluster ssh cluster_name`, this allows you to easily ssh into your clusters. It allows arbitrary command execution and extra ssh flags to be provided after the command. See https://aws-parallelcluster.readthedocs.io/en/latest/commands.html#ssh
 * change:``cfncluster``: Moved global cli flags to the command specific flags. For example `cfncluster --region us-east-1 create` now becomes `cfncluster create --region us-east-1`
 * bugfix:``cfncluster-cookbook``: Fix bug that prevented c5d/m5d instances from working
 * bugfix:``cfncluster-cookbook``: Set CPU as a consumable resource in slurm
@@ -135,7 +142,7 @@ by incompatible dependencies.
 =====
 Offiical release of the CfnCluster 1.x CLI, templates and AMIs. Available in all regions except BJS, with
 support for Amazon Linux, CentOS 6 & 7 and Ubuntu 14.04 LTS. All AMIs are built via packer from the CfnCluster
-Cookbook project (https://github.com/awslabs/cfncluster-cookbook).
+Cookbook project (https://github.com/aws/aws-parallelcluster-cookbook).
 
 1.0.0-beta
 ==========

CONTRIBUTING.md

@@ -11,7 +11,7 @@ information to effectively respond to your bug report or contribution.
 
 We welcome you to use the GitHub issue tracker to report bugs or suggest features.
 
-When filing an issue, please check [existing open](https://github.com/awslabs/cfncluster/issues), or [recently closed](https://github.com/awslabs/cfncluster/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
+When filing an issue, please check [existing open](https://github.com/aws/aws-parallelcluster/issues), or [recently closed](https://github.com/aws/aws-parallelcluster/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20), issues to make sure somebody else hasn't already 
 reported the issue. Please try to include as much information as you can. Details like these are incredibly useful:
 
 * A reproducible test case or series of steps
@@ -41,7 +41,7 @@ GitHub provides additional document on [forking a repository](https://help.githu
 
 
 ## Finding contributions to work on
-Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/awslabs/cfncluster/labels/help%20wanted) issues is a great place to start. 
+Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels ((enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws/aws-parallelcluster/labels/help%20wanted) issues is a great place to start. 
 
 
 ## Code of Conduct
@@ -56,6 +56,6 @@ If you discover a potential security issue in this project we ask that you notif
 
 ## Licensing
 
-See the [LICENSE](https://github.com/awslabs/cfncluster/blob/develop/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
+See the [LICENSE](https://github.com/aws/aws-parallelcluster/blob/develop/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution.
 
 We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes.

NOTICE.txt

@@ -1,2 +1,2 @@
-cfncluster
-Copyright 2013-2014 Amazon.com, Inc. or its affiliates. All Rights Reserved. 
+aws-parallelcluster
+Copyright 2013-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.

README.rst

@@ -1,30 +1,29 @@
-==============================
-CfnCluster - HPC for the Cloud
-==============================
+=======================================
+AWS ParallelCluster - HPC for the Cloud
+=======================================
 
 |Build Status| |Version|
 
-.. |Build Status| image:: https://travis-ci.org/awslabs/cfncluster.png?branch=develop
-   :target: https://travis-ci.org/awslabs/cfncluster/
+.. |Build Status| image:: https://travis-ci.org/aws/aws-parallelcluster.png?branch=develop
+   :target: https://travis-ci.org/aws/aws-parallelcluster/
    :alt: Build Status
-.. |Version| image:: https://badge.fury.io/py/cfncluster.png
-    :target: https://badge.fury.io/py/cfncluster
+.. |Version| image:: https://badge.fury.io/py/aws-parallelcluster.png
+    :target: https://badge.fury.io/py/aws-parallelcluster
 
-CfnCluster ("cloud formation cluster") is a framework that deploys and
-maintains high performance computing clusters on Amazon Web Services
-(AWS). Developed by AWS, CfnCluster facilitates both quick start proof
-of concepts (POCs) and production deployments. CfnCluster supports
-many different types of clustered applications and can easily be
-extended to support different frameworks. The CLI is stateless,
-everything is done using CloudFormation or resources within AWS.\
+AWS ParallelCluster is an AWS supported Open Source cluster management tool that makes it easy for you to deploy and
+manage High Performance Computing (HPC) clusters in the AWS cloud.
+Built on the Open Source CfnCluster project, AWS ParallelCluster enables you to quickly build an HPC compute environment in AWS.
+It automatically sets up the required compute resources and a shared filesystem and offers a variety of batch schedulers such as AWS Batch, SGE, Torque, and Slurm.
+AWS ParallelCluster facilitates both quick start proof of concepts (POCs) and production deployments.
+You can build higher level workflows, such as a Genomics portal that automates the entire DNA sequencing workflow, on top of AWS ParallelCluster.\
 
 Quick Start
 -----------
 First, install the library:
 
 .. code-block:: sh
 
-    $ pip install cfncluster
+    $ pip install aws-parallelcluster
 
 Next, configure your aws credentials and default region:
 
@@ -36,11 +35,11 @@ Next, configure your aws credentials and default region:
     Default region name [us-east-1]:
     Default output format [None]:
 
-Then, run cfncluster configure:
+Then, run pcluster configure:
 
 .. code-block:: ini
 
-  $ cfncluster configure
+  $ pcluster configure
   Cluster Template [default]:
   AWS Access Key ID []:
   AWS Secret Access Key ID []:
@@ -69,14 +68,14 @@ Now you can create your first cluster;
 
 .. code-block:: sh
 
-  $ cfncluster create myfirstcluster
+  $ pcluster create myfirstcluster
 
 
 After the cluster finishes creating, log in:
 
 .. code-block:: sh
 
-  $ cfncluster ssh myfirstcluster
+  $ pcluster ssh myfirstcluster
 
 You can view the running compute hosts:
 
@@ -86,21 +85,21 @@ You can view the running compute hosts:
 
 For more information on any of these steps see the `Getting Started Guide`_.
 
-.. _`Getting Started Guide`: https://cfncluster.readthedocs.io/en/latest/getting_started.html
+.. _`Getting Started Guide`: https://aws-parallelcluster.readthedocs.io/en/latest/getting_started.html
 
 Documentation
 -------------
 
 Documentation is part of the project and is published to -
-https://cfncluster.readthedocs.io/. Of most interest to new users is
+https://aws-parallelcluster.readthedocs.io/. Of most interest to new users is
 the Getting Started Guide -
-https://cfncluster.readthedocs.io/en/latest/getting_started.html.
+https://aws-parallelcluster.readthedocs.io/en/latest/getting_started.html.
 
 Issues
 ------
 
 Please open a GitHub issue for any feedback or issues:
-https://github.com/awslabs/cfncluster.  There is also an active AWS
+https://github.com/aws/aws-parallelcluster.  There is also an active AWS
 HPC forum which may be helpful:https://forums.aws.amazon.com/forum.jspa?forumID=192.
 
 Changes
@@ -112,11 +111,11 @@ Between CfnCluster 1.5.4 and 1.6.0 we made a change to the CfnClusterInstancePol
 on objects in <REGION>-cfncluster bucket, "autoscaling:SetDesiredCapacity", "autoscaling:DescribeTags" permissions and
 "cloudformation:DescribeStacks" permissions on <REGION>:<ACCOUNT_ID>:stack/cfncluster-*.
 
-If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://cfncluster.readthedocs.io/en/latest/iam.html
+If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://aws-parallelcluster.readthedocs.io/en/latest/iam.html
 
 CfnCluster 1.5 IAM Change
 =========================
-Between CfnCluster 1.4.2 and 1.5.0 we made a change to the CfnClusterInstancePolicy that adds “ec2:DescribeVolumes” permissions. If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://cfncluster.readthedocs.io/en/latest/iam.html
+Between CfnCluster 1.4.2 and 1.5.0 we made a change to the CfnClusterInstancePolicy that adds “ec2:DescribeVolumes” permissions. If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://aws-parallelcluster.readthedocs.io/en/latest/iam.html
 
 CfnCluster 1.2 and Earlier
 ==========================

amis.txt

@@ -1,88 +1,88 @@
 # alinux
-ap-northeast-1: ami-0775ab7ad13c0e74f
-ap-northeast-2: ami-09fdf91b4a93730f0
-ap-northeast-3: ami-03a0e4ad407dd7f55
-ap-south-1: ami-008d194619d177765
-ap-southeast-1: ami-08eef0c7d23753d49
-ap-southeast-2: ami-08d6ef8690d843ceb
-ca-central-1: ami-00f519fdeb5feb438
-eu-central-1: ami-07fa07aba1f6fc571
-eu-west-1: ami-09cbb5230da5b6cc2
-eu-west-2: ami-006cd2956238a6dd7
-eu-west-3: ami-0fce6e16d9970d71b
-sa-east-1: ami-0165d34de1afd1618
-us-east-1: ami-0fd8b1245d1435bf3
-us-east-2: ami-0cb3d04ae4b4ca8db
-us-gov-west-1: ami-e0fd6781
-us-west-1: ami-02f24f766b5b7502f
-us-west-2: ami-01e7f12fc989e7004
+ap-northeast-1: ami-0fb9daffb5833ffc4
+ap-northeast-2: ami-0b167d9bbfde62d2d
+ap-northeast-3: ami-0e9881249ab6eabd3
+ap-south-1: ami-0b2c8ee36792079df
+ap-southeast-1: ami-0c0290b9dbf8c8bd9
+ap-southeast-2: ami-06a2306b0b6184107
+ca-central-1: ami-0c8d004dda0a11aa4
+eu-central-1: ami-02495439420c1db34
+eu-west-1: ami-05e3881929ee33ba0
+eu-west-2: ami-0339820b76b2327dc
+eu-west-3: ami-0e331cfd2773c0e74
+sa-east-1: ami-0dfb418ac5bf741bd
+us-east-1: ami-05e07d30f513f27a2
+us-east-2: ami-0d1983025f3f753c0
+us-gov-west-1: ami-13492d72
+us-west-1: ami-044945136c783dbdc
+us-west-2: ami-037856388ec78af89
 # centos6
-ap-northeast-1: ami-0f676dc2bebea5b1a
-ap-northeast-2: ami-0adf39100d009ad31
-ap-northeast-3: ami-0da12c87d2aee0ef5
-ap-south-1: ami-05a8a28e5c8b74642
-ap-southeast-1: ami-02ce4e46ca873697e
-ap-southeast-2: ami-013201a7fda637e83
-ca-central-1: ami-0ee6c95ad927b2da4
-eu-central-1: ami-0de3eb16b173ea6b3
-eu-west-1: ami-0c8eca7deb52169cc
-eu-west-2: ami-013d7f40fc791f845
-eu-west-3: ami-0cac2d11252d18c27
-sa-east-1: ami-074dbf929dc0c8de0
-us-east-1: ami-044fd430f1fd3b163
-us-east-2: ami-03c508534d6f54859
-us-west-1: ami-05ee27a886e16ba39
-us-west-2: ami-08675896633a1d9fb
+ap-northeast-1: ami-0f011ff8ac9f6747e
+ap-northeast-2: ami-019893c1e62bbbe4c
+ap-northeast-3: ami-0962d5772eaffc71b
+ap-south-1: ami-0ccacbf180ed01af6
+ap-southeast-1: ami-012f0ce0586712f3b
+ap-southeast-2: ami-0247c080617d7647e
+ca-central-1: ami-0173b3a7df2c7fe02
+eu-central-1: ami-01de586f622c82f50
+eu-west-1: ami-05d89791118e3da01
+eu-west-2: ami-080489d48a0a89fc4
+eu-west-3: ami-014fd03a63b8d5e77
+sa-east-1: ami-07200200ab715e696
+us-east-1: ami-071dd743cb11df395
+us-east-2: ami-02b9b39628ae6df5f
+us-west-1: ami-017551e9c12e498fd
+us-west-2: ami-0a10c94576b3ecb09
 # centos7
-ap-northeast-1: ami-0b13eb5fa94691483
-ap-northeast-2: ami-015bf65572df2d94d
-ap-northeast-3: ami-0479721fa287d3cf7
-ap-south-1: ami-087bda89547e089dc
-ap-southeast-1: ami-0d339c926f9d35ba3
-ap-southeast-2: ami-0ac15e60210a10310
-ca-central-1: ami-068d6a6fc5e7987f3
-eu-central-1: ami-0f73d5d5bb22f7609
-eu-west-1: ami-05479870b31fa535e
-eu-west-2: ami-08c5c6f990f8253e1
-eu-west-3: ami-0d6c4b2dde29e341f
-sa-east-1: ami-00cfa447d16329380
-us-east-1: ami-0fb2dca992af734f2
-us-east-2: ami-03421fa006b964767
-us-west-1: ami-012c5239be2beb7f1
-us-west-2: ami-0053bef6747ca58bf
+ap-northeast-1: ami-090d018e88f37c1ae
+ap-northeast-2: ami-00d21772254147f94
+ap-northeast-3: ami-008ec06778d170786
+ap-south-1: ami-00cc7c0cc22c258fd
+ap-southeast-1: ami-0a85db3d9cda2d8d2
+ap-southeast-2: ami-014a4f4fcc73ab85b
+ca-central-1: ami-0b68fd6ce4b4a6d50
+eu-central-1: ami-02e343666efe2dccd
+eu-west-1: ami-067629e4a0b0728a5
+eu-west-2: ami-014e7688066be8dbe
+eu-west-3: ami-02788c7972bd10d9d
+sa-east-1: ami-03c901096ca366979
+us-east-1: ami-0d9cb7ba04c4bbede
+us-east-2: ami-0d2b2af3c7a732c30
+us-west-1: ami-0409f18b67cb6710c
+us-west-2: ami-08d9ffbe01a799d21
 # ubuntu1404
-ap-northeast-1: ami-05b67b05deba61e8c
-ap-northeast-2: ami-06aa825c33309fdfd
-ap-northeast-3: ami-0c03a23da1606fa3a
-ap-south-1: ami-096a418446cd1eeb1
-ap-southeast-1: ami-0d413135984f7bd97
-ap-southeast-2: ami-079737fc9c635c09a
-ca-central-1: ami-00cec9c492f236616
-eu-central-1: ami-0d6242ced07309e7b
-eu-west-1: ami-06581a7f68c539440
-eu-west-2: ami-00621b5f273783bce
-eu-west-3: ami-07da72e5a66a93c71
-sa-east-1: ami-0f4a298763a012d8e
-us-east-1: ami-07df96c1d8e62161d
-us-east-2: ami-0f7c7f69478093925
-us-gov-west-1: ami-38f36959
-us-west-1: ami-077e3b1efa9443476
-us-west-2: ami-0581577b81f865ae9
+ap-northeast-1: ami-0fa07bfb48436656a
+ap-northeast-2: ami-01520d0422206114f
+ap-northeast-3: ami-0e21c77589336b5c4
+ap-south-1: ami-0147366ed5d293883
+ap-southeast-1: ami-05f63ebee1896fe32
+ap-southeast-2: ami-0d4b5f0b7af25bdaf
+ca-central-1: ami-045ba5e8c740fc2c1
+eu-central-1: ami-08e581f6d5df7b0f2
+eu-west-1: ami-0727593a99bb922e2
+eu-west-2: ami-06116de8f1691dc3c
+eu-west-3: ami-0d9be8cd40a8afdeb
+sa-east-1: ami-025b9814ede84b028
+us-east-1: ami-0ac1cd261b0bc4d3d
+us-east-2: ami-0939ee584b81081ce
+us-gov-west-1: ami-4e4a2e2f
+us-west-1: ami-072ad76b3dfc8a1b9
+us-west-2: ami-04f7f82e706117342
 # ubuntu1604
-ap-northeast-1: ami-0232ea8f3f82712a2
-ap-northeast-2: ami-08d64fa4a89f85427
-ap-northeast-3: ami-0cfdcf1603db9a672
-ap-south-1: ami-009f971485befe66a
-ap-southeast-1: ami-02ab2d97bea65a826
-ap-southeast-2: ami-0fc63c095a0c31f0a
-ca-central-1: ami-0705c0b15666e7d60
-eu-central-1: ami-08ab664e40e331729
-eu-west-1: ami-096183a7e31fc4e35
-eu-west-2: ami-0a4fbb1dd660013b5
-eu-west-3: ami-0fcf3de0f1913a190
-sa-east-1: ami-06fa4a172ebe11a10
-us-east-1: ami-0b9750a5856891754
-us-east-2: ami-0a2c6075e91d8bbc3
-us-gov-west-1: ami-3bf3695a
-us-west-1: ami-00b0c33ebc9576bf8
-us-west-2: ami-00e5e4cc71cf05d6e
+ap-northeast-1: ami-0e8e2b0e96662ca6b
+ap-northeast-2: ami-0c30bc54925d838fa
+ap-northeast-3: ami-0e70a099e50905585
+ap-south-1: ami-0f535a2041e9b94c8
+ap-southeast-1: ami-090160e13ca207e65
+ap-southeast-2: ami-076f3636f63298dda
+ca-central-1: ami-01e064e69b7c2aca6
+eu-central-1: ami-09708ba08fb4eaa9f
+eu-west-1: ami-02098801bc93be814
+eu-west-2: ami-0ae70974baddc132f
+eu-west-3: ami-04840af5545e98072
+sa-east-1: ami-07b8d3b5a51038201
+us-east-1: ami-0b64b4f7022c87277
+us-east-2: ami-07136e4972929b2e3
+us-gov-west-1: ami-a14b2fc0
+us-west-1: ami-037fe607e2ef6f027
+us-west-2: ami-06f6aa06055c554aa

cli/MANIFEST.in

@@ -1,2 +1,3 @@
-recursive-include cfncluster/cloudformation *
-recursive-include cfncluster/examples *
+recursive-include pcluster/cloudformation *
+recursive-include pcluster/examples *
+recursive-include pcluster/resources *