Skip to content

Commit 11f29da

Browse files
sean-smithsean-smith
authored
Release 1.6.0
Release 1.6.0
2 parents b1714d8 + 1833134 commit 11f29da

31 files changed

+1455
-940
lines changed

‎CHANGELOG.rst

+8
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22
CHANGELOG
33
=========
44

5+
1.6.0
6+
=====
7+
* Refactor scaling up to take into account the number of pending/requested jobs/slots and instance slots.
8+
* Refactor scaling down to scale down faster and take advantage of per-second billing.
9+
* Add `scaledown_idletime` parameter as part of scale-down refactoring
10+
* Lock hosts before termination to ensure removal of dead compute nodes from host list
11+
* Fix HTTP proxy support
12+
513
1.5.4
614
=====
715
* Add option to disable ganglia `extra_json = { "cfncluster" : { "ganglia_enabled" : "no" } }`

‎README.rst

+89-8
Original file line numberDiff line numberDiff line change
@@ -1,36 +1,117 @@
1-
CfnCluster
2-
==========
1+
==============================
2+
CfnCluster - HPC for the Cloud
3+
==============================
34

4-
.. image:: https://travis-ci.org/awslabs/cfncluster.png?branch=develop
5-
:target: https://travis-ci.org/awslabs/cfncluster
5+
|Build Status| |Version|
6+
7+
.. |Build Status| image:: https://travis-ci.org/awslabs/cfncluster.png?branch=develop
8+
:target: https://travis-ci.org/awslabs/cfncluster/
69
:alt: Build Status
10+
.. |Version| image:: https://badge.fury.io/py/cfncluster.png
11+
:target: https://badge.fury.io/py/cfncluster
712

813
CfnCluster ("cloud formation cluster") is a framework that deploys and
914
maintains high performance computing clusters on Amazon Web Services
1015
(AWS). Developed by AWS, CfnCluster facilitates both quick start proof
1116
of concepts (POCs) and production deployments. CfnCluster supports
1217
many different types of clustered applications and can easily be
1318
extended to support different frameworks. The CLI is stateless,
14-
everything is done using CloudFormation or resources within AWS.
19+
everything is done using CloudFormation or resources within AWS.\
20+
21+
Quick Start
22+
-----------
23+
First, install the library:
24+
25+
.. code-block:: sh
26+
27+
$ pip install cfncluster
28+
29+
Next, configure your aws credentials and default region:
30+
31+
.. code-block:: sh
32+
33+
$ aws configure
34+
AWS Access Key ID [None]: YOUR_KEY
35+
AWS Secret Access Key [None]: YOUR_SECRET
36+
Default region name [us-east-1]:
37+
Default output format [None]:
38+
39+
Then, run cfncluster configure:
40+
41+
.. code-block:: ini
42+
43+
$ cfncluster configure
44+
Cluster Template [default]:
45+
AWS Access Key ID []:
46+
AWS Secret Access Key ID []:
47+
Acceptable Values for AWS Region ID:
48+
ap-south-1
49+
...
50+
us-west-2
51+
AWS Region ID [us-east-1]:
52+
VPC Name [myvpc]:
53+
Acceptable Values for Key Name:
54+
keypair1
55+
keypair-test
56+
production-key
57+
Key Name []:
58+
Acceptable Values for VPC ID:
59+
vpc-1kd24879
60+
vpc-blk4982d
61+
VPC ID []:
62+
Acceptable Values for Master Subnet ID:
63+
subnet-9k284a6f
64+
subnet-1k01g357
65+
subnet-b921nv04
66+
Master Subnet ID []:
67+
68+
Now you can create your first cluster;
69+
70+
.. code-block:: sh
71+
72+
$ cfncluster create myfirstcluster
73+
74+
75+
After the cluster finishes creating, log in:
76+
77+
.. code-block:: sh
78+
79+
$ cfncluster ssh myfirstcluster
80+
81+
You can view the running compute hosts:
82+
83+
.. code-block:: sh
84+
85+
$ qhost
86+
87+
For more information on any of these steps see the `Getting Started Guide`_.
88+
89+
.. _`Getting Started Guide`: https://cfncluster.readthedocs.io/en/latest/getting_started.html
1590

1691
Documentation
17-
=============
92+
-------------
1893

1994
Documentation is part of the project and is published to -
2095
https://cfncluster.readthedocs.io/. Of most interest to new users is
2196
the Getting Started Guide -
2297
https://cfncluster.readthedocs.io/en/latest/getting_started.html.
2398

2499
Issues
25-
======
100+
------
26101

27102
Please open a GitHub issue for any feedback or issues:
28103
https://github.com/awslabs/cfncluster. There is also an active AWS
29104
HPC forum which may be helpful:https://forums.aws.amazon.com/forum.jspa?forumID=192.
30105

106+
Changes
107+
-------
108+
31109
CfnCluster 1.6 IAM Change
32110
=========================
33-
Between CfnCluster 1.5.3 and 1.6.0 we made a change to the CfnClusterInstancePolicy that adds “s3:GetObject” permissions on objects in <REGION>-cfncluster bucket.
111+
Between CfnCluster 1.5.3 and 1.6.0 we made a change to the CfnClusterInstancePolicy that adds “s3:GetObject” permissions
112+
on objects in <REGION>-cfncluster bucket, "autoscaling:SetDesiredCapacity", "autoscaling:DescribeTags" permissions and
113+
"cloudformation:DescribeStacks" permissions on <REGION>:<ACCOUNT_NAME>:<STACK_NAME>.
114+
34115
If you’re using a custom policy (e.g. you specify "ec2_iam_role" in your config) be sure it includes this new permission. See https://cfncluster.readthedocs.io/en/latest/iam.html
35116

36117
CfnCluster 1.5 IAM Change

‎amis.txt

+83-83
Original file line numberDiff line numberDiff line change
@@ -1,88 +1,88 @@
11
# alinux
2-
ap-northeast-1: ami-00047986f877e6760
3-
ap-northeast-2: ami-0ca5afbc9d0b20f3d
4-
ap-northeast-3: ami-0b3af1c2fe35b1ae4
5-
ap-south-1: ami-0250a8315b099785d
6-
ap-southeast-1: ami-038312aa862774599
7-
ap-southeast-2: ami-0dce078cd2c343a5c
8-
ca-central-1: ami-04e27bd54e171fbad
9-
eu-central-1: ami-075bf1c36cf6b855f
10-
eu-west-1: ami-08d507a759b9ee0e4
11-
eu-west-2: ami-03ddfb1462e2c92fd
12-
eu-west-3: ami-0e72015cf2b21159f
13-
sa-east-1: ami-02bd755bfe8704cef
14-
us-east-1: ami-07f61c3e18538c70c
15-
us-east-2: ami-096b7925b1f9c8cbb
16-
us-gov-west-1: ami-cfd34dae
17-
us-west-1: ami-04794da830cb9dc86
18-
us-west-2: ami-03f86a2920360dfbe
2+
ap-northeast-1: ami-0775ab7ad13c0e74f
3+
ap-northeast-2: ami-09fdf91b4a93730f0
4+
ap-northeast-3: ami-03a0e4ad407dd7f55
5+
ap-south-1: ami-008d194619d177765
6+
ap-southeast-1: ami-08eef0c7d23753d49
7+
ap-southeast-2: ami-08d6ef8690d843ceb
8+
ca-central-1: ami-00f519fdeb5feb438
9+
eu-central-1: ami-07fa07aba1f6fc571
10+
eu-west-1: ami-09cbb5230da5b6cc2
11+
eu-west-2: ami-006cd2956238a6dd7
12+
eu-west-3: ami-0fce6e16d9970d71b
13+
sa-east-1: ami-0165d34de1afd1618
14+
us-east-1: ami-0fd8b1245d1435bf3
15+
us-east-2: ami-0cb3d04ae4b4ca8db
16+
us-gov-west-1: ami-e0fd6781
17+
us-west-1: ami-02f24f766b5b7502f
18+
us-west-2: ami-01e7f12fc989e7004
1919
# centos6
20-
ap-northeast-1: ami-00c9ad360d255b462
21-
ap-northeast-2: ami-08e84ffe9aead7873
22-
ap-northeast-3: ami-056531169587ffc23
23-
ap-south-1: ami-03a57d339b58b2f23
24-
ap-southeast-1: ami-00063e5635939c9d1
25-
ap-southeast-2: ami-015e730525001bf53
26-
ca-central-1: ami-0fea713db092ac77f
27-
eu-central-1: ami-0f6b91d011f44aaf6
28-
eu-west-1: ami-029accac823a2a844
29-
eu-west-2: ami-032b6b895dab544c6
30-
eu-west-3: ami-039be4bd46c878e2c
31-
sa-east-1: ami-0e507bdec09b8a091
32-
us-east-1: ami-091bbc83b27f92634
33-
us-east-2: ami-0fa89e68b0222f50b
34-
us-west-1: ami-06f3909be633646b9
35-
us-west-2: ami-087d4f29315fd19d2
20+
ap-northeast-1: ami-0f676dc2bebea5b1a
21+
ap-northeast-2: ami-0adf39100d009ad31
22+
ap-northeast-3: ami-0da12c87d2aee0ef5
23+
ap-south-1: ami-05a8a28e5c8b74642
24+
ap-southeast-1: ami-02ce4e46ca873697e
25+
ap-southeast-2: ami-013201a7fda637e83
26+
ca-central-1: ami-0ee6c95ad927b2da4
27+
eu-central-1: ami-0de3eb16b173ea6b3
28+
eu-west-1: ami-0c8eca7deb52169cc
29+
eu-west-2: ami-013d7f40fc791f845
30+
eu-west-3: ami-0cac2d11252d18c27
31+
sa-east-1: ami-074dbf929dc0c8de0
32+
us-east-1: ami-044fd430f1fd3b163
33+
us-east-2: ami-03c508534d6f54859
34+
us-west-1: ami-05ee27a886e16ba39
35+
us-west-2: ami-08675896633a1d9fb
3636
# centos7
37-
ap-northeast-1: ami-09dc3b175ed2905f8
38-
ap-northeast-2: ami-03f494fed9f0f9c98
39-
ap-northeast-3: ami-04094dd3b4013b75f
40-
ap-south-1: ami-0c20297a2c00b0528
41-
ap-southeast-1: ami-0f610508513f2012f
42-
ap-southeast-2: ami-09d0b89439c0c1b65
43-
ca-central-1: ami-0362a3ef26ab5dfce
44-
eu-central-1: ami-03d5519f5f4e7619f
45-
eu-west-1: ami-0575304add4986551
46-
eu-west-2: ami-08263362b21b98916
47-
eu-west-3: ami-0befb803ea7ce1b83
48-
sa-east-1: ami-0860c11596f954bbc
49-
us-east-1: ami-06c93be112f880acd
50-
us-east-2: ami-0493261a1b2adfd27
51-
us-west-1: ami-0af95249ebb8de7a5
52-
us-west-2: ami-0a259cf8220fabdcb
37+
ap-northeast-1: ami-0b13eb5fa94691483
38+
ap-northeast-2: ami-015bf65572df2d94d
39+
ap-northeast-3: ami-0479721fa287d3cf7
40+
ap-south-1: ami-087bda89547e089dc
41+
ap-southeast-1: ami-0d339c926f9d35ba3
42+
ap-southeast-2: ami-0ac15e60210a10310
43+
ca-central-1: ami-068d6a6fc5e7987f3
44+
eu-central-1: ami-0f73d5d5bb22f7609
45+
eu-west-1: ami-05479870b31fa535e
46+
eu-west-2: ami-08c5c6f990f8253e1
47+
eu-west-3: ami-0d6c4b2dde29e341f
48+
sa-east-1: ami-00cfa447d16329380
49+
us-east-1: ami-0fb2dca992af734f2
50+
us-east-2: ami-03421fa006b964767
51+
us-west-1: ami-012c5239be2beb7f1
52+
us-west-2: ami-0053bef6747ca58bf
5353
# ubuntu1404
54-
ap-northeast-1: ami-09dc9e9da730c0248
55-
ap-northeast-2: ami-06346bcc8b825a458
56-
ap-northeast-3: ami-0ae6e02016436404a
57-
ap-south-1: ami-0e4294a5b9d5de425
58-
ap-southeast-1: ami-01e6359021e3bac23
59-
ap-southeast-2: ami-0cfebd51a3470f1d4
60-
ca-central-1: ami-0097472502a04776d
61-
eu-central-1: ami-0a82ac2872821deef
62-
eu-west-1: ami-0d6c90f092266b787
63-
eu-west-2: ami-0f23dffc4db122b53
64-
eu-west-3: ami-02b6ca8fd14f58265
65-
sa-east-1: ami-04e943b561cd32659
66-
us-east-1: ami-0920fa86d5a85a07a
67-
us-east-2: ami-0d63a602f26709e70
68-
us-gov-west-1: ami-c2d34da3
69-
us-west-1: ami-06fe4f837d8715dbb
70-
us-west-2: ami-01170828a21d86a72
54+
ap-northeast-1: ami-05b67b05deba61e8c
55+
ap-northeast-2: ami-06aa825c33309fdfd
56+
ap-northeast-3: ami-0c03a23da1606fa3a
57+
ap-south-1: ami-096a418446cd1eeb1
58+
ap-southeast-1: ami-0d413135984f7bd97
59+
ap-southeast-2: ami-079737fc9c635c09a
60+
ca-central-1: ami-00cec9c492f236616
61+
eu-central-1: ami-0d6242ced07309e7b
62+
eu-west-1: ami-06581a7f68c539440
63+
eu-west-2: ami-00621b5f273783bce
64+
eu-west-3: ami-07da72e5a66a93c71
65+
sa-east-1: ami-0f4a298763a012d8e
66+
us-east-1: ami-07df96c1d8e62161d
67+
us-east-2: ami-0f7c7f69478093925
68+
us-gov-west-1: ami-38f36959
69+
us-west-1: ami-077e3b1efa9443476
70+
us-west-2: ami-0581577b81f865ae9
7171
# ubuntu1604
72-
ap-northeast-1: ami-0b0bf645cde38fe52
73-
ap-northeast-2: ami-0b2c6c9e988689d7c
74-
ap-northeast-3: ami-0dae0e941e3bce10b
75-
ap-south-1: ami-011d655005fe854e9
76-
ap-southeast-1: ami-03428422e2cb20c9f
77-
ap-southeast-2: ami-0bbef9e2b1170757f
78-
ca-central-1: ami-0916119fde83c29b7
79-
eu-central-1: ami-003273c4072efbe43
80-
eu-west-1: ami-009af0d819973e1d6
81-
eu-west-2: ami-029f01bbfec52a10e
82-
eu-west-3: ami-0ae405adcb3d1477c
83-
sa-east-1: ami-0b0057ecd5c465d09
84-
us-east-1: ami-0dfb1662fcc845c55
85-
us-east-2: ami-01bd1361428c384c3
86-
us-gov-west-1: ami-0fd6486e
87-
us-west-1: ami-0e144dd24634766a2
88-
us-west-2: ami-02087bf0ff99cc127
72+
ap-northeast-1: ami-0232ea8f3f82712a2
73+
ap-northeast-2: ami-08d64fa4a89f85427
74+
ap-northeast-3: ami-0cfdcf1603db9a672
75+
ap-south-1: ami-009f971485befe66a
76+
ap-southeast-1: ami-02ab2d97bea65a826
77+
ap-southeast-2: ami-0fc63c095a0c31f0a
78+
ca-central-1: ami-0705c0b15666e7d60
79+
eu-central-1: ami-08ab664e40e331729
80+
eu-west-1: ami-096183a7e31fc4e35
81+
eu-west-2: ami-0a4fbb1dd660013b5
82+
eu-west-3: ami-0fcf3de0f1913a190
83+
sa-east-1: ami-06fa4a172ebe11a10
84+
us-east-1: ami-0b9750a5856891754
85+
us-east-2: ami-0a2c6075e91d8bbc3
86+
us-gov-west-1: ami-3bf3695a
87+
us-west-1: ami-00b0c33ebc9576bf8
88+
us-west-2: ami-00e5e4cc71cf05d6e

0 commit comments

Comments
 (0)