Skip to content

Commit 47b8751

Browse files
enrico-usaienrico-usai
authored
Release 2.3.1
Merge Release 2.3.1
2 parents d2f78ad + 546d1b6 commit 47b8751

File tree

74 files changed

+1912
-3789
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

74 files changed

+1912
-3789
lines changed

Diff for: ‎.github/issue_template.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,5 @@ Any other context about the problem. E.g.:
1919
- configuration file without any credentials or personal data.
2020
- pre/post-install scripts, if any
2121
- screenshots, if useful
22-
- if the cluster fails creation, please re-execute `create` action using `--norollback` option and attach `/var/cfn-init.log`, `/var/log/cloud-init.log` and `/var/log/cloud-init-output.log` files from the Master node
22+
- if the cluster fails creation, please re-execute `create` action using `--norollback` option and attach `/var/log/cfn-init.log`, `/var/log/cloud-init.log` and `/var/log/cloud-init-output.log` files from the Master node
2323
- if you encounter scaling problems please attach `/var/log/nodewatcher` from the Compute node and `/var/log/jobwatcher` and `/var/log/sqswatcher` from the Master node

Diff for: ‎.github/no-response.yml

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Configuration for probot-no-response - https://github.com/probot/no-response
2+
3+
# Number of days of inactivity before an Issue is closed for lack of response
4+
daysUntilClose: 14
5+
# Label requiring a response
6+
responseRequiredLabel: closing-soon-if-no-response
7+
# Comment to post when closing an Issue for lack of response. Set to `false` to disable
8+
closeComment: >
9+
This issue has been automatically closed because there has been no response
10+
to our request for more information from the original author. With only the
11+
information that is currently in the issue, we don't have enough information
12+
to take action. Please reach out if you have or find the answers we need so
13+
that we can investigate further.

Diff for: ‎.travis.yml

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
11
language: python
2-
sudo: false
2+
sudo: required
3+
dist: xenial
34

45
python:
5-
- "2.6"
66
- "2.7"
77
- "3.4"
88
- "3.5"
99
- "3.6"
10+
- "3.7"
1011

1112
matrix:
1213
include:

Diff for: ‎CHANGELOG.rst

+32
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,38 @@
22
CHANGELOG
33
=========
44

5+
2.3.1
6+
=====
7+
8+
**ENHANCEMENTS**
9+
10+
* Add support for FSx Lustre with Amazon Linux. In case of custom AMI,
11+
The kernel will need to be ``>= 4.14.104-78.84.amzn1.x86_64``
12+
* Slurm
13+
* set compute nodes to DRAIN state before removing them from cluster. This prevents the scheduler from submitting a job to a node that is being terminated.
14+
* dynamically adjust max cluster size based on ASG settings
15+
* dynamically change the number of configured FUTURE nodes based on the actual nodes that join the cluster. The max size of the cluster seen by the scheduler always matches the max capacity of the ASG.
16+
* process nodes added to or removed from the cluster in batches. This speeds up cluster scaling which is able to react with a delay of less than 1 minute to variations in the ASG capacity.
17+
* add support for job dependencies and pending reasons. The cluster won't scale up if the job cannot start due to an unsatisfied dependency.
18+
* set ``ReturnToService=1`` in scheduler config in order to recover instances that were initially marked as down due to a transient issue.
19+
* Validate FSx parameters. Fixes `#896 <https://github.com/aws/aws-parallelcluster/issues/896>`_ .
20+
21+
**CHANGES**
22+
23+
* Slurm - Upgrade version to 18.08.6.2
24+
* NVIDIA - update drivers to version 418.56
25+
* CUDA - update toolkit to version 10.0
26+
* Increase default EBS volume size from 15GB to 17GB
27+
* Disabled updates to FSx File Systems, updates to most parameters would cause the filesystem, and all it's data, to be deleted
28+
29+
**BUG FIXES**
30+
31+
* Cookbook wasn't fetched when `custom_ami` parameter specified in the config
32+
* Cfn-init is now fetched from us-east-1, this bug effected non-alinux custom ami's in regions other than us-east-1.
33+
* Account limit check not done for SPOT or AWS Batch Clusters
34+
* Account limit check fall back to master subnet. Fixes `#910 <https://github.com/aws/aws-parallelcluster/issues/910>`_ .
35+
* Boto3 upperbound removed
36+
537
2.2.1
638
=====
739

Diff for: ‎amis.txt

+95-95
Original file line numberDiff line numberDiff line change
@@ -1,100 +1,100 @@
11
# alinux
2-
ap-northeast-1: ami-01e7e307e734daa9b
3-
ap-northeast-2: ami-096da9252851971b7
4-
ap-northeast-3: ami-0fde1cce0915721ec
5-
ap-south-1: ami-0856113449f34cfde
6-
ap-southeast-1: ami-019762c344b80574a
7-
ap-southeast-2: ami-09ddff457fb3815ea
8-
ca-central-1: ami-0ef3d34adf231688a
9-
cn-north-1: ami-053a5a11a4ac83842
10-
cn-northwest-1: ami-0c5a99564e44467e7
11-
eu-central-1: ami-01d7252afc45b0d8b
12-
eu-north-1: ami-02224ba7786413561
13-
eu-west-1: ami-0a32ae196621ce1cd
14-
eu-west-2: ami-0e8d810df84d61f8b
15-
eu-west-3: ami-09a1847f7683c6351
16-
sa-east-1: ami-03c4f6e67dcea925d
17-
us-east-1: ami-096b5898281e68ea3
18-
us-east-2: ami-0c8b41c511db3c17c
19-
us-gov-east-1: ami-08cff3f9ef830bdfb
20-
us-gov-west-1: ami-afa0c8ce
21-
us-west-1: ami-08e9806b160f9aa44
22-
us-west-2: ami-04eba5b9de0d94ec6
2+
ap-northeast-1: ami-0af8c1a29f58c3b91
3+
ap-northeast-2: ami-036c289fda8701f9d
4+
ap-northeast-3: ami-000902aa3082732ce
5+
ap-south-1: ami-00ff6216daa4b0a69
6+
ap-southeast-1: ami-03b015a13daa9ff8d
7+
ap-southeast-2: ami-0c2528255cc7c4cec
8+
ca-central-1: ami-05bad5df22b9502e5
9+
cn-north-1: ami-0227bedfc6798cba1
10+
cn-northwest-1: ami-08143603c5f390f20
11+
eu-central-1: ami-003262ea853b26050
12+
eu-north-1: ami-06cac8aed0729f14c
13+
eu-west-1: ami-0691d6d6d4d209e09
14+
eu-west-2: ami-0d241a5c57ee3421d
15+
eu-west-3: ami-0e59dd1d2794a857c
16+
sa-east-1: ami-07b044055a13cf93e
17+
us-east-1: ami-0f8b01b1377483305
18+
us-east-2: ami-049afa5b53a7880d8
19+
us-gov-east-1: ami-02ee5c66a10526bd1
20+
us-gov-west-1: ami-7da7d01c
21+
us-west-1: ami-02c87842ea944292e
22+
us-west-2: ami-09b457d5cba24514a
2323
# centos6
24-
ap-northeast-1: ami-09c0d7b7eba653962
25-
ap-northeast-2: ami-04bb0577f1425f61f
26-
ap-northeast-3: ami-092c70a47ecb4f730
27-
ap-south-1: ami-0b6b4a5c5952ba214
28-
ap-southeast-1: ami-0d8fee73833faed87
29-
ap-southeast-2: ami-03e0e45d59cf130b3
30-
ca-central-1: ami-0f5cabbce86422cc0
31-
eu-central-1: ami-0c47523135cb69662
32-
eu-north-1: ami-064af23ad3c8bac88
33-
eu-west-1: ami-048edb0ed06f91ef8
34-
eu-west-2: ami-098b95acdef72fd6c
35-
eu-west-3: ami-0e25f1a2ef5f8d237
36-
sa-east-1: ami-07bcda88df87d3db2
37-
us-east-1: ami-0fd0ae64eb509cc23
38-
us-east-2: ami-09c42f6a076482eeb
39-
us-west-1: ami-060bcf74fb6278986
40-
us-west-2: ami-009c08111f57a2d4c
24+
ap-northeast-1: ami-0476984f547d1f4f2
25+
ap-northeast-2: ami-06ecb1e81881cd450
26+
ap-northeast-3: ami-04d195b55ddf56228
27+
ap-south-1: ami-0b1abd2bf8810487c
28+
ap-southeast-1: ami-0576b4b2db8272abf
29+
ap-southeast-2: ami-09a18baab0a142123
30+
ca-central-1: ami-0aa03a3f1b737c651
31+
eu-central-1: ami-092bd9c46746d940b
32+
eu-north-1: ami-07b83433077d8345b
33+
eu-west-1: ami-09880c7e25df69af8
34+
eu-west-2: ami-0eba961d9f30431b2
35+
eu-west-3: ami-0d0b243ac76765544
36+
sa-east-1: ami-0dfdc6ab8bf7935ea
37+
us-east-1: ami-00f71e3be938f3077
38+
us-east-2: ami-0b29637d31cf774aa
39+
us-west-1: ami-08dc392067bcf9807
40+
us-west-2: ami-0fa309858f6ce66ee
4141
# centos7
42-
ap-northeast-1: ami-0307ad022fdd9bd50
43-
ap-northeast-2: ami-01610b2d147974908
44-
ap-northeast-3: ami-04dd95194c8990d40
45-
ap-south-1: ami-0d848603d9bf30b76
46-
ap-southeast-1: ami-084b71a47f7c857b7
47-
ap-southeast-2: ami-0b06de8c266942eff
48-
ca-central-1: ami-0a2b442fa56fe5db0
49-
eu-central-1: ami-089c27327ed56f33c
50-
eu-north-1: ami-016fe6e29bff94e38
51-
eu-west-1: ami-00e9df6764697605f
52-
eu-west-2: ami-00884b56887c9585a
53-
eu-west-3: ami-0252465d6dff43cfb
54-
sa-east-1: ami-02f4e6da1e0de2347
55-
us-east-1: ami-08b43bfe7a0d16eeb
56-
us-east-2: ami-0c71e12e95cc937ef
57-
us-west-1: ami-0f925f166f349eaa4
58-
us-west-2: ami-05f6a646767bf0c87
42+
ap-northeast-1: ami-0f13f45e966236e46
43+
ap-northeast-2: ami-016c726d8902d133c
44+
ap-northeast-3: ami-037c3a13cd142c8f8
45+
ap-south-1: ami-06b7212503b9d9637
46+
ap-southeast-1: ami-0c39937e9ae643ecd
47+
ap-southeast-2: ami-0164dbfb6b7b938f5
48+
ca-central-1: ami-0ee7cb4d2673e78de
49+
eu-central-1: ami-0bcced571d9cc0142
50+
eu-north-1: ami-00255a59ce6bd8147
51+
eu-west-1: ami-00c07933e0ea22f7d
52+
eu-west-2: ami-09aa34259643c50eb
53+
eu-west-3: ami-04ce6f74e1070a795
54+
sa-east-1: ami-0a625e9dcf563db57
55+
us-east-1: ami-0658a809b3e89b0c9
56+
us-east-2: ami-07cef254f8886ea4e
57+
us-west-1: ami-0454b933360a077e4
58+
us-west-2: ami-03b7e311ae2f4aacb
5959
# ubuntu1404
60-
ap-northeast-1: ami-0141f9239dd88f7eb
61-
ap-northeast-2: ami-069d8e1e9b74ee48d
62-
ap-northeast-3: ami-07853be3b108fb14c
63-
ap-south-1: ami-02744b942f4094210
64-
ap-southeast-1: ami-070fef5acc9fd3de2
65-
ap-southeast-2: ami-097c066ff342e12fa
66-
ca-central-1: ami-075be3409c2c2738d
67-
cn-north-1: ami-0e90250aa31ac4a12
68-
eu-central-1: ami-093d2529752e5e2f2
69-
eu-north-1: ami-01ea61e8b26299de1
70-
eu-west-1: ami-0d1cb938a6d6b693c
71-
eu-west-2: ami-0487ce94890b5869b
72-
eu-west-3: ami-01a321e226dbf081a
73-
sa-east-1: ami-02366044e8d7372fa
74-
us-east-1: ami-00f8fb8ed06007c1f
75-
us-east-2: ami-0e23f662335e169a6
76-
us-gov-east-1: ami-0a34c5de0403642d2
77-
us-gov-west-1: ami-e4a3cb85
78-
us-west-1: ami-021201edbaab31f29
79-
us-west-2: ami-085bb9cfafd2e3b3a
60+
ap-northeast-1: ami-0ce1c5516c087ef8d
61+
ap-northeast-2: ami-0744c53e9582abcd4
62+
ap-northeast-3: ami-0d0faa548bcca5fac
63+
ap-south-1: ami-00721e9f7f8235dba
64+
ap-southeast-1: ami-03df9d0a89a448c63
65+
ap-southeast-2: ami-06116e2159f6ba6bf
66+
ca-central-1: ami-0d180013cf3d07fc9
67+
cn-north-1: ami-0ef85bbc4ba66c301
68+
eu-central-1: ami-04b116ae9a44c861f
69+
eu-north-1: ami-0de1c666987bbdb1f
70+
eu-west-1: ami-01b114f6a268d6a42
71+
eu-west-2: ami-0f9ad3c001b80325a
72+
eu-west-3: ami-0f921986737ab8306
73+
sa-east-1: ami-0d1d30ad051235185
74+
us-east-1: ami-0422aa8ec2e452870
75+
us-east-2: ami-02447e477105886bd
76+
us-gov-east-1: ami-03538e53996b83762
77+
us-gov-west-1: ami-90a2d5f1
78+
us-west-1: ami-0f4a99f972b9b4882
79+
us-west-2: ami-04caeb57df33aba89
8080
# ubuntu1604
81-
ap-northeast-1: ami-00bacec2848062b6b
82-
ap-northeast-2: ami-0a6edfea96ed6c9db
83-
ap-northeast-3: ami-09eeb02948489e793
84-
ap-south-1: ami-0dcc47340ec3a0e45
85-
ap-southeast-1: ami-0771cf73e73259040
86-
ap-southeast-2: ami-0252c6d83fa35183c
87-
ca-central-1: ami-090b3ee62a0b5412b
88-
cn-north-1: ami-0b89a3058ea57b76f
89-
eu-central-1: ami-0c7ccc7ec89bd0d75
90-
eu-north-1: ami-0675d81a659cb530e
91-
eu-west-1: ami-010af7d7788dac778
92-
eu-west-2: ami-01290e9a5298ae134
93-
eu-west-3: ami-0996ed0efb2b8cc4a
94-
sa-east-1: ami-0a3ef670136cc81ee
95-
us-east-1: ami-05f79ab77fc13e20a
96-
us-east-2: ami-08a675767416c627a
97-
us-gov-east-1: ami-08b15aaa28124fac1
98-
us-gov-west-1: ami-eba3cb8a
99-
us-west-1: ami-059b8cd1b0041071a
100-
us-west-2: ami-034bffb8da06d6951
81+
ap-northeast-1: ami-041f6050eff86f024
82+
ap-northeast-2: ami-0df4c1dafbfee5031
83+
ap-northeast-3: ami-08d3ef362e1d06e56
84+
ap-south-1: ami-0ef148f6ae69767d7
85+
ap-southeast-1: ami-0b63a13236ce5b8d9
86+
ap-southeast-2: ami-0f5a3072f23556b07
87+
ca-central-1: ami-0c88262f6fd2738fc
88+
cn-north-1: ami-017ea2a40c48f9af4
89+
eu-central-1: ami-06a21b6e0815065a4
90+
eu-north-1: ami-0418320f06192d788
91+
eu-west-1: ami-0809bc00666e41cfa
92+
eu-west-2: ami-04d8578267aaa2ac4
93+
eu-west-3: ami-02de781189ccb9f92
94+
sa-east-1: ami-088d6a838e8dc6b11
95+
us-east-1: ami-0a8c4ea1bd1ff7651
96+
us-east-2: ami-04d5c390495e0509f
97+
us-gov-east-1: ami-0bfb76fbbbb68030d
98+
us-gov-west-1: ami-eeaed98f
99+
us-west-1: ami-0a33d79d5f920cc2c
100+
us-west-2: ami-00050b3048393bc12

Diff for: ‎cli/README

+13-13
Original file line numberDiff line numberDiff line change
@@ -13,20 +13,20 @@ You can build higher level workflows, such as a Genomics portal that automates t
1313

1414
positional arguments:
1515
{create,update,delete,start,stop,status,list,instances,ssh,configure,version,createami}
16-
create creates a cluster
17-
update update a running cluster
18-
delete delete a cluster
19-
start start the compute fleet that has been stopped
20-
stop stop the compute fleet, but leave the master server running for debugging/development
21-
status pull the current status of the cluster
22-
list display a list of stacks associated with AWS ParallelCluster
23-
instances display a list of all instances in a cluster
24-
ssh connect to the master server using SSH
25-
configure creating initial AWS ParallelCluster configuration
26-
version display version of AWS ParallelCluster
27-
createami (Linux/OSX) create a custom AMI to use with AWS ParallelCluster
16+
create Creates a cluster.
17+
update Updates a running cluster.
18+
delete Deletes a cluster.
19+
start Starts the compute fleet that has been stopped.
20+
stop Stops the compute fleet, but leave the master server running for debugging/development.
21+
status Pulls the current status of the cluster.
22+
list Displays a list of stacks associated with AWS ParallelCluster.
23+
instances Displays a list of all instances in a cluster.
24+
ssh Connects to the master server using SSH.
25+
configure Starts the AWS ParallelCluster configuration.
26+
version Displays version of AWS ParallelCluster.
27+
createami (Linux/macOS) Creates a custom AMI to use with AWS ParallelCluster.
2828

2929
optional arguments:
30-
-h, --help show this help message and exit
30+
-h, --help Shows this help message and exits.
3131

3232
For command specific flags run pcluster [command] --help

Diff for: ‎cli/awsbatch/awsbsub.py

+4-2
Original file line numberDiff line numberDiff line change
@@ -630,19 +630,21 @@ def main():
630630
depends_on = _get_depends_on(args)
631631

632632
# select submission (standard vs MNP)
633-
if args.nodes:
633+
if args.nodes and args.nodes > 1:
634634
if not hasattr(config, "job_definition_mnp"):
635635
fail("Current cluster does not support MNP jobs submission")
636636
job_definition = config.job_definition_mnp
637+
nodes = args.nodes
637638
else:
638639
job_definition = config.job_definition
640+
nodes = None
639641

640642
AWSBsubCommand(log, boto3_factory).run(
641643
job_definition=job_definition,
642644
job_name=job_name,
643645
job_queue=config.job_queue,
644646
command=command,
645-
nodes=args.nodes,
647+
nodes=nodes,
646648
vcpus=args.vcpus,
647649
memory=args.memory,
648650
array_size=args.array_size,

0 commit comments

Comments
 (0)