diff --git a/docs/user-guide/cli.md b/docs/user-guide/cli.md index ff16c762..c21e0931 100644 --- a/docs/user-guide/cli.md +++ b/docs/user-guide/cli.md @@ -139,12 +139,19 @@ ELB failures which will result in instance replacements. ## `moonshot update` -Update an environment with the latest local CloudFormation template, -and any Parameter updates specified via `--answer-file` or -`--parameter`. If there are new parameters in the template and they -are not specified, the user will be prompted for their values, unless -`--no-interactive` is specified, in which case an error will be -displayed. +Update an environment with the latest local CloudFormation template +using a ChangeSet. Keep all existing parameters, unless they are +specified by `--answer-file` and/or `--parameter`. If there are new +parameters in the template and they are not specified, the user will +be prompted for their values, unless `--no-interactive` is specified, +in which case an error will be displayed. + +The user is prompted interactively to accept the ChangeSet, unless +`--no-interactive` is set. `--force` can be specified to automatically +accept the changes. If `--dry-run` is set, the changes are +automatically rejected after being displayed, which can be useful for +seeing what the impact of a template change might be in a given +environment. ### Options @@ -164,6 +171,17 @@ See [create][#moonshot-create]. See [create][#moonshot-create]. +#### `--force` + +Automatically accept the ChangeSet that was generated, without +prompting the user. The changes are still displayed in the log +output. + +#### `--dry-run` + +Automatically reject the ChangeSet that was generated, after +displaying the changes. + ### Examples Update a single Stack parameter, using the latest template: diff --git a/docs/user-guide/stack_parameter_strategies.md b/docs/user-guide/stack_parameter_strategies.md deleted file mode 100644 index 7de1ca2e..00000000 --- a/docs/user-guide/stack_parameter_strategies.md +++ /dev/null @@ -1,95 +0,0 @@ -# Stack Parameter strategies - -Currently used, valid strategy types are: -- `default` -- `merge` - -Strategy type can be set both via command line, or overriding -the default behaviour in your `Moonshot::CLI` subclass. - -**Command line:** `bin/environment update --parameter-strategy=merge` - -**Inline:** `parameter_strategy :merge` - -Accepted both as a string or a symbol. - -Setting precedence is the following: -- command line option -- inline default behaviour override -- falling back to `default` - -## Default strategy - -Default strategy is the legacy behaviour, works exactly as previously, -prior to introducing different strategies. Each parameter is loaded from the -parameter file (eg. `cloud_formation/parameters/environment-name.yml`), -and *all of them* are overridden on update. - -**Word of caution:** when using this strategy we encourage you to keep all -per-environment tunings in the source repository acting as both a safety net -and documentation of existing environments. A possible solution for using this -strategy in a safe manner is the following: - -When a stack update is performed, a *parameter file* is checked in -`cloud_formation/parameters/environment-name.yml`. This file is YAML formatted -and takes a hash of stack parameter names and values, for example: -```yaml ---- -AsgDesiredCap: 12 -AsgMaxCap: 15 -ELBCertificate: iam::something:star_example_com -``` - -If a file exists, it's used every time a CloudFormation change request is sent, -so no configuration can revert back to defaults through this tool. It's highly -recommended that you add these files back to source control as soon as possible -and be in the habit of pulling latest changes before applying any infrastructure -updates. - -## Merge strategy - -Merge strategy is a new way of dealing with stack parameters on stack update. -You only have to declare **parameters you want to update** in your parameter file, -the remaining parameters are not updated, meaning it stays as it's current uploaded (live) state. -This behaviour is achived by using CloudFormation's `UsePreviousValue` feature. -This way you can avoid accidentally reverting parameter values with your outdated local -parameter file. - -# Defining custom parameter strategy class -Defining and using your own custom parameter strategy class is possible if you are -using Moonshot without the provided CLI. A parameter strategy class is a class which responds -to a method called `parameters`. It receives two parameters: the first contains a hash -of the existing, currently deployed stack parameters, the second one (also a hash) contains -the parameters defined in the parameters file. The method should return an array of -hashes of the following format: - -```json -{ - parameter_key: key, - parameter_value: value, - use_previous_value: false -} -``` - -You either supply a value for `parameter_value` or set -`use_previous_value` to `true`, which leaves the parameter as it -currently is. - -Example: - -```ruby -class CustomStrategy - def parameters(parameters, stack_parameters, template) - parameters.map do |k, v| - { - parameter_key: k, - parameter_value: v, - use_previous_value: false - } - end - end -end -``` - -In order to use your custom strategy class, set a new instance of your -class to `ControllerConfig`'s `parameter_strategy` attribute. diff --git a/lib/moonshot.rb b/lib/moonshot.rb index 2c2ec8cb..1dc38f47 100644 --- a/lib/moonshot.rb +++ b/lib/moonshot.rb @@ -54,8 +54,6 @@ module Plugins 'stack_config', 'stack_lister', 'stack_events_poller', - 'merge_strategy', - 'default_strategy', 'ask_user_source', 'always_use_default_source', diff --git a/lib/moonshot/change_set.rb b/lib/moonshot/change_set.rb new file mode 100644 index 00000000..b28ef859 --- /dev/null +++ b/lib/moonshot/change_set.rb @@ -0,0 +1,100 @@ +module Moonshot + class ChangeSet + attr_reader :name + attr_reader :stack_name + + def initialize(name, stack_name) + @name = name + @stack_name = stack_name + @change_set = nil + @cf_client = Aws::CloudFormation::Client.new + end + + def confirm? + unless Moonshot.config.interactive + raise 'Cannot confirm ChangeSet when interactive mode is disabled!' + end + + loop do + print 'Apply changes? ' + resp = gets.chomp.downcase + + return true if resp == 'yes' + return false if resp == 'no' + puts "Please enter 'yes' or 'no'!" + end + end + + def valid? + @change_set.status == 'CREATE_COMPLETE' + end + + def invalid_reason + @change_set.status_reason + end + + def display_changes + wait_for_change_set unless @change_set + + @change_set.changes.map(&:resource_change).each do |c| + puts "* #{c.action} #{c.logical_resource_id} (#{c.resource_type})" + + if c.replacement == 'True' + puts ' - Will be replaced' + elsif c.replacement == 'Conditional' + puts ' - May be replaced (Conditional)' + end + + c.details.each do |d| + case d.change_source + when 'ResourceReference', 'ParameterReference' + puts " - Caused by #{d.causing_entity.blue} (#{d.change_source})" + when 'DirectModification' + puts " - Caused by template change (#{d.target.attribute}: #{d.target.name})" + end + end + end + end + + def execute + wait_for_change_set unless @change_set + @cf_client.execute_change_set( + change_set_name: @name, + stack_name: @stack_name) + end + + def delete + wait_for_change_set unless @change_set + @cf_client.delete_change_set( + change_set_name: @name, + stack_name: @stack_name) + rescue Aws::CloudFormation::Errors::InvalidChangeSetStatus + sleep 1 + retry + end + + # NOTE: At the time of this patch, AWS-SDK native Waiters do not + # have support for ChangeSets. Once they add this, we can make + # this code much better. + def wait_for_change_set + start = Time.now.to_i + + loop do + resp = @cf_client.describe_change_set( + change_set_name: @name, + stack_name: @stack_name) + + if %w(CREATE_COMPLETE FAILED).include?(resp.status) + @change_set = resp + return + end + + if Time.now.to_i > start + 30 + raise 'ChangeSet did not complete creation within 30 seconds!' + end + + sleep 0.25 # http://bit.ly/1qY1ZXJ + end + end + end +end diff --git a/lib/moonshot/commands/update.rb b/lib/moonshot/commands/update.rb index 337c7ae0..adb34b7b 100644 --- a/lib/moonshot/commands/update.rb +++ b/lib/moonshot/commands/update.rb @@ -8,22 +8,22 @@ class Update < Moonshot::Command self.usage = 'update [options]' self.description = 'Update the CloudFormation stack within an environment.' - def execute - controller.update - end + def parser + parser = super - private + parser.on('--dry-run', TrueClass, 'Show the changes that would be applied, but do not execute them') do |v| # rubocop:disable LineLength + @dry_run = v + end - def parameter_strategy_factory(value) - case value.to_sym - when :default - Moonshot::ParameterStrategy::DefaultStrategy.new - when :merge - Moonshot::ParameterStrategy::MergeStrategy.new - else - raise "Unknown parameter strategy: #{value}" + parser.on('--force', '-f', TrueClass, 'Apply ChangeSet without confirmation') do |v| + @force = v end end + + def execute + @force = true unless Moonshot.config.interactive + controller.update(dry_run: @dry_run, force: @force) + end end end end diff --git a/lib/moonshot/controller.rb b/lib/moonshot/controller.rb index 358c9910..bbe2dc7c 100644 --- a/lib/moonshot/controller.rb +++ b/lib/moonshot/controller.rb @@ -71,7 +71,7 @@ def create # rubocop:disable AbcSize end end - def update # rubocop:disable AbcSize + def update(dry_run:, force:) # rubocop:disable AbcSize # Scan the template for all required parameters and configure # the ParameterCollection. @config.parameters = ParameterCollection.from_template(stack.template) @@ -119,7 +119,7 @@ def update # rubocop:disable AbcSize end run_hook(:deploy, :pre_update) - stack.update + stack.update(dry_run: dry_run, force: force) run_hook(:deploy, :post_update) run_plugins(:post_update) end diff --git a/lib/moonshot/controller_config.rb b/lib/moonshot/controller_config.rb index f39e57f2..88acf58d 100644 --- a/lib/moonshot/controller_config.rb +++ b/lib/moonshot/controller_config.rb @@ -1,4 +1,3 @@ -require_relative 'default_strategy' require_relative 'ssh_config' require_relative 'task' require_relative 'ask_user_source' diff --git a/lib/moonshot/default_strategy.rb b/lib/moonshot/default_strategy.rb deleted file mode 100644 index 4aae997e..00000000 --- a/lib/moonshot/default_strategy.rb +++ /dev/null @@ -1,16 +0,0 @@ -module Moonshot - module ParameterStrategy - # Default strategy: use parameter defined in the parameter file - class DefaultStrategy - def parameters(params, _, _) - params.map do |key, _| - { - parameter_key: key, - parameter_value: params[key], - use_previous_value: false - } - end - end - end - end -end diff --git a/lib/moonshot/merge_strategy.rb b/lib/moonshot/merge_strategy.rb deleted file mode 100644 index ea9fbfde..00000000 --- a/lib/moonshot/merge_strategy.rb +++ /dev/null @@ -1,31 +0,0 @@ -require 'highline/import' -require_relative 'unicode_table' - -module Moonshot - module ParameterStrategy - # Merge strategy: prefer parameter values defined in the parameter file, - # otherwise use the previously set value on the existing stack. - class MergeStrategy - def parameters(params, stack_params, template) - stack_keys = stack_params.keys.select do |k| - template.parameters.any? { |p| p.name == k } - end - - (params.keys + stack_keys).uniq.map do |key| - if params[key] - { - parameter_key: key, - parameter_value: params[key], - use_previous_value: false - } - else - { - parameter_key: key, - use_previous_value: true - } - end - end - end - end - end -end diff --git a/lib/moonshot/stack.rb b/lib/moonshot/stack.rb index a0e38412..930318a5 100644 --- a/lib/moonshot/stack.rb +++ b/lib/moonshot/stack.rb @@ -7,6 +7,7 @@ require_relative 'stack_output_printer' require_relative 'stack_asg_printer' require_relative 'unicode_table' +require_relative 'change_set' require 'yaml' module Moonshot @@ -44,22 +45,21 @@ def create should_wait ? wait_for_stack_state(:stack_create_complete, 'created') : true end - def update + def update(dry_run:, force:) raise "No stack found #{@name.blue}!" unless stack_exists? - should_wait = true - @ilog.start "Updating #{stack_name}." do |s| - if update_stack - s.success "Initiated update for #{stack_name}." - else - s.success 'No Stack update required.' - should_wait = false - end + change_set = ChangeSet.new(new_change_set, @name) + wait_for_change_set(change_set) + return unless change_set.valid? + + if dry_run + change_set.display_changes + elsif !force + change_set.display_changes + change_set.confirm? || raise('ChangeSet rejected!') end - success = should_wait ? wait_for_stack_state(:stack_update_complete, 'updated') : true - raise 'Failed to update the CloudFormation Stack.' unless success - success + execute_change_set(change_set) end def delete @@ -232,20 +232,23 @@ def create_stack raise 'You are not authorized to perform create_stack calls.' end - # @return [Boolean] - # true if a stack update was required and initiated, false otherwise. - def update_stack - cf_client.update_stack( + def new_change_set + change_set_name = [ + 'moonshot', + @name, + Time.now.utc.to_i.to_s + ].join('-') + + cf_client.create_change_set( + change_set_name: change_set_name, + description: "Moonshot update command for application '#{Moonshot.config.app_name}'", stack_name: @name, template_body: template.body, capabilities: ['CAPABILITY_IAM'], parameters: @config.parameters.values.map(&:to_cf) ) - true - rescue Aws::CloudFormation::Errors::ValidationError => e - raise e.message unless - e.message == 'No updates are to be performed.' - false + + change_set_name end # TODO: Refactor this into it's own class. @@ -335,5 +338,28 @@ def doctor_check_template_against_aws rescue => e critical('Invalid CloudFormation template!', e.message) end + + def wait_for_change_set(change_set) + @ilog.start_threaded "Waiting for ChangeSet #{change_set.name.blue} to be created." do |s| + change_set.wait_for_change_set + + if change_set.valid? + s.success "ChangeSet #{change_set.name.blue} ready!" + else + s.failure "ChangeSet failed to create: #{change_set.invalid_reason}" + end + end + end + + def execute_change_set(change_set) + @ilog.start_threaded "Executing ChangeSet #{change_set.name.blue} for #{stack_name}." do |s| + change_set.execute + s.success "Executed ChangeSet #{change_set.name.blue} for #{stack_name}." + end + + success = wait_for_stack_state(:stack_update_complete, 'updated') + raise 'Failed to update the CloudFormation Stack.' unless success + success + end end end diff --git a/lib/moonshot/stack_config.rb b/lib/moonshot/stack_config.rb index 9378ae1f..7093ce1a 100644 --- a/lib/moonshot/stack_config.rb +++ b/lib/moonshot/stack_config.rb @@ -3,7 +3,6 @@ module Moonshot class StackConfig attr_accessor :parent_stacks attr_accessor :show_all_events - attr_accessor :parameter_strategy def initialize @parent_stacks = [] diff --git a/mkdocs.yml b/mkdocs.yml index 815428fe..9fff98f0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -7,7 +7,6 @@ pages: - 'CLI Commands': 'user-guide/cli.md' - 'Including Moonshot in your project': 'user-guide/include_in_your_project.md' - 'Working with parent stacks': 'user-guide/parent_stacks.md' - - 'Stack parameter strategies': 'user-guide/stack_parameter_strategies.md' - Mechanisms: - 'Artifact Repository': 'mechanisms/artifact-repository.md' - 'Build': 'mechanisms/build.md' diff --git a/spec/moonshot/controller_spec.rb b/spec/moonshot/controller_spec.rb index 9a12282a..914c3e89 100644 --- a/spec/moonshot/controller_spec.rb +++ b/spec/moonshot/controller_spec.rb @@ -152,7 +152,7 @@ expect(stack).to receive(:parameters).and_return(existing_parameters) expect(stack).to receive(:update) - subject.update + subject.update(dry_run: false, force: false) pc = subject.config.parameters expect(pc.keys).to eq(%w(InputParameter1 InputParameter2 InputParameter3 InputParameter4)) diff --git a/spec/moonshot/merge_strategy_spec.rb b/spec/moonshot/merge_strategy_spec.rb deleted file mode 100644 index f592c768..00000000 --- a/spec/moonshot/merge_strategy_spec.rb +++ /dev/null @@ -1,80 +0,0 @@ -describe Moonshot::ParameterStrategy::MergeStrategy do - subject { described_class.new } - - describe '#parameters' do - subject do - super().parameters( - parameters, - stack_parameters, - template - ) - end - - let(:parameters) do - { - file_parameter_1: 'file_parameter_1', - file_parameter_2: 'file_parameter_2' - } - end - - let(:stack_parameters) do - { - stack_parameter_1: 'stack_parameter_1', - stack_parameter_2: 'stack_parameter_2' - } - end - - let(:template_parameters) do - [ - double('template_param', name: :stack_parameter_1), - double('template_param', name: :stack_parameter_2) - ] - end - - let(:template) do - template = double(Moonshot::StackTemplate) - allow(template).to receive(:parameters).and_return(template_parameters) - template - end - - let(:actual_keys) do - subject.map { |p| p[:parameter_key] } - end - - it 'includes the stack and YAML file parameters' do - expected_keys = (parameters.keys + stack_parameters.keys).uniq - expect(actual_keys).to eq(expected_keys) - end - - context 'when the template no longer includes a stack parameter' do - let(:removed_element) do - template_parameters.pop - end - - it 'does not include the removed parameter' do - removed_key = removed_element.name - expect(actual_keys).not_to include(removed_key) - end - end - - context 'when the template adds a new parameter and it is provided' do - let(:added_parameter) do - double('template_param', name: :template_parameter_1) - end - - let(:template_parameters) do - super().insert(added_parameter) - end - - let(:parameters) do - parameters = super() - parameters[added_parameter.name] = 'template_parameter_1' - parameters - end - - it 'includes the added parameter' do - expect(actual_keys).to include(added_parameter.name) - end - end - end -end