Skip to content

feat(s3): allow specifying a custom IAM Role for bucket replication #33978

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

hassaku63
Copy link
Contributor

Issue # (if applicable)

Closes #33974

Reason for this change

Currently, the aws-s3 module automatically creates and manages the IAM Role used for S3 replication. This limits integration flexibility, especially in environments where IAM Roles are provisioned externally or reused across stacks/accounts.

This change addresses that limitation by allowing users to provide a custom IAM Role for replication.

Description of changes

  • Introduced an optional replicationRole?: iam.IRole property in BucketProps.
  • When replicationRole is provided, the CDK uses it instead of creating a new role.
  • Required permissions are NOT automatically attached to the provided role. It is the user's responsibility to attach the necessary IAM policies.
  • Added validation to ensure that if replicationRole is specified, replicationRules must also be defined and non-empty, since both are required by CloudFormation when configuring replication.

Describe any new or updated permissions being added

No new IAM actions are introduced. When a custom role is provided, CDK does not attach any permissions automatically. Users are expected to grant the appropriate replication-related permissions manually.

Description of how you validated changes

Added unit and integ tests.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added the beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK label Mar 30, 2025
@aws-cdk-automation aws-cdk-automation requested a review from a team March 30, 2025 11:14
@github-actions github-actions bot added effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 labels Mar 30, 2025
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@hassaku63
Copy link
Contributor Author

hassaku63 commented Mar 30, 2025

Additional information

Behavior when replicationRole is specified without replicationRules

See: #33974 (comment)

As described in the comment above, this PR implements validation to throw an error when a replicationRole is provided without corresponding replicationRules. This is aligned with CloudFormation requirements, where both Role and Rules must be specified when enabling replication.

Whether CDK should automatically attach permissions to a user-supplied IAM Role

In the current implementation, the CDK does not automatically attach replication permissions to a user-supplied IAM Role.

While it might be reasonable to support this via a method like grantReplicationPermission(), I believe it would be best handled in a separate issue and follow-up PR. (That said, I’m also open to including it in this PR if the reviewers feel it makes sense to do so.)

Since replication-related settings are finalized during the construction of the Bucket object, another viable approach could be to automatically attach the required permissions inside the renderReplicationConfiguration method.

I'm currently undecided on which approach would be best, so I’d greatly appreciate any feedback or recommendations from the reviewers.

@hassaku63 hassaku63 changed the title feat(aws-s3): Allow specifying a custom IAM Role for bucket replication feat(s3): Allow specifying a custom IAM Role for bucket replication Mar 30, 2025
@hassaku63 hassaku63 changed the title feat(s3): Allow specifying a custom IAM Role for bucket replication feat(s3): allow specifying a custom IAM Role for bucket replication Mar 30, 2025
@aws-cdk-automation aws-cdk-automation dismissed their stale review March 30, 2025 11:30

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Copy link

codecov bot commented Mar 30, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.00%. Comparing base (74cbe27) to head (98bf3ff).
Report is 35 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #33978      +/-   ##
==========================================
+ Coverage   83.98%   84.00%   +0.01%     
==========================================
  Files         120      121       +1     
  Lines        6976     6984       +8     
  Branches     1178     1179       +1     
==========================================
+ Hits         5859     5867       +8     
  Misses       1005     1005              
  Partials      112      112              
Flag Coverage Δ
suite.unit 84.00% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk ∅ <ø> (∅)
packages/aws-cdk-lib/core 84.00% <ø> (+0.01%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Mar 30, 2025
Copy link
Contributor

@badmintoncryer badmintoncryer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution!! I added two minor comments.

Comment on lines 73 to 75
postCliContext: {
[SET_UNIQUE_REPLICATION_ROLE_NAME]: false,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably don't need to explicitly disable this function flag. It will be set true implicitly.

Suggested change
postCliContext: {
[SET_UNIQUE_REPLICATION_ROLE_NAME]: false,
},

This fix should not cause any differences in the snapshot generation in this integration test, but if it does, I would appreciate it if you could run the snapshot update again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your feedback. You're right, it doesn't seem necessary to consider this in the current test scenario. I will remove it.

replicationRole: new iam.Role(stack, 'ReplicationRole', {
assumedBy: new iam.ServicePrincipal('s3.amazonaws.com'),
}),
replicationRules: [],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably correct not to specify replicationRules, right?

Suggested change
replicationRules: [],

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are absolutely right. This replicationRule did not reflect the original intent of testing the unspecified state. I have removed it in the latest commit.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Mar 31, 2025
Copy link
Contributor

@badmintoncryer badmintoncryer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!!

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Mar 31, 2025
@shikha372 shikha372 self-assigned this Apr 3, 2025
@shikha372
Copy link
Contributor

shikha372 commented Apr 10, 2025

Thank you @hassaku63 for this contribution, I agree with abstracting the permissions under a new method , which can be added in a follow up PR.

Note: added do-not-merge just to confirm internally that this doesn't require any security review.

Copy link
Contributor

mergify bot commented Apr 10, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@shikha372 shikha372 added the pr/do-not-merge This PR should not be merged at this time. label Apr 10, 2025
Copy link
Contributor

mergify bot commented Apr 10, 2025

This pull request has been removed from the queue for the following reason: pull request dequeued.

Pull request #33978 has been dequeued. The pull request rule doesn't match anymore. The following conditions don't match anymore:

  • -label~=(blocked|do-not-merge|no-squash|two-approvers|priority-pr)
  • status-success=validate-pr
  • status-success~=AWS CodeBuild us-east-1
  • any of: [🔀 queue conditions]
    • all of: [📌 queue conditions of queue default-squash]
      • any of: [🛡 GitHub branch protection]
        • check-neutral = validate-pr
        • check-skipped = validate-pr
        • check-success = validate-pr
      • any of: [🛡 GitHub branch protection]
        • check-neutral = AWS CodeBuild us-east-1 (AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv)
        • check-skipped = AWS CodeBuild us-east-1 (AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv)
        • check-success = AWS CodeBuild us-east-1 (AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv).

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.
If you do update this pull request, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio requeue comment.

@hstern
Copy link

hstern commented Apr 14, 2025

I have authored a commit that changes the integ test to validate the cross-stack use case for explicitly-specified replication roles. Would it be possible to include it here? It will close #34104.

@shikha372 shikha372 removed the pr/do-not-merge This PR should not be merged at this time. label Apr 14, 2025
Copy link
Contributor

mergify bot commented Apr 14, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@shikha372
Copy link
Contributor

shikha372 commented Apr 14, 2025

I have authored a commit that changes the integ test to validate the cross-stack use case for explicitly-specified replication roles. Would it be possible to include it here? It will close #34104 .

Thank you @hstern, there is already an integration test to check the cross-account functionality in this PR, do you see anything missing from this integration test that you would like to add?

@hassaku63
Copy link
Contributor Author

Thanks for the note. I don't have decision-making authority on this, so I generally intend to follow the maintainers' guidance. That said, while I'm not entirely sure whether this pull request is the right place to have the discussion, I'd like to share my personal thoughts.

From my perspective, if the proposed integ test is well-motivated and works as expected, I think there's value in including the commit. In that case, I assume I should merge your commit into my remote branch (please let me know if there's a preferred approach you'd recommend instead).

At the same time, I do have a few concerns about the proposed integ test, which I'd like to bring up.

First, I find the assumed setup somewhat unusual. In most cases, wouldn't it be the source stack that takes responsibility for creating the replication role? These stacks will likely deploy successfully since they belong to the same account. However, I'm not sure this setup accurately reflects a typical cross-stack use case.

Personally, I think it's more common for either the source or destination stack to be deployed independently first. If the goal here is to detect circular dependencies, that feels like a separate topic—perhaps one that warrants its own validation logic and discussion.

Second, I'm wondering whether this test assumes that the replication role is created automatically in the source stack. Based on the comment, I understand this test is intended for the case where the replication role is explicitly specified. However, looking at the SourceStack implementation, it seems to rely on the automatic creation of the IAM role used for S3 replication. Meanwhile, what appears to be the explicitly created IAM role is defined in the DestStack, which seems a bit counterintuitive to me.

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 98bf3ff
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

mergify bot commented Apr 14, 2025

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 615f626 into aws:main Apr 14, 2025
20 checks passed
Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 14, 2025
@hassaku63 hassaku63 deleted the feat/custom-iam-role-for-bucket-replication branch April 14, 2025 17:04
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(aws-s3): Allow specifying a custom IAM Role for bucket replication
5 participants