diff --git a/README.md b/README.md index becbed1..73e62a1 100644 --- a/README.md +++ b/README.md @@ -13,30 +13,30 @@ Henceforth, so long as the plugin remains enabled, all Pipeline builds will stre Viewing the **Console** link in Jenkins (or the equivalent in Blue Ocean) will load log content directly from CloudWatch Logs. The first line of a (classic) console view will be a link to the CloudWatch Logs section of the AWS Console -with the log stream name for all messages coming from the Jenkins master and a filter predefined to show that build. +with the log stream name for all messages coming from the Jenkins controller and a filter predefined to show that build. (Currently you would need to manually browse to related log streams to see output produced by agents.) ## Security With this plugin active, log content generated by processes running on agents, such as `sh` steps, -will be sent to CloudWatch Logs directly from that agent machine, without passing through the Jenkins master. -For that to work, the master will send AWS credentials to the agent sufficient to write logs. +will be sent to CloudWatch Logs directly from that agent machine, without passing through the Jenkins controller. +For that to work, the controller will send AWS credentials to the agent sufficient to write logs. That prevents a rogue build from writing to logs of another job (though not another build of the same job), or otherwise causing problems. It will attempt to use either the `AssumeRole` or `GetFederationToken` API calls to create a fresh session token with a policy restriction allowing the agent only to write logs to the streams for this job and nothing else. -One of these is selected by inspecting the configured master credentials according to +One of these is selected by inspecting the configured controller credentials according to [this table](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_request.html#stsapi_comparison). -If such a policy restriction fails, the master falls back to transmitting its own credentials. +If such a policy restriction fails, the controller falls back to transmitting its own credentials. **Validate configuration** will indicate whether these calls are likely to work before actually running a build. Note that if agents are themselves running in EC2, you must use tools to block them from having any default AWS credentials. For example, if the Jenkins system is running in EKS, you could try [kube2iam](https://github.com/jtblin/kube2iam). -### Permissions for the master +### Permissions for the controller -The Jenkins master will need permissions for at least these API calls, scoped to the log group name: +The Jenkins controller will need permissions for at least these API calls, scoped to the log group name: * `FilterLogEvents` * `DescribeLogStreams` @@ -71,7 +71,7 @@ Only `AssumeRule` starting with temporary security credentials has been tested f `GetFederationToken` has not been tested. The retrieval of log content from CloudWatch Logs for display inside Jenkins is not well optimized. -API requests made from the master filter down to the level of log stream group, log stream name pattern, +API requests made from the controller filter down to the level of log stream group, log stream name pattern, build number, and (where applicable) step identifier. However due to the use of bytestream-based cursors in Jenkins (core and Blue Ocean), the entire build may be downloaded even if the console display is truncated; @@ -79,7 +79,7 @@ and incremental display of a running build will perform multiple downloads. Scalability to large numbers of builds, gigantic build logs, or extremely long lines of text has not been evaluated. -**Validate configuration** does not check availability of all master permissions. +**Validate configuration** does not check availability of all controller permissions. Job names involving unusual characters may not work. @@ -102,9 +102,13 @@ but the meat of the testing is in `PipelineBridgeTest` which is a live test (con and as such requires several environment variables to be set: * `AWS_PROFILE` should select a profile in `~/.aws/credentials`. - This may be a session token, but should _not_ have assumed a role. + This may be a session token. * `AWS_ROLE` should refer to a role (`arn:aws:iam::…:role/…` syntax) which this profile may assume. - The role should grant at least the master permissions detailed above. + The role should grant at least the controller permissions detailed above. +* `AWS_CHAINED_ROLE` may be set as an alternative to `AWS_ROLE` + in case the profile already has controller permissions, + for example after using `aws sso login`, + but may assume the same or a separate role to apply a more restrictive policy for agents. * `AWS_REGION` should be set to, e.g., `us-east-1`. * `CLOUDWATCH_LOG_GROUP_NAME` should be set to a test log group name, which you may need to precreate. @@ -168,7 +172,7 @@ When you run a build, you should see an initial log line ``` with a hyperlink to the Console. -Look at the two log streams created, one for messages originating from the Jenkins master, +Look at the two log streams created, one for messages originating from the Jenkins controller, one for messages produced and sent from the agent. Each log event will have a JSON block with at least a `message` and `build` field, and sometimes also `node` and/or `annotations` fields. @@ -193,7 +197,7 @@ and then just offers a Pipeline step (which your `Jenkinsfile` must call) which uploads the log text up to that point to CloudWatch Logs (any output after the step is called will not be uploaded). Logs are still displayed from the traditional location, -and log text is still streamed from agents to the master via the Remoting channel. +and log text is still streamed from agents to the controller via the Remoting channel. # Other materials diff --git a/pom.xml b/pom.xml index f2393a2..1bf299d 100644 --- a/pom.xml +++ b/pom.xml @@ -112,6 +112,12 @@ 109.vac7845f10470 test + + software.amazon.awssdk + sso + 2.20.69 + test + diff --git a/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/CloudWatchAwsGlobalConfiguration.java b/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/CloudWatchAwsGlobalConfiguration.java index 439383e..2c9ff7d 100644 --- a/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/CloudWatchAwsGlobalConfiguration.java +++ b/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/CloudWatchAwsGlobalConfiguration.java @@ -24,24 +24,12 @@ package io.jenkins.plugins.pipeline_cloudwatch_logs; -import java.io.IOException; - -import edu.umd.cs.findbugs.annotations.NonNull; - -import org.apache.commons.lang.StringUtils; -import org.jenkinsci.Symbol; -import org.kohsuke.accmod.Restricted; -import org.kohsuke.accmod.restrictions.NoExternalUse; -import org.kohsuke.stapler.DataBoundSetter; -import org.kohsuke.stapler.QueryParameter; -import org.kohsuke.stapler.interceptor.RequirePOST; - +import com.amazonaws.auth.AWSCredentialsProviderChain; import com.amazonaws.auth.AWSStaticCredentialsProvider; import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.amazonaws.services.logs.AWSLogs; import com.amazonaws.services.logs.AWSLogsClientBuilder; import com.amazonaws.services.logs.model.FilterLogEventsRequest; - import edu.umd.cs.findbugs.annotations.NonNull; import hudson.Extension; import hudson.Util; @@ -49,7 +37,15 @@ import hudson.util.FormValidation; import io.jenkins.plugins.aws.global_configuration.AbstractAwsGlobalConfiguration; import io.jenkins.plugins.aws.global_configuration.CredentialsAwsGlobalConfiguration; +import java.io.IOException; import jenkins.model.Jenkins; +import org.apache.commons.lang.StringUtils; +import org.jenkinsci.Symbol; +import org.kohsuke.accmod.Restricted; +import org.kohsuke.accmod.restrictions.NoExternalUse; +import org.kohsuke.stapler.DataBoundSetter; +import org.kohsuke.stapler.QueryParameter; +import org.kohsuke.stapler.interceptor.RequirePOST; /** * Store the AWS configuration to save it on a separate file @@ -58,6 +54,9 @@ @Extension public class CloudWatchAwsGlobalConfiguration extends AbstractAwsGlobalConfiguration { + // mutable for tests + static AWSCredentialsProviderChain awsCredentialsProviderChain = DefaultAWSCredentialsProviderChain.getInstance(); + /** * Name of the CloudWatch log group. */ @@ -117,7 +116,7 @@ static AWSLogsClientBuilder getAWSLogsClientBuilder(String region, String creden CredentialsAwsGlobalConfiguration.get().sessionCredentials(builder, region, credentialsId)); return builder.withCredentials(credentialsProvider); } else { - return builder.withCredentials(new DefaultAWSCredentialsProviderChain()); + return builder.withCredentials(awsCredentialsProviderChain); } } @@ -133,18 +132,18 @@ public FormValidation doCheckLogGroupName(@QueryParameter String logGroupName) { public FormValidation doValidate(@QueryParameter String logGroupName, @QueryParameter String region, @QueryParameter String credentialsId) { Jenkins.get().checkPermission(Jenkins.ADMINISTER); - return validate(logGroupName, Util.fixEmptyAndTrim(region), Util.fixEmptyAndTrim(credentialsId)); + return validate(logGroupName, Util.fixEmptyAndTrim(region), Util.fixEmptyAndTrim(credentialsId), true); } @Restricted(NoExternalUse.class) - FormValidation validate(String logGroupName, String region, String credentialsId) { + FormValidation validate(String logGroupName, String region, String credentialsId, boolean abbreviate) { AWSLogs client; try { AWSLogsClientBuilder builder = getAWSLogsClientBuilder(region, credentialsId); client = builder.build(); } catch (Exception x) { String msg = processExceptionMessage(x); - return FormValidation.error("Unable to validate credentials: " + StringUtils.abbreviate(msg, 200)); + return FormValidation.error("Unable to validate credentials: " + (abbreviate ? StringUtils.abbreviate(msg, 200) : msg)); } try { @@ -161,7 +160,7 @@ FormValidation validate(String logGroupName, String region, String credentialsId } } catch (Exception x) { String msg = processExceptionMessage(x); - return FormValidation.error("Unable to simulate policy restriction: " + StringUtils.abbreviate(msg, 200)); + return FormValidation.error("Unable to simulate policy restriction: " + (abbreviate ? StringUtils.abbreviate(msg, 200) : msg)); } return FormValidation.ok("success"); } diff --git a/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/LogStreamState.java b/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/LogStreamState.java index 24a34cd..d74cacb 100644 --- a/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/LogStreamState.java +++ b/src/main/java/io/jenkins/plugins/pipeline_cloudwatch_logs/LogStreamState.java @@ -46,6 +46,7 @@ import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClientBuilder; import com.amazonaws.services.securitytoken.model.AssumeRoleRequest; import com.amazonaws.services.securitytoken.model.Credentials; +import com.amazonaws.services.securitytoken.model.GetCallerIdentityRequest; import com.amazonaws.services.securitytoken.model.GetFederationTokenRequest; import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsImpl; import com.cloudbees.jenkins.plugins.awscredentials.AmazonWebServicesCredentials; @@ -71,6 +72,7 @@ import edu.umd.cs.findbugs.annotations.CheckForNull; import edu.umd.cs.findbugs.annotations.NonNull; import edu.umd.cs.findbugs.annotations.Nullable; +import java.util.UUID; import jenkins.security.HMACConfidentialKey; import jenkins.security.SlaveToMasterCallable; import jenkins.util.JenkinsJVM; @@ -190,6 +192,8 @@ Auth authenticate() throws IOException { if (jenkinsCredentials != null) { AWSStaticCredentialsProvider credentialsProvider = new AWSStaticCredentialsProvider(jenkinsCredentials.getCredentials()); builder.withCredentials(credentialsProvider); + } else { + builder.withCredentials(CloudWatchAwsGlobalConfiguration.awsCredentialsProviderChain); } AWSCredentialsProvider credentialsProvider = builder.getCredentials(); AWSCredentials masterCredentials = credentialsProvider != null ? credentialsProvider.getCredentials() : null; @@ -205,7 +209,7 @@ Auth authenticate() throws IOException { } if (masterCredentials instanceof AWSSessionCredentials) { // otherwise would just throw AWSSecurityTokenServiceException: Cannot call GetFederationToken with session credentials - String role = null; + String role = System.getenv("AWS_CHAINED_ROLE"); // TODO define in CloudWatchAwsGlobalConfiguration? if (jenkinsCredentials instanceof AWSCredentialsImpl) { role = Util.fixEmpty(((AWSCredentialsImpl) jenkinsCredentials).getIamRoleArn()); } @@ -227,15 +231,17 @@ Auth authenticate() throws IOException { private Auth assumeRole(String role, String region, String agentLogStreamName) { // TODO would be cleaner if AmazonWebServicesCredentials had a getCredentials overload taking a policy AWSSecurityTokenServiceClientBuilder builder = AWSSecurityTokenServiceClientBuilder.standard(); + builder.withCredentials(CloudWatchAwsGlobalConfiguration.awsCredentialsProviderChain); if (region != null) { builder = builder.withRegion(region); } - Auth auth = new Auth(builder.build().assumeRole(new AssumeRoleRequest(). + Credentials credentials = builder.build().assumeRole(new AssumeRoleRequest(). withRoleArn(role). - withRoleSessionName("CloudWatchSender"). // TODO does this need to be unique? + withRoleSessionName("CloudWatchSender-" + UUID.randomUUID()). withPolicy(policy(agentLogStreamName))). - getCredentials(), region, agentLogStreamName); - LOGGER.log(Level.FINE, "AssumeRole succeeded; using {0}", auth.accessKeyId); + getCredentials(); + Auth auth = new Auth(credentials, region, agentLogStreamName); + LOGGER.fine(() -> "AssumeRole succeeded; using " + AWSSecurityTokenServiceClientBuilder.standard().withCredentials(new AWSStaticCredentialsProvider(new BasicSessionCredentials(credentials.getAccessKeyId(), credentials.getSecretAccessKey(), credentials.getSessionToken()))).build().getCallerIdentity(new GetCallerIdentityRequest())); return auth; } @@ -244,7 +250,7 @@ private Auth assumeRole(String role, String region, String agentLogStreamName) { */ private Auth getFederationToken(AWSSecurityTokenServiceClientBuilder builder, String region, String agentLogStreamName) { Auth auth = new Auth(builder.build().getFederationToken(new GetFederationTokenRequest(). - withName("CloudWatchSender"). // TODO as above? + withName("CloudWatchSender-" + UUID.randomUUID()). withPolicy(policy(agentLogStreamName))). getCredentials(), region, agentLogStreamName); LOGGER.log(Level.FINE, "GetFederationToken succeeded; using {0}", auth.accessKeyId); @@ -272,7 +278,8 @@ void notifyShutdown(String agentLogStreamName) { if (auth.restricted) { return null; } else if (auth.accessKeyId != null) { - return "Giving up on limiting session credentials to a policy; using " + auth.accessKeyId + " as is"; + return "Giving up on limiting session credentials to a policy; using " + auth.accessKeyId + " as is: " + + AWSSecurityTokenServiceClientBuilder.standard().withCredentials(CloudWatchAwsGlobalConfiguration.awsCredentialsProviderChain).build().getCallerIdentity(new GetCallerIdentityRequest()); } else { return "No AWS credentials to be found, giving up on limiting to a policy"; } diff --git a/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/PipelineBridgeTest.java b/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/PipelineBridgeTest.java index ce95038..49ddd6a 100644 --- a/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/PipelineBridgeTest.java +++ b/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/PipelineBridgeTest.java @@ -24,6 +24,8 @@ package io.jenkins.plugins.pipeline_cloudwatch_logs; +import com.amazonaws.auth.AWSCredentialsProviderChain; +import com.amazonaws.auth.DefaultAWSCredentialsProviderChain; import com.cloudbees.jenkins.plugins.awscredentials.AWSCredentialsImpl; import com.cloudbees.plugins.credentials.CredentialsScope; import com.cloudbees.plugins.credentials.SystemCredentialsProvider; @@ -59,8 +61,9 @@ static void globalConfiguration() throws Exception { SystemCredentialsProvider.getInstance().getCredentials().add(new AWSCredentialsImpl(CredentialsScope.GLOBAL, credentialsId, null, null, null, role, null)); CredentialsAwsGlobalConfiguration.get().setCredentialsId(credentialsId); } + CloudWatchAwsGlobalConfiguration.awsCredentialsProviderChain = new AWSCredentialsProviderChain(new V2ProfileCredentialsProvider(), DefaultAWSCredentialsProviderChain.getInstance()); CloudWatchAwsGlobalConfiguration configuration = ExtensionList.lookupSingleton(CloudWatchAwsGlobalConfiguration.class); - FormValidation logGroupNameValidation = configuration.validate(logGroupName, null, credentialsId); + FormValidation logGroupNameValidation = configuration.validate(logGroupName, null, credentialsId, false); assumeThat(logGroupNameValidation.toString(), logGroupNameValidation.kind, is(FormValidation.Kind.OK)); configuration.setLogGroupName(logGroupName); } diff --git a/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/V2ProfileCredentialsProvider.java b/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/V2ProfileCredentialsProvider.java new file mode 100644 index 0000000..c1d62ac --- /dev/null +++ b/src/test/java/io/jenkins/plugins/pipeline_cloudwatch_logs/V2ProfileCredentialsProvider.java @@ -0,0 +1,57 @@ +/* + * The MIT License + * + * Copyright 2023 CloudBees, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +package io.jenkins.plugins.pipeline_cloudwatch_logs; + +import com.amazonaws.auth.AWSCredentials; +import com.amazonaws.auth.AWSCredentialsProvider; +import com.amazonaws.auth.BasicAWSCredentials; +import com.amazonaws.auth.BasicSessionCredentials; +import software.amazon.awssdk.auth.credentials.AwsCredentials; +import software.amazon.awssdk.auth.credentials.AwsSessionCredentials; +import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider; + +/** + * Allows use of {@code aws sso login} when running tests. + *

TODO copied from {@code io.jenkins.plugins.artifact_manager_jclouds.s3}; + * perhaps move to {@code com.cloudbees.jenkins.plugins.awscredentials} + */ +public class V2ProfileCredentialsProvider implements AWSCredentialsProvider { + + private final ProfileCredentialsProvider delegate = ProfileCredentialsProvider.create(); + + @Override public AWSCredentials getCredentials() { + AwsCredentials credentials = delegate.resolveCredentials(); + if (credentials instanceof AwsSessionCredentials) { + AwsSessionCredentials sessionCredentials = (AwsSessionCredentials) credentials; + return new BasicSessionCredentials(sessionCredentials.accessKeyId(), sessionCredentials.secretAccessKey(), sessionCredentials.sessionToken()); + } else { + return new BasicAWSCredentials(credentials.accessKeyId(), credentials.secretAccessKey()); + } + } + + @Override public void refresh() { + assert false; + } +}