[JENKINS-70101] Revive ability to snapshot() the CertificateCredentials so they can be used on remote agents #391
Conversation
…des [JENKINS-70101]
…tials() et al to standalone cases
…able*() to make sure it is right
…tial" to differentiate from expected other credential types
…equest() in the same pipeline (using same bytes)
|
Problem exposed in build logs, e.g. in https://github.com/jenkinsci/credentials-plugin/pull/391/checks?check_run_id=9617279731 : Will post the fix commits now :) |
…r getUploadedKeystore()
…dedKeystore" be used to serialize
…d, must use Secret not SecretBytes
…urce() depend on Channel.current() == null
|
Tests fixed. Now posting the final touch, to make them quieter. This passes - and good to merge (on my side) :) |
|
@jenkinsci/core-security-review Should probably review this. |
|
Yep. In particular, I wonder if it is okay to leave it like this (possibly storing keystore data "plaintext" in a I have a further effort to force conversion from |
|
@jenkinsci/core-security-review Gentle bump :) |
|
Hey @jimklimov, we've noticed the issue. We'll try to review it if/when time permits. |
…esting [JENKINS-70101] Signed-off-by: Jim Klimov <[email protected]>
jimklimov
force-pushed
the
JENKINS-70101
branch
from
696d93c to
d56e1bc
Compare
Signed-off-by: Jim Klimov <[email protected]>
…about the agent setup Signed-off-by: Jim Klimov <[email protected]>
Signed-off-by: Jim Klimov <[email protected]>
…enkinsci#391 discussion Specifically the jenkinsci#391 (comment) note that the original comment about `values[]` seems wrong, and comments before that. Signed-off-by: Jim Klimov <[email protected]>
jimklimov
force-pushed
the
JENKINS-70101
branch
from
d56e1bc to
6e808d1
Compare
…) instead of custom code [JENKINS-70101] Signed-off-by: Jim Klimov <[email protected]>
…nAgent() proposed in PR jenkinsci#391 discussion Rearranged from commit 4ee16f3 posted in https://github.com/dwnusbaum/credentials-plugin/tree/JENKINS-70101 (initially as a simpler bug repro case than CredentialsInPipelineTest). Co-authored-by: Devin Nusbaum <[email protected]> Signed-off-by: Jim Klimov <[email protected]>
… agent after test case [JENKINS-70101] Signed-off-by: Jim Klimov <[email protected]>
|
Note for posterity: for a long time, it sufficed in practice to merge master-branch increments and reap the benefits of the PR'ed change for our local workflows (regardless of it being or not pedantically "wrong", per review comments above, it worked). Alas in the past couple of weeks, with the ban on JUnit 4 the existing test cases broke, a quick refactor did not help, so finally (long overdue) I'm trying to dig into the requested fixes to the PR, piece by piece. At least with the New Year holidays got a few cycles away from dayjob churn to dedicate some attention to this issue, once in a few years. As I am still not much of a Java developer, I am struggling to make practical sense of some short suggestions posted earlier; consider me obtuse if needed:
|
|
Odd... locally test runs in IDEA failed with padding problems (maybe wrong key), but CI above passed as of https://ci.jenkins.io/job/Plugins/job/credentials-plugin/job/PR-391/20/pipeline-overview/ both on Linux and Windows workers, so I vented the questions above. FWIW, locally it complained thusly: ...so I was afraid something in the changes to test setup (or with recent 2 weeks worth of changes merged from master branch) broke the cross-JVM data transfer back. Retrying locally with |
|
...and |
…ialsImplOnRemoteAgent() vs useCertificateCredentialsImplOnBuiltinAgent() Signed-off-by: Jim Klimov <[email protected]>
…er file names for test keystores Note: `createTempFile()` deals with patterns like shell `mktemp`, not with exact file names. Signed-off-by: Jim Klimov <[email protected]>
Signed-off-by: Jim Klimov <[email protected]>
…lugin on a remote agent JVM, note that in the thrown Error Signed-off-by: Jim Klimov <[email protected]>
|
Good news, even though still puzzling: as I worked on adding Maybe somehow somewhere it loaded an older build of the plugin (the standard one without the fix) until the repetitive recompilation pushed it out?.. This makes little sense, but is the best explanation I've still got. |
|
As for dropping the dependency on http_request plugin (in tests), the setup can be inverted by moving the tests there: https://github.com/jenkinsci/http-request-plugin/blame/master/src/test/java/jenkins/plugins/http_request/HttpRequestStepCredentialsTest.java was updated with the passing half (local JVM) of this test suite years ago. But this is inherently harder to test before this PR gets merged on one hand, and essentially tests that credentials plugin is functional by code that is part of http request plugin then (it still make sense, so plugin users know which version combo works on both local and remote build agents). |
… refactor to optionally juggle withReentrability and to debug-trace withLocalCertLookup [JENKINS-70101] Import and adapt code evicted from jenkinsci/credentials-plugin#391 Signed-off-by: Jim Klimov <[email protected]>
…t code for use of HTTP Request plugin with Credentials [JENKINS-70101] Tests to make sure the complex call stack works properly are offloaded into that plugin, see jenkinsci/http-request-plugin#231 Signed-off-by: Jim Klimov <[email protected]>
…d as deprecated ...to avoid a maven warning Signed-off-by: Jim Klimov <[email protected]>
Consumers should not be using
It should not require any changes on downstream plugins - if a plugin requires changes this to be performed then that plugin is broken and should be fixed. We should not be working around it with serialisation tricks (as this will also affect persisting to disk, causing a lot of fun and games). see https://github.com/jenkinsci/credentials-plugin/blob/master/docs/consumer.adoc#retrieve-the-secret-from-a-specific-credentials-instance for the documentation around this.
If they expect some implementation of an interface they have a broken implementation. The interfaces exist to allow different implementations to be interchangeable. For example there are other credential providers, these may supply their own implementation of
https://javadoc.jenkins.io/component/jenkins-test-harness/org/jvnet/hudson/test/TestExtension.html
|
|
Thanks a lot for the reasonably fast reply so this did not get shelved again :-D
Great, hoped so. Just the original comment seemed like I could read it several ways :)
Gotcha, I hope. To be sure I did :) reiterating the plan in my words:
Is that change a "hard requirement" to eventually merge the PR? Would it suffice to just use Devin's tests as imported and augmented in this PR currently, as they seem to already tick most of those points you listed? This touches on maybe dropping the whole remainder of the |
5 participants
As detailed in https://issues.jenkins.io/browse/JENKINS-70101 I've hit a problem using certificate-based user authentication in http-request-plugin (noticed as part of my work on https://issues.jenkins.io/browse/JENKINS-70000 in jenkinsci/http-request-plugin#120 but also reproduced with latest 1.16 release of that plugin).
After a week of reproductions and tracing across different plugins, the root of the problem crystallized as the use of
SecretBytesinUploadedKeyStoreSourceand effective lack of specialsnapshot()support. As a consequence, serialized copies of the key store used the sameSecretBytesas on the Jenkins Controller, but being on a different JVM in the remote agent case, they used a different instance of the staticKEYfield with its ownsecretfield. Although theSecretBytesare diligently copied (took a lot of effort to confirm that as seen in sibling branch https://github.com/jimklimov/credentials-plugin/tree/JENKINS-70101-trace), they are unreadable on the remote agent.During investigation I found that credentials-plugin did have an implementation for the Certificate Credentials Snapshot Taker, but it was removed by 40d0b5c as part of deprecation of
FileOnMasterKeyStoreSourcesubclass. Also an earlier history of the plugin involved theSecretclass which effectively stores a base64 string and only encodes/decodes it on demand -- this was superseded bySecretBytesbut some handling for conversion from older versions remained. This code proved to be a good starting point for fixing the problem, although not without some further work:isSnapshotSource(forcedtrueforUploadedKeyStoreSourcesubclass) and just returned the original credential if so;SecretBytesfor the copied key store and suffered the same fate - not usable on agent.This PR adds tests (failing at first to confirm the problem), makes use of older codebases and new verified fixes:
UploadedKeyStoreSource.isSnapshotSource()depends onChannel.current() == null(so for remoting-related snapshots it isfalseand shortcuts toreturn thisare not taken)CertificateCredentialsSnapshotTakerwas revived as a separate source file and standalone class, similar to existingUsernamePasswordCredentialsSnapshotTakerUploadedKeyStoreSourcewas modified to handle again aSecret uploadedKeyStorefield (and provide data from it ifSecretBytes uploadedKeyStoreBytesare currentlynull). As part of that,transientandfinalmodifiers on these fields had to go.CertificateCredentialsSnapshotTakerwas modified to create the newUploadedKeyStoreSourceinstance withSecretversion rather thanSecretBytes(asCredentialsSnapshotTakerdocs stipulate, "all the details are captured within the credential")Thanks to @mawinter69 and @slide for bright ideas and pointers, and general sympathy as I woed on the chat :)
Separately note that this PR adds tests using a separate JVM for the build agent to reproduce the problem. This code may be worth exporting into some
JenkinsAgentRuleif someone is up for it.NOTE: I'll post commits in several phases, so CI has its chance to fail with the tests and show the original problem in logs.