Disable DisablePluginCommandTest

[daniel-beck]

Steps to reproduce (for me):

• Check out e3af0f3
• mvn -Dfindbugs.skip=true -Dtest=DisablePluginCommandTest clean install

Result:
https://gist.github.com/daniel-beck/59cddf409b384483c002f162a3b35f65

It continued to run for a while until I aborted it. I have no idea whether it ever finishes, or will just continue to run the test over and over.

I have no idea what's going on here. Perhaps the author @MRamonLeon knows?

[oleg-nenashev]

Let's merge it if it blocks the release

[daniel-beck]

Let's merge it if it blocks the release

Given the existence of 2.152, that's unlikely.

[MRamonLeon]

Being in:

git branch -vv |grep danielBeck
* danielBeckInfinite                  e3af0f3bb1 [maven-release-plugin] prepare for next development iteration

Executing:

mvn -Dfindbugs.skip=True -Dtest=DisablePluginCommandTest clean install

The output is fine:
https://gist.github.com/MRamonLeon/e9d0c13670120d2dbc31c5aa049a3c12#file-log-txt

Do you have any extra information to reproduce the problem?
We can sync tomorrow early @daniel-beck or whenever you need.

[daniel-beck]

https://ci.jenkins.io/job/Core/job/jenkins/job/PR-3728/16/execution/node/69/log/?consoleFull exhibited the same problem (the pipeline part running on Linux JDK8, unsure whether that's relevant). Excerpt:

22:27:21 [INFO] Running hudson.cli.DisablePluginCommandTest
22:27:33 [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.6 s - in hudson.cli.WaitNodeOnlineCommandTest
22:27:33 [INFO] Running hudson.cli.DeleteJobCommandTest
22:27:51 [INFO] Running hudson.cli.DisablePluginCommandTest
22:28:01 [INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 28.776 s - in hudson.cli.DeleteJobCommandTest
22:28:03 [INFO] Running hudson.cli.ListPluginsCommandTest
22:28:18 [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.525 s - in hudson.cli.ListPluginsCommandTest
22:28:18 [INFO] Running hudson.cli.RemoveJobFromViewCommandTest
22:28:18 [INFO] Running hudson.cli.DisablePluginCommandTest
22:28:50 [INFO] Running hudson.cli.DisablePluginCommandTest
22:28:54 [INFO] Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.298 s - in hudson.cli.RemoveJobFromViewCommandTest
22:28:55 [INFO] Running hudson.cli.BuildCommandTest
22:29:22 [INFO] Running hudson.cli.DisablePluginCommandTest
22:29:48 [INFO] Running hudson.cli.DisablePluginCommandTest
22:30:15 [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.246 s - in hudson.cli.BuildCommandTest
22:30:15 [INFO] Running hudson.cli.DisconnectNodeCommandTest
22:30:17 [INFO] Running hudson.cli.DisablePluginCommandTest
22:30:56 [INFO] Running hudson.cli.DisablePluginCommandTest
22:31:28 [INFO] Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 71.053 s - in hudson.cli.DisconnectNodeCommandTest
22:31:28 [INFO] Running hudson.cli.EnablePluginCommandTest
22:31:28 [INFO] Running hudson.cli.DisablePluginCommandTest
22:31:49 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 21.72 s - in hudson.cli.EnablePluginCommandTest
22:31:49 [INFO] Running hudson.cli.DeleteBuildsCommandTest
22:32:02 [INFO] Running hudson.cli.DisablePluginCommandTest
22:32:28 [INFO] Running hudson.cli.DisablePluginCommandTest
22:32:32 [INFO] Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.995 s - in hudson.cli.DeleteBuildsCommandTest
22:32:34 [INFO] Running hudson.cli.SetBuildDisplayNameCommandTest
22:32:49 [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.122 s - in hudson.cli.SetBuildDisplayNameCommandTest
22:32:49 [INFO] Running hudson.cli.OfflineNodeCommandTest
22:32:59 [INFO] Running hudson.cli.DisablePluginCommandTest
22:33:38 [INFO] Running hudson.cli.DisablePluginCommandTest
22:34:17 [INFO] Running hudson.cli.DisablePluginCommandTest
22:34:44 [INFO] Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.523 s - in hudson.cli.OfflineNodeCommandTest
22:34:44 [INFO] Running hudson.cli.ClearQueueCommandTest
22:34:45 [INFO] Running hudson.cli.DisablePluginCommandTest
22:35:03 [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.985 s - in hudson.cli.ClearQueueCommandTest
22:35:03 [INFO] Running hudson.cli.DeleteNodeCommandTest
22:35:18 [INFO] Running hudson.cli.DisablePluginCommandTest
22:35:50 [INFO] Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.336 s - in hudson.cli.DeleteNodeCommandTest
22:35:50 [INFO] Running hudson.cli.SetBuildParameterCommandTest
22:36:09 [INFO] Running hudson.cli.DisablePluginCommandTest
22:36:41 [INFO] Running hudson.cli.DisablePluginCommandTest
22:36:43 [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 52.815 s - in hudson.cli.SetBuildParameterCommandTest
22:36:44 [INFO] Running hudson.cli.SetBuildDescriptionCommandTest
22:37:06 [INFO] Running hudson.cli.DisablePluginCommandTest
22:37:08 [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.06 s - in hudson.cli.SetBuildDescriptionCommandTest
22:37:09 [INFO] Running hudson.cli.OnlineNodeCommandTest
22:37:41 [INFO] Running hudson.cli.DisablePluginCommandTest
22:38:20 [INFO] Running hudson.cli.DisablePluginCommandTest
22:38:32 [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 81.166 s - in hudson.cli.OnlineNodeCommandTest
22:38:32 [INFO] Running hudson.cli.InstallPluginCommandTest
22:38:42 [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.501 s - in hudson.cli.InstallPluginCommandTest
22:38:42 [INFO] Running hudson.cli.ComputerStateTest
22:38:46 [INFO] Running hudson.cli.DisablePluginCommandTest
22:39:25 [INFO] Running hudson.cli.DisablePluginCommandTest
22:39:57 [INFO] Running hudson.cli.DisablePluginCommandTest
22:39:59 [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 76.844 s - in hudson.cli.ComputerStateTest
22:40:01 [INFO] Running hudson.cli.RunRangeCommand2Test
22:40:23 [INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.396 s - in hudson.cli.RunRangeCommand2Test
22:40:23 [INFO] Running hudson.cli.ReloadConfigurationCommandTest
22:40:23 [INFO] Running hudson.cli.DisablePluginCommandTest
22:40:50 [WARNING] Tests run: 7, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 25.555 s - in hudson.cli.ReloadConfigurationCommandTest
22:40:50 [INFO] Running hudson.cli.GetJobCommandTest
22:40:56 [INFO] Running hudson.cli.DisablePluginCommandTest
22:41:00 [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.341 s - in hudson.cli.GetJobCommandTest
22:41:00 [INFO] Running hudson.cli.CLITest
22:41:27 [INFO] Running hudson.cli.DisablePluginCommandTest
22:41:59 [INFO] Running hudson.cli.DisablePluginCommandTest

@MRamonLeon I still don't know how to reproduce this, but it's clearly happening on systems that aren't just mine.

[daniel-beck]

Also seems to have affected this core master build.

Here's what the previous, successful build looked like:

Here's the one with the failing test:

[daniel-beck]

I think @varyvol looked at this a while ago and identified the specific problem, so pinging him.

KK tells me the staging of a security update earlier this week was affected by this as well, although it finally went through for him.

[varyvol]

@daniel-beck I had this issue, but I think it was not raised by this specific test. The problem then was the test was trying to restart the test instance, and that makes JUnit to go crazy.

[MRamonLeon]

@daniel-beck The errors raised due to the Jenkins instance being null. The DisablePluginCommandTest has two methods where the Jenkins.get().safeRestart() is invoked. It seems it could be the cause of the problem. Is it feasible to have two tests with such invocation? If so, it could be a kind of race condition being hit during the clean up and restart of Jenkins. WDYT?

[batmat]

To add some color here, I confirm I've seen this test causing a few build failures here in the past few days/weeks, but not that often.
I suppose we're also going to see a clearer picture now we merged #3795, and hopefully we put #3797 in too soonish.

[MRamonLeon]

If the test is still failing after these both PRs get merged, I'll try removing the tests with restart.

[jglick]

I discovered this problem independently in #3804.

[jglick]

I think I have this.