Fix a deadlock between WorkflowRun#save and WorkflowRun#reload

[Vlatombe]

Ensure locking order

metadataGuard > WorkflowRun > CpsFlowExecution

"some-thread" id=130 (0x82) state=BLOCKED cpu=84%
    - waiting to lock <0x54046fbb> (a org.jenkinsci.plugins.workflow.job.WorkflowRun)
      owned by "Running CpsFlowExecution[Owner[some-project/1:some-project #1]]" id=27474 (0x6b52)
    at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.save(WorkflowRun.java:1252)
    at hudson.BulkChange.commit(BulkChange.java:98)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1557)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.FlowHead.lambda$notifyNewHead$0(FlowHead.java:164)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.FlowHead$$Lambda/0x000000080191daa0.run(Unknown Source)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.FlowHead.notifyNewHead(FlowHead.java:169)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.FlowHead.setNewHead(FlowHead.java:146)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.FlowHead.setNewHead(FlowHead.java:123)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.createPlaceholderNodes(CpsFlowExecution.java:751)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.onLoad(CpsFlowExecution.java:815)
    at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.reload(WorkflowRun.java:561)
[...]

"Running CpsFlowExecution[Owner[some-project/1:some-project #1]]" id=27474 (0x6b52) state=BLOCKED cpu=64%
    - waiting to lock <0x29b5111b> (a org.jenkinsci.plugins.workflow.cps.CpsFlowExecution)
      owned by "some-thread" id=130 (0x82)
    at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$ConverterImpl.marshal(CpsFlowExecution.java:1737)
    at hudson.util.XStream2$AssociatedConverterImpl.marshal(XStream2.java:556)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:83)
    at hudson.util.RobustReflectionConverter.marshallField(RobustReflectionConverter.java:283)
    at hudson.util.RobustReflectionConverter$2.writeField(RobustReflectionConverter.java:270)
    at hudson.util.RobustReflectionConverter$2.visit(RobustReflectionConverter.java:241)
    at com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider.visitSerializableFields(PureJavaReflectionProvider.java:174)
    at hudson.util.RobustReflectionConverter.doMarshal(RobustReflectionConverter.java:226)
    at hudson.util.RobustReflectionConverter.marshal(RobustReflectionConverter.java:163)
    at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:68)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59)
    at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:44)
    at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:83)
    at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37)
    at com.thoughtworks.xstream.XStream.marshal(XStream.java:1307)
    at com.thoughtworks.xstream.XStream.marshal(XStream.java:1296)
    at com.thoughtworks.xstream.XStream.toXML(XStream.java:1269)
    at hudson.XmlFile.write(XmlFile.java:216)
    at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.PipelineIOUtils.writeByXStream(PipelineIOUtils.java:30)
    at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.save(WorkflowRun.java:1253)
[...]

Testing done

Submitter checklist

• Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
• Ensure that the pull request title represents the desired changelog entry
• Please describe what you did
• Link to relevant issues in GitHub or Jira
• Link to relevant pull requests, esp. upstream and downstream changes
• Ensure you have provided tests - that demonstrates feature works or fixes the issue

[jglick]

Amending #472 / #510 FTR

[jglick]

🤞

[felipecrs]

From a user perspective, can this help with pipelines not recovering after Jenkins restart?

[jglick]

@felipecrs no, whatever problem you may be having is almost certainly unrelated.

[felipecrs]

Got it, thank you @jglick.

[felipecrs]

Out of curiosity, @jglick, I found out what was causing my builds not to recover after a Jenkins restart: JENKINS-75731.