Attempts to update project failing

[hazendaz]

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jdk.internal.loader.ClassLoaders$AppClassLoader@5ffd2b27-org.sonar.server.plugins.ServerPluginManager': Initialization of bean failed; nested exception is Plugin Findbugs [findbugs] requires at least Sonar Plugin API version 10.14.0.2599 (current: 10.11.0.2468)

Getting issues like this and not sure how its getting downgraded as dependency tree does not show that. Happens with 9.x as well.

Additionally if I try latest sonar version it fails with non descriptive error.

Not sure what is causing the issues but would like to get fixed as project was considerably out of date.

[gtoison]

How are you getting these errors (and what are they exactly)? I mean, how are you running SonarQube (with orchestrator from the integration tests? which version?)
There were so many changes recently that I lost track

[hazendaz]

ok so on this, if I update the sonar artifacts (9/10) to the latest which are currently sitting in renovate dashboard as closed, the error pops up that it says it wants older version. I cannot see exactly where that was happening from. The message up there is what it states, switching it slightly back then it works fine. So using 10.14 for example, I'm struggling to see where 10.11 comes from.

Yes there were many changes, renovate stopped working because there was account issues. I fixed those and builds started working so I launched all. Looks like still many more to go. nodejs section in this tends to be that way as it seems nodejs teams release constantly every time a file is touched.

[gtoison]

The sonar artifacts should stay on the version corresponding to the earliest SonarQube supported, which is 9.9 LTS. For a given SonarQube version there are several related dependencies (the plugin API, the java plugin, the java version) and they need to be on the right version.
Upgrading the plugin API or the java plugin to the latest version is not a good idea because they are usually released before a compatible SonarQube server exists.
The matrix build here overrides these maven properties so there's a full build (compile, unit tests, integration tests) on multiple SonarQube version, currently 9.9.7 and 10.7
In the end the release is executed the default (non overridden) versions from the pom.xml so these should for the earliest version, or it will break compatibility with SonarQube 9.9

Now regarding the nodejs section, it is not functional in its current state and I could not find the time to look into it.
It was added to automate the "SonarSource part" of the release process which used to involve posting a release message on their forum, plus creating a pull request here: https://github.com/SonarSource/sonar-update-center-properties/pulls
Now they have dropped the requirement for the forum post, and I'm doing the pull request manually.
At the very least the renovate pull request should be throttled (or grouped?) because there isn't much point cluttering the git history with all these commits

[gtoison]

Hey @hazendaz
Would it be possible to adjust the renovate settings? I see that you're merging a lot renovate PRs but most of them aren't right:

• All the PR touching the typescript action aren't right because that thing does not work at all, nobody stepped in it fix so I think it should be removed
• The PR to upgrade SpotBugs or its plugins aren't right because the metadata and docs aren't updated at the same time

I am all for automation but the way this is currently working is not right: the vast majority of the renovate commits are broken in some way

[hazendaz]

Hey @hazendaz Would it be possible to adjust the renovate settings? I see that you're merging a lot renovate PRs but most of them aren't right:

* All the PR touching the typescript action aren't right because that thing does not work at all, nobody stepped in it fix so I think it should be removed

* The PR to upgrade SpotBugs or its plugins aren't right because the metadata and docs aren't updated at the same time

I am all for automation but the way this is currently working is not right: the vast majority of the renovate commits are broken in some way

On the second item, there is a way to add a script to run a post action. That could address items you are wanting updated at same time so that it is not a manual event. Can you outline or possibly point to a commit where that was done last and I'll see if I can get a script together that will address that.

On the first, I agree they are not really tested but nearly every one was minor updates so they are probably right, just noisy. I can add an ignore I think to stop suggesting in that package since its non working anyways. If that part isn't really necessary, then maybe the right option is just deleting it as you note.

In meantime, I stopped merging things until these concerns can be addressed. Thanks.

[gtoison]

Here are commits where renovate upgraded SpotBugs or a plugin:
8a8ecb8
670334e

And the corresponding commits with all the changes needed:
35926e7
540ca0c

I have started working on simplifying the metadata build in the sq-10 branch but haven't touched it in a while due to time constraints on my part.

[gtoison]

Hello @hazendaz
I've just realized that you were sending pull requests to sonar-update-center-properties
I'm not sure if you saw but they aren't approved, because the properties aren't right.

Any objection removing that build action creating these pull requests? At this point it is completely broken and isn't saving anyone's time.

[hazendaz]

I"m super confused, I wondered why I got sent notices from them. How the heck is this opening pull requests from a fork I have that I haven't once updated. My fork is showing all kinds of updates on it. How is this job somehow doing that? I've only ever updated the files a few times and not once was it related to my account. Any idea?

Whatever its doing, they were ok with it but stated it needed an update fix which was trivial, now they are complaining that I keep sending these but I haven't sent a single one. At least not as myself.

[hazendaz]

Seems this started happening last month. I'm still very confused, only thing that happened last month was getting access to sonar dashboard for the project. The sonar credential looks updated a month back here but suspect that was you rather than me.

[hazendaz]

But since it seems almost here and they asked to just fix the script, what would be downside of fixing it once understanding how its working as that seems the correct way to upload stuff and its barely off.

[hazendaz]

@gtoison So your PR to them appears to have the same context in description but different data, are you doing that manually? I'm still struggling to understand how this branded under my name from my fork that doesn't make any sense to me. Also based on what you did with assumption that was manual, why would we not just get this working, its so utterly close to the same content with little effort to fix. The bigger thing is how is it happening. I don't see how the github action on one project is pulling stuff onto my fork then sending PR on my behalf.

[hazendaz]

e that you're merging

What do you really think isn't working there? Clearly its running the action and its actually working. So I'm a bit concerned on the comment renovate isn't working there. So the changes I've applied are in fact accurate with all the data I now have. I'm still at a loss right now as to why every run is updating my forked repo. I have some 500+ branches as a result but the point is that its clearly working so the renovate PRs were in fact valid and worked. That usually is the case with nodejs. Those libraries release far more frequently than any other platform I'm aware of an rarely do they cause issues. If the larger concern is the file structure is off, it looks to me a few lines and it would be working matching what you sent sonar. The key I think is figuring out how my fork started sending PRs to them in february when I would not have expected it to even know about my fork. Maybe its getting my user id from the merges but I didn't even merge days ago, you did though and it still wrote to me repo then sent a PR from my repo :(

[hazendaz]

So to that I'm kind of saying I want to tackle and fix this so it works going forwards. I do a ton of dev ops at my work and run entire platform for renovate there over thousands of repos, its extremely rare that was ever an issue, I just went with what you were saying and dropped out of the items in that area but again, its clearly working or it would have failed.

[gtoison]

Yes, I did the PR in sonar-update-center-properties manually, it just takes a minute or two.
The branches in your fork are because the integration tests of the action are doing them.

It's great that it works again, but would it be possible to tone down the renovate PR? Maybe group the version upgrades together, or reduce the frequency so it does not pick up every minor upgrade.

The builds for the action are still failing apparently, I'm not sure why:

[hazendaz]

For the failure, its 2 tests using nock that are failing mocking. Those tests only run on github. So if running locally it works fine. I'm looking into those as I'm not familiar with the library and the first test was initially failing as it wanted discovered max sonar version 25.3.x. After fixing that the next statement is to do the assertion from nock and it seems to not think it was mocked, probably because it was a real call I suspect that worked but I need to understand better on what that is and do some 101 learning on it.

The only other two outstanding items currently on the nodejs part is eslint needs upgraded in a one shot deal, I've got that locally but it required migration and the migration guide I don't fully understand as it pertains to it stating it found the typescript files to lint but refuses to do so as it says configuration is missing but the migration created the configuration which more or less matches the original. That will clear bulk of what is not updated there by renovate. The second issue is with promises library and that requires some code changes that I initially tried to update back in November and left as WIP when I got stuck. Once I get those, at least from a current status, renovate would have nothing to do.

Now from a configuration for renovate, it may be a tricky trick to make things work. I think we both agree nodejs in general is far too noisy. Not only is it fixing the pinned dependencies in package.json, its also updating the lock file every time something vulnerable is patched. With nodejs that is constant daily churn. Now I personally don't mind the churn given workloads I already see on github and at work where I'm merging typically 100s of PRs a day for same reason. So I'm thinking, I split that out with an ignores against the master branch and I use a separate branch on my fork that collects those up and I bring over in a far smaller rate. That would possible make that work so you only see what is necessary.

Other considerations I have at the moment

• Why does the sonar update center sometimes run but other times not? Its weird to me that it shows ok sometimes only because it did not actually run so some gha issue there.
• Need to add logic to branch cleanup all the junk it adds, its adding every single build. As I had noted, your repo copy of that has some 200+ branches, mine had 500+. I'll review that logic and make it such that it deletes all those prefixed branch types entirely before submitting a new one.
• The actual logic that is writing that branch needs to be updated to pull of what you were doing. Maybe at that point its feasible to then use that.

My plan is to spend some cycles on this for next week or so to try to hammer that out aspects of this. But for our general sake of churn here, if I don't have it figured out in all ways mentioned, I'll at least turn off renovate from suggesting in that area, I can look those up given that process isn't fully functional yet. I'll try to get that on there in next day or two so you just stop seeing those. I'll also look to put a timer on it so its not running every time something in the world changes but rather maybe once a week instead.