-
Notifications
You must be signed in to change notification settings - Fork 314
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(npm): Remove unused parallelization constructs #9316
Conversation
41c90e0
to
6f80410
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9316 +/- ##
============================================
- Coverage 67.49% 67.49% -0.01%
- Complexity 1198 1200 +2
============================================
Files 241 241
Lines 8495 8493 -2
Branches 900 899 -1
============================================
- Hits 5734 5732 -2
Misses 2399 2399
Partials 362 362
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Mind having a look, @oheger-bosch? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes look good to me. Nice finding, that the async
logic was rather useless.
@@ -24,11 +24,6 @@ package org.ossreviewtoolkit.plugins.packagemanagers.node | |||
import java.io.File |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commit message:
Nit: "...strictly sequentially."
So, if I understand correctly, the whole fiddling with async
had no real effect, since only a single package was processed, and the parsePackage
function is only called sequentially? Maybe make this a bit more explicit in the commit message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since only a single package was processed
hm, I made the observation when running the fun tests, so not a single package but multiple package were involved. (A single package cannot be processed in parallel, so I guess this is a mis-understanding)
and the parsePackage function is only called sequentially
I in fact verified that getRemotePackageDetails()
executes only sequentally. But, parsePackage()
is the only caller of that one, so I guess the same applied to parsePackage()
.
What change to the commit message do you suggest?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From what I understand, there are multiple aspects:
- The
withContext
construct ingetRemotePackageDetailsAsync
is only exited after all coroutines started within it are finished. Therefore, theasync
invocations are useless, since the function waits until the asynchronous processing is complete. - But even on a higher level, no parallelization is applied;
parsePackage
is invoked sequentially for each package and waits for the result. So, even if asynchronous processing worked, it would not help here.
Do you agree with this?
Regarding the commit message, it is probably not necessary to describe this in detail. So, your text is probably fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you agree with this?
Yes, makes sense to me.
@@ -245,10 +245,6 @@ open class Npm( | |||
* content via the `npm view` command. The result is a [Pair] with the raw identifier and the new package. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Commit message nit: "...would have searched..."
783b8cc
to
cfdcadb
Compare
cfdcadb
to
4dc3f13
Compare
The code looks like it would run things in parallel, but it executes `getRemotePackageDetails()` strictly sequentially. The pull request which had introduced these constructs [1] also introduced the caching. So, the speed-up observed back then was probably solely due to the added cache. Remove the parallelization constructs to not mislead the reader and to simplify. [1]: #6009 Signed-off-by: Frank Viernau <[email protected]>
The log message sounds as if the function would have searched the 'package.json' and now found one. However, the function does not search anything. Simply remove that log output to simplify an upcoming change. Signed-off-by: Frank Viernau <[email protected]>
4dc3f13
to
697903b
Compare
The code looks like it would run things in parallel, but it executes
getRemotePackageDetails()
strictly sequentially. The pull request which had introduced these constructs 1 also introduced the caching. So, the speed-up observed back then was probably solely due to the added cache. Remove the parallelization constructs to not mislead the reader and to simplify.Part of #9261.