Make simple license mappings more strict #10334

sschuberth · 2025-05-15T11:04:03Z

Please have a look at the individual commit messages for the details.

sschuberth · 2025-05-15T11:09:44Z

utils/spdx/src/main/resources/simple-license-mapping.yml

@@ -22,34 +22,8 @@
 # Sort the entries below via IntelliJ's "Edit" -> "Sort Lines".
 ---

-# Ambiguous mappings (mapping reason not obvious without additional information)


As you can see, the new check accurately removes all entries that were already deemed to be ambiguous. Additionally, it removes some entries that probably should have been regarded to be ambiguous in the first place as well.

utils/spdx/src/main/kotlin/SpdxSimpleLicenseMapping.kt

sschuberth · 2025-05-15T11:11:47Z

utils/spdx/src/main/resources/simple-license-mapping.yml

@@ -23,41 +23,22 @@
 ---

 AFLv2.1: AFL-2.1
-ALv2: Apache-2.0


This further strictens the check to not tolerate differences in trailing zeros when extract digits. This avoids the theoretical issue that e.g. Apache2 could either refer to Apache-2.0 or Apache-2.1 if the latter came out at some point.

Make these explicit for clarity. Signed-off-by: Sebastian Schuberth <[email protected]>

Make mroe clear what distinguished these test containers. Signed-off-by: Sebastian Schuberth <[email protected]>

Tolerate differences in trailing zeros. Signed-off-by: Sebastian Schuberth <[email protected]>

Signed-off-by: Sebastian Schuberth <[email protected]>

sschuberth · 2025-05-15T11:17:01Z

@oss-review-toolkit/core-devs I would really like to finally get rid of the ambiguous simple license mappings that we already identified quite some time ago. I have implemented two variants via a check: A lenient and a stricter one. Let's discuss whether we can at least get the lenient check in. To me it's important that the criteria for checking ambiguity are objective and reproducible by tests.

fviernau · 2025-05-15T11:16:11Z

utils/spdx/src/test/kotlin/SpdxSimpleLicenseMappingTest.kt

@@ -57,6 +57,23 @@ class SpdxSimpleLicenseMappingTest : WordSpec({
                if (license.id.endsWith("-only")) key should containADigit()
            }
        }
+
+        "contain equal digits omitting trailing zeros" {
+            val exceptions = mapOf(


There are license IDs which have dates in their identifier, e.g. some LicenseRef-scancode-* licenses.
There may be further cases, where this heuristic fails.

I'd be fine with adding anything that start with LicenseRef- to the list of exceptions. But we don't have such mappings currently anyway.

fviernau · 2025-05-15T11:18:01Z

utils/spdx/src/main/resources/simple-license-mapping.yml

-ALv2: Apache-2.0
-ASL: Apache-2.0
-Apache-style: Apache-2.0
-Apache: Apache-2.0


commit: The title says Check for...

But the commit also removes a bunch of entries. I recall in the past, that we removed only some BSD related entry which caused huge amounts of issues popping up.

So, I believe this is critical and should be made more prominent by dedicating the commit title to it.

I can tweak commit messages later if we agree on the general idea and I move this PR out of draft state.

fviernau · 2025-05-15T11:18:48Z

utils/spdx/src/main/resources/simple-license-mapping.yml

@@ -23,41 +23,22 @@
 ---

 AFLv2.1: AFL-2.1
-ALv2: Apache-2.0


commit-msg: I believe the title should make clear that entries from the mapping are removed.

fviernau · 2025-05-15T11:20:09Z

utils/spdx/src/main/resources/simple-license-mapping.yml

 BSD-3: BSD-3-Clause
 BSD2: BSD-2-Clause
 BSD3: BSD-3-Clause
 Bouncy: MIT
 EDL-1.0: BSD-3-Clause
 FreeBSD: BSD-2-Clause-Views
-GPL-2: GPL-2.0-only
-GPL2: GPL-2.0-only


I prefer not to remove these entries, but instead separate them.
So, their use can be configured.

Making simple license mappings configurable is certainly a good idea, but out of scope of this PR. I first like to get the baseline clean.

Agree with @fviernau we can't remove these declared license mappings - this will cause major service disruption for many ORT users.

I'm fine with simply dropping this last, stricter commit from the PR. Would that resolve any of your concerns?

I would propose a splitting out the (not so risky) ambiguous ones and introduce a ambiguousMappingsEnabled configuration option.

Please, not yet more configuration options 🙄

codecov · 2025-05-15T11:31:20Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 56.48%. Comparing base (88015b4) to head (b922c7d).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main   #10334   +/-   ##
=========================================
  Coverage     56.48%   56.48%           
+ Complexity     1611     1609    -2     
=========================================
  Files           331      331           
  Lines         12268    12268           
  Branches       1138     1138           
=========================================
  Hits           6929     6929           
  Misses         4896     4896           
  Partials        443      443

Flag	Coverage Δ
funTest-non-docker	`33.43% <ø> (-0.18%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

tsteenbe · 2025-05-15T12:00:04Z

utils/spdx/src/main/resources/simple-license-mapping.yml

 BSD-3: BSD-3-Clause
 BSD2: BSD-2-Clause
 BSD3: BSD-3-Clause
 Bouncy: MIT
 EDL-1.0: BSD-3-Clause
 FreeBSD: BSD-2-Clause-Views
-GPL-2: GPL-2.0-only
-GPL2: GPL-2.0-only


Agree with @fviernau we can't remove these declared license mappings - this will cause major service disruption for many ORT users.

sschuberth · 2025-05-15T12:15:08Z

Splitting out the first two commits to #10336.

sschuberth commented May 15, 2025

View reviewed changes

fviernau reviewed May 15, 2025

View reviewed changes

utils/spdx/src/main/kotlin/SpdxSimpleLicenseMapping.kt Outdated Show resolved Hide resolved

sschuberth commented May 15, 2025

View reviewed changes

sschuberth added 4 commits May 15, 2025 13:14

chore(spdx-utils): Add types to SpdxSimpleLicenseMapping properties

d777dd6

Make these explicit for clarity. Signed-off-by: Sebastian Schuberth <[email protected]>

test(spdx-utils): Improve test container naming

6a1c0ee

Make mroe clear what distinguished these test containers. Signed-off-by: Sebastian Schuberth <[email protected]>

feat(spdx-utils)!: Check for equals digits in simple license mappings

ba5e1b8

Tolerate differences in trailing zeros. Signed-off-by: Sebastian Schuberth <[email protected]>

feat(spdx-utils)!: Remove tolerating trailing zero differences

b922c7d

Signed-off-by: Sebastian Schuberth <[email protected]>

sschuberth force-pushed the simple-license-mappings branch from f43a1fa to b922c7d Compare May 15, 2025 11:14

fviernau requested changes May 15, 2025

View reviewed changes

fviernau requested a review from tsteenbe May 15, 2025 11:20

tsteenbe requested changes May 15, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make simple license mappings more strict #10334

Make simple license mappings more strict #10334

sschuberth commented May 15, 2025

sschuberth May 15, 2025

sschuberth May 15, 2025

sschuberth commented May 15, 2025

fviernau May 15, 2025

sschuberth May 15, 2025

fviernau May 15, 2025

sschuberth May 15, 2025

fviernau May 15, 2025

fviernau May 15, 2025

sschuberth May 15, 2025

tsteenbe May 15, 2025

sschuberth May 15, 2025

fviernau May 15, 2025

sschuberth May 15, 2025

codecov bot commented May 15, 2025

tsteenbe May 15, 2025

sschuberth commented May 15, 2025

Make simple license mappings more strict #10334

Are you sure you want to change the base?

Make simple license mappings more strict #10334

Conversation

sschuberth commented May 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sschuberth commented May 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented May 15, 2025

Codecov Report

Choose a reason for hiding this comment

sschuberth commented May 15, 2025