maven-heaven: map_shaded_classes_to_maven_packages ignores POMs present only in the from/ codebase, leaving all shaded .class resources unmapped

This is a follow-up issue for #2247

When scanning a Maven package whose binary JAR excludes META-INF/maven/** (a common shade‑plugin configuration) but whose source JAR still contains the POM, the shaded .class resources in the binary are never mapped back to their originating Maven dependencies. They remain with an application-package status instead of shaded-class.

Sample

PURL: pkg:maven/org.apache.kyuubi/kyuubi-relocated-thrift@0.4.1

There is no pom.xml in the deployment (to/) codebase. The only POM present is: from/META-INF/maven/org.apache.kyuubi/kyuubi-relocated-thrift/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <parent>
    <artifactId>kyuubi-relocated-parent</artifactId>
    <groupId>org.apache.kyuubi</groupId>
    <version>0.4.1</version>
    <relativePath>../../pom.xml</relativePath>
  </parent>
  <modelVersion>4.0.0</modelVersion>
  <artifactId>kyuubi-relocated-thrift</artifactId>
  <description>Relocated Thrift classes used by Kyuubi internally.</description>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-shade-plugin</artifactId>
        <executions>
          <execution>
            <id>shade-kyuubi</id>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <createDependencyReducedPom>true</createDependencyReducedPom>
              <dependencyReducedPomLocation>${project.basedir}/target/dependency-reduced-pom.xml</dependencyReducedPomLocation>
              <artifactSet>
                <includes>
                  <include>*</include>
                </includes>
              </artifactSet>
              <filters>
                <filter>
                  <artifact>*:*</artifact>
                  <excludes>
                    <exclude>**/*.proto</exclude>
                    <exclude>META-INF/*.SF</exclude>
                    <exclude>META-INF/*.DSA</exclude>
                    <exclude>META-INF/*.RSA</exclude>
                    <exclude>META-INF/DEPENDENCIES</exclude>
                    <exclude>META-INF/LICENSE.txt</exclude>
                    <exclude>META-INF/NOTICE.txt</exclude>
                    <exclude>META-INF/maven/**</exclude>
                    <exclude>LICENSE.txt</exclude>
                    <exclude>NOTICE.txt</exclude>
                    <exclude>mozilla/**</exclude>
                    <exclude>**/module-info.class</exclude>
                  </excludes>
                </filter>
              </filters>
              <relocations>
                <relocation>
                  <pattern>com.facebook.fb303</pattern>
                  <shadedPattern>${shading.prefix}.fb303</shadedPattern>
                  <includes>
                    <include>com.facebook.fb303.**</include>
                  </includes>
                </relocation>
                <relocation>
                  <pattern>org.apache.thrift</pattern>
                  <shadedPattern>${shading.prefix}.thrift</shadedPattern>
                  <includes>
                    <include>org.apache.thrift.**</include>
                  </includes>
                </relocation>
              </relocations>
            </configuration>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>flatten-maven-plugin</artifactId>
      </plugin>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <executions>
          <execution>
            <id>compile</id>
            <phase>package</phase>
            <goals>
              <goal>attach-artifact</goal>
            </goals>
            <configuration>
              <artifacts>
                <artifact>
                  <file>${basedir}/target/${project.artifactId}-${project.version}.jar</file>
                  <type>jar</type>
                  <classifier>optional</classifier>
                </artifact>
              </artifacts>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
  <properties>
    <thrift.version>0.16.0</thrift.version>
    <fb303.version>0.9.3</fb303.version>
  </properties>
</project>

Note that the binary archive excludes META-INF/maven/** and therefore no pom.xml in the deployment codebase.
As a result, the map_shaded_classes_to_maven_packages is not mapping anything.

Issues:

The pom.xml was excluded from the deployment codebase:
https://repo1.maven.org/maven2/org/apache/kyuubi/kyuubi-relocated-thrift/0.4.1/kyuubi-relocated-thrift-0.4.1.pom
It contains the ${shading.prefix} variable, which requires fetching the parent package to resolve:
https://repo1.maven.org/maven2/org/apache/kyuubi/kyuubi-relocated-parent/0.4.1/kyuubi-relocated-parent-0.4.1.pom

Not addressed by this issue — see #2292

No dependency information is defined in the Maven pom.xml; the dependency details exist only in the project's GitHub repository:
https://raw.githubusercontent.com/apache/kyuubi-shaded/refs/tags/v0.4.1/kyuubi-relocated-thrift/pom.xml

@chinyeungli

Hi @chinyeungli,

I've been looking into this issue and traced the relevant logic in scanpipe/pipes/maven.py, particularly get_main_maven_pom() and get_maven_shade_relocations().

One possible cause I found is that get_main_maven_pom() searches only the to/ codebase. In the deploy-to-develop workflow, the binary JAR may not contain the Maven POM because META-INF/maven/** was excluded, even though the corresponding POM is available in the from/ codebase. If that happens, the relocation information cannot be extracted, leaving the shaded classes unmapped.

I also noticed two potentially related cases worth investigating: unresolved property expressions in <shadedPattern> values and packages missing from the discovered dependency inventory. I haven't assumed these should all be addressed in the same change.

My initial proposal would be to start with a focused fix that falls back to the from/ codebase when the POM is absent from to/, with regression tests covering this scenario and the existing behavior.

Before proceeding, could you clarify a few things?

Should the lookup prioritize to/, then from/, while preserving the existing behavior for other matching POMs?
Is property interpolation for shade relocation patterns already being addressed, or would that be a separate follow-up?
Since you're assigned to this issue, are you already working on a fix, or would you be open to a targeted contribution for the POM lookup and its regression tests?

I'd be happy to align with your preferred approach and keep the scope consistent with the existing Maven pipeline.

Thanks for your time!

Resolve Maven property placeholders and recover dependency-reduced POMs

@Rp-13-byt

Hi @Rp-13-byt ,

Thanks for the interest on the issue, but I've already worked on the issue.

@chinyeungli

Thanks for letting me know, @chinyeungli

I'd still really like to contribute to ScanCode.io and become more involved with the project. If there are any other open issues that need attention, I'd appreciate it if you could point me toward one that would be a good fit.

I'm particularly interested in Python, pipeline improvements, package analysis, and writing regression tests. I'm happy to start with a smaller issue, understand the existing implementation, and follow the project's contribution guidelines.

Also, is there an official community channel, such as Slack or a chat group, where ScanCode.io maintainers and contributors discuss development and coordinate contributions? I'd love to join and learn more about the project's ongoing work.

Thanks again for your time and guidance!

Remove the feature to load dependencies from source repo #2284

Address feedback on Maven shaded class mapping #2284

FIx the test by adding mock data #2284

Fixed typo and the `test_scanpipe_maven_resolve_shading_properties_ma…

Fix the `test_scanpipe_maven_resolve_shading_properties_main_pom_wins…

Add the `test_scanpipe_maven_walk_parent_poms_depth_limit()` #2284

Addressed PR feedback #2284

Removed the unused test file #2284