[SP-2874] feat: add licenses sub-command, add support for ingesting CDX, add CDX input validation #131
Conversation
WalkthroughThis update introduces a new Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant Components
participant CycloneDx
participant ScanossGrpc
participant LicenseService
User->>CLI: scanoss component licenses --purl/--input
CLI->>Components: comp_licenses(args)
Components->>CycloneDx: is_cyclonedx_json(input)
alt CycloneDX input
CycloneDx-->>Components: get_purls_request_from_cdx(input)
end
Components->>ScanossGrpc: get_licenses(purls)
ScanossGrpc->>LicenseService: GetLicenses(ComponentBatchRequest)
LicenseService-->>ScanossGrpc: BasicResponse
ScanossGrpc-->>Components: response
Components-->>CLI: output license details
CLI-->>User: display/write output
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (39)
✅ Files skipped from review due to trivial changes (2)
🚧 Files skipped from review as they are similar to previous changes (37)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (12)
CHANGELOG.md(2 hunks)requirements.txt(1 hunks)setup.cfg(1 hunks)src/scanoss/__init__.py(1 hunks)src/scanoss/api/common/v2/scanoss_common_pb2.py(1 hunks)src/scanoss/api/common/v2/scanoss_common_pb2_grpc.py(1 hunks)src/scanoss/api/licenses/v2/scanoss_licenses_pb2.py(1 hunks)src/scanoss/api/licenses/v2/scanoss_licenses_pb2_grpc.py(1 hunks)src/scanoss/cli.py(8 hunks)src/scanoss/components.py(4 hunks)src/scanoss/cyclonedx.py(2 hunks)src/scanoss/scanossgrpc.py(4 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/scanoss/scanossgrpc.py (2)
src/scanoss/api/licenses/v2/scanoss_licenses_pb2_grpc.py (3)
LicenseStub(29-59)GetLicenses(74-79)GetLicenses(159-183)src/scanoss/components.py (1)
get_licenses(373-403)
src/scanoss/components.py (1)
src/scanoss/scanossgrpc.py (1)
get_licenses(668-683)
🪛 GitHub Actions: Build/Test Local Python Package
src/scanoss/api/licenses/v2/scanoss_licenses_pb2.py
[error] 27-27: ModuleNotFoundError: No module named 'protoc_gen_openapiv2'. The scanoss-py command failed with exit code 1.
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build
🔇 Additional comments (27)
setup.cfg (1)
42-42: LGTM: CycloneDX dependency addition is appropriate.The addition of
cyclonedx-python-lib[validation]supports the new CycloneDX JSON validation and processing features. The[validation]extra ensures JSON schema validation capabilities are available.src/scanoss/__init__.py (1)
25-25: LGTM: Version bump aligns with new feature additions.The version increment from 1.27.1 to 1.28.0 is appropriate for a minor release introducing new license sub-command and CycloneDX support features.
requirements.txt (1)
16-16: LGTM: Dependency addition is consistent with setup.cfg.The
cyclonedx-python-lib[validation]dependency addition matches the setup.cfg configuration and supports the new CycloneDX functionality.CHANGELOG.md (2)
12-17: LGTM: Changelog entry is well-formatted and comprehensive.The version 1.28.0 changelog entry clearly documents the three main additions: licenses sub-command, CycloneDX ingestion support, and CDX input validation. The format follows the existing changelog structure.
592-592: LGTM: Version comparison link is consistent.The version comparison link for 1.28.0 follows the established pattern and maintains consistency with existing entries.
src/scanoss/scanossgrpc.py (5)
39-39: LGTM: License stub import added correctly.The import follows the same pattern as other gRPC stub imports in the file.
45-45: LGTM: ComponentBatchRequest import added correctly.This import is needed for the new license functionality and follows the existing import pattern.
175-175: LGTM: License stub initialization for insecure channels.The initialization follows the same pattern as other stubs in the class.
188-188: LGTM: License stub initialization for secure channels.The initialization follows the same pattern as other stubs in the class.
668-683: LGTM: Well-implemented get_licenses method.The method follows the established patterns in the class:
- Uses the
_call_rpchelper for consistent error handling- Proper type hints with
Optional[Dict]return type- Appropriate debug message template
- Correct usage of
ComponentBatchRequesttypeThe implementation is consistent with other gRPC methods in the class.
src/scanoss/cyclonedx.py (3)
31-32: LGTM: Required imports for CycloneDX validation.The imports are necessary for the new validation functionality and follow the existing import pattern.
311-329: LGTM: Well-implemented CycloneDX validation method.The method provides proper validation functionality:
- Uses CycloneDX schema v1.6 for validation
- Includes comprehensive error handling with try-catch
- Consistent error messaging using
print_stderr- Returns boolean indicating validation success
- Follows the existing class patterns
The implementation is robust and handles validation errors appropriately.
331-349: LGTM: Effective PURL extraction from CycloneDX.The method correctly extracts PURL requests from CycloneDX components:
- Properly handles optional version requirements
- Creates appropriate request structure with 'purls' key
- Handles both cases (with and without version) correctly
- Returns the expected dictionary format
The implementation aligns with the expected input format for the license API.
src/scanoss/cli.py (3)
309-316: LGTM: Licenses sub-command properly definedThe new
licensessub-command follows the established pattern for other component sub-commands. The aliaslicsand handler function assignment are correctly implemented.
419-427: LGTM: Licenses sub-command properly included in common optionsThe
c_licensesparser is correctly added to the list of parsers that receive common purl and input options, maintaining consistency with other component sub-commands.
1888-1920: LGTM: Licenses handler function correctly implementedThe
comp_licensesfunction follows the established pattern for other component handlers:
- Proper input validation (purl XOR input)
- Certificate file existence check
- PAC file processing
- Components instance creation with all necessary parameters
- Error handling with appropriate exit codes
The implementation is consistent with similar handlers like
comp_vulnsandcomp_semgrep.src/scanoss/api/common/v2/scanoss_common_pb2.py (2)
12-19: LGTM: Runtime version validation addedThe protobuf runtime version validation ensures compatibility with protobuf version 5.29.0, which is a good practice for generated code.
49-52: LGTM: New message types properly definedThe serialization offsets for the new
ComponentBatchRequestandComponentRequestmessage types are correctly generated and will be used by the licenses API.src/scanoss/components.py (4)
32-33: LGTM: Proper imports for CycloneDX supportThe imports for
CycloneDxandvalidate_json_fileare correctly added to support CycloneDX input validation and processing.
89-89: LGTM: CycloneDx instance properly initializedThe
CycloneDxinstance is correctly initialized with the debug flag from the parent class, enabling CycloneDX processing capabilities.
100-108: LGTM: Enhanced load_purls method with CycloneDX supportThe updated
load_purlsmethod properly:
- Uses
validate_json_filefor robust JSON validation- Detects CycloneDX format using
self.cdx.is_cyclonedx_json- Converts CycloneDX data to PURL request format via
self.cdx.get_purls_request_from_cdx- Falls back to direct JSON data usage for non-CycloneDX inputs
This implementation maintains backward compatibility while adding CycloneDX support.
373-403: LGTM: get_licenses method correctly implementedThe new
get_licensesmethod follows the established pattern of otherget_*methods in the class:
- Proper input validation and loading
- File/stdout handling with error checking
- Correct construction of
ComponentBatchRequestdictionary- Appropriate gRPC API call to
self.grpc_api.get_licenses- JSON output formatting and success handling
- Consistent error handling and file cleanup
The method integrates well with the existing codebase architecture.
src/scanoss/api/licenses/v2/scanoss_licenses_pb2_grpc.py (5)
1-27: Well-implemented version compatibility checking.The version checking logic is robust and follows gRPC best practices. It gracefully handles import errors and provides clear upgrade/downgrade instructions for version mismatches.
29-60: Correctly implemented client-side gRPC stub.The LicenseStub class properly implements all four RPC methods with correct endpoint paths, serialization, and deserialization. The implementation follows standard gRPC client patterns.
62-94: Appropriate placeholder implementations for server-side servicer.The LicenseServicer class correctly implements placeholder methods that raise NotImplementedError with UNIMPLEMENTED status. This is the expected pattern for generated gRPC servicers, allowing developers to provide actual implementations by subclassing or overriding these methods.
96-123: Correctly implemented server registration function.The add_LicenseServicer_to_server function properly registers all RPC method handlers with appropriate serialization/deserialization configuration. The implementation follows standard gRPC server setup patterns.
125-238: Well-implemented experimental gRPC API.The License experimental class provides static methods for direct RPC calls with full parameter support. The implementation correctly uses grpc.experimental.unary_unary and follows the expected patterns for experimental gRPC APIs.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/scanoss/cryptography.py(3 hunks)
🧰 Additional context used
🪛 GitHub Actions: Lint
src/scanoss/cryptography.py
[error] 30-30: Ruff: Too many branches (14 > 12) in function 'post_init' (PLR0912)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: build
- GitHub Check: build
🔇 Additional comments (2)
src/scanoss/cryptography.py (2)
5-5: LGTM: Clean import addition.The import of
CycloneDxis properly placed and follows the existing import structure.
193-193: LGTM: Method signature formatting improvement.The single-line method signature is cleaner and more consistent with Python conventions.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
src/scanoss/api/common/v2/scanoss_common_pb2_grpc.py (1)
7-24: Version mismatch issue persists and has worsened.The generated code now requires grpcio>=1.73.1, but based on previous review comments, the project dependencies only specify grpcio>=1.70.0. This creates a version mismatch where users with grpcio versions 1.70.0-1.73.0 will encounter a RuntimeError.
This is a continuation of the issue flagged in previous reviews, but it has worsened as the required version has increased from 1.70.0 to 1.73.1.
Please ensure the project dependencies are updated to align with the generated code requirement:
# In setup.cfg and requirements.txt -grpcio>=1.70.0 +grpcio>=1.73.1Or alternatively, regenerate the gRPC stubs with a lower version threshold that matches the current project dependencies.
🧹 Nitpick comments (1)
src/scanoss/api/common/v2/scanoss_common_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsimport is not used in the code and should be removed.-import warnings
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
src/scanoss/api/common/v2/scanoss_common_pb2.py(1 hunks)src/scanoss/api/common/v2/scanoss_common_pb2_grpc.py(1 hunks)src/scanoss/api/licenses/v2/scanoss_licenses_pb2.py(1 hunks)src/scanoss/api/licenses/v2/scanoss_licenses_pb2_grpc.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- src/scanoss/api/licenses/v2/scanoss_licenses_pb2.py
- src/scanoss/api/common/v2/scanoss_common_pb2.py
- src/scanoss/api/licenses/v2/scanoss_licenses_pb2_grpc.py
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (6)
src/scanoss/api/vulnerabilities/v2/scanoss_vulnerabilities_pb2_grpc.py (3)
1-1: Manual modifications to generated code may be lost.This file is marked as generated code with "DO NOT EDIT!" but contains changes. Ensure these modifications are made in the source proto files or generation scripts to prevent loss during regeneration.
4-4: Remove unused import.The
warningsmodule is imported but never used in the code.
9-27: Version check may be overly restrictive.The runtime version check enforces exactly
grpcio>=1.73.1, which could cause issues for users with newer compatible versions.src/scanoss/api/dependencies/v2/scanoss_dependencies_pb2_grpc.py (3)
1-1: Manual modifications to generated code may be lost.This file is marked as generated code with "DO NOT EDIT!" but contains changes. Ensure these modifications are made in the source proto files or generation scripts to prevent loss during regeneration.
4-4: Remove unused import.The
warningsmodule is imported but never used in the code.
9-27: Version check may be overly restrictive.The runtime version check enforces exactly
grpcio>=1.73.1, which could cause issues for users with newer compatible versions.
🧹 Nitpick comments (6)
src/protoc_gen_swagger/options/openapiv2_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but never used in the code.-import warningssrc/protoc_gen_swagger/options/annotations_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but not used in this generated file.-import warningssrc/scanoss/api/cryptography/v2/scanoss_cryptography_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but not used in this generated file.-import warningssrc/scanoss/api/semgrep/v2/scanoss_semgrep_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but not used in this generated file.-import warningssrc/scanoss/api/scanning/v2/scanoss_scanning_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but not used in this generated file.-import warningssrc/scanoss/api/components/v2/scanoss_components_pb2_grpc.py (1)
4-4: Remove unused import.The
warningsmodule is imported but not used in this generated file.-import warnings
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (20)
src/protoc_gen_swagger/options/annotations_pb2.py(2 hunks)src/protoc_gen_swagger/options/annotations_pb2.pyi(1 hunks)src/protoc_gen_swagger/options/annotations_pb2_grpc.py(1 hunks)src/protoc_gen_swagger/options/openapiv2_pb2.py(2 hunks)src/protoc_gen_swagger/options/openapiv2_pb2.pyi(1 hunks)src/protoc_gen_swagger/options/openapiv2_pb2_grpc.py(1 hunks)src/scanoss/api/components/v2/scanoss_components_pb2.py(1 hunks)src/scanoss/api/components/v2/scanoss_components_pb2_grpc.py(7 hunks)src/scanoss/api/cryptography/v2/scanoss_cryptography_pb2.py(1 hunks)src/scanoss/api/cryptography/v2/scanoss_cryptography_pb2_grpc.py(9 hunks)src/scanoss/api/dependencies/v2/scanoss_dependencies_pb2.py(1 hunks)src/scanoss/api/dependencies/v2/scanoss_dependencies_pb2_grpc.py(6 hunks)src/scanoss/api/geoprovenance/v2/scanoss_geoprovenance_pb2.py(1 hunks)src/scanoss/api/geoprovenance/v2/scanoss_geoprovenance_pb2_grpc.py(6 hunks)src/scanoss/api/scanning/v2/scanoss_scanning_pb2.py(2 hunks)src/scanoss/api/scanning/v2/scanoss_scanning_pb2_grpc.py(5 hunks)src/scanoss/api/semgrep/v2/scanoss_semgrep_pb2.py(1 hunks)src/scanoss/api/semgrep/v2/scanoss_semgrep_pb2_grpc.py(5 hunks)src/scanoss/api/vulnerabilities/v2/scanoss_vulnerabilities_pb2.py(1 hunks)src/scanoss/api/vulnerabilities/v2/scanoss_vulnerabilities_pb2_grpc.py(6 hunks)
✅ Files skipped from review due to trivial changes (11)
- src/protoc_gen_swagger/options/annotations_pb2.py
- src/scanoss/api/components/v2/scanoss_components_pb2.py
- src/scanoss/api/vulnerabilities/v2/scanoss_vulnerabilities_pb2.py
- src/scanoss/api/cryptography/v2/scanoss_cryptography_pb2.py
- src/protoc_gen_swagger/options/openapiv2_pb2.py
- src/scanoss/api/semgrep/v2/scanoss_semgrep_pb2.py
- src/scanoss/api/scanning/v2/scanoss_scanning_pb2.py
- src/protoc_gen_swagger/options/annotations_pb2.pyi
- src/scanoss/api/dependencies/v2/scanoss_dependencies_pb2.py
- src/scanoss/api/geoprovenance/v2/scanoss_geoprovenance_pb2.py
- src/protoc_gen_swagger/options/openapiv2_pb2.pyi
🔇 Additional comments (11)
src/protoc_gen_swagger/options/openapiv2_pb2_grpc.py (2)
7-10: LGTM: Version constants are properly defined.The version constants and initialization are correctly implemented for version compatibility checking.
11-24: LGTM: Robust version compatibility checking.The version checking logic is well-implemented with proper error handling:
- Uses try-except to handle cases where
first_version_is_lowerutility might not be available- Provides a comprehensive error message with actionable guidance
- Gracefully falls back to marking version as unsupported if the utility is unavailable
This ensures that users are informed about version incompatibilities and know how to resolve them.
src/protoc_gen_swagger/options/annotations_pb2_grpc.py (1)
7-24: LGTM: Version compatibility check is well implemented.The version compatibility check correctly handles both the case where
first_version_is_loweris available and when it's not, providing clear error messages to guide users on how to resolve version conflicts.src/scanoss/api/cryptography/v2/scanoss_cryptography_pb2_grpc.py (2)
44-69: LGTM: Explicit method registration is properly implemented.The addition of
_registered_method=Trueflags to all unary-unary RPC method calls in the stub class correctly implements explicit method registration as per the updated gRPC Python API.
156-156: LGTM: Server-side method registration is correctly added.The call to
server.add_registered_method_handlerscomplements the existing generic handler registration, ensuring methods are properly registered on the server side.src/scanoss/api/semgrep/v2/scanoss_semgrep_pb2_grpc.py (1)
44-49: LGTM: Method registration flags are properly added.The
_registered_method=Trueflags are correctly added to all RPC method calls in the stub class.src/scanoss/api/scanning/v2/scanoss_scanning_pb2_grpc.py (1)
44-49: LGTM: Method registration is correctly implemented.The explicit method registration flags are properly added to all RPC methods in the stub class.
src/scanoss/api/components/v2/scanoss_components_pb2_grpc.py (2)
44-59: LGTM: Method registration flags are consistently applied.All RPC method calls in the stub class correctly include the
_registered_method=Trueflag for explicit method registration.
122-122: LGTM: Server method registration is properly implemented.The server-side method registration call is correctly added to ensure methods are properly registered.
src/scanoss/api/geoprovenance/v2/scanoss_geoprovenance_pb2_grpc.py (2)
1-1: Manual modifications to generated code may be lost.This file is marked as generated code with "DO NOT EDIT!" but contains changes. Ensure these modifications are made in the source proto files or generation scripts to prevent loss during regeneration.
9-27: Keep the strict gRPC version check as-isThe gRPC Python team intentionally added a runtime enforcement (since v1.65.0) to ensure that generated stubs run against the same—or a newer—
grpciorelease. This prevents subtle incompatibilities (e.g. Protobuf runtime updates in v1.73.0+) and avoids runtime errors caused by mismatched code and library versions. Feel free to retain the currentgrpcio>=1.73.1check and accompanying error message.Likely an incorrect or invalid review comment.
| @@ -1,10 +1,30 @@ | |||
| # Generated by the gRPC Python protocol compiler plugin. DO NOT EDIT! | |||
| """Client and server classes corresponding to protobuf-defined services.""" | |||
| import grpc | |||
| import warnings | |||
There was a problem hiding this comment.
🛠️ Refactor suggestion
Remove unused import.
The warnings module is imported but never used in the code.
import grpc
-import warnings📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import warnings | |
| import grpc |
🤖 Prompt for AI Agents
In src/scanoss/api/geoprovenance/v2/scanoss_geoprovenance_pb2_grpc.py at line 4,
the warnings module is imported but not used anywhere in the file. Remove the
import statement for warnings to clean up unused imports.
…DX, add CDX input validation
matiasdaloia
force-pushed
the
feature/mdaloia/SP-2874-Add-support-for-ingesting-CDX-to-all-scanoss-py-decoration-comamnds
branch
from
b167952 to
4c37f13
Compare
Summary by CodeRabbit
New Features
Dependency Updates
Improvements
Documentation
Chores