diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 5ab58f3b..49b27c02 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -8,9 +8,9 @@ Mixlib::Install is a library for interacting with Chef Software Inc's software d ## Ruby Version Support Strategy ### Supported Ruby Versions -- **Minimum**: Ruby 2.3+ -- **Target Range**: Ruby 2.3 through Ruby 3.4+ -- **Testing Focus**: Maintain backward compatibility with Ruby 2.3+ while supporting latest Ruby releases +- **Minimum**: Ruby 2.6+ +- **Target Range**: Ruby 2.6 through Ruby 3.4+ +- **Testing Focus**: Maintain backward compatibility with Ruby 2.6+ while supporting latest Ruby releases ### Critical Compatibility Rules @@ -18,30 +18,33 @@ Mixlib::Install is a library for interacting with Chef Software Inc's software d - NO numbered parameters `_1, _2` (Ruby 2.7+) - NO pattern matching (Ruby 2.7+) - NO endless methods (Ruby 3.0+) - - Use Ruby 2.3-compatible syntax as the baseline + - Use Ruby 2.6-compatible syntax as the baseline -2. **Dependency Version Constraints** +1. **Dependency Version Constraints** - Always use version-conditional dependency constraints in gemspec - Follow the existing pattern for Ruby version-specific dependencies (see `openssl` gem constraints in gemspec) - Consider backward compatibility when adding new dependencies - Check Gemfile for Ruby version-specific gem constraints before adding dependencies -3. **Standard Library Compatibility** +1. **Standard Library Compatibility** - Be cautious with stdlib changes across Ruby versions - - Test with methods available in Ruby 2.3 - - Avoid relying on gems that dropped support for Ruby 2.3+ - - Ruby 2.3 features that are safe to use: + - Test with methods available in Ruby 2.6 + - Avoid relying on gems that dropped support for Ruby 2.6+ + - Ruby 2.6 features that are safe to use: - Safe navigation operator (`&.`) - Squiggly heredoc (`<<~`) - `dig` method on Hash and Array - `grep_v` on Enumerable - Frozen string literal comment + - Endless ranges: `(1..)` + - `Enumerable#chain` + - `Kernel#then` ## Code Style & Conventions ### RuboCop Configuration -- TargetRubyVersion: 2.7 (set in `.rubocop.yml`) -- Note: While RuboCop targets 2.7, code must remain compatible with Ruby 2.3+ +- TargetRubyVersion: 2.6 (set in `.rubocop.yml`) +- Note: RuboCop targets 2.6 to match the minimum supported Ruby version - Uses `chefstyle` gem version ~> 0.4.0 - Run style checks: `bundle exec rake style` @@ -73,26 +76,33 @@ All Ruby files should include the Apache 2.0 license header: - Provides `artifact_info`, `available_versions`, `install_command`, `download_artifact` methods - Delegates to Backend for API interactions -2. **Options** (`lib/mixlib/install/options.rb`) +1. **Options** (`lib/mixlib/install/options.rb`) - Validates and normalizes user input - Supports EXTRA_PRODUCTS_FILE environment variable for custom products - - Key options: channel, product_name, product_version, platform, platform_version, architecture + - Key options: channel, product_name, product_version, platform, platform_version, architecture, license_id + - **license_id**: Enables commercial/trial API access for licensed Chef products -3. **Product Matrix** (`lib/mixlib/install/product_matrix.rb`) +1. **Product Matrix** (`lib/mixlib/install/product_matrix.rb`) - DSL for defining product metadata - Extensible via EXTRA_PRODUCTS_FILE - Run `bundle exec rake matrix` to update PRODUCT_MATRIX.md after changes -4. **Backend** (`lib/mixlib/install/backend/`) +1. **Backend** (`lib/mixlib/install/backend/`) - Package Router backend for Chef's package API - Handles API communication with packages.chef.io -5. **Generators** (`lib/mixlib/install/generator/`) - - Bourne shell (install.sh) generator - - PowerShell (install.ps1) generator - - Supports proxy configuration and download_url_override - -6. **Artifact Info** (`lib/mixlib/install/artifact_info.rb`) +1. **Generators** (`lib/mixlib/install/generator/`) + - Bourne shell (install.sh) generator with Content-Disposition header support + - PowerShell (install.ps1) generator with JSON API response parsing + - Supports proxy configuration, download_url_override, and license_id + - **Commercial/Trial API Support**: When license_id is provided, uses specialized download endpoints + - Trial API: `https://chefdownload-trial.chef.io` (for license IDs starting with `free-` or `trial-`) + - Commercial API: `https://chefdownload-commercial.chef.io` (for other license IDs) + - Returns JSON responses instead of text format + - Uses Content-Disposition headers for filename extraction + - Implements temp file download approach with multiple filename extraction methods + +1. **Artifact Info** (`lib/mixlib/install/artifact_info.rb`) - Represents package metadata - Includes platform, version, URL, checksum, license info @@ -140,12 +150,12 @@ When adding test dependencies, follow this pattern. - Run `bundle exec rake matrix` to update documentation - Add tests in `spec/unit/mixlib/install/product_spec.rb` -2. **Platform Support** +1. **Platform Support** - Update `lib/mixlib/install/options.rb` SUPPORTED_ARCHITECTURES if needed - Add platform detection logic in `lib/mixlib/install/util.rb` - Update install script generators if platform-specific logic needed -3. **API Changes** +1. **API Changes** - Maintain backward compatibility - Add deprecation warnings before removing features - Update README.md with examples @@ -162,8 +172,8 @@ When adding test dependencies, follow this pattern. #### Adding Dependencies to Gemspec 1. Consider minimum Ruby version compatibility -2. Use version constraints with Ruby version conditionals if needed -3. Example pattern (from gemspec): +1. Use version constraints with Ruby version conditionals if needed +1. Example pattern (from gemspec): ```ruby if RUBY_VERSION < "2.7.0" spec.add_dependency "openssl", ">= 3.1.2", "< 3.2.0" @@ -210,17 +220,31 @@ The library includes sophisticated platform version compatibility logic: - Supports: http_proxy, https_proxy, ftp_proxy, no_proxy - Platform detection for Linux/Unix systems - Generated via `lib/mixlib/install/generator/bourne.rb` +- **Content-Disposition Support**: When `license_id` is provided: + - Downloads to temp file: `chef-download-temp.$$` + - Extracts filename from HTTP response headers (3 methods): + 1. Content-Disposition header: `attachment; filename="..."` + 1. Location redirect header: Extract from redirect URL + 1. URL pattern matching: Search for `.rpm|.deb|.pkg|.msi|.dmg` patterns + - Fallback: Constructs filename from platform metadata if extraction fails + - Renames temp file to extracted/constructed filename + - Works with all download methods: wget, curl, fetch, perl, python ### PowerShell (install.ps1) - Supports: http_proxy - Windows platform support - TLS negotiation for older .NET versions - Generated via `lib/mixlib/install/generator/powershell.rb` +- **JSON API Response**: When `license_id` is provided: + - Parses JSON responses with `ConvertFrom-Json` + - Extracts `url` and `sha256` from JSON object + - Automatically routes to trial or commercial API based on license_id prefix ### Script Options - `download_url_override`: Direct URL instead of API lookup - `checksum`: SHA256 for verification - `install_strategy`: "once" to skip if already installed +- `license_id`: License ID for commercial/trial API access (format: `free-*`, `trial-*`, or standard license ID) ## API Usage Patterns @@ -262,24 +286,85 @@ When implementing features, ensure this extensibility is maintained. - Linux tests: `.expeditor/run_linux_tests.sh` - Windows tests: `.expeditor/run_windows_tests.ps1` +## Commercial and Trial API Integration + +### Overview +Mixlib::Install supports Chef's commercial and trial licensing APIs, which provide authenticated access to Chef products for licensed customers. + +### API Endpoints +- **Trial API**: `https://chefdownload-trial.chef.io` + - Used when `license_id` starts with `free-` or `trial-` + - Returns JSON responses with download URLs +- **Commercial API**: `https://chefdownload-commercial.chef.io` + - Used for standard license IDs + - Returns JSON responses with download URLs +- **Traditional Omnitruck**: `https://omnitruck.chef.io` + - Used when no `license_id` is provided + - Returns text-based metadata responses + +### Response Format Differences +- **Commercial/Trial APIs**: JSON format + ```json + { + "url": "https://...", + "sha256": "abc123..." + } + ``` +- **Omnitruck API**: Text format + ``` + url\thttp://... + sha256\tabc123... + ``` + +### Content-Disposition Header Handling +Commercial and trial APIs return endpoint URLs that use HTTP Content-Disposition headers to specify the actual filename, rather than including the filename in the URL path. + +**Implementation Details**: +1. **Detection**: `use_content_disposition="true"` when `license_id` is present +1. **Download Strategy**: Use temp file with process ID suffix: `chef-download-temp.$$` +1. **Filename Extraction** (3 methods, attempted in order): + - Parse `Content-Disposition` header: `filename="chef-18.8.54-1.el9.x86_64.rpm"` + - Parse `Location` redirect header: Extract filename from redirect URL + - Pattern matching: Search stderr output for `.rpm|.deb|.pkg|.msi|.dmg` extensions +1. **Fallback Construction**: Build filename from platform metadata if extraction fails +1. **File Rename**: Move temp file to final location with extracted/constructed filename + +**Cross-Platform Compatibility**: This approach works with all download methods: +- `wget` (with `--content-disposition` flag as secondary approach) +- `curl` (with `-O -J` flags as secondary approach) +- `fetch` (FreeBSD) +- `perl` (LWP::Simple) +- `python` (urllib2) + +### Testing Commercial/Trial API Features +When adding or modifying commercial/trial API functionality: +1. Test with `license_id` starting with `free-` (trial API) +1. Test with `license_id` starting with `trial-` (trial API) +1. Test with standard license ID format (commercial API) +1. Verify JSON parsing in both Bourne shell (sed) and PowerShell (ConvertFrom-Json) +1. Test filename extraction with various response header formats +1. Verify fallback filename construction for each platform type + ## Common Pitfalls to Avoid -1. **Don't use Ruby 2.4+ features** - Always consider Ruby 2.3 compatibility -2. **Don't assume gem availability** - Check version constraints in Gemfile first -3. **Don't break the Product Matrix DSL** - It's critical for product definitions -4. **Don't skip `rake matrix`** - Must run after modifying product_matrix.rb -5. **Don't hardcode URLs** - Use product definitions and API lookups -6. **Don't ignore platform compatibility** - Test across platforms when possible -7. **Don't add dependencies without version constraints** - Especially for Ruby 2.3+ support +1. **Don't use Ruby 2.7+ features** - Always consider Ruby 2.6 compatibility +1. **Don't assume gem availability** - Check version constraints in Gemfile first +1. **Don't break the Product Matrix DSL** - It's critical for product definitions +1. **Don't skip `rake matrix`** - Must run after modifying product_matrix.rb +1. **Don't hardcode URLs** - Use product definitions and API lookups +1. **Don't ignore platform compatibility** - Test across platforms when possible +1. **Don't add dependencies without version constraints** - Especially for Ruby 2.6+ support +1. **Don't assume filename in URL** - Commercial/trial APIs use Content-Disposition headers +1. **Don't break temp file download approach** - Required for license_id support across all download methods ## Documentation Requirements When making changes: 1. Update README.md with API examples if public interface changes -2. Update CHANGELOG.md (handled by Expeditor) -3. Run `rake matrix` if products changed -4. Add code comments for complex compatibility logic -5. Document Ruby version requirements for new features +1. Update CHANGELOG.md (handled by Expeditor) +1. Run `rake matrix` if products changed +1. Add code comments for complex compatibility logic +1. Document Ruby version requirements for new features ## Performance Considerations @@ -291,18 +376,18 @@ When making changes: ## Security Considerations 1. **Checksum Verification**: Always provide/verify SHA256 checksums -2. **HTTPS**: Use secure connections to packages.chef.io -3. **OpenSSL**: Maintain up-to-date openssl gem constraints (see gemspec) -4. **Proxy Support**: Respect proxy settings in secure environments -5. **License Content**: Handle license_content securely (may contain sensitive info) +1. **HTTPS**: Use secure connections to packages.chef.io +1. **OpenSSL**: Maintain up-to-date openssl gem constraints (see gemspec) +1. **Proxy Support**: Respect proxy settings in secure environments +1. **License Content**: Handle license_content securely (may contain sensitive info) ## Release Process 1. Merge PR to main branch -2. Expeditor automatically bumps version (unless skip label) -3. Expeditor builds gem -4. Manual promotion triggers RubyGems publish -5. GitHub release created with version tag (v{{version}}) +1. Expeditor automatically bumps version (unless skip label) +1. Expeditor builds gem +1. Manual promotion triggers RubyGems publish +1. GitHub release created with version tag (v{{version}}) ## Getting Help @@ -333,11 +418,11 @@ When making changes: --- -**Remember**: When in doubt about Ruby version compatibility, check the Gemfile and gemspec for version-specific patterns, and test with Ruby 2.3+ when possible. The goal is maximum compatibility (Ruby 2.3+) without sacrificing functionality. +**Remember**: When in doubt about Ruby version compatibility, check the Gemfile and gemspec for version-specific patterns, and test with Ruby 2.6+ when possible. The goal is maximum compatibility (Ruby 2.6+) without sacrificing functionality. -### Ruby 2.3+ Feature Reference +### Ruby 2.6+ Feature Reference -#### Safe to Use (Ruby 2.3+) +#### Safe to Use (Ruby 2.6+) - Safe navigation operator: `object&.method` - Squiggly heredoc: `<<~TEXT` - `Hash#dig`, `Array#dig` @@ -345,25 +430,19 @@ When making changes: - `Hash#fetch_values` - `Hash#to_proc` - Frozen string literal pragma: `# frozen_string_literal: true` - -#### Avoid (Ruby 2.4+) +- Endless ranges: `(1..)` +- `Enumerable#chain` +- `Kernel#then` - `Integer#digits` - `Comparable#clamp` - `String#match?`, `Regexp#match?` - Multiple assignment in conditionals - -#### Avoid (Ruby 2.5+) - `yield_self` / `then` -- `Kernel#yield_self` - `rescue` in blocks without `begin` -#### Avoid (Ruby 2.6+) -- Endless ranges: `(1..)` -- `Enumerable#chain` -- `Kernel#then` - #### Avoid (Ruby 2.7+) - Numbered parameters: `_1`, `_2` - Pattern matching - `Enumerable#filter_map` - `Enumerable#tally` +- Method reference operator: `.:` diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml index 27ff0c78..fb6c7190 100644 --- a/.github/workflows/integration.yml +++ b/.github/workflows/integration.yml @@ -43,5 +43,7 @@ jobs: with: ruby-version: ${{ matrix.ruby }} bundler-cache: true - - name: Chef Install Test + - name: Chef Omnibus download Test run: bundle exec mixlib-install download chef + - name: Chef Licensed download Test + run: bundle exec mixlib-install download chef -L free-79df705d-b685-419a-8b68-88401f74ff72-3999 diff --git a/.rubocop.yml b/.rubocop.yml index 11489396..b0242b53 100644 --- a/.rubocop.yml +++ b/.rubocop.yml @@ -1,16 +1,16 @@ --- AllCops: - TargetRubyVersion: 2.3 + TargetRubyVersion: 2.6 # The following cops are disabled because their autocorrect features could -# potentially introduce syntax that breaks Ruby 2.3 compatibility, even though -# the cops themselves support Ruby 2.3 +# potentially introduce syntax that breaks Ruby 2.6 compatibility, even though +# the cops themselves support Ruby 2.6 -# Keep encoding comments for Ruby 2.3 compatibility (they're harmless) +# Keep encoding comments for Ruby 2.6 compatibility (they're harmless) Style/Encoding: Enabled: false -# Disable unless we're certain all suggestions work in Ruby 2.3 +# Disable unless we're certain all suggestions work in Ruby 2.6 Style/MutableConstant: Enabled: false @@ -18,10 +18,6 @@ Style/MutableConstant: Security/Eval: Enabled: false -# Chef-specific cop that suggests require optimizations -Chef/Ruby/UnlessDefinedRequire: - Enabled: false - # Added for Chefstyle 0.12 compatibility, can be removed with switch to cookstyle Bundler/DuplicatedGem: Enabled: false diff --git a/ADR/README.md b/ADR/README.md new file mode 100644 index 00000000..a3c9fee2 --- /dev/null +++ b/ADR/README.md @@ -0,0 +1,91 @@ +# Architectural Decision Records (ADRs) + +## What is an ADR? + +An Architectural Decision Record (ADR) is a document that captures an important architectural decision made along with its context and consequences. ADRs are a way to document how and why a decision was reached within a project. + +## Why use ADRs? + +- **Historical record**: ADRs provide a historical record of decisions, helping new team members understand past choices +- **Knowledge sharing**: They capture the context, options, and reasoning that isn't visible in code alone +- **Avoiding repeated discussions**: They document decisions that have already been made +- **Making the decision process explicit**: They clarify the decision-making process by documenting alternatives considered + +## When to write an ADR + +Write an ADR when you make a significant decision that impacts: + +- System architecture +- API design and changes +- Platform support decisions +- Dependency choices +- Security implementations +- Performance optimizations + +## ADR Format + +Each ADR should follow this structure: + +1. **Title**: A short, descriptive title (e.g., "ADR 0002: Migration to Redis Cluster") +2. **Date**: When the decision was made +3. **Context**: Background information explaining the need for a decision +4. **Decision**: The chosen solution with reasoning +5. **Implementation**: Code examples or diagrams explaining how the decision is implemented +6. **Alternatives Considered**: What other options were evaluated and why they weren't selected +7. **Benefits**: The advantages of the chosen approach +8. **Future Considerations**: Potential impacts or future work + +## How to Create a New ADR + +1. Copy the template below +2. Create a new file named `adrXXXX.md` where XXXX is the next sequential number +3. Fill in the sections +4. Submit a PR for review + +## Template + +```markdown +# Architectural Decision Record (ADR) + +## [Date]: [Title] + +### Context + +[Describe the problem and context for this decision] + +### Decision + +[What is the change being proposed? How will it solve the problem?] + +### Implementation + +```[code language] +[Add code examples or diagrams] +``` + +### Sequence Diagram (if applicable) + +```mermaid +[Add a sequence diagram] +``` + +### Benefits + +[List benefits of this approach] + +### Alternative Solutions Considered + +[Describe alternatives and why they weren't chosen] + +### Future Considerations + +[Note any future implications or potential follow-up work] + +## Existing ADRs + +- [ADR 0001](adr0001.md): Commercial and Trial API Support with License-Based Authentication + +## References + +- [ADR GitHub Organization](https://adr.github.io/) +- [Michael Nygard's article on ADRs](https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions) diff --git a/ADR/adr0001.md b/ADR/adr0001.md new file mode 100644 index 00000000..b5bae631 --- /dev/null +++ b/ADR/adr0001.md @@ -0,0 +1,508 @@ +# Architectural Decision Record (ADR) + +## November 2025: Commercial and Trial API Support with License-Based Authentication + +### Context + +Mixlib::Install is a library for interacting with Progress Software: Chef's software distribution systems, providing APIs and command-line tools to download Chef products and generate installation scripts for various platforms. Previously, all package downloads were facilitated through the Omnitruck API (`https://omnitruck.chef.io`), which provided unauthenticated access to Chef products. + +The need arose to implement a more robust distribution system that could: + +1. **Support licensed/enterprise customers** with authenticated downloads +2. **Enable trial licenses** for evaluation purposes with controlled access +3. **Implement usage tracking and analytics** to understand product adoption +4. **Provide better security and access control** for commercial distributions +5. **Prepare for future deprecation** of unauthenticated Omnitruck access + +The challenge was to implement this authentication mechanism while: + +- Maintaining backward compatibility with existing Omnitruck-based downloads +- Supporting all existing platforms and download methods (wget, curl, PowerShell, etc.) +- Handling API response format differences between commercial and legacy systems +- Working with Content-Disposition headers for dynamic filename resolution + +### Decision + +We implemented a dual-API system with license-based authentication through a `license_id` parameter. The solution introduces two new API endpoints alongside the existing Omnitruck API: + +1. **Trial API** (`https://chefdownload-trial.chef.io`): For license IDs starting with `free-` or `trial-` +2. **Commercial API** (`https://chefdownload-commercial.chef.io`): For standard license IDs +3. **Omnitruck API** (`https://omnitruck.chef.io`): Continues to work when no license_id is provided (backward compatible) + +The implementation adds a new `license_id` option throughout the mixlib-install stack: + +- **Ruby API**: `Mixlib::Install.new(license_id: "...")` +- **CLI**: `mixlib-install download chef -L ` +- **Generated Scripts**: Both Bourne shell and PowerShell scripts accept and use license_id + +**API Routing Logic**: + +```ruby +if license_id.start_with?("free-", "trial-") + endpoint = "https://chefdownload-trial.chef.io" +elsif license_id.present? + endpoint = "https://chefdownload-commercial.chef.io" +else + endpoint = "https://omnitruck.chef.io" # Traditional behavior +end +``` + +**Response Format Handling**: + +- **Commercial/Trial APIs**: Return JSON responses with download URLs as endpoints + + ```json + { + "url": "https://endpoint/download?params", + "sha256": "abc123...", + "version": "18.8.54" + } + ``` + +- **Omnitruck API**: Returns text format with direct file URLs + + ```none + url\thttps://packages.chef.io/files/.../package.rpm + sha256\tabc123... + ``` + +**Content-Disposition Handling**: +Due to the commercial/trial API metadata not containing full download URLs with filenames but rather API endpoints, we implemented a sophisticated Content-Disposition header extraction system: + +1. Download to temporary file: `chef-download-temp.$$` +2. Extract filename using three methods (in order of preference): + - Parse `Content-Disposition` header: `filename="chef-18.8.54-1.el9.x86_64.rpm"` + - Parse `Location` redirect header for filename in redirect URL + - Pattern match for common package extensions (`.rpm`, `.deb`, `.pkg`, `.msi`, `.dmg`) +3. Fallback to constructed filename from platform metadata if extraction fails +4. Rename temporary file to final destination with extracted/constructed filename + +### Implementation + +#### 1. New API Endpoint Constants + +```ruby +# lib/mixlib/install/dist.rb +module Mixlib + class Install + class Dist + PRODUCT_ENDPOINT = "https://packages.chef.io".freeze + OMNITRUCK_ENDPOINT = "https://omnitruck.chef.io".freeze + # New endpoints + COMMERCIAL_API_ENDPOINT = "https://chefdownload-commercial.chef.io".freeze + TRIAL_API_ENDPOINT = "https://chefdownload-trial.chef.io".freeze + end + end +end +``` + +#### 2. License ID Option + +```ruby +# lib/mixlib/install/options.rb +SUPPORTED_OPTIONS = [ + # ... existing options ... + :license_id, +] +``` + +#### 3. Backend API Routing + +```ruby +# lib/mixlib/install/backend/package_router.rb +def endpoint + @endpoint ||= if use_trial_api? + Mixlib::Install::Dist::TRIAL_API_ENDPOINT + elsif use_commercial_api? + Mixlib::Install::Dist::COMMERCIAL_API_ENDPOINT + else + PRODUCT_MATRIX.lookup(options.product_name, options.product_version).api_url + end +end + +def use_trial_api? + !options.license_id.nil? && !options.license_id.to_s.empty? && + options.license_id.start_with?("free-", "trial-") +end + +def use_commercial_api? + !options.license_id.nil? && !options.license_id.to_s.empty? && !use_trial_api? +end + +def use_licensed_api? + use_trial_api? || use_commercial_api? +end +``` + +#### 4. Versions API Adaptation + +```ruby +def versions + if use_licensed_api? + # Commercial/trial APIs return simple JSON array + version_list = get("/#{options.channel}/#{omnibus_project}/versions/all") + items = version_list.map do |version| + { "properties" => [{ "key" => "omnibus.version", "value" => version }] } + end + else + # Traditional Omnitruck response + items = get("/api/v1/#{options.channel}/#{omnibus_project}/versions")["results"] + end + items +end +``` + +#### 5. Artifacts API with License ID in Query Parameters + +```ruby +def artifacts_for_version(version) + if use_licensed_api? + query = "v=#{version}" + packages_hash = get("/#{options.channel}/#{omnibus_project}/packages?#{query}") + # Flatten nested hash: platform -> platform_version -> architecture -> package_info + results = [] + packages_hash.each do |platform, platform_versions| + platform_versions.each do |platform_version, architectures| + architectures.each do |arch, pkg_info| + results << { + "omnibus.version" => pkg_info["version"], + "omnibus.platform" => platform, + "omnibus.platform_version" => platform_version, + "omnibus.architecture" => arch, + "omnibus.sha256" => pkg_info["sha256"], + # ... other metadata + } + end + end + end + else + # Traditional Omnitruck API + results = get("/api/v1/#{options.channel}/#{omnibus_project}/#{version}/artifacts")["results"] + end + results +end +``` + +#### 6. Download URL Construction + +```ruby +def create_artifact(artifact_map) + if use_licensed_api? + # Build query parameter URL with license_id + download_url = "#{endpoint}/#{options.channel}/#{omnibus_project}/download?" + + "p=#{platform}&pv=#{platform_version}&m=#{architecture}&" + + "v=#{version}&license_id=#{options.license_id}" + else + # Traditional direct file URL + download_url = "#{base_url}/#{chef_standard_path}" + end + + ArtifactInfo.new(url: download_url, ...) +end +``` + +#### 7. Bourne Shell Script Changes + +```bash +# lib/mixlib/install/generator/bourne/scripts/fetch_metadata.sh.erb + +# Use commercial API if license_id is provided, otherwise use omnitruck +if test "x$license_id" != "x"; then + # Check if license_id starts with 'free-' or 'trial-' for trial API + case "$license_id" in + free-*|trial-*) + base_api_url="https://chefdownload-trial.chef.io" + ;; + *) + base_api_url="https://chefdownload-commercial.chef.io" + ;; + esac + metadata_url="$base_api_url/$channel/$project/metadata?v=$version&p=$platform&pv=$platform_version&m=$machine&license_id=$license_id" +else + # Omnitruck endpoint + metadata_url="<%= base_url %>/$channel/$project/metadata?v=$version&p=$platform&pv=$platform_version&m=$machine" +fi + +# Parse response based on format +if test "x$license_id" != "x"; then + # Parse JSON response from commercial/trial API using sed + download_url=`sed -n 's/.*"url":"\([^"]*\)".*/\1/p' "$metadata_filename"` + sha256=`sed -n 's/.*"sha256":"\([^"]*\)".*/\1/p' "$metadata_filename"` +else + # Parse text response from omnitruck + download_url=`awk '$1 == "url" { print $2 }' "$metadata_filename"` + sha256=`awk '$1 == "sha256" { print $2 }' "$metadata_filename"` +fi +``` + +#### 8. Content-Disposition Handling in Shell Script + +```bash +# lib/mixlib/install/generator/bourne/scripts/fetch_package.sh + +if test "x$use_content_disposition" = "xtrue"; then + # Download to temp file + temp_download="$download_dir/chef-download-temp.$$" + do_download "$download_url" "$temp_download" + + # Method 1: Extract from Content-Disposition header + actual_filename=`grep -i 'content-disposition' $tmp_dir/stderr | \ + sed -n 's/.*filename="\([^"]*\)".*/\1/p' | head -1` + + # Method 2: Extract from Location redirect header + if test "x$actual_filename" = "x"; then + actual_filename=`grep -i '^location:' $tmp_dir/stderr | head -1 | \ + sed 's/.*\///' | sed 's/?.*//'` + fi + + # Method 3: Pattern matching for package extensions + if test "x$actual_filename" = "x"; then + actual_filename=`grep -i '\.rpm\|\.deb\|\.pkg\|\.msi\|\.dmg' $tmp_dir/stderr | \ + sed -n 's/.*\/\([^/?]*\.\(rpm\|deb\|pkg\|msi\|dmg\)\).*/\1/p' | head -1` + fi + + # Fallback: Construct from metadata + if test "x$actual_filename" = "x"; then + echo "Warning: Could not extract filename from response headers, using fallback" + if test "x$platform" = "xel" || test "x$platform" = "xfedora"; then + actual_filename="chef-${version}-1.${platform}${platform_version}.${machine}.rpm" + elif test "x$platform" = "xdebian" || test "x$platform" = "xubuntu"; then + actual_filename="chef_${version}-1_${machine}.deb" + # ... other platform cases + fi + fi + + # Rename temp file to actual filename + download_filename="$download_dir/$actual_filename" + mv "$temp_download" "$download_filename" +fi +``` + +#### 9. PowerShell Script Changes + +```powershell +# lib/mixlib/install/generator/powershell/scripts/get_project_metadata.ps1.erb + +function Get-ProjectMetadata { + param( + # ... existing parameters ... + [string]$license_id + ) + + # Use commercial API if license_id is provided + if ($license_id) { + if ($license_id -match '^(free-|trial-)') { + $base_server_uri = 'https://chefdownload-trial.chef.io' + } else { + $base_server_uri = 'https://chefdownload-commercial.chef.io' + } + } + + # Add license_id to query parameters if provided + if ($license_id) { + $metadata_array += "license_id=$license_id" + } + + $response = (Get-WebContent $metadata_url).trim() + + # Parse based on API type + if ($license_id) { + # Parse JSON response + $json = $response | ConvertFrom-Json + $package_metadata = @{ + url = $json.url + sha256 = $json.sha256 + version = $json.version + } + } else { + # Parse text response + $package_metadata = $response -split '\n' | + foreach { $hash = @{} } {$key, $value = $_ -split '\s+'; $hash.Add($key, $value)} {$hash} + } +} +``` + +#### 10. CLI Support + +```ruby +# lib/mixlib/install/cli.rb +option :license_id, + desc: "License ID for commercial API downloads", + aliases: ["-L"] + +def download(product_name) + mixlib_install_options = { + # ... existing options ... + }.tap do |opt| + opt[:license_id] = options[:license_id] if options[:license_id] + end +end +``` + +### Sequence Diagram + +```mermaid +sequenceDiagram + participant User + participant Mixlib Install + participant Package Router + participant Commercial API + participant Trial API + participant Omnitruck API + + User->>Mixlib Install: new(license_id: "free-abc-123") + Mixlib Install->>Package Router: detect API endpoint + + alt license_id starts with "free-" or "trial-" + Package Router->>Trial API: GET /stable/chef/metadata?license_id=... + Trial API-->>Package Router: JSON response + else license_id provided (commercial) + Package Router->>Commercial API: GET /stable/chef/metadata?license_id=... + Commercial API-->>Package Router: JSON response + else no license_id + Package Router->>Omnitruck API: GET /stable/chef/metadata + Omnitruck API-->>Package Router: Text response + end + + Package Router->>Package Router: Parse response (JSON vs Text) + Package Router-->>Mixlib Install: ArtifactInfo with download URL + + User->>Mixlib Install: download_artifact() + + alt Commercial/Trial API (endpoint URL) + Mixlib Install->>Commercial API: GET download endpoint with license_id + Commercial API-->>Mixlib Install: Response with Content-Disposition header + Mixlib Install->>Mixlib Install: Extract filename from headers + Mixlib Install->>Mixlib Install: Save to extracted filename + else Omnitruck API (direct URL) + Mixlib Install->>Package Server: GET direct file URL + Package Server-->>Mixlib Install: File download + Mixlib Install->>Mixlib Install: Save to URL basename + end + + Mixlib Install-->>User: Downloaded package file +``` + +### Benefits + +1. **Enterprise Support**: Licensed/enterprise customers can now access authenticated downloads with proper tracking and access control + +2. **Trial Licenses**: Enables time-limited trial licenses for product evaluation with controlled access through the trial API + +3. **Usage Analytics**: License-based downloads enable comprehensive usage tracking and analytics to understand product adoption patterns and customer behavior + +4. **Enhanced Security**: Provides better access control and security for commercial distributions by requiring authentication via license IDs + +5. **Cross-Platform Support**: Works seamlessly across all supported platforms and download methods: + - Bourne shell scripts (wget, curl, fetch, perl, python) + - PowerShell scripts + - Direct Ruby API usage + - CLI tool + +6. **Flexible Authentication**: Supports multiple license types through simple prefix-based routing (free-/trial- vs commercial) + +7. **Content-Disposition Resilience**: Implements multiple fallback methods for filename extraction, ensuring downloads work even with varying server response formats + +8. **Transparent API Selection**: Automatically routes to the appropriate API based on license_id presence and format, requiring no additional configuration + +9. **Future-Ready**: Prepares the codebase for the planned deprecation of unauthenticated Omnitruck access + +### Alternative Solutions Considered + +While specific alternative approaches were evaluated during the design phase, the implementation team considered various options including: + +1. **API Key vs License ID**: Using generic API keys versus license-specific identifiers +2. **Authentication Mechanisms**: Various authentication methods including tokens, certificates, or license IDs +3. **Omnitruck Modification vs New Endpoints**: Extending the existing Omnitruck API versus creating dedicated commercial/trial endpoints +4. **Single Endpoint vs Multiple Endpoints**: Using one unified endpoint with authentication versus separate trial and commercial endpoints + +The selected approach of using license IDs with separate trial and commercial endpoints was chosen based on: + +- Clear separation of concerns between free trials and commercial licenses +- Easier routing and backend infrastructure management +- Better analytics and usage tracking capabilities +- Simplified migration path from existing unauthenticated Omnitruck API + +### Performance Considerations + +**Content-Disposition Approach**: +The decision to use Content-Disposition headers rather than including filenames directly in URLs was driven by the backend API design where metadata does not contain full download URLs with filenames. Instead, the APIs provide endpoint URLs that return the actual file with filename information in the Content-Disposition header. + +This approach: + +- Allows backend flexibility in file storage and naming +- Enables dynamic file serving without hardcoded paths in metadata +- Supports better caching and CDN strategies +- Provides abstraction between API URLs and actual file storage + +The implementation mitigates any performance concerns through: + +- Efficient header parsing with multiple extraction methods +- Minimal additional HTTP roundtrips (header information comes with the download response) +- Fallback filename construction to ensure downloads always succeed + +### Future Considerations + +1. **Omnitruck Deprecation**: Official Chef binary downloads will require authentication via the new commercial/trial APIs in the future. Unauthenticated Omnitruck downloads will be blocked to ensure proper license compliance and usage tracking. + +2. **Migration Timeline**: Organizations using mixlib-install should plan to: + - Obtain appropriate license IDs for their installations + - Update automation and CI/CD pipelines to include license_id parameters + - Test the new API endpoints before Omnitruck deprecation + - Update documentation and internal procedures for licensed downloads + +3. **Enhanced Analytics**: The license-based authentication enables future enhancements for: + - Detailed usage reporting and analytics dashboards + - License expiration warnings and notifications + - Automated license renewal workflows + - Product usage insights for better customer support + +4. **API Evolution**: As the commercial and trial APIs mature, consider: + - Additional metadata in API responses (license status, expiration dates) + - Improved error messages for license issues + - Support for offline license validation + - Integration with Chef Software's customer portal for license management + +5. **Security Enhancements**: Future improvements could include: + - License ID rotation capabilities + - Rate limiting based on license tiers + - Audit logging for download events + - Integration with identity management systems + +6. **Compatibility Matrix**: Maintain clear documentation of: + - Minimum mixlib-install version required for commercial API support (3.13.0+) + - Ruby version requirements (2.6+) + - Platform-specific considerations for script generation + - Migration guides for users upgrading from pre-3.13 versions + +7. **Testing and Validation**: Establish comprehensive testing for: + - All three API endpoints (omnitruck, trial, commercial) + - Content-Disposition header extraction across various HTTP clients + - Fallback filename construction for all supported platforms + - JSON vs text response parsing in both shell and PowerShell scripts + - Error handling for invalid or expired license IDs + +### Impact Assessment + +**Breaking Changes**: None. The implementation is fully backward compatible with existing usage patterns. + +**Required Updates**: Organizations wishing to use commercial/trial API features must: + +1. Update to mixlib-install 3.13.0 or later +2. Obtain license IDs from Progress Software: Chef +3. Update scripts and automation to include license_id parameters (optional but recommended) + +**Documentation Updates**: + +- Updated README.md with license_id usage examples +- Added Copilot instructions documenting commercial/trial API integration +- Updated PRODUCT_MATRIX.md with new chef-ice product +- Enhanced CLI help text with license_id option + +**Testing Coverage**: Comprehensive tests added for: + +- API endpoint detection and routing logic +- JSON vs text response parsing +- Content-Disposition header extraction +- License ID parameter handling in CLI, Ruby API, and generated scripts +- Backward compatibility with omnitruck-based downloads diff --git a/PRODUCT_MATRIX.md b/PRODUCT_MATRIX.md index df6a1f8e..aab04a2f 100644 --- a/PRODUCT_MATRIX.md +++ b/PRODUCT_MATRIX.md @@ -7,6 +7,7 @@ | Chef Infra Client | chef | | Chef Backend | chef-backend | | Chef Foundation | chef-foundation | +| Chef Infra Client Enterprise | chef-ice | | Chef Infra Server | chef-server | | Chef Infra Server HA Provisioning for AWS | chef-server-ha-provisioning | | Chef Infra Client MacOS Universal | chef-universal | diff --git a/lib/mixlib/install.rb b/lib/mixlib/install.rb index 67be8f42..0d153ec0 100644 --- a/lib/mixlib/install.rb +++ b/lib/mixlib/install.rb @@ -98,14 +98,53 @@ def download_artifact(directory = Dir.pwd) artifact = artifact_info FileUtils.mkdir_p directory - file = File.join(directory, File.basename(artifact.url)) + # Handle the full URL including query string and redirects uri = URI.parse(artifact.url) - Net::HTTP.start(uri.host) do |http| - resp = http.get(uri.path) - open(file, "wb") do |io| - io.write(resp.body) + filename = nil + final_body = nil + + Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") do |http| + # Build the request path including query string + request_path = uri.path + request_path += "?#{uri.query}" if uri.query + + # Get the response, following redirects + response = http.request_get(request_path) + + # Follow redirects + redirect_limit = 5 + while response.is_a?(Net::HTTPRedirection) && redirect_limit > 0 + redirect_uri = URI.parse(response["location"]) + # Handle relative redirects + redirect_uri = uri + redirect_uri if redirect_uri.relative? + + Net::HTTP.start(redirect_uri.host, redirect_uri.port, use_ssl: redirect_uri.scheme == "https") do |redirect_http| + redirect_path = redirect_uri.path + redirect_path += "?#{redirect_uri.query}" if redirect_uri.query + response = redirect_http.request_get(redirect_path) + + # Try to get filename from Content-Disposition or final URL + if response["content-disposition"] + filename = response["content-disposition"][/filename="?([^"]+)"?/, 1] + else + filename = File.basename(redirect_uri.path) + end + end + + redirect_limit -= 1 end + + final_body = response.body + end + + # Use the extracted filename or fall back to basename of original URL + filename ||= File.basename(uri.path) + file = File.join(directory, filename) + + # Write the final response body to file + File.open(file, "wb") do |io| + io.write(final_body) end file diff --git a/lib/mixlib/install/backend/package_router.rb b/lib/mixlib/install/backend/package_router.rb index 80636740..4c83638f 100644 --- a/lib/mixlib/install/backend/package_router.rb +++ b/lib/mixlib/install/backend/package_router.rb @@ -64,7 +64,17 @@ def available_versions # # @return [Array] Build records for available versions def versions - items = get("/api/v1/#{options.channel}/#{omnibus_project}/versions")["results"] + # Commercial and trial APIs use a different URL structure + if use_licensed_api? + # Response is a JSON array of version strings + version_list = get("/#{options.channel}/#{omnibus_project}/versions/all") + # Convert to the expected format with properties + items = version_list.map do |version| + { "properties" => [{ "key" => "omnibus.version", "value" => version }] } + end + else + items = get("/api/v1/#{options.channel}/#{omnibus_project}/versions")["results"] + end # Circumvent early when there are no product artifacts in a specific channel if items.empty? @@ -80,7 +90,7 @@ def versions # always complete. In fact we should not do this since for some arcane # builds like Chef Client 10.X we do not have build record created in # artifactory. - if options.channel == :unstable + if options.channel == :unstable && !use_licensed_api? # We check if "artifacts" field contains something since it is only # populated with the build record if "artifact.module.build" exists. items.reject! { |i| i["artifacts"].nil? } @@ -123,24 +133,49 @@ def extract_version_from_response(response) # @return [Array] Array of info about found artifacts def artifacts_for_version(version) begin - results = get("/api/v1/#{options.channel}/#{omnibus_project}/#{version}/artifacts")["results"] + if use_licensed_api? + # Commercial/trial APIs use the packages endpoint which returns metadata for all platforms + query = "v=#{version}" + packages_hash = get("/#{options.channel}/#{omnibus_project}/packages?#{query}") + # Response is a nested hash: platform -> platform_version -> architecture -> package_info + # Flatten it to an array of package metadata objects + results = [] + packages_hash.each do |platform, platform_versions| + platform_versions.each do |platform_version, architectures| + architectures.each do |arch, pkg_info| + results << { + "omnibus.version" => pkg_info["version"], + "omnibus.platform" => platform, + "omnibus.platform_version" => platform_version, + "omnibus.architecture" => arch, + "omnibus.project" => omnibus_project, + "omnibus.license" => "Apache-2.0", + "omnibus.sha256" => pkg_info["sha256"], + "omnibus.sha1" => pkg_info.fetch("sha1", ""), + "omnibus.md5" => pkg_info.fetch("md5", ""), + } + end + end + end + else + results = get("/api/v1/#{options.channel}/#{omnibus_project}/#{version}/artifacts")["results"] + # Merge artifactory properties to a flat Hash + results.collect! do |result| + { + "filename" => result["name"], + }.merge( + map_properties(result["properties"]) + ) + end + end rescue Net::HTTPServerException => e - if e.message =~ /404/ + if e.message.match?(/404/) return [] else raise e end end - # Merge artifactory properties to a flat Hash - results.collect! do |result| - { - "filename" => result["name"], - }.merge( - map_properties(result["properties"]) - ) - end - # Convert results to build records results.map { |a| create_artifact(a) } end @@ -153,6 +188,13 @@ def get(url) http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = (uri.scheme == "https") full_path = File.join(uri.path, url) + + # Add license_id as query parameter if using commercial or trial API + if use_licensed_api? + separator = full_path.include?("?") ? "&" : "?" + full_path = "#{full_path}#{separator}license_id=#{options.license_id}" + end + res = http.request(create_http_request(full_path)) res.value JSON.parse(res.body) @@ -191,7 +233,7 @@ def create_artifact(artifact_map) software_dependencies = metadata.fetch("version_manifest", {}) .fetch("software", nil) rescue Net::HTTPServerException => e - if e.message =~ /404/ + if e.message.match?(/404/) license_content, software_dependencies = nil else raise e @@ -202,11 +244,22 @@ def create_artifact(artifact_map) end # create the download path with the correct endpoint - base_url = if use_compat_download_url_endpoint?(platform, platform_version) - COMPAT_DOWNLOAD_URL_ENDPOINT - else - endpoint - end + if use_licensed_api? + # Commercial/trial APIs use the download endpoint with query parameters + # Construct platform parameters + p_param = platform + pv_param = platform_version + m_param = Util.normalize_architecture(artifact_map["omnibus.architecture"]) + v_param = artifact_map["omnibus.version"] + download_url = "#{endpoint}/#{options.channel}/#{omnibus_project}/download?p=#{p_param}&pv=#{pv_param}&m=#{m_param}&v=#{v_param}&license_id=#{options.license_id}" + else + base_url = if use_compat_download_url_endpoint?(platform, platform_version) + COMPAT_DOWNLOAD_URL_ENDPOINT + else + endpoint + end + download_url = "#{base_url}/#{chef_standard_path}" + end ArtifactInfo.new( architecture: Util.normalize_architecture(artifact_map["omnibus.architecture"]), @@ -220,7 +273,7 @@ def create_artifact(artifact_map) sha1: artifact_map["omnibus.sha1"], sha256: artifact_map["omnibus.sha256"], software_dependencies: software_dependencies, - url: "#{base_url}/#{chef_standard_path}", + url: download_url, version: artifact_map["omnibus.version"] ) end @@ -241,6 +294,29 @@ def use_compat_download_url_endpoint?(platform, platform_version) end end + # Public API detection methods for testing + def endpoint + @endpoint ||= if use_trial_api? + Mixlib::Install::Dist::TRIAL_API_ENDPOINT + elsif use_commercial_api? + Mixlib::Install::Dist::COMMERCIAL_API_ENDPOINT + else + PRODUCT_MATRIX.lookup(options.product_name, options.product_version).api_url + end + end + + def use_trial_api? + !options.license_id.nil? && !options.license_id.to_s.empty? && options.license_id.start_with?("free-", "trial-") + end + + def use_commercial_api? + !options.license_id.nil? && !options.license_id.to_s.empty? && !use_trial_api? + end + + def use_licensed_api? + use_trial_api? || use_commercial_api? + end + private # Converts Array where the Hash is a key pair and @@ -267,10 +343,6 @@ def generate_chef_standard_path(channel, project, version, platform, platform_ve path.join("/") end - def endpoint - @endpoint ||= PRODUCT_MATRIX.lookup(options.product_name, options.product_version).api_url - end - def omnibus_project @omnibus_project ||= PRODUCT_MATRIX.lookup(options.product_name, options.product_version).omnibus_project end diff --git a/lib/mixlib/install/cli.rb b/lib/mixlib/install/cli.rb index 941522de..aa0e15a3 100644 --- a/lib/mixlib/install/cli.rb +++ b/lib/mixlib/install/cli.rb @@ -71,6 +71,9 @@ def list_products option :attributes, desc: "Print artifact attributes", type: :boolean + option :license_id, + desc: "License ID for commercial API downloads", + aliases: ["-L"] def download(product_name) # Set minimum options @@ -83,6 +86,7 @@ def download(product_name) }.tap do |opt| opt[:platform] = options[:platform] if options[:platform] opt[:platform_version] = options[:platform_version] if options[:platform_version] + opt[:license_id] = options[:license_id] if options[:license_id] end # auto detect platform options if not configured diff --git a/lib/mixlib/install/dist.rb b/lib/mixlib/install/dist.rb index 57916a24..8c36fb92 100644 --- a/lib/mixlib/install/dist.rb +++ b/lib/mixlib/install/dist.rb @@ -8,6 +8,10 @@ class Dist PRODUCT_ENDPOINT = "https://packages.chef.io".freeze # Omnitruck endpoint OMNITRUCK_ENDPOINT = "https://omnitruck.chef.io".freeze + # Commercial API endpoint + COMMERCIAL_API_ENDPOINT = "https://chefdownload-commercial.chef.io".freeze + # Trial API endpoint + TRIAL_API_ENDPOINT = "https://chefdownload-trial.chef.io".freeze # Default product name DEFAULT_PRODUCT = "chef".freeze # Default download page URL diff --git a/lib/mixlib/install/generator/bourne.rb b/lib/mixlib/install/generator/bourne.rb index e16b075c..367b6c82 100644 --- a/lib/mixlib/install/generator/bourne.rb +++ b/lib/mixlib/install/generator/bourne.rb @@ -57,16 +57,21 @@ def install_command end def render_variables - < /dev/null && grep '^sha256' $metadata_filename > /dev/null; then - echo "downloaded metadata file looks valid..." + + # Commercial and trial APIs return JSON, omnitruck returns text format + if test "x$license_id" != "x"; then + # Parse JSON response from commercial/trial API + # Check if response looks like JSON + if grep -q '^{' "$metadata_filename" 2>/dev/null; then + # Extract url and sha256 from JSON + # Try using sed for simple JSON parsing (more portable than jq) + download_url=`sed -n 's/.*"url":"\([^"]*\)".*/\1/p' "$metadata_filename"` + sha256=`sed -n 's/.*"sha256":"\([^"]*\)".*/\1/p' "$metadata_filename"` + + if test "x$download_url" != "x" && test "x$sha256" != "x"; then + echo "downloaded metadata file looks valid..." + else + echo "downloaded metadata file is corrupted or an uncaught error was encountered in downloading the file..." + report_bug + exit 1 + fi + else + echo "downloaded metadata file is corrupted or an uncaught error was encountered in downloading the file..." + report_bug + exit 1 + fi else - echo "downloaded metadata file is corrupted or an uncaught error was encountered in downloading the file..." - # this generally means one of the download methods downloaded a 404 or something like that and then reported a successful exit code, - # and this should be fixed in the function that was doing the download. - report_bug - exit 1 + # Parse text response from omnitruck + if grep '^url' $metadata_filename > /dev/null && grep '^sha256' $metadata_filename > /dev/null; then + echo "downloaded metadata file looks valid..." + download_url=`awk '$1 == "url" { print $2 }' "$metadata_filename"` + sha256=`awk '$1 == "sha256" { print $2 }' "$metadata_filename"` + else + echo "downloaded metadata file is corrupted or an uncaught error was encountered in downloading the file..." + # this generally means one of the download methods downloaded a 404 or something like that and then reported a successful exit code, + # and this should be fixed in the function that was doing the download. + report_bug + exit 1 + fi fi - - download_url=`awk '$1 == "url" { print $2 }' "$metadata_filename"` - sha256=`awk '$1 == "sha256" { print $2 }' "$metadata_filename"` else download_url=$download_url_override # Set sha256 to empty string if checksum not set diff --git a/lib/mixlib/install/generator/bourne/scripts/fetch_package.sh b/lib/mixlib/install/generator/bourne/scripts/fetch_package.sh index a324d39e..bee6bb1e 100644 --- a/lib/mixlib/install/generator/bourne/scripts/fetch_package.sh +++ b/lib/mixlib/install/generator/bourne/scripts/fetch_package.sh @@ -8,26 +8,48 @@ # Optional Inputs: # $cmdline_filename: Name of the package downloaded on local disk. # $cmdline_dl_dir: Name of the directory downloaded package will be saved to on local disk. +# $license_id: If set, indicates we're using commercial/trial API with content-disposition headers # # Outputs: # $download_filename: Name of the downloaded file on local disk. # $filetype: Type of the file downloaded. ############ -filename=`echo $download_url | sed -e 's/?.*//' | sed -e 's/^.*\///'` -filetype=`echo $filename | sed -e 's/^.*\.//'` - -# use either $tmp_dir, the provided directory (-d) or the provided filename (-f) -if test "x$cmdline_filename" != "x"; then - download_filename="$cmdline_filename" -elif test "x$cmdline_dl_dir" != "x"; then - download_filename="$cmdline_dl_dir/$filename" +# For licensed APIs (commercial/trial), the URL is an endpoint, not a direct file URL +# The actual filename will come from the Content-Disposition header +if test "x$license_id" != "x"; then + # Use content-disposition to get the filename + use_content_disposition="true" + # We don't know the filename yet - it will come from Content-Disposition + # Just set the download directory + if test "x$cmdline_filename" != "x"; then + download_filename="$cmdline_filename" + download_dir=`dirname $download_filename` + use_content_disposition="false" # User specified exact filename + elif test "x$cmdline_dl_dir" != "x"; then + download_dir="$cmdline_dl_dir" + download_filename="" # Will be determined after download + else + download_dir="$tmp_dir" + download_filename="" # Will be determined after download + fi + filetype="" # Will be determined after we get the actual filename else - download_filename="$tmp_dir/$filename" -fi + # Traditional omnitruck URLs have the filename in the URL + use_content_disposition="false" + filename=`echo $download_url | sed -e 's/?.*//' | sed -e 's/^.*\///'` + filetype=`echo $filename | sed -e 's/^.*\.//'` -# ensure the parent directory where we download the installer always exists -download_dir=`dirname $download_filename` + # use either $tmp_dir, the provided directory (-d) or the provided filename (-f) + if test "x$cmdline_filename" != "x"; then + download_filename="$cmdline_filename" + elif test "x$cmdline_dl_dir" != "x"; then + download_filename="$cmdline_dl_dir/$filename" + else + download_filename="$tmp_dir/$filename" + fi + download_dir=`dirname $download_filename` +fi (umask 077 && mkdir -p $download_dir) || exit 1 # check if we have that file locally available and if so verify the checksum @@ -41,12 +63,16 @@ download_dir=`dirname $download_filename` cached_file_available="false" verify_checksum="true" -if test -f $download_filename; then +# Skip caching checks when using content-disposition since we don't know the real filename yet +if test "x$use_content_disposition" = "xtrue"; then + cached_file_available="false" + verify_checksum="true" +elif test "x$download_filename" != "x" && test -f "$download_filename"; then echo "$download_filename exists" cached_file_available="true" fi -if test "x$download_url_override" != "x"; then +if test "x$download_url_override" != "x" && test "x$use_content_disposition" = "xfalse"; then echo "Download URL override specified" if test "x$cached_file_available" = "xtrue"; then echo "Verifying local file" @@ -75,7 +101,63 @@ if test "x$download_url_override" != "x"; then fi if test "x$cached_file_available" != "xtrue"; then - do_download "$download_url" "$download_filename" + if test "x$use_content_disposition" = "xtrue"; then + # For licensed APIs, download to a temporary file and extract filename from response headers + # The download_dir was already set during initialization above + + # Create temp file for download + temp_download="$download_dir/chef-download-temp.$$" + + # Download to temp file + do_download "$download_url" "$temp_download" + + # Extract filename from response headers (try multiple methods for compatibility) + if test -f "$tmp_dir/stderr"; then + # Method 1: Try to extract filename from content-disposition header + # Format: content-disposition: attachment; filename="chef-18.8.54-1.el9.x86_64.rpm" + actual_filename=`grep -i 'content-disposition' $tmp_dir/stderr | sed -n 's/.*filename="\([^"]*\)".*/\1/p' | head -1` + + # Method 2: If content-disposition failed, try to extract from location redirect header + # Format: location: https://packages.chef.io/files/stable/chef/18.8.54/el/9/chef-18.8.54-1.el9.x86_64.rpm?licenseId=... + if test "x$actual_filename" = "x"; then + actual_filename=`grep -i '^location:' $tmp_dir/stderr | head -1 | sed 's/.*\///' | sed 's/?.*//'` + fi + + # Method 3: Try extracting from any URL-like pattern in stderr + if test "x$actual_filename" = "x"; then + actual_filename=`grep -i '\.rpm\|\.deb\|\.pkg\|\.msi\|\.dmg' $tmp_dir/stderr | sed -n 's/.*\/\([^/?]*\.\(rpm\|deb\|pkg\|msi\|dmg\)\).*/\1/p' | head -1` + fi + fi + + # If we still couldn't extract from headers, construct filename from metadata + if test "x$actual_filename" = "x"; then + echo "Warning: Could not extract filename from response headers, using fallback" + # Construct a reasonable filename from available metadata + # This is a fallback and may not match the exact package name + if test "x$platform" = "xel" || test "x$platform" = "xfedora" || test "x$platform" = "xamazon"; then + actual_filename="chef-${version}-1.${platform}${platform_version}.${machine}.rpm" + elif test "x$platform" = "xdebian" || test "x$platform" = "xubuntu"; then + actual_filename="chef_${version}-1_${machine}.deb" + elif test "x$platform" = "xmac_os_x"; then + actual_filename="chef-${version}.dmg" + else + actual_filename="chef-${version}.pkg" + fi + fi + + download_filename="$download_dir/$actual_filename" + + # Move temp file to final location + mv "$temp_download" "$download_filename" + + # Extract filetype from actual filename + filetype=`echo $actual_filename | sed -e 's/^.*\.//'` + + echo "Downloaded as: $download_filename (type: $filetype)" + else + # Traditional download with known filename + do_download "$download_url" "$download_filename" + fi fi if test "x$verify_checksum" = "xtrue"; then diff --git a/lib/mixlib/install/generator/bourne/scripts/helpers.sh.erb b/lib/mixlib/install/generator/bourne/scripts/helpers.sh.erb index f204a709..98e9cc5c 100644 --- a/lib/mixlib/install/generator/bourne/scripts/helpers.sh.erb +++ b/lib/mixlib/install/generator/bourne/scripts/helpers.sh.erb @@ -115,7 +115,12 @@ capture_tmp_stderr() { # do_wget URL FILENAME do_wget() { echo "trying wget..." - wget --user-agent="User-Agent: <%= user_agent_string %>" -O "$2" "$1" 2>$tmp_dir/stderr + # If filename is empty, use --content-disposition to get filename from server + if test "x$2" = "x"; then + wget --user-agent="User-Agent: <%= user_agent_string %>" --content-disposition "$1" 2>$tmp_dir/stderr + else + wget --user-agent="User-Agent: <%= user_agent_string %>" -O "$2" "$1" 2>$tmp_dir/stderr + fi rc=$? # check for 404 grep "ERROR 404" $tmp_dir/stderr 2>&1 >/dev/null @@ -124,8 +129,14 @@ do_wget() { http_404_error fi - # check for bad return status or empty output - if test $rc -ne 0 || test ! -s "$2"; then + # check for bad return status (skip empty output check if using content-disposition) + if test $rc -ne 0; then + capture_tmp_stderr "wget" + return 1 + fi + + # Only check for empty output if we specified a filename + if test "x$2" != "x" && test ! -s "$2"; then capture_tmp_stderr "wget" return 1 fi @@ -136,8 +147,15 @@ do_wget() { # do_curl URL FILENAME do_curl() { echo "trying curl..." - curl -A "User-Agent: <%= user_agent_string %>" --retry 5 -sL -D $tmp_dir/stderr "$1" > "$2" - rc=$? + # If filename is empty, use -O and -J to get filename from Content-Disposition header + if test "x$2" = "x"; then + curl -A "User-Agent: <%= user_agent_string %>" --retry 5 -sL -D $tmp_dir/stderr -O -J "$1" + rc=$? + else + curl -A "User-Agent: <%= user_agent_string %>" --retry 5 -sL -D $tmp_dir/stderr "$1" > "$2" + rc=$? + fi + # check for 404 grep "404 Not Found" $tmp_dir/stderr 2>&1 >/dev/null if test $? -eq 0; then @@ -145,8 +163,14 @@ do_curl() { http_404_error fi - # check for bad return status or empty output - if test $rc -ne 0 || test ! -s "$2"; then + # check for bad return status + if test $rc -ne 0; then + capture_tmp_stderr "curl" + return 1 + fi + + # Only check for empty output if we specified a filename + if test "x$2" != "x" && test ! -s "$2"; then capture_tmp_stderr "curl" return 1 fi diff --git a/lib/mixlib/install/generator/bourne/scripts/script_cli_parameters.sh.erb b/lib/mixlib/install/generator/bourne/scripts/script_cli_parameters.sh.erb index 8c6f4684..fa3ef2ca 100644 --- a/lib/mixlib/install/generator/bourne/scripts/script_cli_parameters.sh.erb +++ b/lib/mixlib/install/generator/bourne/scripts/script_cli_parameters.sh.erb @@ -12,13 +12,14 @@ # $install_strategy: Method of package installations. default strategy is to always install upon exec. Set to "once" to skip if project is installed # $download_url_override: Install package downloaded from a direct URL. # $checksum: SHA256 for download_url_override file (optional) +# $license_id: License ID for commercial API access (optional) ############ # Defaults channel="stable" project="<%= default_product %>" -while getopts pnv:c:f:P:d:s:l:a opt +while getopts pnv:c:f:P:d:s:l:a:L: opt do case "$opt" in @@ -32,9 +33,10 @@ do s) install_strategy="$OPTARG";; l) download_url_override="$OPTARG";; a) checksum="$OPTARG";; + L) license_id="$OPTARG";; \?) # unknown flag echo >&2 \ - "usage: $0 [-P project] [-c release_channel] [-v version] [-f filename | -d download_dir] [-s install_strategy] [-l download_url_override] [-a checksum]" + "usage: $0 [-P project] [-c release_channel] [-v version] [-f filename | -d download_dir] [-s install_strategy] [-l download_url_override] [-a checksum] [-L license_id]" exit 1;; esac done diff --git a/lib/mixlib/install/generator/powershell.rb b/lib/mixlib/install/generator/powershell.rb index 848b1f6b..44b96530 100644 --- a/lib/mixlib/install/generator/powershell.rb +++ b/lib/mixlib/install/generator/powershell.rb @@ -28,7 +28,7 @@ def self.install_ps1(context) install_project_module << get_script("install_project.ps1") install_command = [] - install_command << ps1_modularize(install_project_module.join("\n"), "Omnitruck") + install_command << ps1_modularize(install_project_module.join("\n"), "Installer-Module") install_command.join("\n\n") end @@ -49,7 +49,7 @@ def install_command install_project_module << get_script("get_project_metadata.ps1") install_project_module << get_script("install_project.ps1") install_command = [] - install_command << ps1_modularize(install_project_module.join("\n"), "Omnitruck") + install_command << ps1_modularize(install_project_module.join("\n"), "Installer-Module") install_command << render_command install_command.join("\n\n") end @@ -71,6 +71,7 @@ def render_command cmd << " -version #{options.product_version}" cmd << " -channel #{options.channel}" cmd << " -architecture #{options.architecture}" if options.architecture + cmd << " -license_id #{options.license_id}" if options.license_id && !options.license_id.to_s.empty? cmd << install_command_params if options.install_command_options cmd << "\n" end diff --git a/lib/mixlib/install/generator/powershell/scripts/get_project_metadata.ps1.erb b/lib/mixlib/install/generator/powershell/scripts/get_project_metadata.ps1.erb index c6da3c10..21a0a1d9 100644 --- a/lib/mixlib/install/generator/powershell/scripts/get_project_metadata.ps1.erb +++ b/lib/mixlib/install/generator/powershell/scripts/get_project_metadata.ps1.erb @@ -39,7 +39,10 @@ function Get-ProjectMetadata { $nightlies, [validateset('auto', 'i386', 'x86_64')] [string] - $architecture = 'auto' + $architecture = 'auto', + # License ID for commercial API access + [string] + $license_id ) # The following legacy switches are just aliases for the current channel @@ -60,17 +63,56 @@ function Get-ProjectMetadata { Write-Verbose "Architecture: $architecture" Write-Verbose "Project: $project" + # Use commercial API if license_id is provided, otherwise use omnitruck + if ($license_id) { + # Check if license_id starts with 'free-' or 'trial-' for trial API + if ($license_id -match '^(free-|trial-)') { + $base_server_uri = 'https://chefdownload-trial.chef.io' + Write-Verbose "Using Trial API with license ID" + } else { + $base_server_uri = 'https://chefdownload-commercial.chef.io' + Write-Verbose "Using Commercial API with license ID" + } + } + $metadata_base_url = "/$($channel)/$($project)/metadata" $metadata_array = ("?v=$($version)", "p=$platform", "pv=$platform_version", "m=$architecture") + + # Add license_id to query parameters if provided + if ($license_id) { + $metadata_array += "license_id=$license_id" + } + $metadata_base_url += [string]::join('&', $metadata_array) $metadata_url = new-uri $base_server_uri $metadata_base_url Write-Verbose "Downloading $project details from $metadata_url" - $package_metadata = (Get-WebContent $metadata_url).trim() -split '\n' | - foreach { $hash = @{} } {$key, $value = $_ -split '\s+'; $hash.Add($key, $value)} {$hash} + $response = (Get-WebContent $metadata_url).trim() + + # Commercial and trial APIs return JSON, omnitruck returns text format + if ($license_id) { + # Parse JSON response from commercial/trial API + try { + $json = $response | ConvertFrom-Json + $package_metadata = @{ + url = $json.url + sha256 = $json.sha256 + version = $json.version + } + if ($json.sha1) { + $package_metadata['sha1'] = $json.sha1 + } + } catch { + throw "Failed to parse JSON response from API: $_" + } + } else { + # Parse text response from omnitruck + $package_metadata = $response -split '\n' | + foreach { $hash = @{} } {$key, $value = $_ -split '\s+'; $hash.Add($key, $value)} {$hash} + } Write-Verbose "Project details: " foreach ($key in $package_metadata.keys) { diff --git a/lib/mixlib/install/generator/powershell/scripts/helpers.ps1.erb b/lib/mixlib/install/generator/powershell/scripts/helpers.ps1.erb index e7df3a55..f6542ece 100644 --- a/lib/mixlib/install/generator/powershell/scripts/helpers.ps1.erb +++ b/lib/mixlib/install/generator/powershell/scripts/helpers.ps1.erb @@ -6,13 +6,12 @@ function Get-PlatformVersion { $platform_version = switch ($osVersion) { # Windows Server build numbers from: https://betawiki.net/wiki/Microsoft_Windows + { $_ -ge [version]'10.0.26100' } { '2025'; break } { $_ -ge [version]'10.0.20145' } { '2022'; break } { $_ -ge [version]'10.0.17609' } { '2019'; break } { $_ -ge [version]'10.0.0' } { '2016'; break } { $_ -ge [version]'6.3.0' } { '2012r2'; break } { $_ -ge [version]'6.2.0' } { '2012'; break } - { $_ -ge [version]'6.1.0' } { '2008r2'; break } - { $_ -ge [version]'6.0.0' } { '2008'; break } } if(Test-Path 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Server\ServerLevels') { @@ -58,10 +57,10 @@ function Get-WebContent { try { if($PSVersionTable.PSEdition -eq 'Core') { - Get-WebContentOnCore $uri $filepath + return Get-WebContentOnCore $uri $filepath } else { - Get-WebContentOnFullNet $uri $filepath + return Get-WebContentOnFullNet $uri $filepath } } catch { @@ -98,6 +97,28 @@ function Get-WebContentOnFullNet { } else { $wc.downloadfile($uri, $filepath) + + # Try to extract filename from Content-Disposition header + $contentDisposition = $wc.ResponseHeaders["Content-Disposition"] + $extractedFilename = $null + + if ($contentDisposition) { + # Try to match filename="..." or filename*=UTF-8''... + if ($contentDisposition -match 'filename="([^"]+)"') { + $extractedFilename = $matches[1] + } + elseif ($contentDisposition -match "filename='([^']+)'") { + $extractedFilename = $matches[1] + } + elseif ($contentDisposition -match "filename=([^;]+)") { + $extractedFilename = $matches[1].Trim() + } + } + + return @{ + Filename = $extractedFilename + Downloaded = $true + } } } @@ -125,6 +146,19 @@ function Get-WebContentOnCore { if ($copyStreamOp.Exception -ne $null) { throw $copyStreamOp.Exception } + + # Try to extract filename from Content-Disposition header + $extractedFilename = $null + $contentDisposition = $response.Content.Headers.ContentDisposition + + if ($contentDisposition -and $contentDisposition.FileName) { + $extractedFilename = $contentDisposition.FileName.Trim('"') + } + + return @{ + Filename = $extractedFilename + Downloaded = $true + } } } } diff --git a/lib/mixlib/install/generator/powershell/scripts/install_project.ps1.erb b/lib/mixlib/install/generator/powershell/scripts/install_project.ps1.erb index 1e59bf79..c8767242 100644 --- a/lib/mixlib/install/generator/powershell/scripts/install_project.ps1.erb +++ b/lib/mixlib/install/generator/powershell/scripts/install_project.ps1.erb @@ -56,7 +56,10 @@ function Install-Project { $checksum, # Set to 'once' to skip install if project is detected [string] - $install_strategy + $install_strategy, + # License ID for commercial API access + [string] + $license_id ) # Check for chef-client command in various locations @@ -86,7 +89,7 @@ function Install-Project { $download_url = $download_url_override $sha256 = $checksum } else { - $package_metadata = Get-ProjectMetadata -project $project -channel $channel -version $version -prerelease:$prerelease -nightlies:$nightlies -architecture $architecture + $package_metadata = Get-ProjectMetadata -project $project -channel $channel -version $version -prerelease:$prerelease -nightlies:$nightlies -architecture $architecture -license_id $license_id $download_url = $package_metadata.url $sha256 = $package_metadata.sha256 } @@ -99,7 +102,13 @@ function Install-Project { } } else { - $filename = (([System.Uri]$download_url).AbsolutePath -split '/')[-1] + # For licensed downloads, we won't know the filename until after download + if ([string]::IsNullOrEmpty($license_id)) { + $filename = (([System.Uri]$download_url).AbsolutePath -split '/')[-1] + } else { + $filename = "chef-download-temp-$PID" + Write-Host "Using temporary filename for licensed download: $filename" + } } Write-Host "Download directory: $download_directory" Write-Host "Filename: $filename" @@ -137,7 +146,28 @@ function Install-Project { if (-not ($cached_installer_available)) { if ($pscmdlet.ShouldProcess("$($download_url)", "Download $project")) { Write-Host "Downloading $project from $($download_url) to $download_destination." - Get-WebContent $download_url -filepath $download_destination + $download_result = Get-WebContent $download_url -filepath $download_destination + + # For licensed downloads, extract actual filename from Content-Disposition + if (-not [string]::IsNullOrEmpty($license_id) -and $download_result -and $download_result.Filename) { + $actual_filename = $download_result.Filename + Write-Host "Extracted filename from Content-Disposition: $actual_filename" + + $final_destination = join-path $download_directory $actual_filename + + # Move temp file to final location with correct name + if ($download_destination -ne $final_destination) { + Write-Host "Moving to final location: $final_destination" + if (test-path $final_destination) { + remove-item $final_destination -force + } + move-item $download_destination $final_destination -force + $download_destination = $final_destination + } + } elseif (-not [string]::IsNullOrEmpty($license_id)) { + Write-Host "Warning: Could not extract filename from Content-Disposition header for licensed download." + Write-Host "Using temporary filename. Package installation may fail." + } } } diff --git a/lib/mixlib/install/options.rb b/lib/mixlib/install/options.rb index 0da880cb..33187367 100644 --- a/lib/mixlib/install/options.rb +++ b/lib/mixlib/install/options.rb @@ -63,6 +63,7 @@ class InvalidOptions < ArgumentError; end :include_metadata, :user_agent_headers, :install_command_options, + :license_id, ] SUPPORTED_WINDOWS_DESKTOP_VERSIONS = %w{10} diff --git a/lib/mixlib/install/product_matrix.rb b/lib/mixlib/install/product_matrix.rb index 16edd820..0a692e63 100644 --- a/lib/mixlib/install/product_matrix.rb +++ b/lib/mixlib/install/product_matrix.rb @@ -47,6 +47,11 @@ package_name "chef" end + product "chef-ice" do + product_name "Chef Infra Client Enterprise" + package_name "chef-ice" + end + product "chef-foundation" do product_name "Chef Foundation" package_name "chef-foundation" diff --git a/lib/mixlib/install/script_generator.rb b/lib/mixlib/install/script_generator.rb index 82b52a9d..af095583 100644 --- a/lib/mixlib/install/script_generator.rb +++ b/lib/mixlib/install/script_generator.rb @@ -224,7 +224,7 @@ def metadata_endpoint_from_project(project = nil) end def windows_metadata_url - base = if omnibus_url =~ %r{/install.sh$} + base = if omnibus_url.match?(%r{/install.sh$}) "#{File.dirname(omnibus_url)}/" end diff --git a/spec/functional/mixlib/install/cli_spec.rb b/spec/functional/mixlib/install/cli_spec.rb index 08f403ed..4e904a51 100644 --- a/spec/functional/mixlib/install/cli_spec.rb +++ b/spec/functional/mixlib/install/cli_spec.rb @@ -237,5 +237,15 @@ expect(last_command_output).to match /Download saved to .*mydir\/chef/ end end + + context "with license_id" do + let(:additional_args) { "-L test-license-key-123 --url" } + + it "accepts license_id parameter" do + # This will fail with actual API call, but we're testing that the parameter is accepted + # In a real scenario with a valid license, it would use the commercial API + expect { Mixlib::ShellOut.new("mixlib-install #{command} #{args} ").run_command }.not_to raise_error + end + end end end diff --git a/spec/unit/mixlib/install/backend/package_router_spec.rb b/spec/unit/mixlib/install/backend/package_router_spec.rb index be063add..32616e39 100644 --- a/spec/unit/mixlib/install/backend/package_router_spec.rb +++ b/spec/unit/mixlib/install/backend/package_router_spec.rb @@ -31,6 +31,7 @@ let(:user_agent_headers) { nil } let(:pv_compat) { nil } let(:include_metadata) { nil } + let(:license_id) { nil } let(:options) do {}.tap do |opt| @@ -43,6 +44,7 @@ opt[:platform] = platform if platform opt[:platform_version] = platform_version if platform_version opt[:architecture] = architecture if architecture + opt[:license_id] = license_id if license_id end end @@ -67,6 +69,139 @@ end end + context "for commercial API with license_id" do + let(:channel) { :stable } + let(:product_name) { "chef" } + let(:product_version) { "18.0.0" } + let(:license_id) { "test-license-key-123" } + + it "uses commercial API endpoint" do + expect(package_router.endpoint).to eq Mixlib::Install::Dist::COMMERCIAL_API_ENDPOINT + end + + it "detects commercial API usage" do + expect(package_router.use_commercial_api?).to be true + end + + context "with platform info" do + let(:platform) { "ubuntu" } + let(:platform_version) { "20.04" } + let(:architecture) { "x86_64" } + + it "includes license_id in download URL" do + # Mock the HTTP request to prevent actual API calls + # Commercial/trial APIs return nested hash: platform -> platform_version -> architecture -> package_info + allow(package_router).to receive(:get).and_return({ + "ubuntu" => { + "20.04" => { + "x86_64" => { + "version" => "18.0.0", + "sha256" => "abc123", + "sha1" => "ghi789", + }, + }, + }, + }) + + artifact = artifact_info + expect(artifact.url).to include("license_id=#{license_id}") + end + end + end + + context "for trial API with free- license_id" do + let(:channel) { :stable } + let(:product_name) { "chef" } + let(:product_version) { "18.0.0" } + let(:license_id) { "free-abc-def-123" } + + it "uses trial API endpoint" do + expect(package_router.endpoint).to eq Mixlib::Install::Dist::TRIAL_API_ENDPOINT + end + + it "detects trial API usage" do + expect(package_router.use_trial_api?).to be true + end + + it "does not detect commercial API usage" do + expect(package_router.use_commercial_api?).to be false + end + + it "detects licensed API usage" do + expect(package_router.use_licensed_api?).to be true + end + + context "with platform info" do + let(:platform) { "ubuntu" } + let(:platform_version) { "20.04" } + let(:architecture) { "x86_64" } + + it "includes license_id in download URL" do + # Mock the HTTP request to prevent actual API calls + # Commercial/trial APIs return nested hash: platform -> platform_version -> architecture -> package_info + allow(package_router).to receive(:get).and_return({ + "ubuntu" => { + "20.04" => { + "x86_64" => { + "version" => "18.0.0", + "sha256" => "abc123", + "sha1" => "ghi789", + }, + }, + }, + }) + + artifact = artifact_info + expect(artifact.url).to include("license_id=#{license_id}") + end + end + end + + context "for trial API with trial- license_id" do + let(:channel) { :stable } + let(:product_name) { "chef" } + let(:product_version) { "18.0.0" } + let(:license_id) { "trial-xyz-789-456" } + + it "uses trial API endpoint" do + expect(package_router.endpoint).to eq Mixlib::Install::Dist::TRIAL_API_ENDPOINT + end + + it "detects trial API usage" do + expect(package_router.use_trial_api?).to be true + end + + it "does not detect commercial API usage" do + expect(package_router.use_commercial_api?).to be false + end + + it "detects licensed API usage" do + expect(package_router.use_licensed_api?).to be true + end + end + + context "without license_id" do + let(:channel) { :stable } + let(:product_name) { "chef" } + let(:product_version) { "18.0.0" } + + it "uses standard endpoint" do + expect(package_router.endpoint).to eq Mixlib::Install::Dist::PRODUCT_ENDPOINT + end + + it "detects no commercial API usage" do + expect(package_router.use_commercial_api?).to be false + end + + it "detects no trial API usage" do + expect(package_router.use_trial_api?).to be false + end + + it "detects no licensed API usage" do + expect(package_router.use_licensed_api?).to be false + end + end + context "for chef/stable with specific version" do let(:channel) { :stable } let(:product_name) { "chef" } diff --git a/spec/unit/mixlib/install/backend_spec.rb b/spec/unit/mixlib/install/backend_spec.rb index 9d1483ee..91a2a905 100644 --- a/spec/unit/mixlib/install/backend_spec.rb +++ b/spec/unit/mixlib/install/backend_spec.rb @@ -178,4 +178,84 @@ def check_platform_info(data) end end end + + context "with license_id for commercial API" do + let(:product_name) { "chef" } + let(:channel) { :stable } + let(:product_version) { :latest } + let(:license_id) { "test-license-key-789" } + + let(:info_with_license) do + Mixlib::Install.new( + channel: channel, + product_name: product_name, + product_version: product_version, + license_id: license_id + ) + end + + it "accepts license_id parameter" do + expect(info_with_license.options.license_id).to eq license_id + end + + it "uses commercial API backend" do + backend = Mixlib::Install::Backend.backend(info_with_license.options) + expect(backend.use_commercial_api?).to be true + expect(backend.endpoint).to eq Mixlib::Install::Dist::COMMERCIAL_API_ENDPOINT + end + end + + context "with free- license_id for trial API" do + let(:product_name) { "chef" } + let(:channel) { :stable } + let(:product_version) { :latest } + let(:license_id) { "free-trial-license-123" } + + let(:info_with_trial_license) do + Mixlib::Install.new( + channel: channel, + product_name: product_name, + product_version: product_version, + license_id: license_id + ) + end + + it "accepts license_id parameter" do + expect(info_with_trial_license.options.license_id).to eq license_id + end + + it "uses trial API backend" do + backend = Mixlib::Install::Backend.backend(info_with_trial_license.options) + expect(backend.use_trial_api?).to be true + expect(backend.use_commercial_api?).to be false + expect(backend.endpoint).to eq Mixlib::Install::Dist::TRIAL_API_ENDPOINT + end + end + + context "with trial- license_id for trial API" do + let(:product_name) { "chef" } + let(:channel) { :stable } + let(:product_version) { :latest } + let(:license_id) { "trial-xyz-456" } + + let(:info_with_trial_license) do + Mixlib::Install.new( + channel: channel, + product_name: product_name, + product_version: product_version, + license_id: license_id + ) + end + + it "accepts license_id parameter" do + expect(info_with_trial_license.options.license_id).to eq license_id + end + + it "uses trial API backend" do + backend = Mixlib::Install::Backend.backend(info_with_trial_license.options) + expect(backend.use_trial_api?).to be true + expect(backend.use_commercial_api?).to be false + expect(backend.endpoint).to eq Mixlib::Install::Dist::TRIAL_API_ENDPOINT + end + end end diff --git a/spec/unit/mixlib/install/generator_spec.rb b/spec/unit/mixlib/install/generator_spec.rb index f3754e41..c06bf202 100644 --- a/spec/unit/mixlib/install/generator_spec.rb +++ b/spec/unit/mixlib/install/generator_spec.rb @@ -78,13 +78,187 @@ end it_behaves_like "the correct sh script" + + it "uses traditional text parsing for omnitruck without license_id" do + expect(install_script).to include("awk '$1 == \"url\" { print $2 }'") + expect(install_script).to include("grep '^url' $metadata_filename") + end + end + + context "with license_id" do + let(:add_options) do + { + license_id: "test-license-key-123", + } + end + + it "includes license_id in the script variables" do + expect(install_script).to include("license_id=test-license-key-123") + end + + it "uses commercial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-commercial.chef.io") + end + + it "includes JSON parsing logic for commercial API" do + expect(install_script).to include("sed -n 's/.*\"url\":\"\\([^\"]*\\)\".*/\\1/p'") + expect(install_script).to include("sed -n 's/.*\"sha256\":\"\\([^\"]*\\)\".*/\\1/p'") + end + + it "checks for JSON format when license_id is present" do + expect(install_script).to include("grep -q '^{' \"$metadata_filename\"") + end + + it "sets use_content_disposition flag when license_id is present" do + expect(install_script).to include("use_content_disposition=\"true\"") + end + + it "includes content-disposition handling in wget" do + expect(install_script).to include("--content-disposition") + end + + it "includes content-disposition handling in curl" do + expect(install_script).to include("-O -J") + end + + it "skips caching checks when using content-disposition" do + expect(install_script).to include('if test "x$use_content_disposition" = "xtrue"; then') + expect(install_script).to include('cached_file_available="false"') + end + + it "downloads to temp file and extracts filename from headers" do + expect(install_script).to include('temp_download="$download_dir/chef-download-temp.$$"') + expect(install_script).to include("grep -i 'content-disposition'") + expect(install_script).to include("mv \"$temp_download\" \"$download_filename\"") + end + + it "extracts filetype from actual downloaded filename" do + expect(install_script).to include("filetype=`echo $actual_filename | sed -e 's/^.*\\.//'`") + end + end + + context "with free- license_id" do + let(:add_options) do + { + license_id: "free-trial-abc-123", + } + end + + it "includes license_id in the script variables" do + expect(install_script).to include("license_id=free-trial-abc-123") + end + + it "uses trial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-trial.chef.io") + end + + it "includes JSON parsing logic for trial API" do + expect(install_script).to include("sed -n 's/.*\"url\":\"\\([^\"]*\\)\".*/\\1/p'") + expect(install_script).to include("sed -n 's/.*\"sha256\":\"\\([^\"]*\\)\".*/\\1/p'") + end + + it "sets use_content_disposition flag for trial API" do + expect(install_script).to include("use_content_disposition=\"true\"") + end + + it "skips caching checks when using content-disposition" do + expect(install_script).to include('if test "x$use_content_disposition" = "xtrue"; then') + expect(install_script).to include('cached_file_available="false"') + end + end + + context "with trial- license_id" do + let(:add_options) do + { + license_id: "trial-xyz-456", + } + end + + it "includes license_id in the script variables" do + expect(install_script).to include("license_id=trial-xyz-456") + end + + it "uses trial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-trial.chef.io") + end + + it "includes JSON parsing logic for trial API" do + expect(install_script).to include("sed -n 's/.*\"url\":\"\\([^\"]*\\)\".*/\\1/p'") + expect(install_script).to include("sed -n 's/.*\"sha256\":\"\\([^\"]*\\)\".*/\\1/p'") + end + + it "sets use_content_disposition flag for trial API" do + expect(install_script).to include("use_content_disposition=\"true\"") + end + + it "skips caching checks when using content-disposition" do + expect(install_script).to include('if test "x$use_content_disposition" = "xtrue"; then') + expect(install_script).to include('cached_file_available="false"') + end + + it "downloads to temp file and extracts filename from headers" do + expect(install_script).to include('temp_download="$download_dir/chef-download-temp.$$"') + expect(install_script).to include("grep -i 'content-disposition'") + expect(install_script).to include("mv \"$temp_download\" \"$download_filename\"") + end + + it "includes multiple filename extraction methods" do + # Method 1: Content-Disposition header + expect(install_script).to include("grep -i 'content-disposition'") + expect(install_script).to include("sed -n 's/.*filename=\"\\([^\"]*\\)\".*/\\1/p'") + + # Method 2: Location redirect header + expect(install_script).to include("grep -i '^location:'") + expect(install_script).to include("sed 's/.*\\///'") + expect(install_script).to include("sed 's/?.*//'") + + # Method 3: URL pattern matching + expect(install_script).to include("grep -i '\\.rpm\\|\\.deb\\|\\.pkg\\|\\.msi\\|\\.dmg'") + end + + it "includes fallback filename construction" do + expect(install_script).to include('echo "Warning: Could not extract filename from response headers, using fallback"') + expect(install_script).to include('actual_filename="chef-${version}-1.${platform}${platform_version}.${machine}.rpm"') + expect(install_script).to include('actual_filename="chef_${version}-1_${machine}.deb"') + expect(install_script).to include('actual_filename="chef-${version}.dmg"') + expect(install_script).to include('actual_filename="chef-${version}.pkg"') + end + + it "extracts filetype from actual downloaded filename" do + expect(install_script).to include("filetype=`echo $actual_filename | sed -e 's/^.*\\.//'`") + end + end + + context "filename extraction for content-disposition" do + let(:add_options) do + { + license_id: "test-license-key-123", + } + end + + it "includes all three extraction methods in order" do + # Verify the extraction logic is ordered correctly + script_lines = install_script.split("\n") + + # Find the indices of each method + content_disposition_idx = script_lines.index { |l| l.include?("grep -i 'content-disposition'") && l.include?("sed -n") } + location_idx = script_lines.index { |l| l.include?("grep -i '^location:'") && l.include?("sed") } + fallback_idx = script_lines.index { |l| l.include?("Warning: Could not extract filename from response headers") } + + # Verify they exist and are in the correct order + expect(content_disposition_idx).not_to be_nil + expect(location_idx).not_to be_nil + expect(fallback_idx).not_to be_nil + expect(content_disposition_idx).to be < location_idx + expect(location_idx).to be < fallback_idx + end end context "for windows" do shared_examples_for "the correct ps1 script" do it "generates a ps1 script" do expect(install_script).to be_a(String) - expect(install_script).to start_with("new-module -name Omnitruck -scriptblock") + expect(install_script).to start_with("new-module -name Installer-Module -scriptblock") expect(install_script).to include("set-alias install -value Install-Project") end end @@ -118,6 +292,95 @@ it "adds omits the architecture param" do expect(install_script).to match(/install -project #{options[:product_name]} -version .* -channel #{options[:channel]}\n/) end + + it "uses traditional text parsing for omnitruck without license_id" do + expect(install_script).to include("-split '\\n'") + expect(install_script).to include("$key, $value = $_ -split '\\s+'") + end + end + + context "with license_id for PowerShell" do + let(:add_options) do + { + shell_type: :ps1, + license_id: "test-license-key-456", + } + end + + it_behaves_like "the correct ps1 script" + + it "includes license_id in install command" do + expect(install_script).to match(/install -project #{options[:product_name]} -version .* -channel #{options[:channel]} -license_id test-license-key-456\n/) + end + + it "includes license_id parameter in Get-ProjectMetadata function" do + expect(install_script).to include("[string]") + expect(install_script).to include("$license_id") + end + + it "uses commercial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-commercial.chef.io") + end + + it "includes JSON parsing logic for commercial API" do + expect(install_script).to include("ConvertFrom-Json") + expect(install_script).to include("$json.url") + expect(install_script).to include("$json.sha256") + end + + it "includes conditional parsing based on license_id" do + expect(install_script).to include("if ($license_id)") + end + end + + context "with free- license_id for PowerShell" do + let(:add_options) do + { + shell_type: :ps1, + license_id: "free-trial-789", + } + end + + it_behaves_like "the correct ps1 script" + + it "includes license_id in install command" do + expect(install_script).to match(/install -project #{options[:product_name]} -version .* -channel #{options[:channel]} -license_id free-trial-789\n/) + end + + it "uses trial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-trial.chef.io") + end + + it "includes JSON parsing logic for trial API" do + expect(install_script).to include("ConvertFrom-Json") + expect(install_script).to include("$json.url") + expect(install_script).to include("$json.sha256") + end + end + + context "with trial- license_id for PowerShell" do + let(:add_options) do + { + shell_type: :ps1, + license_id: "trial-abc-xyz", + } + end + + it_behaves_like "the correct ps1 script" + + it "includes license_id in install command" do + expect(install_script).to match(/install -project #{options[:product_name]} -version .* -channel #{options[:channel]} -license_id trial-abc-xyz\n/) + end + + it "uses trial API in metadata fetch" do + expect(install_script).to include("https://chefdownload-trial.chef.io") + end + + it "includes JSON parsing logic for trial API" do + expect(install_script).to include("ConvertFrom-Json") + expect(install_script).to include("$json.url") + expect(install_script).to include("$json.sha256") + end end end end diff --git a/spec/unit/mixlib/install/options_spec.rb b/spec/unit/mixlib/install/options_spec.rb index d0631dbe..df32e983 100644 --- a/spec/unit/mixlib/install/options_spec.rb +++ b/spec/unit/mixlib/install/options_spec.rb @@ -137,4 +137,25 @@ end end end + + context "for license_id option" do + let(:product_name) { "chef" } + let(:channel) { :stable } + let(:license_id) { "test-license-123" } + + it "accepts license_id parameter" do + mi = Mixlib::Install.new(product_name: product_name, channel: channel, license_id: license_id) + expect(mi.options.license_id).to eq license_id + end + + it "allows nil license_id" do + mi = Mixlib::Install.new(product_name: product_name, channel: channel) + expect(mi.options.license_id).to be_nil + end + + it "allows empty license_id" do + mi = Mixlib::Install.new(product_name: product_name, channel: channel, license_id: "") + expect(mi.options.license_id).to eq "" + end + end end diff --git a/spec/unit/mixlib/install/product_spec.rb b/spec/unit/mixlib/install/product_spec.rb index 4ecf1f74..00c5ecb9 100644 --- a/spec/unit/mixlib/install/product_spec.rb +++ b/spec/unit/mixlib/install/product_spec.rb @@ -137,6 +137,7 @@ angrychef automate chef + chef-ice chef-foundation chef-universal chef-backend diff --git a/spec/unit/mixlib/install_spec.rb b/spec/unit/mixlib/install_spec.rb index bbfebb4c..1d372748 100644 --- a/spec/unit/mixlib/install_spec.rb +++ b/spec/unit/mixlib/install_spec.rb @@ -150,7 +150,7 @@ end it "should render a script with cli parameters" do - expect(install_sh).to include("while getopts pnv:c:f:P:d:s:l:a opt") + expect(install_sh).to include("while getopts pnv:c:f:P:d:s:l:a:L: opt") end context "with custom base_url" do