diff --git a/.github/workflows/content-sources-actions.yml b/.github/workflows/content-sources-actions.yml index 075090191..9d3beb13f 100644 --- a/.github/workflows/content-sources-actions.yml +++ b/.github/workflows/content-sources-actions.yml @@ -28,7 +28,6 @@ jobs: make openapi - run: | git diff --exit-code api/openapi.json - openapivalidate: name: openapi validate runs-on: ubuntu-latest @@ -38,7 +37,15 @@ jobs: with: generator: python openapi-file: api/openapi.json - + deploymentyamldiff: + name: Check for deployment.yaml changes + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - run: | + make deployment-generate + - run: | + git diff --exit-code deployments/deployment.yaml golangci: name: Lint runs-on: ubuntu-latest diff --git a/deployments/README.md b/deployments/README.md index c6e4ecbab..c13c28fd0 100644 --- a/deployments/README.md +++ b/deployments/README.md @@ -1,3 +1,135 @@ -# `/deployments` +# Deployment Template System -IaaS, PaaS, system and container orchestration deployment configurations and templates (docker-compose, kubernetes/helm, mesos, terraform, bosh). +This directory contains a template system for generating the `deployment.yaml` file from a template and organized environment variables. This system helps reduce duplication and makes it easier to manage environment variables across different deployments and jobs. + +## Files + +- `build/deployment.template.yaml` - The base template file with placeholders for environment variables +- `build/env-variables.yaml` - Organized environment variables grouped by usage +- `build/process-template.py` - Python script that combines the template and environment variables +- `deployment.yaml` - Generated deployment file (created by the template processor) + +## Environment Variables Organization + +The environment variables in `env-variables.yaml` are organized into the following groups: + +### Common Variables (`common`) +Variables used across all deployments and jobs: +- Core application settings (CLOWDER_ENABLED, LOGGING_LEVEL, etc.) +- Certificates and secrets (RH_CDN_CERT_PAIR, SENTRY_DSN) +- Pulp client settings (CLIENTS_PULP_*) +- RBAC settings (CLIENTS_RBAC_BASE_URL) +- Feature flags (FEATURES_*) +- Candlepin settings (CLIENTS_CANDLEPIN_*) +- Feature service settings (CLIENTS_FEATURE_SERVICE_*) +- Roadmap settings (CLIENTS_ROADMAP_SERVER) + +### Service-Specific Variables (`service_specific`) +Variables used only by the API service: +- Repository import filter (OPTIONS_REPOSITORY_IMPORT_FILTER) +- Pulp database settings (CLIENTS_PULP_DATABASE_*) + +### Job-Specific Variables (`job_specific`) +Variables used by specific cron jobs: + +#### `process_repos` +- Snapshot retention settings (OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT) + +#### `transform_pulp_logs` +- CloudWatch settings (CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_*) +- S3 settings (CLIENTS_PULP_LOG_PARSER_S3_*) + +## Template Placeholders + +The template file uses the following placeholders that get replaced with environment variables: + +- `{{ENV_COMMON}}` - Common environment variables +- `{{ENV_SERVICE_SPECIFIC}}` - Service-specific environment variables +- `{{ENV_JOB_PROCESS_REPOS}}` - Process repos job-specific variables +- `{{ENV_JOB_TRANSFORM_PULP_LOGS}}` - Transform pulp logs job-specific variables + +## Usage + +### Generate deployment.yaml +```bash +make deployment-generate +``` + +This command: +1. Reads the `deployment.template.yaml` file +2. Reads the `env-variables.yaml` file +3. Combines them to generate the final `deployment.yaml` file + +### Clean up generated files +```bash +make deployment-clean +``` + +This removes the generated `deployment.yaml` file. + +### Validate the generated template +```bash +make deployment-validate +``` + +This validates the generated deployment template (requires OpenShift CLI). + +### Show help +```bash +make deployment-help +``` + +This shows all available deployment-related make targets. + +## Adding New Environment Variables + +To add new environment variables: + +1. **Common variables**: Add them to the `common` section in `env-variables.yaml` +2. **Service-specific variables**: Add them to the `service_specific` section +3. **Job-specific variables**: Add them to the appropriate job section in `job_specific` + +### Example: Adding a new common variable + +```yaml +common: + # ... existing variables ... + - name: NEW_VARIABLE + value: ${NEW_VARIABLE} +``` + +### Example: Adding a new job-specific variable + +```yaml +job_specific: + # ... existing jobs ... + new_job: + - name: NEW_JOB_VARIABLE + value: ${NEW_JOB_VARIABLE} +``` + +Then add the placeholder `{{ENV_JOB_NEW_JOB}}` to the template and update the Python script to handle it. + +## File Structure + +``` +deployments/ +├── README.md # This documentation +├── deployment.yaml # Generated deployment file +└── build/ # Build artifacts and source files + ├── deployment.template.yaml # Template file + ├── env-variables.yaml # Environment variables + └── process-template.py # Template processor script +``` + +## Benefits + +1. **Reduced duplication**: Environment variables are defined once and reused +2. **Better organization**: Variables are grouped by their usage +3. **Easier maintenance**: Changes to common variables only need to be made in one place +4. **Template flexibility**: Easy to add new job types or modify existing ones +5. **Validation**: The generated YAML can be validated for syntax correctness + +## Migration from Original deployment.yaml + +The original `deployment.yaml` file has been replaced with this template system. The generated file should be functionally equivalent to the original, but with better organization and reduced duplication. diff --git a/deployments/build/deployment.template.yaml b/deployments/build/deployment.template.yaml new file mode 100644 index 000000000..22e298039 --- /dev/null +++ b/deployments/build/deployment.template.yaml @@ -0,0 +1,465 @@ +# https://consoledot.pages.redhat.com/clowder/dev/api_reference.html +--- +apiVersion: template.openshift.io/v1 +kind: Template +metadata: + name: content-sources-backend +objects: + - apiVersion: cloud.redhat.com/v1alpha1 + kind: ClowdApp + metadata: + name: content-sources-backend + spec: + envName: ${ENV_NAME} + testing: + iqePlugin: content-sources + dependencies: + - rbac + - pulp + # https://consoledot.pages.redhat.com/clowder/dev/providers/kafka.html + kafkaTopics: + - partitions: 3 + replicas: 3 + topicName: platform.content-sources.template + - partitions: 3 + replicas: 3 + topicName: platform.notifications.ingress + deployments: + - name: task-worker + replicas: ${{TASK_WORKER_REPLICAS}} + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + command: + - /content-sources + - consumer + - instrumentation + initContainers: + - name: db-migrate + inheritEnv: true + args: + - /dbmigrate + - up + - name: external-repos-import + inheritEnv: true + args: + - /external-repos + - import + - name: introspect-single-repo + inheritEnv: true + args: + - /external-repos + - introspect + - --url + - https://cdn.redhat.com/content/dist/rhel9/9/aarch64/codeready-builder/os/,https://cdn.redhat.com/content/dist/rhel8/8.8/x86_64/baseos/os + - name: snapshot-single-repo + inheritEnv: true + args: + - /external-repos + - snapshot + - --url + - https://cdn.redhat.com/content/dist/rhel9/9/aarch64/codeready-builder/os/ + image: ${IMAGE}:${IMAGE_TAG} + livenessProbe: + failureThreshold: 3 + httpGet: + path: /ping + port: 8000 + scheme: HTTP + initialDelaySeconds: 35 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 120 + readinessProbe: + failureThreshold: 3 + httpGet: + path: /ping + port: 8000 + scheme: HTTP + initialDelaySeconds: 35 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 120 + env: {{ENV_COMMON}} + resources: + limits: + cpu: ${CPU_LIMIT_CS_WORKER} + memory: ${MEMORY_LIMIT_CS_WORKER} + requests: + cpu: ${CPU_REQUEST_CS_WORKER} + memory: ${MEMORY_REQUEST_CS_WORKER} + volumes: + - name: sel-downloads + emptyDir: + medium: Memory + - emptyDir: {} + name: tmpdir + volumeMounts: + - mountPath: /tmp + name: tmpdir + - name: sel-downloads + mountPath: /home/selenium/Downloads + - name: service + replicas: ${{API_REPLICAS}} + webServices: + public: + enabled: true + apiPath: content-sources + whitelistPaths: + - /api/content-sources/v1/repository_gpg_key/* + - /api/content-sources/v1.0/repository_gpg_key/* + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + command: + - /content-sources + - api + - instrumentation + livenessProbe: + failureThreshold: 3 + httpGet: + path: /ping + port: 8000 + scheme: HTTP + initialDelaySeconds: 35 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 120 + readinessProbe: + failureThreshold: 3 + httpGet: + path: /ping + port: 8000 + scheme: HTTP + initialDelaySeconds: 35 + periodSeconds: 5 + successThreshold: 1 + timeoutSeconds: 120 + env: {{ENV_COMMON}}{{ENV_SERVICE_SPECIFIC}} + resources: + limits: + cpu: ${CPU_LIMIT_CS_API} + memory: ${MEMORY_LIMIT_CS_API} + requests: + cpu: ${CPU_REQUEST_CS_API} + memory: ${MEMORY_REQUEST_CS_API} + volumes: + - emptyDir: {} + name: tmpdir + volumeMounts: + - mountPath: /tmp + name: tmpdir + jobs: + - name: create-latest-distributions + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - create-latest-distributions + env: {{ENV_COMMON}} + - name: retry-failed-tasks + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - retry-failed-tasks + env: {{ENV_COMMON}} + - name: process-repos + # https://crontab.guru/ + schedule: ${NIGHTLY_CRON_JOB} + suspend: ${{SUSPEND_CRON_JOB}} + concurrencyPolicy: "Forbid" + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /external-repos + - process-repos + - --interval + - "96" + env: {{ENV_COMMON}}{{ENV_JOB_PROCESS_REPOS}} + - name: cleanup + # https://crontab.guru/ + schedule: ${CLEANUP_CRON_JOB} + suspend: ${{SUSPEND_CRON_JOB}} + concurrencyPolicy: "Forbid" + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /external-repos + - cleanup + - --exclude + - "pulp-orphan" + env: {{ENV_COMMON}} + - name: pulp-orphan-cleanup + # https://crontab.guru/ + schedule: ${WEEKLY_CRON_JOB} + suspend: ${{SUSPEND_CRON_JOB}} + concurrencyPolicy: "Forbid" + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /external-repos + - cleanup + - --type + - "pulp-orphan" + env: {{ENV_COMMON}} + - name: transform-pulp-logs + schedule: "0 1 * * *" + suspend: ${{SUSPEND_TRANSFORM_PULP_LOGS}} + concurrencyPolicy: "Forbid" + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - transform-pulp-logs + env: {{ENV_COMMON}}{{ENV_JOB_TRANSFORM_PULP_LOGS}} + - name: hotfix-transform-pulp-logs-fix + suspend: ${{SUSPEND_TRANSFORM_PULP_LOGS}} + concurrencyPolicy: "Forbid" + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - transform-pulp-logs + - "2025-05-09" + - "24" + env: {{ENV_COMMON}}{{ENV_JOB_TRANSFORM_PULP_LOGS}} + - name: cleanup-missing-domains + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - cleanup-missing-domains + env: {{ENV_COMMON}} + - name: set-domain-label + podSpec: + securityContext: + runAsNonRoot: true + runAsUser: 1001 + image: ${IMAGE}:${IMAGE_TAG} + inheritEnv: true + command: + - /jobs + - set-domain-label + env: {{ENV_COMMON}} + database: + name: content-sources + version: 15 + inMemoryDb: true + objectStore: + - content-sources-central-pulp-s3 + - apiVersion: v1 + kind: Service + metadata: + labels: + app: content-sources-backend + name: content-sources-backend + spec: + ports: + - name: 8000-tcp + port: 8000 + protocol: TCP + targetPort: 8000 + - name: metrics + port: 9000 + targetPort: 9000 + selector: + pod: content-sources-backend-service + sessionAffinity: None + type: ClusterIP + status: + loadBalancer: {} + - apiVersion: metrics.console.redhat.com/v1alpha1 + kind: FloorPlan + metadata: + name: content-sources-backend + labels: + app: content-sources-backend + service: content-sources + spec: + database: + secretName: ${FLOORIST_DB_SECRET_NAME} + objectStore: + secretName: ${FLOORIST_BUCKET_SECRET_NAME} + logLevel: ${FLOORIST_LOGLEVEL} + suspend: ${{FLOORIST_SUSPEND}} + queries: + - prefix: ${FLOORIST_QUERY_PREFIX}/repositories + query: >- + select rc.account_id, rc.org_id, r.url, rc.created_at, rc.updated_at + from repository_configurations rc inner join + repositories r on rc.repository_uuid = r.uuid + +parameters: + - name: ENV_NAME + required: true + - name: IMAGE + value: quay.io/redhat-services-prod/insights-management-tenant/insights-content-sources/content-sources-backend + - name: WEEKLY_CRON_JOB + value: "0 8 * * 4" + - name: NIGHTLY_CRON_JOB + value: "0 0/1 * * *" + - name: CLEANUP_CRON_JOB + value: "0 1 * * *" + - name: SUSPEND_CRON_JOB + value: "false" + - name: IMAGE_TAG + required: true + - name: CPU_LIMIT_CS_API + value: 500m + - name: CPU_REQUEST_CS_API + value: 100m + - name: MEMORY_LIMIT_CS_API + value: 500Mi + - name: MEMORY_REQUEST_CS_API + value: 100Mi + - name: CPU_LIMIT_CS_WORKER + value: 500m + - name: CPU_REQUEST_CS_WORKER + value: 100m + - name: MEMORY_LIMIT_CS_WORKER + value: 1Gi + - name: MEMORY_REQUEST_CS_WORKER + value: 200Mi + - name: API_REPLICAS + value: "3" + - name: TASK_WORKER_REPLICAS + value: "3" + - name: LOGGING_LEVEL + value: debug + - name: CLIENTS_RBAC_BASE_URL + value: http://rbac-service:8000/api/rbac/v1 + required: true + - name: CLIENTS_RBAC_ENABLED + value: "True" + - name: FLOORIST_LOGLEVEL + description: Floorist loglevel config + value: 'INFO' + - name: FLOORIST_SUSPEND + description: Disable Floorist cronjob execution + value: 'false' + - name: FLOORIST_DB_SECRET_NAME + description: Name of the secret for accessing the database for floorist + value: "content-sources-db" + - name: FLOORIST_BUCKET_SECRET_NAME + description: Name of the secret for accessing the bucket for the floorist data dump + value: "floorist-bucket" + - name: FLOORIST_QUERY_PREFIX + description: Prefix for separating query data between prod and stage in the bucket + value: "hms_analytics/content-sources/unknown" + - name: OPTIONS_EXTERNAL_URL + required: true + description: URL (http://console.redhat.com) to access the api from + - name: CLIENTS_PULP_SERVER + description: Pulp Api URL (e.g. http://hostname:8080) + value: "http://pulp-api:8000" + - name: CLIENTS_PULP_USERNAME + description: Username for accessing pulp using basic auth + value: "contentsources" + - name: CLIENTS_PULP_PASSWORD + description: Password for accessing pulp over basic auth + - name: CLIENTS_PULP_DOWNLOAD_POLICY + description: the download policy to use in the environment (immediate or on_demand) + - name: FEATURES_SNAPSHOTS_ENABLED + description: Whether the Snapshots feature should be turned on + - name: FEATURES_SNAPSHOTS_ACCOUNTS + description: Comma separated list of account number that can access the feature + - name: FEATURES_SNAPSHOTS_ORGANIZATIONS + description: Comma separated list of org ids that can access the feature + - name: FEATURES_ADMIN_TASKS_ENABLED + description: Whether the Admin Tasks feature should be turned on + - name: FEATURES_ADMIN_TASKS_ACCOUNTS + description: Comma separated list of account number that can access the feature + - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS + description: Comma separated list of org ids that can access the feature + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + description: Option to make testing nightly snapshotting & introspection easier + default: 'false' + - name: OPTIONS_ENABLE_NOTIFICATIONS + description: Send notifications via kafka + default: 'false' + - name: OPTIONS_REPOSITORY_IMPORT_FILTER + description: Optionally filter preset repos that are imported + default: '' + - name: OPTIONS_CLIENTS_PULP_DATABASE_HOST + default: '' + - name: CLIENTS_PULP_DATABASE_PORT + default: '' + - name: CLIENTS_PULP_DATABASE_USER + default: '' + - name: CLIENTS_PULP_DATABASE_PASSWORD + default: '' + - name: CLIENTS_PULP_DATABASE_NAME + default: '' + - name: CLIENTS_PULP_DATABASE_POOL_LIMIT + default: 5 + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + default: 'false' + description: whether to create content guards when creating snapshots + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + default: "" + description: name to allow via turnpike authentication + - name: TASKING_WORKER_COUNT + default: 3 + description: Number of task workers running within a single worker process + - name: TASKING_POOL_LIMIT + default: 20 + description: Size of postgres connection pool for tasking system + - name: CLIENTS_CANDLEPIN_SERVER + default: '' + - name: CLIENTS_FEATURE_SERVICE_SERVER + default: '' + - name: OPTIONS_FEATURE_FILTER + description: Comma separated list of features that determine which repos to import + - name: OPTIONS_ENTITLE_ALL + description: Allow access to all features, only true for ephemeral + default: 'false' + - name: OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT + description: Number of days after which snapshots older then that will be cleaned up + - name: SUSPEND_TRANSFORM_PULP_LOGS + description: whether to not run the daily job to upload pulp logs + required: false + value: "false" + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP + description: cloud watch group name to parse pulp logs from + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_REGION + description: region to pull cloudwatch logs from + - name: CLIENTS_PULP_LOG_PARSER_S3_FILE_PREFIX + description: path prefix to store pulp logs at in s3 + - name: CLIENTS_ROADMAP_SERVER + description: URL to the roadmap service (i.e. https://console.stage.redhat.com/api/roadmap/v1) diff --git a/deployments/build/env-variables.yaml b/deployments/build/env-variables.yaml new file mode 100644 index 000000000..09a54f7b3 --- /dev/null +++ b/deployments/build/env-variables.yaml @@ -0,0 +1,196 @@ +# Environment variables organized by groups for content-sources-backend deployment +# This file contains all environment variables used across different deployments and jobs + +common: + # Core application settings + - name: CLOWDER_ENABLED + value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} + + # Certificates and secrets + - name: RH_CDN_CERT_PAIR + valueFrom: + secretKeyRef: + name: content-sources-certs + key: cdn.redhat.com + - name: SENTRY_DSN + valueFrom: + secretKeyRef: + name: content-sources-glitchtip + key: dsn + optional: true + + # Pulp client settings + - name: CLIENTS_PULP_SERVER + value: ${{CLIENTS_PULP_SERVER}} + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + value: ${{CLIENTS_PULP_GUARD_SUBJECT_DN}} + - name: CLIENTS_PULP_DOWNLOAD_POLICY + value: ${{CLIENTS_PULP_DOWNLOAD_POLICY}} + - name: CLIENTS_PULP_USERNAME + value: ${{CLIENTS_PULP_USERNAME}} + - name: CLIENTS_PULP_PASSWORD + valueFrom: + secretKeyRef: + name: pulp-content-sources-password + key: password + optional: true + + # RBAC settings + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} + + # Feature flags + - name: FEATURES_SNAPSHOTS_ENABLED + value: ${FEATURES_SNAPSHOTS_ENABLED} + - name: FEATURES_SNAPSHOTS_ACCOUNTS + value: ${FEATURES_SNAPSHOTS_ACCOUNTS} + - name: FEATURES_SNAPSHOTS_ORGANIZATIONS + value: ${FEATURES_SNAPSHOTS_ORGANIZATIONS} + - name: FEATURES_ADMIN_TASKS_ENABLED + value: ${FEATURES_ADMIN_TASKS_ENABLED} + - name: FEATURES_ADMIN_TASKS_ACCOUNTS + value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} + - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS + value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} + + # Candlepin settings + - name: CLIENTS_CANDLEPIN_SERVER + value: ${CLIENTS_CANDLEPIN_SERVER} + - name: CLIENTS_CANDLEPIN_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_CANDLEPIN_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + + # Feature service settings + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + + # Roadmap settings + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} + +service_specific: + # Service-specific variables (for API service) + - name: OPTIONS_REPOSITORY_IMPORT_FILTER + value: ${OPTIONS_REPOSITORY_IMPORT_FILTER} + - name: CLIENTS_PULP_DATABASE_HOST + valueFrom: + secretKeyRef: + name: pulp-db + key: db.host + optional: false + - name: CLIENTS_PULP_DATABASE_PORT + valueFrom: + secretKeyRef: + name: pulp-db + key: db.port + optional: false + - name: CLIENTS_PULP_DATABASE_USER + valueFrom: + secretKeyRef: + name: pulp-db + key: db.user + optional: false + - name: CLIENTS_PULP_DATABASE_PASSWORD + valueFrom: + secretKeyRef: + name: pulp-db + key: db.password + optional: false + - name: CLIENTS_PULP_DATABASE_NAME + valueFrom: + secretKeyRef: + name: pulp-db + key: db.name + optional: false + +job_specific: + # Job-specific variables for different cron jobs + process_repos: + - name: OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT + value: ${OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT} + + transform_pulp_logs: + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP + value: ${CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP} + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_REGION + value: ${CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_REGION} + - name: CLIENTS_PULP_LOG_PARSER_S3_FILE_PREFIX + value: ${CLIENTS_PULP_LOG_PARSER_S3_FILE_PREFIX} + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_KEY + valueFrom: + secretKeyRef: + name: content-sources-appsre-log-access-pulp + key: aws_access_key_id + optional: true + - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_SECRET + valueFrom: + secretKeyRef: + name: content-sources-appsre-log-access-pulp + key: aws_secret_access_key + optional: true + - name: CLIENTS_PULP_LOG_PARSER_S3_NAME + valueFrom: + secretKeyRef: + name: ${FLOORIST_BUCKET_SECRET_NAME} + key: bucket + optional: true + - name: CLIENTS_PULP_LOG_PARSER_S3_SECRET_KEY + valueFrom: + secretKeyRef: + name: ${FLOORIST_BUCKET_SECRET_NAME} + key: aws_secret_access_key + - name: CLIENTS_PULP_LOG_PARSER_S3_ACCESS_KEY + valueFrom: + secretKeyRef: + name: ${FLOORIST_BUCKET_SECRET_NAME} + key: aws_access_key_id + - name: CLIENTS_PULP_LOG_PARSER_S3_REGION + valueFrom: + secretKeyRef: + name: ${FLOORIST_BUCKET_SECRET_NAME} + key: aws_region \ No newline at end of file diff --git a/deployments/build/process-template.py b/deployments/build/process-template.py new file mode 100755 index 000000000..0cd66c51d --- /dev/null +++ b/deployments/build/process-template.py @@ -0,0 +1,141 @@ +#!/usr/bin/env python3 +""" +Template processor for content-sources-backend deployment.yaml + +This script reads the deployment.template.yaml file and env-variables.yaml file, +then combines them to generate the final deployment.yaml file with all +environment variables properly organized and deduplicated. +""" + +import yaml +import sys +import os +from typing import Dict, List, Any + + +def load_yaml_file(file_path: str) -> Dict[str, Any]: + """Load a YAML file and return its contents as a dictionary.""" + try: + with open(file_path, 'r', encoding='utf-8') as file: + return yaml.safe_load(file) + except FileNotFoundError: + print(f"Error: File {file_path} not found", file=sys.stderr) + sys.exit(1) + except yaml.YAMLError as e: + print(f"Error parsing YAML file {file_path}: {e}", file=sys.stderr) + sys.exit(1) + + +def format_env_vars(env_vars: List[Dict[str, Any]]) -> str: + """Format environment variables list as a YAML string.""" + if not env_vars: + return "" + + # Convert to YAML string + yaml_str = yaml.dump(env_vars, default_flow_style=False, sort_keys=False) + + # Add proper indentation for the env block + lines = yaml_str.split('\n') + indented_lines = [] + for line in lines: + if line.strip(): # Skip empty lines + indented_lines.append(' ' + line) + + return '\n'.join(indented_lines).rstrip() + + +def process_template(template_content: str, env_vars: Dict[str, Any]) -> str: + """Process the template content by replacing placeholders with environment variables.""" + + # Get the common environment variables + common_env = env_vars.get('common', []) + service_specific_env = env_vars.get('service_specific', []) + job_specific_env = env_vars.get('job_specific', {}) + + # Format environment variables + common_env_str = format_env_vars(common_env) + service_specific_env_str = format_env_vars(service_specific_env) + + # Get job-specific environment variables + process_repos_env = job_specific_env.get('process_repos', []) + transform_pulp_logs_env = job_specific_env.get('transform_pulp_logs', []) + + process_repos_env_str = format_env_vars(process_repos_env) + transform_pulp_logs_env_str = format_env_vars(transform_pulp_logs_env) + + # Replace placeholders in template + processed_content = template_content + + # Replace common environment variables (add newline if not empty) + if common_env_str: + processed_content = processed_content.replace('{{ENV_COMMON}}', '\n' + common_env_str) + else: + processed_content = processed_content.replace('{{ENV_COMMON}}', '') + + # Replace service-specific environment variables (add newline if not empty) + if service_specific_env_str: + processed_content = processed_content.replace('{{ENV_SERVICE_SPECIFIC}}', '\n' + service_specific_env_str) + else: + processed_content = processed_content.replace('{{ENV_SERVICE_SPECIFIC}}', '') + + # Replace job-specific environment variables (add newline if not empty) + if process_repos_env_str: + processed_content = processed_content.replace('{{ENV_JOB_PROCESS_REPOS}}', '\n' + process_repos_env_str) + else: + processed_content = processed_content.replace('{{ENV_JOB_PROCESS_REPOS}}', '') + + if transform_pulp_logs_env_str: + processed_content = processed_content.replace('{{ENV_JOB_TRANSFORM_PULP_LOGS}}', '\n' + transform_pulp_logs_env_str) + else: + processed_content = processed_content.replace('{{ENV_JOB_TRANSFORM_PULP_LOGS}}', '') + + return processed_content + + +def main(): + """Main function to process the template and generate deployment.yaml.""" + + # Get script directory + script_dir = os.path.dirname(os.path.abspath(__file__)) + + # Define file paths + template_file = os.path.join(script_dir, 'deployment.template.yaml') + env_vars_file = os.path.join(script_dir, 'env-variables.yaml') + output_file = os.path.join(os.path.dirname(script_dir), 'deployment.yaml') + + # Check if template file exists + if not os.path.exists(template_file): + print(f"Error: Template file {template_file} not found", file=sys.stderr) + sys.exit(1) + + # Check if env-variables file exists + if not os.path.exists(env_vars_file): + print(f"Error: Environment variables file {env_vars_file} not found", file=sys.stderr) + sys.exit(1) + + # Load files + print(f"Loading template file: {template_file}") + with open(template_file, 'r', encoding='utf-8') as file: + template_content = file.read() + + print(f"Loading environment variables file: {env_vars_file}") + env_vars = load_yaml_file(env_vars_file) + + # Process template + print("Processing template...") + processed_content = process_template(template_content, env_vars) + + # Write output + print(f"Writing output to: {output_file}") + with open(output_file, 'w', encoding='utf-8') as file: + file.write("# DO NOT EDIT THIS FILE DIRECTLY\n") + file.write("# This yaml file is generated from deployment.template.yaml\n") + file.write("# and env-variables.yaml in deployments/build.\n") + file.write("# Run 'make deployment-generate' to regenerate\n") + file.write(processed_content) + + print("Template processing completed successfully!") + + +if __name__ == "__main__": + main() diff --git a/deployments/deployment.yaml b/deployments/deployment.yaml index 3b0b9d56d..427361887 100644 --- a/deployments/deployment.yaml +++ b/deployments/deployment.yaml @@ -1,3 +1,7 @@ +# DO NOT EDIT THIS FILE DIRECTLY +# This yaml file is generated from deployment.template.yaml +# and env-variables.yaml in deployments/build. +# Run 'make deployment-generate' to regenerate # https://consoledot.pages.redhat.com/clowder/dev/api_reference.html --- apiVersion: template.openshift.io/v1 @@ -81,9 +85,25 @@ objects: periodSeconds: 5 successThreshold: 1 timeoutSeconds: 120 - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -110,13 +130,9 @@ objects: secretKeyRef: name: pulp-content-sources-password key: password - optional: false - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} + optional: true - name: CLIENTS_RBAC_BASE_URL value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -129,20 +145,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - - name: OPTIONS_REPOSITORY_IMPORT_FILTER - value: ${OPTIONS_REPOSITORY_IMPORT_FILTER} - - name: OPTIONS_FEATURE_FILTER - value: ${OPTIONS_FEATURE_FILTER} - - name: OPTIONS_ENTITLE_ALL - value: ${OPTIONS_ENTITLE_ALL} - - name: TASKING_WORKER_COUNT - value: ${TASKING_WORKER_COUNT} - - name: TASKING_POOL_LIMIT - value: ${TASKING_POOL_LIMIT} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -235,9 +237,25 @@ objects: periodSeconds: 5 successThreshold: 1 timeoutSeconds: 120 - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -265,12 +283,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - name: CLIENTS_RBAC_BASE_URL value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -283,42 +297,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - - name: OPTIONS_FEATURE_FILTER - value: ${OPTIONS_FEATURE_FILTER} - - name: CLIENTS_PULP_DATABASE_HOST - valueFrom: - secretKeyRef: - name: pulp-db - key: db.host - optional: false - - name: CLIENTS_PULP_DATABASE_PORT - valueFrom: - secretKeyRef: - name: pulp-db - key: db.port - optional: false - - name: CLIENTS_PULP_DATABASE_USER - valueFrom: - secretKeyRef: - name: pulp-db - key: db.user - optional: false - - name: CLIENTS_PULP_DATABASE_PASSWORD - valueFrom: - secretKeyRef: - name: pulp-db - key: db.password - optional: false - - name: CLIENTS_PULP_DATABASE_NAME - valueFrom: - secretKeyRef: - name: pulp-db - key: db.name - optional: false - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -355,6 +333,38 @@ objects: optional: true - name: CLIENTS_ROADMAP_SERVER value: ${CLIENTS_ROADMAP_SERVER} + - name: OPTIONS_REPOSITORY_IMPORT_FILTER + value: ${OPTIONS_REPOSITORY_IMPORT_FILTER} + - name: CLIENTS_PULP_DATABASE_HOST + valueFrom: + secretKeyRef: + name: pulp-db + key: db.host + optional: false + - name: CLIENTS_PULP_DATABASE_PORT + valueFrom: + secretKeyRef: + name: pulp-db + key: db.port + optional: false + - name: CLIENTS_PULP_DATABASE_USER + valueFrom: + secretKeyRef: + name: pulp-db + key: db.user + optional: false + - name: CLIENTS_PULP_DATABASE_PASSWORD + valueFrom: + secretKeyRef: + name: pulp-db + key: db.password + optional: false + - name: CLIENTS_PULP_DATABASE_NAME + valueFrom: + secretKeyRef: + name: pulp-db + key: db.name + optional: false resources: limits: cpu: ${CPU_LIMIT_CS_API} @@ -379,9 +389,25 @@ objects: command: - /jobs - create-latest-distributions - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -390,11 +416,15 @@ objects: - name: SENTRY_DSN valueFrom: secretKeyRef: - name: content-sources-sentry + name: content-sources-glitchtip key: dsn optional: true - name: CLIENTS_PULP_SERVER value: ${{CLIENTS_PULP_SERVER}} + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + value: ${{CLIENTS_PULP_GUARD_SUBJECT_DN}} - name: CLIENTS_PULP_DOWNLOAD_POLICY value: ${{CLIENTS_PULP_DOWNLOAD_POLICY}} - name: CLIENTS_PULP_USERNAME @@ -405,10 +435,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -421,12 +449,42 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: CLIENTS_CANDLEPIN_SERVER + value: ${CLIENTS_CANDLEPIN_SERVER} + - name: CLIENTS_CANDLEPIN_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_CANDLEPIN_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: retry-failed-tasks podSpec: securityContext: @@ -437,9 +495,25 @@ objects: command: - /jobs - retry-failed-tasks - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -448,7 +522,7 @@ objects: - name: SENTRY_DSN valueFrom: secretKeyRef: - name: content-sources-sentry + name: content-sources-glitchtip key: dsn optional: true - name: CLIENTS_PULP_SERVER @@ -467,10 +541,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -483,12 +555,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -503,6 +569,28 @@ objects: name: content-sources-candlepin key: key optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: process-repos # https://crontab.guru/ schedule: ${NIGHTLY_CRON_JOB} @@ -519,22 +607,38 @@ objects: - process-repos - --interval - "96" - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} - - name: RH_CDN_CERT_PAIR - valueFrom: - secretKeyRef: - name: content-sources-certs - key: cdn.redhat.com - - name: SENTRY_DSN - valueFrom: - secretKeyRef: - name: content-sources-sentry - key: dsn - optional: true - - name: CLIENTS_PULP_SERVER - value: ${{CLIENTS_PULP_SERVER}} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} + - name: RH_CDN_CERT_PAIR + valueFrom: + secretKeyRef: + name: content-sources-certs + key: cdn.redhat.com + - name: SENTRY_DSN + valueFrom: + secretKeyRef: + name: content-sources-glitchtip + key: dsn + optional: true + - name: CLIENTS_PULP_SERVER + value: ${{CLIENTS_PULP_SERVER}} - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} - name: CLIENTS_PULP_GUARD_SUBJECT_DN @@ -549,10 +653,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -565,14 +667,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - - name: OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT - value: ${OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -587,6 +681,30 @@ objects: name: content-sources-candlepin key: key optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} + - name: OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT + value: ${OPTIONS_SNAPSHOT_RETAIN_DAYS_LIMIT} - name: cleanup # https://crontab.guru/ schedule: ${CLEANUP_CRON_JOB} @@ -603,9 +721,44 @@ objects: - cleanup - --exclude - "pulp-orphan" - env: + env: + - name: CLOWDER_ENABLED + value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} + - name: RH_CDN_CERT_PAIR + valueFrom: + secretKeyRef: + name: content-sources-certs + key: cdn.redhat.com + - name: SENTRY_DSN + valueFrom: + secretKeyRef: + name: content-sources-glitchtip + key: dsn + optional: true - name: CLIENTS_PULP_SERVER value: ${{CLIENTS_PULP_SERVER}} + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + value: ${{CLIENTS_PULP_GUARD_SUBJECT_DN}} + - name: CLIENTS_PULP_DOWNLOAD_POLICY + value: ${{CLIENTS_PULP_DOWNLOAD_POLICY}} - name: CLIENTS_PULP_USERNAME value: ${{CLIENTS_PULP_USERNAME}} - name: CLIENTS_PULP_PASSWORD @@ -614,8 +767,56 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} + - name: FEATURES_SNAPSHOTS_ENABLED + value: ${FEATURES_SNAPSHOTS_ENABLED} + - name: FEATURES_SNAPSHOTS_ACCOUNTS + value: ${FEATURES_SNAPSHOTS_ACCOUNTS} + - name: FEATURES_SNAPSHOTS_ORGANIZATIONS + value: ${FEATURES_SNAPSHOTS_ORGANIZATIONS} + - name: FEATURES_ADMIN_TASKS_ENABLED + value: ${FEATURES_ADMIN_TASKS_ENABLED} + - name: FEATURES_ADMIN_TASKS_ACCOUNTS + value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} + - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS + value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} + - name: CLIENTS_CANDLEPIN_SERVER + value: ${CLIENTS_CANDLEPIN_SERVER} + - name: CLIENTS_CANDLEPIN_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_CANDLEPIN_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: pulp-orphan-cleanup # https://crontab.guru/ schedule: ${WEEKLY_CRON_JOB} @@ -632,9 +833,25 @@ objects: - cleanup - --type - "pulp-orphan" - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -643,7 +860,7 @@ objects: - name: SENTRY_DSN valueFrom: secretKeyRef: - name: content-sources-sentry + name: content-sources-glitchtip key: dsn optional: true - name: CLIENTS_PULP_SERVER @@ -662,10 +879,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -678,12 +893,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -698,6 +907,28 @@ objects: name: content-sources-candlepin key: key optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: transform-pulp-logs schedule: "0 1 * * *" suspend: ${{SUSPEND_TRANSFORM_PULP_LOGS}} @@ -711,11 +942,102 @@ objects: command: - /jobs - transform-pulp-logs - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} - name: LOGGING_LEVEL value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} + - name: RH_CDN_CERT_PAIR + valueFrom: + secretKeyRef: + name: content-sources-certs + key: cdn.redhat.com + - name: SENTRY_DSN + valueFrom: + secretKeyRef: + name: content-sources-glitchtip + key: dsn + optional: true + - name: CLIENTS_PULP_SERVER + value: ${{CLIENTS_PULP_SERVER}} + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + value: ${{CLIENTS_PULP_GUARD_SUBJECT_DN}} + - name: CLIENTS_PULP_DOWNLOAD_POLICY + value: ${{CLIENTS_PULP_DOWNLOAD_POLICY}} + - name: CLIENTS_PULP_USERNAME + value: ${{CLIENTS_PULP_USERNAME}} + - name: CLIENTS_PULP_PASSWORD + valueFrom: + secretKeyRef: + name: pulp-content-sources-password + key: password + optional: true + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} + - name: FEATURES_SNAPSHOTS_ENABLED + value: ${FEATURES_SNAPSHOTS_ENABLED} + - name: FEATURES_SNAPSHOTS_ACCOUNTS + value: ${FEATURES_SNAPSHOTS_ACCOUNTS} + - name: FEATURES_SNAPSHOTS_ORGANIZATIONS + value: ${FEATURES_SNAPSHOTS_ORGANIZATIONS} + - name: FEATURES_ADMIN_TASKS_ENABLED + value: ${FEATURES_ADMIN_TASKS_ENABLED} + - name: FEATURES_ADMIN_TASKS_ACCOUNTS + value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} + - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS + value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} + - name: CLIENTS_CANDLEPIN_SERVER + value: ${CLIENTS_CANDLEPIN_SERVER} + - name: CLIENTS_CANDLEPIN_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_CANDLEPIN_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP value: ${CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP} - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_REGION @@ -769,11 +1091,102 @@ objects: - transform-pulp-logs - "2025-05-09" - "24" - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} - name: LOGGING_LEVEL value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} + - name: RH_CDN_CERT_PAIR + valueFrom: + secretKeyRef: + name: content-sources-certs + key: cdn.redhat.com + - name: SENTRY_DSN + valueFrom: + secretKeyRef: + name: content-sources-glitchtip + key: dsn + optional: true + - name: CLIENTS_PULP_SERVER + value: ${{CLIENTS_PULP_SERVER}} + - name: CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS + value: ${CLIENTS_PULP_CUSTOM_REPO_CONTENT_GUARDS} + - name: CLIENTS_PULP_GUARD_SUBJECT_DN + value: ${{CLIENTS_PULP_GUARD_SUBJECT_DN}} + - name: CLIENTS_PULP_DOWNLOAD_POLICY + value: ${{CLIENTS_PULP_DOWNLOAD_POLICY}} + - name: CLIENTS_PULP_USERNAME + value: ${{CLIENTS_PULP_USERNAME}} + - name: CLIENTS_PULP_PASSWORD + valueFrom: + secretKeyRef: + name: pulp-content-sources-password + key: password + optional: true + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} + - name: FEATURES_SNAPSHOTS_ENABLED + value: ${FEATURES_SNAPSHOTS_ENABLED} + - name: FEATURES_SNAPSHOTS_ACCOUNTS + value: ${FEATURES_SNAPSHOTS_ACCOUNTS} + - name: FEATURES_SNAPSHOTS_ORGANIZATIONS + value: ${FEATURES_SNAPSHOTS_ORGANIZATIONS} + - name: FEATURES_ADMIN_TASKS_ENABLED + value: ${FEATURES_ADMIN_TASKS_ENABLED} + - name: FEATURES_ADMIN_TASKS_ACCOUNTS + value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} + - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS + value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} + - name: CLIENTS_CANDLEPIN_SERVER + value: ${CLIENTS_CANDLEPIN_SERVER} + - name: CLIENTS_CANDLEPIN_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_CANDLEPIN_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP value: ${CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_GROUP} - name: CLIENTS_PULP_LOG_PARSER_CLOUDWATCH_REGION @@ -823,9 +1236,25 @@ objects: command: - /jobs - cleanup-missing-domains - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -834,7 +1263,7 @@ objects: - name: SENTRY_DSN valueFrom: secretKeyRef: - name: content-sources-sentry + name: content-sources-glitchtip key: dsn optional: true - name: CLIENTS_PULP_SERVER @@ -853,10 +1282,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -869,12 +1296,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -889,6 +1310,28 @@ objects: name: content-sources-candlepin key: key optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} - name: set-domain-label podSpec: securityContext: @@ -899,9 +1342,25 @@ objects: command: - /jobs - set-domain-label - env: + env: - name: CLOWDER_ENABLED value: ${CLOWDER_ENABLED} + - name: LOGGING_LEVEL + value: ${{LOGGING_LEVEL}} + - name: OPTIONS_EXTERNAL_URL + value: ${OPTIONS_EXTERNAL_URL} + - name: OPTIONS_ALWAYS_RUN_CRON_TASKS + value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} + - name: OPTIONS_ENABLE_NOTIFICATIONS + value: ${OPTIONS_ENABLE_NOTIFICATIONS} + - name: OPTIONS_FEATURE_FILTER + value: ${OPTIONS_FEATURE_FILTER} + - name: OPTIONS_ENTITLE_ALL + value: ${OPTIONS_ENTITLE_ALL} + - name: TASKING_WORKER_COUNT + value: ${TASKING_WORKER_COUNT} + - name: TASKING_POOL_LIMIT + value: ${TASKING_POOL_LIMIT} - name: RH_CDN_CERT_PAIR valueFrom: secretKeyRef: @@ -910,7 +1369,7 @@ objects: - name: SENTRY_DSN valueFrom: secretKeyRef: - name: content-sources-sentry + name: content-sources-glitchtip key: dsn optional: true - name: CLIENTS_PULP_SERVER @@ -929,10 +1388,8 @@ objects: name: pulp-content-sources-password key: password optional: true - - name: LOGGING_LEVEL - value: ${{LOGGING_LEVEL}} - - name: OPTIONS_EXTERNAL_URL - value: ${OPTIONS_EXTERNAL_URL} + - name: CLIENTS_RBAC_BASE_URL + value: ${{CLIENTS_RBAC_BASE_URL}} - name: FEATURES_SNAPSHOTS_ENABLED value: ${FEATURES_SNAPSHOTS_ENABLED} - name: FEATURES_SNAPSHOTS_ACCOUNTS @@ -945,12 +1402,6 @@ objects: value: ${FEATURES_ADMIN_TASKS_ACCOUNTS} - name: FEATURES_ADMIN_TASKS_ORGANIZATIONS value: ${FEATURES_ADMIN_TASKS_ORGANIZATIONS} - - name: CLIENTS_RBAC_BASE_URL - value: ${{CLIENTS_RBAC_BASE_URL}} - - name: OPTIONS_ALWAYS_RUN_CRON_TASKS - value: ${OPTIONS_ALWAYS_RUN_CRON_TASKS} - - name: OPTIONS_ENABLE_NOTIFICATIONS - value: ${OPTIONS_ENABLE_NOTIFICATIONS} - name: CLIENTS_CANDLEPIN_SERVER value: ${CLIENTS_CANDLEPIN_SERVER} - name: CLIENTS_CANDLEPIN_CLIENT_CERT @@ -965,6 +1416,28 @@ objects: name: content-sources-candlepin key: key optional: true + - name: CLIENTS_CANDLEPIN_CA_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: ca + optional: true + - name: CLIENTS_FEATURE_SERVICE_SERVER + value: ${CLIENTS_FEATURE_SERVICE_SERVER} + - name: CLIENTS_FEATURE_SERVICE_CLIENT_CERT + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: cert + optional: true + - name: CLIENTS_FEATURE_SERVICE_CLIENT_KEY + valueFrom: + secretKeyRef: + name: content-sources-candlepin + key: key + optional: true + - name: CLIENTS_ROADMAP_SERVER + value: ${CLIENTS_ROADMAP_SERVER} database: name: content-sources version: 15 @@ -1150,4 +1623,4 @@ parameters: - name: CLIENTS_PULP_LOG_PARSER_S3_FILE_PREFIX description: path prefix to store pulp logs at in s3 - name: CLIENTS_ROADMAP_SERVER - description: URL to the roadmap service (i.e. https://console.stage.redhat.com/api/roadmap/v1) + description: URL to the roadmap service (i.e. https://console.stage.redhat.com/api/roadmap/v1) diff --git a/mk/deployment.mk b/mk/deployment.mk new file mode 100644 index 000000000..247e2bc60 --- /dev/null +++ b/mk/deployment.mk @@ -0,0 +1,13 @@ +# Deployment-related targets + +.PHONY: deployment-generate deployment-clean deployment-validate deployment-diff deployment-help + +deployment-generate: ## Generate deployment.yaml from template and environment variables + @echo "Generating deployment.yaml from template and environment variables..." + @python3 deployments/build/process-template.py + @echo "Deployment file generated: deployments/deployment.yaml" + +deployment-validate: deployment-generate ## Validate the generated deployment template + @echo "Validating generated deployment template..." + @oc process --local -f deployments/deployment.yaml > /dev/null + @echo "Deployment template validation successful" diff --git a/mk/includes.mk b/mk/includes.mk index 319c2b4e2..b9786c7fa 100644 --- a/mk/includes.mk +++ b/mk/includes.mk @@ -24,6 +24,7 @@ include mk/variables.mk include mk/help.mk include mk/meta-general.mk include mk/go-rules.mk +include mk/playwright.mk include mk/printvars.mk include mk/plantuml.mk include mk/pkg-deps-graph.mk @@ -43,4 +44,5 @@ include mk/prometheus.mk include mk/pulp.mk include mk/meta-compose.mk include mk/compose.mk -include mk/playwright.mk +include mk/meta-deployment.mk +include mk/deployment.mk diff --git a/mk/meta-deployment.mk b/mk/meta-deployment.mk new file mode 100644 index 000000000..e30798f5a --- /dev/null +++ b/mk/meta-deployment.mk @@ -0,0 +1 @@ +##@ Manage deployment.yaml