Skip to content

Conversation

@vycius
Copy link

@vycius vycius commented Oct 20, 2025

When exporting repositories from one Pulp instance and importing them on another, multiple exports may arrive simultaneously. It is currently difficult to identify key details (e.g., repository name, export type: full vs. incremental) without inspecting large tar files. This slows down automation and makes import order logic harder.

Closes #6712

Solution

This change introduces a new optional meta field to the export serializer.

  • Accepts arbitrary key-value metadata at export time.
  • Metadata is written into the toc.json file under the meta section.
  • The same metadata is also stored in toc_info for quick lookup.

This allows exporters/importers to embed identifying information (e.g., repo name, export type, author) directly in the TOC, making exports easier to classify and manage.

Example

Export request

client.post(
    f"{exporter_href}exports/",
    json={
        "full": full_export,
        "chunk_size": chunk_size,
        "meta": {
            "repositories": ["example/repository"],
            "author": "pulp"        
        },
    },
)

TOC json

{
  "meta": {
    "repositories": ["example/repository"],
    "author": "pulp",
    "checksum_type": "crc32",
    "chunk_size": 53687091200
  },
  "files": {
    "export-019a01ce-20251020_1328.tar.0000": "96903fec"
  }
}

Exporter list output

{
  "params": {
    "full": true,
    "meta": {
      "repositories": [
        "example/repository"
      ],
      "author": "pulp"
    },
  },
  "toc_info": {
    "file": "/export-019a01d7-20251020_1338-toc.json",
    "meta": {
      "repositories": [
        "example/repository"
      ],
      "author": "pulp"
    },
    "sha256": "49c9ea862f67efc04c4"
  }
  ...
}

@dralley
Copy link
Contributor

dralley commented Oct 20, 2025

@vycius Could you please rebase on top of the current state of the main branch?

@vycius vycius force-pushed the export-meta-support branch from 0ea538d to 7e999ad Compare October 21, 2025 04:20
@vycius
Copy link
Author

vycius commented Oct 21, 2025

@vycius Could you please rebase on top of the current state of the main branch?

Rebased.

@ggainey ggainey closed this Oct 22, 2025
@ggainey ggainey reopened this Oct 22, 2025
Copy link
Contributor

@ggainey ggainey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable, but def needs tests. You can add here https://github.com/pulp/pulpcore/blob/main/pulp_file/tests/functional/api/test_pulp_export.py

@vycius vycius force-pushed the export-meta-support branch from 7e999ad to 4dd9a41 Compare October 25, 2025 06:46
@github-actions github-actions bot removed the no-issue label Oct 25, 2025
@vycius vycius force-pushed the export-meta-support branch 2 times, most recently from 037749a to 32a6814 Compare October 25, 2025 06:50
Allow adding custom metadata to exports

closes pulp#6712
@vycius vycius force-pushed the export-meta-support branch from 32a6814 to 3d6f317 Compare October 25, 2025 06:59
@vycius
Copy link
Author

vycius commented Oct 25, 2025

This looks reasonable, but def needs tests. You can add here https://github.com/pulp/pulpcore/blob/main/pulp_file/tests/functional/api/test_pulp_export.py

Tests added

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tags inside toc.json for easier export identification

3 participants