Final command-line tweaks and documentation

Author: gershnik

README.md

@@ -13,7 +13,7 @@ A portable Python library to generate binary software repositories
 
 Ever needed to build an APT package repository on Fedora? Or perhaps a DNF repository on Debian? How about FreeBSD repository on Windows or Mac? This library allows you to do all these things and more. And yes, you can do it even on Windows if you are so inclined for some reason.
 
-All binary package repositories have their own tools that usually range from being "non-portable" to "portable with lots of effort to limited platforms only". On the other hand it is often convenient to build software packages in a Map/Reduce fashion where a single host collects multiple packages built for different platforms to produce binary repositories. Such host will necessarily need to be able to build repositories for "foreign" packages. This library is an attempt to enable such scenario.
+All binary package repositories have their own tools that usually range from being "non-portable" to "portable with lots of effort to limited platforms only". On the other hand it is often convenient to build software packages in a Map/Reduce fashion where a single host collects multiple packages built for different platforms to produce binary repositories. Such host will necessarily need to be able to build repositories for "foreign" packages. This library is an attempt to enable such scenario. It provides both programmatic and command-line access.
 
 ## Requirements
 
@@ -35,23 +35,15 @@ All binary package repositories have their own tools that usually range from bei
 pip install repopulator
 ```
 
-### Documentation
+## Documentation
 
-Reference for public API is available at https://gershnik.github.io/repopulator/
+Documentation for API and command-line syntax is available at https://gershnik.github.io/repopulator/
 
-### Sample Usage
+## Examples
 
-The basic outline of the usage is the same for all repository types:
-- Create the repository object
-- Add packages to it. These must be files somewhere on your filesystem *which is not their final destination*
-- Some repositories like APT have additional subdivisions (distributions for APT). If so you need to create them and assign packages added to repository to them
-- Export the repository to the destination folder. This overwrites any repository already there (but not any extra files you might have). 
+### APT
 
-That's all there is to it. Note that there is no ability to "load" existing repositories and change them. This is deliberate. If you want to do incremental repository maintenance it is far easier to keep necessary info yourself in your own format than to parse it out of various repositories. 
-
-Currently repositories are required to be signed and you need to provide signing info for export (see examples below). This requirement might be relaxed in future versions.
-
-#### APT
+#### Programmatic
 
 ```python
 from repopulator import AptRepo, PgpSigner
@@ -61,12 +53,7 @@ repo = AptRepo()
 package1 = repo.add_package('/path/to/awesome_3.14_amd64.deb')
 package2 = repo.add_package('/path/to/awesome_3.14_arm64.deb')
 
-dist = repo.add_distribution('jammy', 
-                             origin='my packages', 
-                             label='my apt repo', 
-                             suite='jammy', 
-                             version='1.2', 
-                             description='my awesome repo')
+dist = repo.add_distribution('jammy')
 
 repo.assign_package(package1, dist, component='main')
 repo.assign_package(package2, dist, component='main')
@@ -77,7 +64,17 @@ repo.export('/path/of/new/repo', signer)
 
 ```
 
-#### RPM
+#### Command-line
+
+```bash
+repopulator apt -o /path/of/new/repo -k name_of_key_to_use -w password_of_that_key \
+  -d jammy -c main \
+  -p /path/to/awesome_3.14_amd64.deb /path/to/awesome_3.14_arm64.deb
+```
+
+### RPM
+
+#### Programmatic
 
 ```python
 from repopulator import RpmRepo, PgpSigner
@@ -92,14 +89,21 @@ repo.export('/path/of/new/repo', signer)
 
 ```
 
-#### Pacman
+#### Command-line
+
+```bash
+repopulator rpm -o /path/of/new/repo -k name_of_key_to_use -w password_of_that_key \
+  -p /path/to/awesome-3.14-1.el9.x86_64.rpm /path/to/awesome-3.14-1.el9.aarch64.rpm
+```
+
+### Pacman
+
+#### Programmatic
 
 ```python
 from repopulator import PacmanRepo, PgpSigner
 
 repo = PacmanRepo('myrepo')
-# if .sig file is present next to the .zst file it will be used for signature
-# otherwise new signature will be generated at export time
 repo.add_package('/path/to/awesome-3.14-1-x86_64.pkg.tar.zst')
 repo.add_package('/path/to/another-1.2-1-x86_64.pkg.tar.zst')
 
@@ -109,7 +113,16 @@ repo.export('/path/of/new/repo', signer)
 
 ```
 
-#### Alpine apk
+#### Command-line
+
+```bash
+repopulator pacman -o /path/of/new/repo -k name_of_key_to_use -w password_of_that_key \
+    -n myrepo -p /path/to/awesome-3.14-1-x86_64.pkg.tar.zst /path/to/another-1.2-1-x86_64.pkg.tar.zst
+```
+
+### Alpine apk
+
+#### Programmatic
 
 ```python
 from repopulator import AlpineRepo, PkiSigner
@@ -126,7 +139,18 @@ repo.export('/path/of/new/repo', signer, signer_name = '[email protected]
 
 ```
 
-#### FreeBSD pkg
+#### Command-line
+
+```bash
+repopulator alpine -o /path/of/new/repo -d 'my repo description' \
+  -k /path/to/private/key.rsa -w password_of_that_key \
+  -s '[email protected]' \
+  -p /path/to/awesome-3.14-r0.apk /path/to/another-1.23-r0.apk
+```
+
+### FreeBSD pkg
+
+#### Programmatic
 
 ```python
 from repopulator import FreeBSDRepo, PkiSigner
@@ -141,3 +165,11 @@ repo.export('/path/of/new/repo', signer)
 
 ```
 
+#### Command-line
+
+```bash
+repopulator freebsd -o /path/of/new/repo \
+  -k /path/to/private/key -w password_of_that_key \
+  -p /path/to/awesome-3.14.pkg /path/to/another-1.2.pkg
+```
+

docs/command-line.md

@@ -0,0 +1,187 @@
+# Command-Line Interface
+
+## General
+
+The `repopulator` module provides a simple command-line interface to create repositories.
+
+You can invoke the command-line either via a script
+```bash
+$ repopulator
+```
+or as a module
+```bash
+$ python3 -m repopulator
+```
+
+The general form of the command line is:
+
+```bash
+$ repopulator TYPE -o DEST [options...] -p package1 package2 ....
+```
+
+where `TYPE` is one of: `alpine`, `apt`, `freebsd`, `pacman`, `rpm` and `DEST` is the destination directory for the repository.
+
+You can obtain overall help by using
+```bash
+$ repopulator -h/--help
+```
+
+and a help about available options for each repository via:
+```bash
+$ repopulator TYPE -h/--help
+```
+
+Options and their effect for each repository type are described below
+
+## Alpine
+
+The general command-line form for Alpine pkg repository is:
+
+```bash
+$ repopulator alpine -o DEST -d DESC -k KEY_PATH [-w KEY_PASSWORD] [-s SIGNER] \
+     -p package1 package2 ... \
+     -a ARCH1 -p package3 package4 ... \
+     -a ARCH2 -p package5 package6 ...
+
+```
+
+Where:
+
+`-d DESC`, `--desc DESC`
+: The repository description
+
+`-k KEY_PATH`, `--key KEY_PATH`
+: The path to private key for signing. If `-s/--signer` option is not supplied the stem of the private key filename is used as the name. So for example a key `[email protected]` will result in `[email protected]` being used as a signer name.
+
+`-w KEY_PASSWORD`, `--password KEY_PASSWORD`
+: The password for the private key, if needed
+
+`-s SIGNER`, `--signer SIGNER`
+: The signer name that overrides automatic deduction from the key filename described above
+
+`-a ARCH`, `--arch ARCH`
+: Forces the architecture of the _following_ packages to be `ARCH`. 
+
+`-p path ...`, `--package path...`
+: `.apk` packages to add
+
+By default, internal architecture of the package is used to decide under which architecture to register it in the repo. 
+Some packages (such as `-doc-`, `-openrc-` etc.) do not have specific architecture and are marked as `noarch`. All Alpine packages in a repo must belong to some architecture so you need to use `-a ARCH` with them. 
+
+If you wish to "stop" the latest `-a ARCH` effect and revert to using architecture of the package use `-a` without an argument.
+
+## APT
+
+The general command-line form for APT repository is:
+
+```bash
+$ repopulator apt -o DEST -k KEY_NAME -w KEY_PASSWORD \
+    -d DISTRO1 \
+      [--origin ORIGIN] [--label LABEL] [--suite SUITE] \
+        [--codename CODENAME] [--version VERSION] [--desc DESC] \
+      [-c COMPONENT1] \
+          -p package1 package2 ... \
+      [-c COMPONENT2] \
+          -p package3 package4 ... \
+    -d DISTRO2 \
+    ...
+```
+
+Where:
+
+`-k KEY_NAME`, `--key KEY_NAME`
+: Name or ID of the GPG key for signing
+
+`-w KEY_PASSWORD`, `--password KEY_PASSWORD`
+: GPG key password
+
+`-d DISTRO`, `--distro DISTRO`
+: Starts a new distribution named `DISTRO` (e.g. `jammy` or `focal`). All subsequent arguments refer to this distribution until the next `-d/--distro` argument. The distribution name can be a path like `stable/updates`
+
+`--origin ORIGIN`
+: Optional `Origin` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Origin
+
+`--label LABEL`
+: Optional `Label` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Label
+
+`--suite SUITE`
+: Optional `Suite` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Suite. If omitted defaults to the last component of distribution path.
+
+`--codename CODENAME`
+: Optional `Codename` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Codename. If omitted defaults to the last component of distribution path.
+
+`--version VERSION`
+: Optional `Version` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Version. If omitted defaults to the last component of distribution path.
+
+`--desc DESC`
+: Optional `Description` field for the distribution. See https://wiki.debian.org/DebianRepository/Format#Description. If omitted defaults to the last component of distribution path.
+
+`-c COMPONENT`, `--comp COMPONENT`
+: Optional component of the _following_ packages. If not specified or component name is omitted defaults to `main`. You can specify multiple components for a distribution. 
+
+`-p path ...`, `--package path...`
+: `.deb` (or `.udeb`) packages to add to the current distribution and component
+
+## FreeBSD
+
+The general command-line form for FreeBSD repository is:
+
+```bash
+$ repopulator freebsd -o DEST -k KEY_PATH [-w KEY_PASSWORD] \
+     -p package1 package2 ... 
+```
+
+Where:
+
+`-k KEY_PATH`, `--key KEY_PATH`
+: The path to private key for signing. 
+
+`-w KEY_PASSWORD`, `--password KEY_PASSWORD`
+: The password for the private key, if needed
+
+`-p path ...`, `--package path...`
+: `.pkg` packages to add
+
+
+## Pacman
+
+The general command-line form for Pacman repository is:
+
+```bash
+$ repopulator pacman -o DEST -k KEY_NAME -w KEY_PASSWORD \
+    -n name -p package1 package2 ...
+```
+
+Where:
+
+`-k KEY_NAME`, `--key KEY_NAME`
+: Name or ID of the GPG key for signing
+
+`-w KEY_PASSWORD`, `--password KEY_PASSWORD`
+: GPG key password
+
+`-n NAME`, `--name NAME`
+: Repository name
+
+`-p path ...`, `--package path...`
+: `.zst` packages to add. If a matching .sig file with the same name exists next to the package, it will be automatically used to supply the package signature
+
+## RPM
+
+The general command-line form for Pacman repository is:
+
+```bash
+$ repopulator rpm -o DEST -k KEY_NAME -w KEY_PASSWORD \
+    -p package1 package2 ...
+```
+
+Where:
+
+`-k KEY_NAME`, `--key KEY_NAME`
+: Name or ID of the GPG key for signing
+
+`-w KEY_PASSWORD`, `--password KEY_PASSWORD`
+: GPG key password
+
+`-p path ...`, `--package path...`
+: `.rpm` packages to add. 

docs/index.md

@@ -18,10 +18,8 @@ This is done in a portable fashion without relying on any platform and distribut
 * Alpine apk
 * FreeBSD pkg
 
+## Documentation
 
-## Reference
-
-::: repopulator
-options:
-    summary:
-    modules: false
+* [Usage](usage.md)
+* [Reference](reference.md)
+* [Command-Line Interface](command-line.md)

docs/license.md

@@ -0,0 +1,3 @@
+# License
+
+{!LICENSE.txt!}

docs/reference.md

@@ -0,0 +1,7 @@
+# Reference
+
+::: repopulator
+options:
+    summary:
+    modules: false
+