APT repository Packages metadata stops getting updated and becomes out of sync with uploaded packages and reindex does not force rebuild of the Packages metadata

[ju2wheels]

• What problem are you trying to solve?

I reported this issue multiple versions ago on the old bug report JIRA, but this issue is still present on 3.70.1-02 . The repo metadata Packages* files sometimes stop getting updated or are updated with invalid data when you upload deb packages to the repo.

In previous versions of Nexus, this out sync metadata in the Packages file sometimes applied to just Packages and not the bz2 or gz versions of the file so it was inconsistent even across the three files present which causes even more weird issues where apt was not able to find packages clearly in the repo because these packages index files did not include the package at all.

In my most recent iteration of hitting this issue, APT gets Hash mismatch errors because the uploaded package will show the proper hash data such as:

package_name
python3-asus-nuc-wmi
package_version
1.1.0-1
index_section
Package: python3-asus-nuc-wmi Source: asus-nuc-wmi Version: 1.1.0-1 Architecture: all Maintainer: Julio Lajara <[email protected]> Installed-Size: 77 Depends: python3:any (>= 3.3.2-2~) Replaces: python-asus-nuc-wmi Section: python Priority: optional Homepage: https://github.com/tvision-insights/intel_nuc_led Description: ASUS NUC WMI CLI userland for asus_nuc_wmi kernel module # asus_nuc_wmi Python userland for ASUS NUC WMI kernel module . ## Compatibility . This `asus_nuc_wmi` userland was written from the available ASUS NUC WMI guide for the NUC 14 (included in the [contrib/reference/](../reference) folder). . It has been tested on NUC 14. . Although we followed the specification documents, we have found that compatibility varies by a number of factors: . * BIOS version (for some settings, the BIOS version may impact what options are available). * BIOS configuration (for some settings, the BIOS configuration for LEDs can affect whether they are usable and in what default state they are in). * BIOS bugs (we have found some device BIOS have bugs where LEDs which should support RGB are only capable of dual color mode through WMI, but are capable of RGB when manually configuring them via the BIOS only). . Aside from the above, command options can change based on the combination of what the BIOS allows and what indicator option mode LEDs are put in. Filename: pool/p/python3-asus-nuc-wmi/python3-asus-nuc-wmi_1.1.0-1_all.deb Size: 11664 MD5Sum: ed0c9c59dcacf86c4a5e1957303890cb SHA1: dd78e65a4425e7c0eb6b0a2d8c6f8a139ef65174 SHA256: 661bea49a78ff7d693ddbb214207b42e2f064822db15fa55ba7f29aa4afdc8f3
asset_kind
DEB
architecture
all
Checksum
sha1
dd78e65a4425e7c0eb6b0a2d8c6f8a139ef65174
sha256
661bea49a78ff7d693ddbb214207b42e2f064822db15fa55ba7f29aa4afdc8f3
md5
ed0c9c59dcacf86c4a5e1957303890cb
Content
last_modified
2024-12-06T18:10:21.430Z
Provenance
hashes_not_verified
false

while when I go look at the Packages metadata file it shows this which doesnt even properly match the package:

Package: python3-asus-nuc-wmi
Source: asus-nuc-wmi
Version: 1.1.0-1
Architecture: all
Maintainer: Julio Lajara <[email protected]>
Installed-Size: 77
Depends: python3:any (>= 3.3.2-2~)
Replaces: python-asus-nuc-wmi
Section: python
Priority: optional
Homepage: https://github.com/tvision-insights/intel_nuc_led
Description: ASUS NUC WMI CLI userland for asus_nuc_wmi kernel module
 # asus_nuc_wmi Python userland for ASUS NUC WMI kernel module
 .
 ## Compatibility
 .
 This `asus_nuc_wmi` userland was written from the available ASUS NUC WMI guide for the NUC 14
 (included in the [contrib/reference/](../reference) folder).
 .
 It has been tested on NUC 14.
 .
 Although we followed the specification documents, we have found that compatibility varies by a number of factors:
 .
 * BIOS version (for some settings, the BIOS version may impact what options are available).
 * BIOS configuration (for some settings, the BIOS configuration for LEDs can affect whether they are usable and
   in what default state they are in).
 * BIOS bugs (we have found some device BIOS have bugs where LEDs which should support RGB are only capable of
   dual color mode through WMI, but are capable of RGB when manually configuring them via the BIOS only).
 .
 Aside from the above, command options can change based on the combination of what the BIOS allows and what
 indicator option mode LEDs are put in.
Filename: pool/p/python3-asus-nuc-wmi/python3-asus-nuc-wmi_1.1.0-1_all.deb
Size: 11664
MD5Sum: 5302529f06e1ef68c45185c5698610b1
SHA1: fb0e1bb90ac8d84098b2f864efc45a5e1b51abb9
SHA256: 248ed1c4e930c4b3a4f6089cf9f76d89edebac624dfed3e6d7d64e40aca959c0

• Do you have a workaround you are using at present?
  The only recourse is to delete the metadata directory and then re-upload a package for that metadata directory to be regenerated.

• What feature or behavior is this required for?
  None

• How could we solve this issue? (Not knowing is okay!)
  The Packages metadata files must be properly kept in sync with whats in the repo and we need a way to force update that metadata without need to upload/re-upload packages. The rebuild index option does not seem to trigger regeneration of the metadata.
  IIRC, this issue was also much more likely to happen when mixing and matching uploading deb packages via UI vs upload deb packages via curl through API calls.

• Tell us about your Nexus Repository deployment: what version, operating system, and database are you using?
  Nexus OSS 3.70.1-02 Docker images running on AWS ECS with the default orientdb and S3 blob store.

• Anything else?
  None

[mardjuro]

Task Apt - Rebuild Apt metadata performs a full Apt metadata rebuild. It may be scheduled to run periodically.

I've found this when many packages were not included in metadata after repository key update and manually deleting metadata directory in order to force metadata rebuild. Correct procedure is to update the key and run the rebuild Apt metadata task.
Hopefully this helps in your case, at least as a workaround.

[ju2wheels]

In my case, I did not update the repository key, I was simply uploading new packages and in causes metadata issues. But ill give the task a go next time it happens.