Skip to content

ci: upload dependency mapping json #734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
+143 −0
Open

ci: upload dependency mapping json #734

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 110 additions & 0 deletions .github/workflows/release.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,13 @@ jobs:
path: ./src/lima/lima-and-qemu.macos*
if-no-files-found: error

- name: Upload dependency mapping ARM64
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: dep-version-mapping-aarch64
path: ./src/lima/dep-version-mapping-aarch64.json
if-no-files-found: error

- name: Make and release source code of dependencies
run: make download-sources
- name: Upload MacOS build
Expand Down Expand Up @@ -119,6 +126,109 @@ jobs:
path: ./src/lima/lima-and-qemu.macos*
if-no-files-found: error

- name: Upload dependency mapping x86_64
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: dep-version-mapping-x86_64
path: ./src/lima/dep-version-mapping-x86_64.json
if-no-files-found: error

macos-arm64-ventura-build:
runs-on: [self-hosted, macos, arm64, "13", release]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make this run on macOS 14 since we recently removed all macOS 13 runners

timeout-minutes: 60
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we make this 120 minutes in accordance with the other workflows?

steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 1
submodules: recursive
persist-credentials: false
- name: 'Fetch submodule tags'
run: |
git submodule foreach --recursive git fetch --tags

- uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: e2e/go.mod

- name: Create Ventura limactl tarball
working-directory: src/lima
run: |
make clean && make exe
tar cfz limactl.ventura.arm64.tar.gz -C _output/bin limactl

- name: Upload Ventura build
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: limactl.ventura.arm64
path: src/lima/limactl.ventura.arm64.tar.gz
if-no-files-found: error

macos-x86_64-ventura-build:
runs-on: [self-hosted, macos, amd64, "13", release]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

timeout-minutes: 60
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 1
submodules: recursive
persist-credentials: false
- name: 'Fetch submodule tags'
run: |
git submodule foreach --recursive git fetch --tags

- uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
with:
go-version-file: e2e/go.mod

- name: Create Ventura limactl tarball
working-directory: src/lima
run: |
make clean && make exe
tar cfz limactl.ventura.x86_64.tar.gz -C _output/bin limactl

- name: Upload Ventura build
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: limactl.ventura.x86_64
path: src/lima/limactl.ventura.x86_64.tar.gz
if-no-files-found: error

upload-dependency-mappings:
runs-on: ubuntu-latest
timeout-minutes: 10
needs:
- macos-x86-build
- macos-arm64-build

steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
with:
fetch-depth: 1
persist-credentials: false

- name: configure aws credentials
uses: aws-actions/configure-aws-credentials@7474bc4690e29a8392af63c5b98e7449536d5c3a # v4.3.1
with:
role-to-assume: ${{ secrets.DEPENDENCY_MAPPING_BUCKET_ROLE }}
role-session-name: dependency-mapping-upload-session
aws-region: ${{ secrets.DEPENDENCY_MAPPING_BUCKET_REGION }}

- name: Download dependency mapping ARM64
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
name: dep-version-mapping-aarch64
path: build

- name: Download dependency mapping x86_64
uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0
with:
name: dep-version-mapping-x86_64
path: build

- name: Upload dependency mappings to S3
run: |
aws s3 cp ./build/dep-version-mapping-aarch64.json s3://${{ secrets.DEPENDENCY_MAPPING_BUCKET_NAME }}/aarch64/dep-version-mapping.json
aws s3 cp ./build/dep-version-mapping-x86_64.json s3://${{ secrets.DEPENDENCY_MAPPING_BUCKET_NAME }}/x86-64/dep-version-mapping.json

release:
runs-on: ubuntu-latest
Expand Down
33 changes: 33 additions & 0 deletions bin/lima-and-qemu.pl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,39 @@
unlink("$repo_root/$dist.tar.gz");
system("tar cvfz $repo_root/$dist.tar.gz -C /tmp/$dist $files");

# Extract package versions from Cellar and export to JSON
my %package_versions;
for my $file (keys %deps) {
# Extract package info from Cellar path: /opt/homebrew/Cellar/package/version/...
if ($file =~ m|/Cellar/([^/]+)/([^/]+)/|) {
my ($package, $version) = ($1, $2);
$package_versions{$package} = {
"package" => $package,
"version" => $version
};
}
# Handle direct bin files (like limactl) that might not be in Cellar
elsif ($file =~ m|^$install_dir/bin/([^/]+)$|) {
my $binary_name = $1;
# Try to get version from the binary itself
my $version_output = qx($file --version 2>/dev/null | head -1);
if ($version_output =~ /(\d+\.\d+\.\d+[^\s]*)/) {
$package_versions{$binary_name} = {
"package" => $binary_name,
"version" => $1
};
}
}
}

# Export to JSON
use JSON qw( encode_json );
my $json_file = "$repo_root/dep-version-mapping-$arch.json";
open(my $json_fh, ">", $json_file) or die "Can't write $json_file: $!";
print $json_fh encode_json(\%package_versions);
close($json_fh);
print "Generated dependency mapping: $json_file\n";

exit;

# File references may involve multiple symlinks that need to be recorded as well, e.g.
Expand Down