Skip to content

Makefile/ko build targets hardcode linux/amd64, breaking arm64 clusters #358

Description

@loafoe

Problem

make deploy and make push only ever produce linux/amd64 images, regardless of the target cluster's architecture:

  • deploy-controller/deploy-server invoke ko apply with no --platform flag, and ko's default build platform is linux/amd64 regardless of host OS/arch.
  • build-task-runner hardcodes GOOS=linux GOARCH=amd64 for the cross-compile and --platform linux/amd64 for the container build.
  • Dockerfile.task-runner hardcodes COPY bin/linux_amd64/ax-task-runner ....

Deploying to an arm64 cluster (e.g. a k3s/Raspberry Pi cluster) with make deploy AX_IMAGE_REPO=... silently builds and pushes amd64 images. Pods then fail with exec format error once they actually pull and try to run.

Fix

I have a fix ready on a branch, but google/ax's pull_request_creation_policy is set to collaborators_only, so I can't open a PR from a fork (github.com/loafoe/ax) — hence this issue instead.

Branch: https://github.com/loafoe/ax/tree/arm64-platform-support
Commit: loafoe@c1d58ba

Summary of the fix:

  • Adds KO_PLATFORM (default linux/amd64, unchanged behavior) used as ko apply --platform=$(KO_PLATFORM) in deploy-controller/deploy-server.
  • Adds TASK_RUNNER_PLATFORM (default linux/amd64) driving GOARCH, --platform, and a new TASK_RUNNER_GOARCH build-arg passed to Dockerfile.task-runner, which now does COPY bin/linux_${TASK_RUNNER_GOARCH}/ax-task-runner ... instead of a hardcoded amd64 path.
  • Verified: make build-task-runner TASK_RUNNER_PLATFORM=linux/arm64 produces a real ELF ... ARM aarch64 binary, and make deploy AX_IMAGE_REPO=... KO_PLATFORM=linux/arm64 was used to successfully deploy ax-controller/ax-server to a live arm64 k3s cluster.

Happy to open a proper PR if I'm added as a collaborator, or if the PR creation policy is relaxed to allow contributions from forks.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions