Hello, and welcome! If you're thinking about contributing to Kuma's code base, you came to the right place. This document serves as guide/reference for technical contributors.
The following dependencies are all necessary. Please follow the installation instructions for any tools/libraries that you may be missing.
Throughout this guide, we will use make to run pre-defined tasks in the Makefile.
Use the following command to list out all the possible commands:
make helpWe use mise to manage dev tools for Kuma. If you had mise installed previously we recommend running:
mise cache clear
rm -rf ~/.local/share/mise/{installs,plugins}/kubectlIf you're experiencing errors with installing kubectl or container-structure-test.
Please see jdx/mise#3110 for details.
To install dev tools you can run:
make installTo run all code formatting, linting and vetting tools use the target:
make checkWe use Ginkgo as our testing framework. To run the existing test suite, you have several options:
For all tests, run:
make testAnd you can run tests that are specific to a part of Kuma by appending the app name as shown below:
make test/kumactlFor even more specific tests you can specify the package you want to run tests from:
make test TEST_PKG_LIST=<pkgPath>pkgPath is a package list selector for example: ./pkg/xds/... will run all tests in the pkg/xds subtree.
There's a large set of integration tests that can be run with:
make test/e2eThese tests are big and slow, it is recommended to read e2e-test-tips before running them.
To build all the binaries run:
make buildLike make test, you can append the app name to the target to build a specific binary. For example, here is how you would build the binary for only kumactl:
make build/kumactlThis could help expedite your development process if you only made changes to the kumactl files.
With Skaffold we can utilize code live reload on k8s cluster. Everytime you make a change in code Kuma will be rebuild and reinstalled on cluster.
- Run
make k3d/cluster/start - Run
make dev/fetch-demoto get the Kuma counter demo app - Run
skaffold dev
Like any other go program Kuma can be debugged using dlv. In this section we'll go into how to trigger a breakpoint both in K8S and Universal.
- Run
make k3d/cluster/start - Run
skaffold debug - Run goland/vscode debugger with remote target on port
56268(Skaffold will log exposed port for debugging) - Put a breakpoint where you want
- Enjoy!
- Add
4000port in UniversalApp so that it's exposed, and the debugger can connect. - Remove "-w -s" from LDFLAGS here so that debugging symbols are not stripped
- Add a
time.Sleepin a place where you want to debug the test - Run the tests
make -j test/e2e/debug EXTRA_GOFLAGS='-gcflags "all=-N -l"' E2E_PKG_LIST=./test/e2e_env/universal/... - Wait to hit the
time.Sleep - Figure out the
kuma-cpcontainer id by running:docker ps | grep kuma-cp - Exec into the container:
docker exec -it kuma-3_kuma-cp_3dkYrT bash - Download the same go as in
go.mod- e.g.curl -o golang https://dl.google.com/go/go1.23.2.linux-arm64.tar.gz - Extract using
tar xzf golang - Install
dlvversion that is closes to thego.modversion in the container, run:go install github.com/go-delve/delve/cmd/dlv@vCLOSEST_DLV_VERSION - Run
dlv --listen=:4000 --headless=true --api-version=2 --accept-multiclient attach 1 - Figure out the port on the host machine
docker ps | grep kuma-3_kuma-cp_3dkYrT, look for the port forward for4000 - Run goland/vscode debugger with remote target on port from point 12
- Enjoy!
By using the JetBrains Goland IDE, you could also debug the E2E tests running in Kubernetes interactively. To do so, follow these steps:
- Prepare the images by running
make build,make imagesandmake docker/tag - Start the Kubernetes clusters by running
make test/e2e/k8s/start - Create a run configuration by clicking the "Play" button aside the package name in a test suite file (e.g.
test/e2e_env/kubernetes/kubernetes_suite_test.go) - Quickly stop the execution to avoid running the tests, we only want the automatically generated run configuration
- Open the "Run/Debug configuration" dialog and locate and edit the configuration created by using the settings mentioned later in this document
- Locate to particular file containing the test case you want to debug, and mark the code block to
FDescribeorFItto run only that test case - Run the configuration in debug mode by clicking the bug 🪲 icon and enjoy debugging
- Stop the clusters after debugging complete by executing
make test/e2e/k8s/stop
Required settings for the run configuration:
- Environment variables
KUMACTLBIN=<path-to-built-kumactl>should point to thekumactlof your local build, e.g./Users/<username>/go/src/github.com/<username>/kuma/build/artifacts-darwin-arm64/kumactl/kumactlKUMA_K8S_TYPE=k3d(default) orkind- must match the tool used to start the clustersK3D_CNI=flannelonly needed if you are using thek3dcluster type, which is the default
- Go tool arguments
- Prepare the values for Envoy version and Kuma product version
- Envoy version: execute
ENVOY_VERSION=$(make build/info | grep Envoy | cut -d '=' -f 2) - Kuma product version: execute
KUMA_PRODUCT_VERSION=$(./tools/releases/version.sh | awk '{print $1}')
- Envoy version: execute
- Set the Go tools arguments using output from this command:
echo "-ldflags='-X github.com/kumahq/kuma/pkg/version.Envoy=$ENVOY_VERSION -X github.com/kumahq/kuma/pkg/version.version=$KUMA_PRODUCT_VERSION'"
- Prepare the values for Envoy version and Kuma product version
When you need to restart the debugging:
- Rebuild the images if needed
- To rebuild images, execute
make build,make imagesandmake docker/tag - To load the new images into the clusters, execute
make test/e2e/k8s/load/images/kuma-1andmake test/e2e/k8s/load/images/kuma-2
- To rebuild images, execute
- Restart the clusters if needed
- To restart the clusters, execute
make test/e2e/k8s/stopand thenmake test/e2e/k8s/start - We usually don't need to restart the clusters when the debugged cases are sharing the cluster environment (under directory
test/e2e_env) because they clean up the cluster correctly - We need to restart the clusters when debugging the cases that do not share the cluster in normal runs or the cluster state is polluted for some reason.
- To restart the clusters, execute
Execute
make k3d/cluster/restart/kumactlTo stop any existing Kuma K3D cluster, start a new K3D cluster, load images, deploy Kuma and Kuma counter demo.
You can test development versions by first pushing to gcr.io:
gcloud auth configure-docker
make images docker/push DOCKER_REGISTRY=gcr.io/proj-123456then setting up kubectl to connect to your cluster and installing Kuma:
gcloud container clusters get-credentials cluster-name
kumactl install control-plane --registry=gcr.io/proj-123456 | kubectl apply -f -