/run/flannel/subnet.env: no such file or directory

[peng666]

Environmental Info:
K3s Version:

k3s version v1.21.12+k3s1 (1db3ab57)
go version go1.16.10

Node(s) CPU architecture, OS, and Version:
Linux xxxx 3.10.0-862.14.1.5.h428.eulerosv2r7.x86_64 #1 SMP Wed Mar 11 06:00:00 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Other types of devices also encounter this problem.
其他类型的设备也遇到该问题

Cluster Configuration:
1 servers
1个master节点，0个worker节点

Describe the bug:
After a cluster is created, the error message "open /run/flannel/subnet.env: no such file or directory" is displayed.
集群创建后，报错open /run/flannel/subnet.env: no such file or directory

Steps To Reproduce:

• Installed K3s:

k3s server \
--datastore-endpoint=${DATASTORE_ENDPOINT} \
--datastore-cafile=${DATASTORE_CAFILE} \
--datastore-certfile=${DATASTORE_CERTFILE} \
--datastore-keyfile=${DATASTORE_KEYFILE} \
--node-name=${NODE_NAME} \
--data-dir=${DATA_DIR} \
--token=${TOKEN} \
--node-label=${LABEL} \
--kube-proxy-arg="proxy-mode=ipvs" \
--write-kubeconfig=${KUBECONFIG_OUTPUT} \
--no-deploy=traefik \
--disable=traefik \
--disable-cloud-controller \
--flannel-backend=vxlan \
--no-deploy=local-storage \
--disable=local-storage \
--docker

Expected behavior:
The deployments are expected to run properly.
希望部署deployment能正常运行

Actual behavior:

NAMESPACE     NAME                             READY   STATUS              RESTARTS   AGE
kube-system   coredns-574bcc6c46-tbh2w         0/1     ContainerCreating   0          10m
kube-system   metrics-server-ff9dbcb6c-drxff   0/1     ContainerCreating   0          10m

The pod event shows：

Events:
  Type     Reason                  Age                   From               Message
  ----     ------                  ----                  ----               -------
... ...
  Normal   SandboxChanged          7m5s (x275 over 12m)  kubelet            Pod sandbox changed, it will be killed and re-created.
  Warning  FailedCreatePodSandBox  2m5s (x541 over 11m)  kubelet            (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "1b2e40c6f9b71250e2ebdbfff55da4b1efbe220fb95dc208fdfbf9b64e445d07" network for pod "coredns-574bcc6c46-tbh2w": networkPlugin cni failed to set up pod "coredns-574bcc6c46-tbh2w_kube-system" network: open /run/flannel/subnet.env: no such file or directory

The following information is also displayed in the cluster log: open /run/flannel/subnet.env: no such file or directory
The /run/flannel/subnet.env file does not exist on the host machine.

If I have enough patience, wait 76 minutes (verified once). After the cluster log records "Starting flannel with backend vxlan", all pods are restored and the pod status is running. The /run/flannel/subnet.env file is created on the host machine. Why is this strange phenomenon?

等了大概76min，集群日志打印了“Starting flannel with backend vxlan”，之后pod都恢复正常了，/run/flannel/subnet.env也被创建出来了。为什么会有这个奇怪现象呢？日志也没明确报错提示。

Additional context / logs:

The following lines of the cluster log are endlessly looped:
集群日志无限循环以下几行：

I0116 20:39:44.815068   17818 docker_sandbox.go:401] "Failed to read pod IP from plugin/docker" err="networkPlugin cni failed on the status hook for pod \"coredns-574bcc6c46-79vv6_kube-system\": CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container \"1ed46921a5a75546fdc3c91dbde143721de4e4f9c6dca1583fb1668d2dc46478\""
I0116 20:39:44.818101   17818 pod_container_deletor.go:79] "Container not found in pod's containers" containerID="1ed46921a5a75546fdc3c91dbde143721de4e4f9c6dca1583fb1668d2dc46478"
I0116 20:39:44.819006   17818 cni.go:333] "CNI failed to retrieve network namespace path" err="cannot find network namespace for the terminated container \"1ed46921a5a75546fdc3c91dbde143721de4e4f9c6dca1583fb1668d2dc46478\""
E0116 20:39:45.010976   17818 cni.go:361] "Error adding pod to network" err="open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-574bcc6c46-79vv6" podSandboxID={Type:docker ID:64d029aefda6cd62f7b344c36d75aff80d720cbf5162af2b7692d6c5eac3a316} podNetnsPath="/proc/17138/ns/net" networkType="flannel" networkName="cbr0"
E0116 20:39:45.111784   17818 remote_runtime.go:116] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to set up sandbox container \"64d029aefda6cd62f7b344c36d75aff80d720cbf5162af2b7692d6c5eac3a316\" network for pod \"coredns-574bcc6c46-79vv6\": networkPlugin cni failed to set up pod \"coredns-574bcc6c46-79vv6_kube-system\" network: open /run/flannel/subnet.env: no such file or directory"
E0116 20:39:45.111836   17818 kuberuntime_sandbox.go:68] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to set up sandbox container \"64d029aefda6cd62f7b344c36d75aff80d720cbf5162af2b7692d6c5eac3a316\" network for pod \"coredns-574bcc6c46-79vv6\": networkPlugin cni failed to set up pod \"coredns-574bcc6c46-79vv6_kube-system\" network: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-574bcc6c46-79vv6"
E0116 20:39:45.111862   17818 kuberuntime_manager.go:790] "CreatePodSandbox for pod failed" err="rpc error: code = Unknown desc = failed to set up sandbox container \"64d029aefda6cd62f7b344c36d75aff80d720cbf5162af2b7692d6c5eac3a316\" network for pod \"coredns-574bcc6c46-79vv6\": networkPlugin cni failed to set up pod \"coredns-574bcc6c46-79vv6_kube-system\" network: open /run/flannel/subnet.env: no such file or directory" pod="kube-system/coredns-574bcc6c46-79vv6"
E0116 20:39:45.111918   17818 pod_workers.go:190] "Error syncing pod, skipping" err="failed to \"CreatePodSandbox\" for \"coredns-574bcc6c46-79vv6_kube-system(95d6a73c-457e-4337-8333-3dd16285805e)\" with CreatePodSandboxError: \"Failed to create sandbox for pod \\\"coredns-574bcc6c46-79vv6_kube-system(95d6a73c-457e-4337-8333-3dd16285805e)\\\": rpc error: code = Unknown desc = failed to set up sandbox container \\\"64d029aefda6cd62f7b344c36d75aff80d720cbf5162af2b7692d6c5eac3a316\\\" network for pod \\\"coredns-574bcc6c46-79vv6\\\": networkPlugin cni failed to set up pod \\\"coredns-574bcc6c46-79vv6_kube-system\\\" network: open /run/flannel/subnet.env: no such file or directory\"" pod="kube-system/coredns-574bcc6c46-79vv6" podUID=95d6a73c-457e-4337-8333-3dd16285805e

[brandond]

Flannel cannot start until the uninitialized taint has been cleared on the node. You must have a cloud controller deployed to clear the uninitialized taint, but you have disabled the embedded default cloud controller. Why did you do this?

Either leave the cloud controller enabled, or deploy an alternative immediately following cluster startup.