1、现象：
login: failure forking | Resource 错误
下面是日志
118]: time="2025-09-29T16:34:02.703370850+08:00" level=error msg="Handler for POST /v1.39/containers/dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd/stop returned error: cannot stop container: dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: Cannot kill container dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:02 localhost kubelet[1562]: E0929 16:34:02.703816 1562 docker_sandbox.go:275] Failed to stop sandbox "dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd": Error response from daemon: cannot stop container: dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: Cannot kill container dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:02 localhost kubelet[1562]: E0929 16:34:02.704155 1562 remote_runtime.go:143] StopPodSandbox "dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "aase-te-54cfcbfdf7-sf7lq_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: Cannot kill container dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:02 localhost kubelet[1562]: E0929 16:34:02.704201 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd"}
Sep 29 16:34:02 localhost kubelet[1562]: E0929 16:34:02.704302 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "aase-te" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 617b5887bc825a11174d34a840d5bc9054cca211be5d71e31a17d5d539c9d561: Cannot kill container 617b5887bc825a11174d34a840d5bc9054cca211be5d71e31a17d5d539c9d561: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "8f348ab0-68f0-42be-8d49-1e061a31e62f" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"aase-te-54cfcbfdf7-sf7lq_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: Cannot kill container dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:02 localhost kubelet[1562]: E0929 16:34:02.704323 1562 pod_workers.go:191] Error syncing pod 8f348ab0-68f0-42be-8d49-1e061a31e62f ("aase-te-54cfcbfdf7-sf7lq_ibs(8f348ab0-68f0-42be-8d49-1e061a31e62f)"), skipping: error killing pod: [failed to "KillContainer" for "aase-te" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 617b5887bc825a11174d34a840d5bc9054cca211be5d71e31a17d5d539c9d561: Cannot kill container 617b5887bc825a11174d34a840d5bc9054cca211be5d71e31a17d5d539c9d561: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "8f348ab0-68f0-42be-8d49-1e061a31e62f" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"aase-te-54cfcbfdf7-sf7lq_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: Cannot kill container dbaeff9b217f8d509c7a88440dac6cd8d25d81adad16382a51a2de04a049e3fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:02 localhost kubelet[1562]: I0929 16:34:02.742602 1562 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/lib/kubelet/plugins/com.tencent.cloud.csi.cfs/csi.sock <nil> 0 <nil>}] <nil> <nil>}
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011488157+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011508287+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011704427+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011763717+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011798947+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.011795477+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.013089049+08:00" level=error msg="Error running exec 40631c2fe50b323351f884dc09e50c9cd09ac4c69c535ca307bef23ceb399856 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.013315660+08:00" level=error msg="Error running exec 960493bb982b64788624d5da712a4aed76c2bc2aabbf1c2f5c4e7f68cd7268e3 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.013524221+08:00" level=error msg="Error running exec 720d1b7094efe45f59c8f52a89452c71d8137ae3cf839c1d33b4958fee4d0eb5 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.014147 1562 kuberuntime_container.go:528] preStop hook for container "aase-mg" failed: command '/bin/bash /opt/aase/shutdown.sh' exited with 126:
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.014375 1562 kuberuntime_container.go:528] preStop hook for container "gateway" failed: command '/bin/bash /opt/gateway/shutdown.sh' exited with 126:
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.014422 1562 kuberuntime_container.go:528] preStop hook for container "afa5" failed: command '/bin/bash /opt/afa/bin/afastop' exited with 126:
Sep 29 16:34:03 localhost kubelet[1562]: W0929 16:34:03.078368 1562 cni.go:204] Error validating CNI config list {"cniVersion":"0.3.1","name":"multus-cni","plugins":[{"capabilities":{"bandwidth":true,"portMappings":true},"cniVersion":"0.3.1","defaultDelegates":"tke-route-eni","kubeconfig":"/root/.kube/config","logLevel":"info","name":"multus-cni","type":"multus"}]}: [netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable]
Sep 29 16:34:03 localhost kubelet[1562]: W0929 16:34:03.078411 1562 cni.go:239] Unable to update cni config: no valid networks found in /etc/cni/net.d
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.701363561+08:00" level=error msg="Handler for POST /v1.39/containers/cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b/stop returned error: cannot stop container: cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: Cannot kill container cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.701369381+08:00" level=error msg="Handler for POST /v1.39/containers/c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41/stop returned error: cannot stop container: c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: Cannot kill container c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.701867 1562 docker_sandbox.go:275] Failed to stop sandbox "cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b": Error response from daemon: cannot stop container: cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: Cannot kill container cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702074 1562 docker_sandbox.go:275] Failed to stop sandbox "c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41": Error response from daemon: cannot stop container: c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: Cannot kill container c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702182 1562 remote_runtime.go:143] StopPodSandbox "cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-te-6b785599b5-6k8dq_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: Cannot kill container cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702276 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b"}
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702354 1562 remote_runtime.go:143] StopPodSandbox "c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-pl-66bf8597fc-pc9zs_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: Cannot kill container c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702404 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e3dc5bc9099c0e3125c4292da406dcd1988c35bac6098dbdee8c8dd225f7b7ec: Cannot kill container e3dc5bc9099c0e3125c4292da406dcd1988c35bac6098dbdee8c8dd225f7b7ec: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "3eba096a-439a-4d00-91d7-057ff33778d2" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-te-6b785599b5-6k8dq_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: Cannot kill container cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702438 1562 pod_workers.go:191] Error syncing pod 3eba096a-439a-4d00-91d7-057ff33778d2 ("afa-te-6b785599b5-6k8dq_ibs(3eba096a-439a-4d00-91d7-057ff33778d2)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e3dc5bc9099c0e3125c4292da406dcd1988c35bac6098dbdee8c8dd225f7b7ec: Cannot kill container e3dc5bc9099c0e3125c4292da406dcd1988c35bac6098dbdee8c8dd225f7b7ec: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "3eba096a-439a-4d00-91d7-057ff33778d2" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-te-6b785599b5-6k8dq_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: Cannot kill container cc7b1db9ec8d2836938ca3b9f21290804a0b960b0a2b190c9479a80148789f3b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702400 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41"}
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.702526 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: a8ce5ca0c73578ebf221cab3b34c68e955ee515f68ab8a0fe7f8334a6412d33b: Cannot kill container a8ce5ca0c73578ebf221cab3b34c68e955ee515f68ab8a0fe7f8334a6412d33b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "6484d167-5af8-4a55-8f0f-9cc7b0badb42" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-pl-66bf8597fc-pc9zs_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: Cannot kill container c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.703552 1562 pod_workers.go:191] Error syncing pod 6484d167-5af8-4a55-8f0f-9cc7b0badb42 ("afa-pl-66bf8597fc-pc9zs_ibs(6484d167-5af8-4a55-8f0f-9cc7b0badb42)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: a8ce5ca0c73578ebf221cab3b34c68e955ee515f68ab8a0fe7f8334a6412d33b: Cannot kill container a8ce5ca0c73578ebf221cab3b34c68e955ee515f68ab8a0fe7f8334a6412d33b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "6484d167-5af8-4a55-8f0f-9cc7b0badb42" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-pl-66bf8597fc-pc9zs_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: Cannot kill container c8785b5f8ce06527704230b6926afed570f1378b0c0e34d233cf4448fb4d2a41: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:04 localhost dockerd[2118]: time="2025-09-29T16:34:04.010275180+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:04 localhost dockerd[2118]: time="2025-09-29T16:34:04.010290420+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:04 localhost dockerd[2118]: time="2025-09-29T16:34:04.011263502+08:00" level=error msg="Error running exec 5e81c617da72fe2ff4a565e00674a56ed8f600e26f6441a7602515e19768b03c in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.012212 1562 kuberuntime_container.go:528] preStop hook for container "afa5" failed: command '/bin/bash /opt/afa/bin/afastop' exited with 126:
Sep 29 16:34:04 localhost dockerd[2118]: time="2025-09-29T16:34:04.016923583+08:00" level=error msg="Handler for POST /v1.39/containers/838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d/stop returned error: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.017446 1562 remote_runtime.go:274] StopContainer "838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.017511 1562 kuberuntime_container.go:642] Container "docker://838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.017534 1562 kuberuntime_container.go:664] killContainer "gateway"(id={"docker" "838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d"}) for pod "gateway-5684db456-zv6d9_ibs(2b856315-3086-431e-bc0d-0ee79f4abaf5)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost dockerd[2118]: time="2025-09-29T16:34:04.017778584+08:00" level=error msg="Handler for POST /v1.39/containers/fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50/stop returned error: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.018196 1562 remote_runtime.go:274] StopContainer "fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.018242 1562 kuberuntime_container.go:642] Container "docker://fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.018261 1562 kuberuntime_container.go:664] killContainer "afa5"(id={"docker" "fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50"}) for pod "afa-comm-c889d7988-hdgq9_ibs(d2d77ba9-3d43-492b-a806-ed9b0e96ba2e)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.020179 1562 cni.go:387] Error deleting ibs_gateway-5684db456-zv6d9/9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:04 localhost kubelet[1562]: W0929 16:34:04.020208 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.020615 1562 cni.go:387] Error deleting ibs_afa-comm-c889d7988-hdgq9/0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:04 localhost kubelet[1562]: W0929 16:34:04.020647 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.020679 1562 cni.go:387] Error deleting ibs_gateway-5684db456-zv6d9/9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:04 localhost kubelet[1562]: E0929 16:34:04.020941 1562 cni.go:387] Error deleting ibs_afa-comm-c889d7988-hdgq9/0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.011067747+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.011113996+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.012478339+08:00" level=error msg="Error running exec 3210ef9fd785bebac39b4c06f5203d03a7c86ce1c7eb1cdc89072f98fb36f421 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.013636 1562 kuberuntime_container.go:528] preStop hook for container "afa5" failed: command '/bin/bash /opt/afa/bin/afastop' exited with 126:
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.017460009+08:00" level=error msg="Handler for POST /v1.39/containers/e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f/stop returned error: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.017546989+08:00" level=error msg="Handler for POST /v1.39/containers/3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd/stop returned error: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:05 localhost dockerd[2118]: time="2025-09-29T16:34:05.017679680+08:00" level=error msg="Handler for POST /v1.39/containers/e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0/stop returned error: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.017973 1562 remote_runtime.go:274] StopContainer "e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018041 1562 kuberuntime_container.go:642] Container "docker://e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018072 1562 kuberuntime_container.go:664] killContainer "afa5"(id={"docker" "e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f"}) for pod "afa-te-6b785599b5-s7ddf_ibs(d5bc0cc5-9fd6-41e1-a9dc-84b327167c63)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.017981 1562 remote_runtime.go:274] StopContainer "3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018119 1562 kuberuntime_container.go:642] Container "docker://3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018139 1562 remote_runtime.go:274] StopContainer "e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018147 1562 kuberuntime_container.go:664] killContainer "aase-mg"(id={"docker" "3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd"}) for pod "aase-mg-67769748d4-tlbxx_ibs(1f344122-1af8-4593-b47d-3bc13b514f45)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018170 1562 kuberuntime_container.go:642] Container "docker://e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.018191 1562 kuberuntime_container.go:664] killContainer "gateway"(id={"docker" "e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0"}) for pod "gateway-5684db456-wvzpf_ibs(dcca8f7d-e4bd-4f90-99f8-8ae7650f6ea6)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.021352 1562 cni.go:387] Error deleting ibs_afa-te-6b785599b5-s7ddf/f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: W0929 16:34:05.021395 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.021682 1562 cni.go:387] Error deleting ibs_aase-mg-67769748d4-tlbxx/45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: W0929 16:34:05.021733 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.029316 1562 cni.go:387] Error deleting ibs_afa-te-6b785599b5-s7ddf/f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.029681 1562 cni.go:387] Error deleting ibs_aase-mg-67769748d4-tlbxx/45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.029863 1562 cni.go:387] Error deleting ibs_gateway-5684db456-wvzpf/8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: W0929 16:34:05.029895 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:05 localhost kubelet[1562]: E0929 16:34:05.030421 1562 cni.go:387] Error deleting ibs_gateway-5684db456-wvzpf/8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:06 localhost dockerd[2118]: time="2025-09-29T16:34:06.015446651+08:00" level=error msg="Handler for POST /v1.39/containers/fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b/stop returned error: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.016165 1562 remote_runtime.go:274] StopContainer "fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.016253 1562 kuberuntime_container.go:642] Container "docker://fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.016282 1562 kuberuntime_container.go:664] killContainer "afa5"(id={"docker" "fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b"}) for pod "afa-au-79cf4467f5-l8gvb_ibs(ce4e003e-4aff-4ae2-a083-580ae8f2d0f4)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.019166 1562 cni.go:387] Error deleting ibs_afa-au-79cf4467f5-l8gvb/899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:06 localhost kubelet[1562]: W0929 16:34:06.019202 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.019473 1562 cni.go:387] Error deleting ibs_afa-au-79cf4467f5-l8gvb/899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:06 localhost dockerd[2118]: time="2025-09-29T16:34:06.023736686+08:00" level=error msg="Handler for POST /v1.39/containers/9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1/stop returned error: cannot stop container: 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: Cannot kill container 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:06 localhost dockerd[2118]: time="2025-09-29T16:34:06.023819706+08:00" level=error msg="Handler for POST /v1.39/containers/0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc/stop returned error: cannot stop container: 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: Cannot kill container 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024034 1562 docker_sandbox.go:275] Failed to stop sandbox "9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1": Error response from daemon: cannot stop container: 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: Cannot kill container 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024241 1562 docker_sandbox.go:275] Failed to stop sandbox "0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc": Error response from daemon: cannot stop container: 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: Cannot kill container 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024322 1562 remote_runtime.go:143] StopPodSandbox "9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "gateway-5684db456-zv6d9_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: Cannot kill container 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024370 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1"}
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024471 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "gateway" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "2b856315-3086-431e-bc0d-0ee79f4abaf5" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"gateway-5684db456-zv6d9_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: Cannot kill container 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024497 1562 pod_workers.go:191] Error syncing pod 2b856315-3086-431e-bc0d-0ee79f4abaf5 ("gateway-5684db456-zv6d9_ibs(2b856315-3086-431e-bc0d-0ee79f4abaf5)"), skipping: error killing pod: [failed to "KillContainer" for "gateway" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: Cannot kill container 838b2590756662268dd27daf053e54f40a7fd54a9fa03a3bdf77f290e35a4d2d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "2b856315-3086-431e-bc0d-0ee79f4abaf5" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"gateway-5684db456-zv6d9_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: Cannot kill container 9deac1605efcf640af09bf86df77fd90878c5aebf3d7e3a08966f9858c1125e1: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024493 1562 remote_runtime.go:143] StopPodSandbox "0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-comm-c889d7988-hdgq9_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: Cannot kill container 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024527 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc"}
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.024605 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "d2d77ba9-3d43-492b-a806-ed9b0e96ba2e" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-comm-c889d7988-hdgq9_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: Cannot kill container 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:06 localhost kubelet[1562]: E0929 16:34:06.025592 1562 pod_workers.go:191] Error syncing pod d2d77ba9-3d43-492b-a806-ed9b0e96ba2e ("afa-comm-c889d7988-hdgq9_ibs(d2d77ba9-3d43-492b-a806-ed9b0e96ba2e)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: Cannot kill container fe17440604df1daa3eecb45318571d739c82baed6857de127adc1e7c2b567b50: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "d2d77ba9-3d43-492b-a806-ed9b0e96ba2e" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-comm-c889d7988-hdgq9_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: Cannot kill container 0f2ed8cc904dfa022132a9ec16fb500a1bf705e0d1bdb9b38153ba68405befbc: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.011154827+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.011226368+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.012583281+08:00" level=error msg="Error running exec 2a5d1ed4dd3c6a35144276e01e96692b088dd11b85afe00800bf1e0bb9750cfe in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.013388 1562 kuberuntime_container.go:528] preStop hook for container "message" failed: command '/bin/bash /opt/message/shutdown.sh' exited with 126:
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.017103339+08:00" level=error msg="Handler for POST /v1.39/containers/8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536/stop returned error: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.017804 1562 remote_runtime.go:274] StopContainer "8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.017872 1562 kuberuntime_container.go:642] Container "docker://8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.017897 1562 kuberuntime_container.go:664] killContainer "afa5"(id={"docker" "8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536"}) for pod "afa-comm-c889d7988-gpvbh_ibs(abeac207-801c-4f6f-beb9-aa4e76e3c1a3)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.020572 1562 cni.go:387] Error deleting ibs_afa-comm-c889d7988-gpvbh/85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:07 localhost kubelet[1562]: W0929 16:34:07.020597 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.020863 1562 cni.go:387] Error deleting ibs_afa-comm-c889d7988-gpvbh/85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.033104749+08:00" level=error msg="Handler for POST /v1.39/containers/f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7/stop returned error: cannot stop container: f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: Cannot kill container f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.033563570+08:00" level=error msg="Handler for POST /v1.39/containers/45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d/stop returned error: cannot stop container: 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: Cannot kill container 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:07 localhost dockerd[2118]: time="2025-09-29T16:34:07.033875891+08:00" level=error msg="Handler for POST /v1.39/containers/8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67/stop returned error: cannot stop container: 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: Cannot kill container 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.033561 1562 docker_sandbox.go:275] Failed to stop sandbox "f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7": Error response from daemon: cannot stop container: f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: Cannot kill container f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.033849 1562 remote_runtime.go:143] StopPodSandbox "f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-te-6b785599b5-s7ddf_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: Cannot kill container f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.033874 1562 docker_sandbox.go:275] Failed to stop sandbox "45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d": Error response from daemon: cannot stop container: 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: Cannot kill container 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.033910 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7"}
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034034 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "d5bc0cc5-9fd6-41e1-a9dc-84b327167c63" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-te-6b785599b5-s7ddf_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: Cannot kill container f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034063 1562 pod_workers.go:191] Error syncing pod d5bc0cc5-9fd6-41e1-a9dc-84b327167c63 ("afa-te-6b785599b5-s7ddf_ibs(d5bc0cc5-9fd6-41e1-a9dc-84b327167c63)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: Cannot kill container e686eed19c5ec8557d4aee3d93f8fdc6dbaec5160e3f33bf881377e8edfa939f: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "d5bc0cc5-9fd6-41e1-a9dc-84b327167c63" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-te-6b785599b5-s7ddf_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: Cannot kill container f1d3b4a32e6b11d3d05fb4d098dc888d28caa2b49e66202edf694e9920d140e7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034099 1562 remote_runtime.go:143] StopPodSandbox "45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "aase-mg-67769748d4-tlbxx_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: Cannot kill container 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034135 1562 docker_sandbox.go:275] Failed to stop sandbox "8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67": Error response from daemon: cannot stop container: 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: Cannot kill container 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034161 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d"}
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034280 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "aase-mg" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "1f344122-1af8-4593-b47d-3bc13b514f45" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"aase-mg-67769748d4-tlbxx_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: Cannot kill container 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034324 1562 remote_runtime.go:143] StopPodSandbox "8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "gateway-5684db456-wvzpf_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: Cannot kill container 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034360 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67"}
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.034440 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "gateway" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "dcca8f7d-e4bd-4f90-99f8-8ae7650f6ea6" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"gateway-5684db456-wvzpf_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: Cannot kill container 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.035436 1562 pod_workers.go:191] Error syncing pod 1f344122-1af8-4593-b47d-3bc13b514f45 ("aase-mg-67769748d4-tlbxx_ibs(1f344122-1af8-4593-b47d-3bc13b514f45)"), skipping: error killing pod: [failed to "KillContainer" for "aase-mg" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: Cannot kill container 3873420d31c53659f0b752c964a9919593e1d98d48454432cd3153e3e24aa0fd: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "1f344122-1af8-4593-b47d-3bc13b514f45" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"aase-mg-67769748d4-tlbxx_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: Cannot kill container 45bc767ed7108fdcb72cd3e5b541289c857232ba47e3287c6376a046c67b476d: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.036471 1562 pod_workers.go:191] Error syncing pod dcca8f7d-e4bd-4f90-99f8-8ae7650f6ea6 ("gateway-5684db456-wvzpf_ibs(dcca8f7d-e4bd-4f90-99f8-8ae7650f6ea6)"), skipping: error killing pod: [failed to "KillContainer" for "gateway" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: Cannot kill container e7317b7dc420bf090caa4a5210afc6bc2bea50670002619363b79a32ba0656c0: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "dcca8f7d-e4bd-4f90-99f8-8ae7650f6ea6" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"gateway-5684db456-wvzpf_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: Cannot kill container 8d0b983d081fe546ca5752f126427ba1f8369d025c9d102e6f62bd9b42d44d67: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:07 localhost kubelet[1562]: E0929 16:34:07.699942 1562 summary_sys_containers.go:47] Failed to get system container stats for "/system.slice/docker.service": failed to get cgroup stats for "/system.slice/docker.service": failed to get container info for "/system.slice/docker.service": unknown container "/system.slice/docker.service"
Sep 29 16:34:08 localhost dockerd[2118]: time="2025-09-29T16:34:08.023065806+08:00" level=error msg="Handler for POST /v1.39/containers/899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43/stop returned error: cannot stop container: 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: Cannot kill container 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:08 localhost kubelet[1562]: E0929 16:34:08.023535 1562 docker_sandbox.go:275] Failed to stop sandbox "899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43": Error response from daemon: cannot stop container: 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: Cannot kill container 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:08 localhost kubelet[1562]: E0929 16:34:08.023838 1562 remote_runtime.go:143] StopPodSandbox "899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-au-79cf4467f5-l8gvb_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: Cannot kill container 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:08 localhost kubelet[1562]: E0929 16:34:08.023915 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43"}
Sep 29 16:34:08 localhost kubelet[1562]: E0929 16:34:08.024023 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "ce4e003e-4aff-4ae2-a083-580ae8f2d0f4" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-au-79cf4467f5-l8gvb_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: Cannot kill container 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:08 localhost kubelet[1562]: E0929 16:34:08.024068 1562 pod_workers.go:191] Error syncing pod ce4e003e-4aff-4ae2-a083-580ae8f2d0f4 ("afa-au-79cf4467f5-l8gvb_ibs(ce4e003e-4aff-4ae2-a083-580ae8f2d0f4)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: Cannot kill container fab9e07f1436a253d17c8f163b8eeef1d4c3985fe714ed0c548701778c240f7b: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "ce4e003e-4aff-4ae2-a083-580ae8f2d0f4" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-au-79cf4467f5-l8gvb_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: Cannot kill container 899043e566a1dc95c3dd4e11368356aaa48450c8d3ea5508596ba8eecaa3bf43: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:08 localhost kubelet[1562]: W0929 16:34:08.079294 1562 cni.go:204] Error validating CNI config list {"cniVersion":"0.3.1","name":"multus-cni","plugins":[{"capabilities":{"bandwidth":true,"portMappings":true},"cniVersion":"0.3.1","defaultDelegates":"tke-route-eni","kubeconfig":"/root/.kube/config","logLevel":"info","name":"multus-cni","type":"multus"}]}: [netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable]
Sep 29 16:34:08 localhost kubelet[1562]: W0929 16:34:08.079330 1562 cni.go:239] Unable to update cni config: no valid networks found in /etc/cni/net.d
Sep 29 16:34:09 localhost dockerd[2118]: time="2025-09-29T16:34:09.014044834+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:09 localhost dockerd[2118]: time="2025-09-29T16:34:09.014046974+08:00" level=error msg="stream copy error: reading from a closed fifo"
Sep 29 16:34:09 localhost dockerd[2118]: time="2025-09-29T16:34:09.015527726+08:00" level=error msg="Error running exec 365cd8acb1b88e022c03eba7770be43fc474a207e941468f81a6c513e5408cce in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.016536 1562 kuberuntime_container.go:528] preStop hook for container "aase-te" failed: command '/bin/bash /opt/aase/shutdown.sh' exited with 126:
Sep 29 16:34:09 localhost dockerd[2118]: time="2025-09-29T16:34:09.016540249+08:00" level=error msg="Handler for POST /v1.39/containers/9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8/stop returned error: cannot stop container: 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: Cannot kill container 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.017138 1562 remote_runtime.go:274] StopContainer "9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8" from runtime service failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: Cannot kill container 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.017214 1562 kuberuntime_container.go:642] Container "docker://9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8" termination failed with gracePeriod 120: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: Cannot kill container 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.017239 1562 kuberuntime_container.go:664] killContainer "message"(id={"docker" "9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8"}) for pod "message-6b54b7f7bf-7snj7_ibs(cd1180cb-3d88-4099-b438-2803a7ef5935)" failed: rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: Cannot kill container 9014debaf8f04da67415aeb19bfabcdbad4d7936763019ddfc3aa0cf9dfbc6b8: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.022209 1562 cni.go:387] Error deleting ibs_message-6b54b7f7bf-7snj7/43b44ccba7378e842d8184fda7da2d265850bd85e141014445f9f9c505f34c17 from network loopback/cni-loopback: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:09 localhost kubelet[1562]: W0929 16:34:09.022257 1562 cni.go:343] CNI failed to delete loopback network: netplugin failed with no error message: fork/exec /opt/cni/bin/loopback: resource temporarily unavailable
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.022615 1562 cni.go:387] Error deleting ibs_message-6b54b7f7bf-7snj7/43b44ccba7378e842d8184fda7da2d265850bd85e141014445f9f9c505f34c17 from network multus/multus-cni: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable
Sep 29 16:34:09 localhost dockerd[2118]: time="2025-09-29T16:34:09.024938654+08:00" level=error msg="Handler for POST /v1.39/containers/85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7/stop returned error: cannot stop container: 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: Cannot kill container 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.025359 1562 docker_sandbox.go:275] Failed to stop sandbox "85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7": Error response from daemon: cannot stop container: 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: Cannot kill container 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.025674 1562 remote_runtime.go:143] StopPodSandbox "85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7" from runtime service failed: rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod "afa-comm-c889d7988-gpvbh_ibs" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: Cannot kill container 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.025747 1562 kuberuntime_manager.go:923] Failed to stop sandbox {"docker" "85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7"}
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.025856 1562 kubelet.go:1572] error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "abeac207-801c-4f6f-beb9-aa4e76e3c1a3" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-comm-c889d7988-gpvbh_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: Cannot kill container 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:09 localhost kubelet[1562]: E0929 16:34:09.025876 1562 pod_workers.go:191] Error syncing pod abeac207-801c-4f6f-beb9-aa4e76e3c1a3 ("afa-comm-c889d7988-gpvbh_ibs(abeac207-801c-4f6f-beb9-aa4e76e3c1a3)"), skipping: error killing pod: [failed to "KillContainer" for "afa5" with KillContainerError: "rpc error: code = Unknown desc = Error response from daemon: cannot stop container: 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: Cannot kill container 8921c3eca18631ba16581728f595166e4443698b446f468df23115467915a536: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown", failed to "KillPodSandbox" for "abeac207-801c-4f6f-beb9-aa4e76e3c1a3" with KillPodSandboxError: "rpc error: code = Unknown desc = [networkPlugin cni failed to teardown pod \"afa-comm-c889d7988-gpvbh_ibs\" network: netplugin failed with no error message: fork/exec /opt/cni/bin/multus: resource temporarily unavailable, Error response from daemon: cannot stop container: 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: Cannot kill container 85b3ef998e945970cc81137aae49b8bfdc059e67267d1f3119244e185607c4f7: unknown error after kill: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown]"]
Sep 29 16:34:10 localhost dockerd[2118]: time="2025-09-29T16:34:10.010522262+08:00" level=error msg="stream copy error: reading from a closed fifo"分析：
a307bef23ceb399856 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.013315660+08:00" level=error msg="Error running exec 960493bb982b64788624d5da712a4aed76c2bc2aabbf1c2f5c4e7f68cd7268e3 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost dockerd[2118]: time="2025-09-29T16:34:03.013524221+08:00" level=error msg="Error running exec 720d1b7094efe45f59c8f52a89452c71d8137ae3cf839c1d33b4958fee4d0eb5 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.014147 1562 kuberuntime_container.go:528] preStop hook for container "aase-mg" failed: command '/bin/bash /opt/aase/shutdown.sh' exited with 126:
Sep 29 16:34:03 localhost kubelet[1562]: E0929 16:34:03.014375 1562 kuberuntime_container.go:528] preStop hook for container "gateway" failed: command '/bin/bash /opt/gateway/shutdown.sh' exited with 126:a307bef23ceb399856 in container: OCI runtime state failed: fork/exec /usr/bin/runc: resource temporarily unavailable: : unknown"
而是表明 系统资源已耗尽，导致无法创建新的进程（容器运行时 runc 需要创建新进程来操作容器）。
fork/exec /usr/bin/runc: resource temporarily unavailable这里指向的是了 Linux 系统的进程数限制，达到极限
出现这种错误最大的可能原因是： 进程数耗尽。
a、Linux 系统对每个用户（或整个系统）可运行的进程/线程数量有上限。
b、Docker 容器本身会运行进程，容器内的应用也会产生多个进程。
c、如果系统上运行了太多容器，或者某个容器内的应用异常创建了大量进程（例如“进程泄漏”或“fork炸弹”），就会达到这个上限，导致新的 runc 进程无法被创建。
PID 耗尽
a、系统的 PID (进程ID) 是有限的（通过 kernel.pid_max 设置）。如果所有可用的 PID 都被占用，系统同样无法创建新进程。
其他资源限制
a、虽然可能性较低，但内存不足、文件描述符耗尽等情况也可能间接导致此问题。
failure forking 通常表示系统资源不足，特别是：
- 内存不足
- 进程数达到上限
- 文件描述符资源耗尽
1. 检查系统进程/线程总数
在宿主机上执行以下命令，查看当前进程数和系统限制：
# 查看当前系统总进程和线程数
ps -eLf | wc -l
# 或者使用更直观的工具
cat /proc/sys/kernel/pid_max # 查看系统最大PID数
cat /proc/sys/kernel/threads-max # 查看系统最大线程数
ulimit -u # 查看当前用户允许的最大进程数
# 系统级别文件描述符
cat /proc/sys/fs/file-nr # 已分配/未使用/最大文件描述符
cat /proc/sys/fs/file-max # 系统最大文件描述符限制
# 进程级别文件描述符
yum -y install lsof
lsof | wc -l # 总打开文件数
ls /proc/*/fd | wc -l # 所有进程的文件描述符总数
# 按进程查看
for pid in $(ps -eo pid | tail -n +2); do
echo "PID $pid: $(ls /proc/$pid/fd 2>/dev/null | wc -l) fds"
done | sort -k3 -nr | head -10
# 当前进程总数
cat /proc/loadavg | awk '{print $4}' # 显示最近运行的进程数/总进程数
ps -eLf | wc -l # 精确统计所有进程和线程
# 按命名空间查看
ps -eLf | grep -c kube
ps -eLf | grep -c docker
ps -eLf | grep -c containerd
# 进程数趋势
watch "ps -eLf | wc -l"2、
