0
点赞
收藏
分享

微信扫一扫

k8s部署ingress遇到的问题

大雁f 2023-05-21 阅读 78
kubernetesPowered by 金山文档nginxGroupbcHtml/CSS前端开发


1、报错：The Ingress "ingress-http" is invalid: spec.rules[0].host: Invalid value: "10.1.0.1:9009": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*')

是由于host配置的有问题，检查下host

2、The Ingress "ingress-erp" is invalid: spec.rules[0].host: Invalid value: "10.1.0.115": must be a DNS name, not an IP address

说的是host必须是一个域名。而不能是一个ip

修改/etc/host 添加一个域名

3、k8s的completed状态说明

你列的这两个就是job运行起来的容器，正常执行完了。就是这个情况。 这两个容器就是在集群起来时自动检查创建某些资源的，运行一次即可，不需要长久运行。所以就是job，其pod运行一次完成后退出就是completed状态

4、下载ingress-nginx文件，我这里直接复制别人的

需要修改的一个点是 hostNetwork: true 不加这项，ingress的端口不会映射到宿主机上

apiVersion: v1
kind: Namespace
metadata:
 labels:
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx
 namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
 namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx
 namespace: ingress-nginx
rules:
- apiGroups:
 - ""
 resources:
 - namespaces
 verbs:
 - get
- apiGroups:
 - ""
 resources:
 - configmaps
 - pods
 - secrets
 - endpoints
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - ""
 resources:
 - services
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
 resources:
 - ingresses
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
 resources:
 - ingresses/status
 verbs:
 - update
- apiGroups:
 - networking.k8s.io
 resources:
 - ingressclasses
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - ""
 resourceNames:
 - ingress-controller-leader
 resources:
 - configmaps
 verbs:
 - get
 - update
- apiGroups:
 - ""
 resources:
 - configmaps
 verbs:
 - create
- apiGroups:
 - ""
 resources:
 - events
 verbs:
 - create
 - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
 namespace: ingress-nginx
rules:
- apiGroups:
 - ""
 resources:
 - secrets
 verbs:
 - get
 - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 labels:
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx
rules:
- apiGroups:
 - ""
 resources:
 - configmaps
 - endpoints
 - nodes
 - pods
 - secrets
 - namespaces
 verbs:
 - list
 - watch
- apiGroups:
 - ""
 resources:
 - nodes
 verbs:
 - get
- apiGroups:
 - ""
 resources:
 - services
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - networking.k8s.io
 resources:
 - ingresses
 verbs:
 - get
 - list
 - watch
- apiGroups:
 - ""
 resources:
 - events
 verbs:
 - create
 - patch
- apiGroups:
 - networking.k8s.io
 resources:
 - ingresses/status
 verbs:
 - update
- apiGroups:
 - networking.k8s.io
 resources:
 - ingressclasses
 verbs:
 - get
 - list
 - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
rules:
- apiGroups:
 - admissionregistration.k8s.io
 resources:
 - validatingwebhookconfigurations
 verbs:
 - get
 - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx
 namespace: ingress-nginx
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: Role
 name: ingress-nginx
subjects:
- kind: ServiceAccount
 name: ingress-nginx
 namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
 namespace: ingress-nginx
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: Role
 name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
 name: ingress-nginx-admission
 namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 labels:
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: ingress-nginx
subjects:
- kind: ServiceAccount
 name: ingress-nginx
 namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
 name: ingress-nginx-admission
 namespace: ingress-nginx
---
apiVersion: v1
data:
 allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-controller
 namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-controller
 namespace: ingress-nginx
spec:
 externalTrafficPolicy: Local
 ports:
 - appProtocol: http
 name: http
 port: 80
 protocol: TCP
 targetPort: http
 - appProtocol: https
 name: https
 port: 443
 protocol: TCP
 targetPort: https
 selector:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-controller-admission
 namespace: ingress-nginx
spec:
 ports:
 - appProtocol: https
 name: https-webhook
 port: 443
 targetPort: webhook
 selector:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 type: ClusterIP
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-controller
 namespace: ingress-nginx
spec:
 minReadySeconds: 0
 revisionHistoryLimit: 10
 selector:
 matchLabels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 template:
 metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 spec:
 hostNetwork: true
 containers:
 - args:
 - /nginx-ingress-controller
 - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
 - --election-id=ingress-controller-leader
 - --controller-class=k8s.io/ingress-nginx
 - --ingress-class=nginx
 - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
 - --validating-webhook=:8443
 - --validating-webhook-certificate=/usr/local/certificates/cert
 - --validating-webhook-key=/usr/local/certificates/key
 env:
 - name: POD_NAME
 valueFrom:
 fieldRef:
 fieldPath: metadata.name
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 - name: LD_PRELOAD
 value: /usr/local/lib/libmimalloc.so
 # 修改为阿里云的镜像
 #image: k8s.gcr.io/ingress-nginx/controller:v1.2.0@sha256:d8196e3bc1e72547c5dec66d6556c0ff92a23f6d0919b206be170bc90d5f9185
 image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:v1.2.0@sha256:d8196e3bc1e72547c5dec66d6556c0ff92a23f6d0919b206be170bc90d5f9185
 imagePullPolicy: IfNotPresent
 lifecycle:
 preStop:
 exec:
 command:
 - /wait-shutdown
 livenessProbe:
 failureThreshold: 5
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 name: controller
 ports:
 - containerPort: 80
 name: http
 protocol: TCP
 - containerPort: 443
 name: https
 protocol: TCP
 - containerPort: 8443
 name: webhook
 protocol: TCP
 readinessProbe:
 failureThreshold: 3
 httpGet:
 path: /healthz
 port: 10254
 scheme: HTTP
 initialDelaySeconds: 10
 periodSeconds: 10
 successThreshold: 1
 timeoutSeconds: 1
 resources:
 requests:
 cpu: 100m
 memory: 90Mi
 securityContext:
 allowPrivilegeEscalation: true
 capabilities:
 add:
 - NET_BIND_SERVICE
 drop:
 - ALL
 runAsUser: 101
 volumeMounts:
 - mountPath: /usr/local/certificates/
 name: webhook-cert
 readOnly: true
 dnsPolicy: ClusterFirst
 nodeSelector:
 kubernetes.io/os: linux
 serviceAccountName: ingress-nginx
 terminationGracePeriodSeconds: 300
 volumes:
 - name: webhook-cert
 secret:
 secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission-create
 namespace: ingress-nginx
spec:
 template:
 metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission-create
 spec:
 hostNetwork: true
 containers:
 - args:
 - create
 - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
 - --namespace=$(POD_NAMESPACE)
 - --secret-name=ingress-nginx-admission
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 # 修改为阿里云的镜像 
 # image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
 image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
 imagePullPolicy: IfNotPresent
 name: create
 securityContext:
 allowPrivilegeEscalation: false
 nodeSelector:
 kubernetes.io/os: linux
 restartPolicy: OnFailure
 securityContext:
 fsGroup: 2000
 runAsNonRoot: true
 runAsUser: 2000
 serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission-patch
 namespace: ingress-nginx
spec:
 template:
 metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission-patch
 spec:
 hostNetwork: true
 containers:
 - args:
 - patch
 - --webhook-name=ingress-nginx-admission
 - --namespace=$(POD_NAMESPACE)
 - --patch-mutating=false
 - --secret-name=ingress-nginx-admission
 - --patch-failure-policy=Fail
 env:
 - name: POD_NAMESPACE
 valueFrom:
 fieldRef:
 fieldPath: metadata.namespace
 # 修改为阿里云的镜像
 #image: k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
 image: registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
 imagePullPolicy: IfNotPresent
 name: patch
 securityContext:
 allowPrivilegeEscalation: false
 nodeSelector:
 kubernetes.io/os: linux
 restartPolicy: OnFailure
 securityContext:
 fsGroup: 2000
 runAsNonRoot: true
 runAsUser: 2000
 serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: nginx
spec:
 controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
 labels:
 app.kubernetes.io/component: admission-webhook
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/part-of: ingress-nginx
 app.kubernetes.io/version: 1.2.0
 name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
 - v1
 clientConfig:
 service:
 name: ingress-nginx-controller-admission
 namespace: ingress-nginx
 path: /networking/v1/ingresses
 failurePolicy: Fail
 matchPolicy: Equivalent
 name: validate.nginx.ingress.kubernetes.io
 rules:
 - apiGroups:
 - networking.k8s.io
 apiVersions:
 - v1
 operations:
 - CREATE
 - UPDATE
 resources:
 - ingresses
 sideEffects: None

5、通过查看ingress-controller-pod的的日志发现，not valid ingress calss

I033104:22:17.728731 7main.go:101] "successfully validated configuration, accepting"ingress="dmp/dmpweb-ingress"


I033104:23:53.130892 7admission.go:149] processedingressviaadmissioncontroller {testedIngressLength:1testedIngressTime:0.052srenderingIngressLength:1renderingIngressTime:0sadmissionTime:25.3kBstestedConfigurationSize:0.052}

除了可能会有多个不同类型的 Ingress Controller 之外，还可能存在多个相同类型的 Ingress Controller，比如部署了两个 NGINX Ingress Controller，一个负责处理外网访问，一个负责处理内网访问。

此时也可以通过上面的方式，为每个 Controller 设定唯一的一个 class。

当多个 controller 的 class 不唯一，或者 controller 和 Ingress 都没有指定 class 又没有默认的 class 时，会导致所有符合条件的 Ingress Controller 竞争满足 Ingress 配置，可能会导致不可预测的结果。

解决方法一：

查看已创建的ingressclass

[root@k8s]# kubectl get ingressclass

NAME CONTROLLER PARAMETERS AGE

nginx k8s.io/ingress-nginx <none> 64m


已创建ingress 添加 IngressClass属性

修改ingress资源 添加spec.ingressClassName: nginx

kubectl edit ingress dmpweb-ingress
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
 name: dmpweb-ingress
 namespace: dmp
 annotations:
 kubesphere.io/creator: admin
spec:
ingressClassName: nginx #加的是这段
 rules:
 - host: wuhan.idss.com
 http:
 paths:
 - path: /
 pathType: ImplementationSpecific
 backend:
 service:
 name: dmpweb-svc
 port:
 number: 9999


解决方法二：

设置默认的 IngressClass 注意：修改后必须新创建的ingress才会默认使用

在集群中，我们可以设定一个默认的 Ingress Class，以便处理所有没有指定 Ingress Class 的 Ingress 资源。

在 IngressClass 资源上，我们可以通过将 ingressclass.kubernetes.io/is-default-class 注解的值设定为 true，来使没有设置 ingressClassName 的 Ingress 使用此默认的 IngressClass。

[root@k8s]# kubectl get ingressclass
NAME CONTROLLER PARAMETERS AGE
nginx k8s.io/ingress-nginx <none> 68m
[root@sjzt-wuhan-1 k8s-yaml]# kubectl edit ingressclass nginx
annotations里加上
ingressclass.kubernetes.io/is-default-class: "true"
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
 annotations:
 ingressclass.kubernetes.io/is-default-class: "true" #添加这段
 kubectl.kubernetes.io/last-applied-configuration: |
 {"apiVersion":"networking.k8s.io/v1","kind":"IngressClass","metadata":{"annotations":{},"labels":{"app.kubernetes.io/component":"controller","app.kubernetes.io/instance":"ingress-nginx","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/version":"1.1.1","helm.sh/chart":"ingress-nginx-4.0.15"},"name":"nginx"},"spec":{"controller":"k8s.io/ingress-nginx"}}
 creationTimestamp: "2022-03-31T03:39:24Z"
 generation: 1
 labels:
 app.kubernetes.io/component: controller
 app.kubernetes.io/instance: ingress-nginx
 app.kubernetes.io/managed-by: Helm
 app.kubernetes.io/name: ingress-nginx
 app.kubernetes.io/version: 1.1.1
 helm.sh/chart: ingress-nginx-4.0.15
 name: nginx
 resourceVersion: "20944816"
 selfLink: /apis/networking.k8s.io/v1/ingressclasses/nginx
 uid: e5a52f22-671e-4e82-87f3-0cee446ffa88
spec:
 controller: k8s.io/ingress-nginx

举报

相关推荐

springcloud in k8s 部署ingress

spring cloud in kubernetes个人学习

k8s部署nginx-ingress

nginxdockergithubk8skubernetes云计算

k8s 部署 ingress-nginx

nginx3dsedLinux系统/运维

K8S中部署apisix(非ingress)

javabig data开发语言

k8s部署Ingress并创建规则

kubernetesingress

K8S之Ingress

kubernetesnginx负载均衡工作原理云计算

k8s 练习 - ingress

ingresskubernetes云计算

k8s之Ingress

kubernetes

理解k8s 的 Ingress

日记本软件架构Kubernetesk8s那点事儿

k8s中的Ingress

应用层传输层用户态HarmonyOS后端开发
0 条评论