java操作k8s api示例：通过java完成对kubenetes原生资源对象（pod、node、namespace、servcie、deployment）和自定义资源对象CRD的增删改查或事件监听

本文目标

基于官方​​kubernetes-client/java​​类库，实现通过java完成对kubenetes原生资源对象（pod、node、namespace、servcie、deployment）和自定义资源对象（如：cluster）的增删改查或事件监听（watch）

k8s-client-java选型

目前通过java操作k8s，开源版本共有两个：

• ​​kubernetes-client/java​​
• ​​fabric8io/kubernetes-client​​

kubernetes-client/java和fabric8io/kubernetes-client对比

鉴于​​kubernetes-client/java​​​和​​官网API​​一致性好，本文决定采用它

kubernetes-client/java的使用

REST API

API 资源使用REST模式。

kube-apiserver 支持同时提供 https（默认监听在 6443 端口）和 http API（默认监听在 127.0.0.1 的 8080 端口），其中 http API 是非安全接口，不做任何认证授权机制，不建议生产环境启用。两个接口提供的 REST API 格式相同

图片来自 ​​OpenShift Blog ​​

1. GET /<资源名的复数格式>：获得某一类型的资源列表，例如GET /pods 返回一个Pod资源列表。
2. POST /<资源名的复数格式>：创建一个资源，该资源来自用户提供的JSON对象。
3. GET /<资源名复数格式>/<名字>：通过给出的名称（Name）获得单个资源，例如GET /pods/first 返回一个名称为“first”的Pod。
4. DELETE /<资源名复数格式>/<名字>：通过给出的名字删除单个资源，删除选项（DeleteOptions）中可以指定的优雅删除（Grace Deletion）的时间（GracePeriodSeconds），该可选项表明了从服务端接收到删除请求到资源被删除的时间间隔（单位为秒）。
5. PUT /<资源名复数格式>/<名字>：通过给出的资源名和客户端提供的JSON对象来更新或创建资源。
6. PATCH /<资源名复数格式>/<名字>：选择修改资源详细指定的域。
7. GET /watch/<资源名复数格式>：随时间变化，不断接收一连串的JSON对象，这些JSON对象记录了给定资源类别内所有资源对象的变化情况。
8. GET /watch/<资源名复数格式>/：随时间变化，不断接收一连串的JSON对象，这些JSON对象记录了某个给定资源对象的变化情况。

REST API版本说明

为了在兼容旧版本的同时不断升级新的API，Kubernetes支持多种API版本，每种API版本都有不同的API路径，例如/api/v1或 /apis/extensions/v1beta1。

Alpha级别：

• 包含alpha名称的版本（例如v1alpha1）。
• 该软件可能包含错误。启用一个功能可能会导致bug。默认情况下，功能可能会被禁用。

Beta级别：

• 包含beta名称的版本（例如v2beta3）。
• 该软件经过很好的测试。启用功能被认为是安全的。默认情况下功能是开启的。
• 大家使用过的Beta版本后，可以多给社区反馈，如果此版本在后续更新后将不会有太大变化。

Stable级别：

• 该版本名称命名方式：vX这里X是一个整数。
• Stable版本的功能特性，将出现在后续发布的软件版本中。

Alpha、Beta、RC、GA版本的区别

• Alpha：是内部测试版,一般不向外部发布,会有很多Bug.一般只有测试人员使用。
• Beta：也是测试版，这个阶段的版本会一直加入新的功能。在Alpha版之后推出。
• RC：(Release　Candidate) 顾名思义么 ! 用在软件上就是候选版本。系统平台上就是发行候选版本。RC版不会再加入新的功能了，主要着重于除错。
• GA:General Availability,正式发布的版本，在国外都是用GA来说明release版本的。

kubectl api-versions

查看 apiserver暴露的接口

kubectl api-versions

或者

curl -H’Authorization: Bearer token’ ​​https://192.168.1.122:6443/apis​​ –insecure

[root@fly]# kubectl api-versions
admissionregistration.k8s.io/v1beta1
apiextensions.k8s.io/v1beta1
apiregistration.k8s.io/v1beta1
apps/v1
apps/v1beta1
apps/v1beta2
authentication.istio.io/v1alpha1
authentication.k8s.io/v1
authentication.k8s.io/v1beta1
authorization.k8s.io/v1
authorization.k8s.io/v1beta1
autoscaling/v1
autoscaling/v2beta1
batch/v1
batch/v1beta1
certificates.k8s.io/v1beta1
config.istio.io/v1alpha2
events.k8s.io/v1beta1
extensions/v1beta1
networking.istio.io/v1alpha3
networking.k8s.io/v1
policy/v1beta1
rbac.authorization.k8s.io/v1
rbac.authorization.k8s.io/v1beta1
rbac.istio.io/v1alpha1
storage.k8s.io/v1
storage.k8s.io/v1beta1
v1

REST API 实例:

​​https://10.10.124.199:6443/apis/apps/v1/deployment​​

1. https ： kube-apiserver 支持同时提供https（默认监听在 6443 端口）和 http API（默认监听在 127.0.0.1 的 8080 端口），由于http api 是不会经过认证授权的，在生产环境中，为了安全性考虑，推荐是不启用。
2. 10.10.124.199 ： 该ip地址为kubernetes master 地址，在集群为高可用设置的场景下，该ip地址为 vip 地址。
3. apis ：kubernetes在这一层级只有api 与 apis ，api只有初步核心资源，包含pod，node，namespace等基本资源
4. apps ： 为api group 名，api group 是kubernetes 对资源类型特性相近的整合。
5. v1： 是作为api group version， 在新资源类型的加入到kubernetes，会经历版本变迁： v1alpha1–>v1alpha2–>….–>v1alphaN–>v1beta1–>v1beta2–>v1 。
6. deployment ： 是kubernetes 的资源名

API Object 整体划分图 (红星符号代表常用资源)

​​kubernetes-client/java​​客户端API接口识别

打开​​kubernetes-client/java​​，只要是以Api结尾，一般就是我们可以调用的Api接口

API接口识别

ApiClient初始化&认证

ApiClient client = new ClientBuilder().setBasePath("ApiServer地址").setVerifyingSsl(false)
                .setAuthentication(new AccessTokenAuthentication("Token")).build();
        Configuration.setDefaultApiClient(client);

在生产环境，建议放在 程序启动前的初始化方法中

CRD资源增删改查

使用 CustomObjectsApi apiInstance = new CustomObjectsApi(); 操作

操作示例

简要描述：

​​集群列表接口​​

请求方式：

• GET

请求URL：

• /apis/{group}/{version}/{plural}

请求URL示例：

• /apis/flycloud.cn/v1/clusters

请求java示例：

CustomObjectsApi apiInstance = new CustomObjectsApi();
        String group = "flycloud.cn"; 
        String version = "v1";
        String plural = "clusters";
        String pretty = "ture";
        try {
            Object result = apiInstance.listClusterCustomObject(group,version,plural,pretty,null,null,null,null);
            String listCluster = JSON.toJSONString(result);
            System.out.println(listCluster);
        } catch (ApiException e) {
            System.err.println("Exception when calling CustomObjectsApi#listClusterCustomObject");
            e.printStackTrace();
        }

返回结果：

{
    "apiVersion": "flycloud.cn/v1",
    "items": [{
        "apiVersion": "flycloud.cn/v1",
        "kind": "Cluster",
        "metadata": {
            "annotations": {
                "name": "top"
            },
            "creationTimestamp": "2019-08-12T07:03:23Z",
            "generation": 1.0,
            "labels": {
                "template": "platform"
            },
            "name": "top",
            "namespace": "cluster-top",
            "resourceVersion": "277020",
            "selfLink": "/apis/flycloud.cn/v1/namespaces/cluster-top/clusters/top",
            "uid": "46528941-bccf-11e9-bfeb-005056bc7cff"
        },
        "spec": {
            "info": {
                "address": "192.168.103.60",
                "harbor": {
                    "address": "192.168.103.65",
                    "password": "123456",
                    "port": 443.0,
                    "protocol": "https",
                    "user": "admin"
                },
                "jenkins": {
                    "password": "admin",
                    "type": "jenkins",
                    "username": "admin"
                },
                "mysql": {
                    "connectionProperties": "druid.stat.mergeSql=true druid.stat.slowSqlMillis=5000",
                    "driverClass": "com.mysql.jdbc.Driver",
                    "filters": "stat",
                    "initialSize": 0.0,
                    "logAbandoned": true,
                    "maxActive": 100.0,
                    "maxOpenPreparedStatements": 50.0,
                    "maxWait": 60000.0,
                    "minIdle": 0.0,
                    "minPoolSize": 2.0,
                    "password": "123456",
                    "poolPreparedStatements": false,
                    "removeAbandoned": true,
                    "removeAbandonedTimeout": 900.0,
                    "type": "api-mysql",
                    "username": "root"
                },
                "network": {
                    "networkFlag": "calico",
                    "version": 1.0
                },
                "nfs": [{
                    "capacity": "1",
                    "ip": "192.168.103.65",
                    "name": "nfs",
                    "path": "/nfs/top",
                    "type": "nfs"
                }],
                "port": 6443.0,
                "prometheusPort": 30003.0,
                "protocol": "https",
                "redis": {
                    "maxTotal": 500.0,
                    "maxWaitMillis": 15000.0,
                    "minIdle": 10.0,
                    "password": "123456",
                    "testOnBorrow": true,
                    "testWhileIdle": true,
                    "timeBetweenEvictionRunsMillis": 600000.0,
                    "type": "api-redis"
                }
            },
            "template": [{
                "namespace": "kube-system",
                "serviceName": "heapster",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "check"
                }],
                "type": "heapster"
            }, {
                "namespace": "kube-system",
                "serviceName": "influxdb",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "web"
                }, {
                    "port": 8086.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "influxdb"
            }, {
                "namespace": "kube-system",
                "serviceName": "elasticsearch-logging-v1",
                "servicePort": [{
                    "port": 9200.0,
                    "protocol": "TCP",
                    "type": "web"
                }, {
                    "port": 9300.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "es"
            }, {
                "namespace": "kube-system",
                "serviceName": "oam-api-service",
                "servicePort": [{
                    "port": 8081.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "oam-api"
            }, {
                "namespace": "kube-system",
                "serviceName": "oam-task-service",
                "type": "oma-task"
            }, {
                "namespace": "kube-system",
                "serviceName": "webapi-service",
                "servicePort": [{
                    "port": 8080.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "webapi"
            }, {
                "namespace": "kube-system",
                "serviceName": "webpage-service",
                "servicePort": [{
                    "port": 8887.0,
                    "protocol": "TCP",
                    "type": "web"
                }],
                "type": "webpage"
            }, {
                "namespace": "kube-system",
                "serviceName": "terminal-service",
                "servicePort": [{
                    "port": 8888.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "terminal"
            }, {
                "namespace": "kube-system",
                "serviceName": "api-mysql-service",
                "servicePort": [{
                    "nodePort": 30306.0,
                    "port": 3306.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "api-mysql"
            }, {
                "namespace": "kube-system",
                "serviceName": "api-redis-service",
                "servicePort": [{
                    "nodePort": 30379.0,
                    "port": 6379.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "api-redis"
            }, {
                "namespace": "kube-system",
                "serviceName": "jenkins",
                "servicePort": [{
                    "nodePort": 30080.0,
                    "port": 8080.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "jenkins"
            }, {
                "namespace": "kube-system",
                "serviceName": "nfs-controller",
                "type": "nfs-controller"
            }, {
                "namespace": "kube-system",
                "serviceName": "auto-scale",
                "type": "auto-scale"
            }, {
                "namespace": "kube-system",
                "serviceName": "node-up-down",
                "type": "node-up-down"
            }, {
                "namespace": "kube-system",
                "serviceName": "calico-node",
                "type": "calico-node"
            }, {
                "namespace": "kube-system",
                "serviceName": "calico-kube-controller",
                "type": "calico-cotnroller"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-apiserver",
                "type": "kube-apiserver"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-controller-manager",
                "type": "kube-controller-manager"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-scheduler",
                "type": "kube-scheduler"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-proxy",
                "type": "kube-proxy"
            }, {
                "namespace": "kube-system",
                "serviceName": "etcd",
                "type": "etcd"
            }, {
                "namespace": "kube-system",
                "serviceName": "cluster-controller",
                "type": "cluster-controller"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-dns",
                "servicePort": [{
                    "port": 53.0,
                    "protocol": "TCP",
                    "type": "check"
                }, {
                    "port": 53.0,
                    "protocol": "UDP",
                    "type": "dns"
                }],
                "type": "kube-dns"
            }]
        },
        "status": {
            "conditions": [{
                "status": false,
                "type": "Ready"
            }]
        }
    }, {
        "apiVersion": "flycloud.cn/v1",
        "kind": "Cluster",
        "metadata": {
            "annotations": {
                "name": "test"
            },
            "creationTimestamp": "2019-09-13T15:54:57Z",
            "generation": 1.0,
            "labels": {
                "template": "dev"
            },
            "name": "test",
            "namespace": "flycloud",
            "resourceVersion": "7687403",
            "selfLink": "/apis/flycloud.cn/v1/namespaces/flycloud/clusters/test",
            "uid": "d5bddb21-d63e-11e9-b5a7-005056bc7cff"
        },
        "spec": {
            "info": {
                "address": "192.168.103.60",
                "domain": [],
                "external": [{
                    "labels": {
                        "lb": "nginx"
                    },
                    "maxPort": 35000.0,
                    "minPort": 33000.0,
                    "tcpConfig": "system-expose-nginx-config-tcp",
                    "topLb": "192.168.103.61",
                    "type": "nginx",
                    "udpConfig": "system-expose-nginx-config-udp"
                }],
                "harbor": {
                    "address": "192.168.103.59",
                    "password": "Harbor12345",
                    "port": 443.0,
                    "protocol": "https",
                    "user": "admin"
                },
                "network": {
                    "networkFlag": "calico",
                    "version": "1"
                },
                "nfs": [{
                    "capacity": "1",
                    "ip": "192.168.103.65",
                    "name": "nfs",
                    "path": "/nfs/top",
                    "type": "nfs"
                }],
                "port": 6443.0,
                "prometheusPort": 30003.0,
                "protocol": "https",
                "storages": []
            },
            "template": [{
                "namespace": "kube-system",
                "serviceName": "auto-scale",
                "type": "auto-scale"
            }, {
                "namespace": "kube-system",
                "serviceName": "calico-kube-controller",
                "type": "calico-cotnroller"
            }, {
                "namespace": "kube-system",
                "serviceName": "calico-node",
                "type": "calico-node"
            }, {
                "namespace": "kube-system",
                "serviceName": "default-http-backend",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "web"
                }],
                "type": "default-http-backend"
            }, {
                "namespace": "kube-system",
                "serviceName": "elasticsearch-logging-v1",
                "servicePort": [{
                    "port": 9200.0,
                    "protocol": "TCP",
                    "type": "web"
                }, {
                    "port": 9300.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "es"
            }, {
                "namespace": "kube-system",
                "serviceName": "etcd",
                "type": "etcd"
            }, {
                "namespace": "kube-system",
                "serviceName": "heapster",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "check"
                }],
                "type": "heapster"
            }, {
                "namespace": "kube-system",
                "serviceName": "influxdb",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "web"
                }, {
                    "port": 8086.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "influxdb"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-apiserver",
                "type": "kube-apiserver"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-controller-manager",
                "type": "kube-controller-manager"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-dns",
                "servicePort": [{
                    "port": 54.0,
                    "protocol": "TCP",
                    "type": "check"
                }, {
                    "port": 53.0,
                    "protocol": "UDP",
                    "type": "dns"
                }],
                "type": "kube-dns"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-proxy",
                "type": "kube-proxy"
            }, {
                "namespace": "kube-system",
                "serviceName": "kube-scheduler",
                "type": "kube-scheduler"
            }, {
                "namespace": "kube-system",
                "serviceName": "nfs-controller",
                "type": "nfs-controller"
            }, {
                "namespace": "kube-system",
                "serviceName": "nginx-controller",
                "servicePort": [{
                    "port": 80.0,
                    "protocol": "TCP",
                    "type": "web"
                }],
                "type": "nginx-controller"
            }, {
                "namespace": "kube-system",
                "serviceName": "node-up-down",
                "type": "node-up-down"
            }, {
                "namespace": "kube-system",
                "serviceName": "oam-api-service",
                "servicePort": [{
                    "port": 8081.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "oam-api"
            }, {
                "namespace": "kube-system",
                "serviceName": "oam-task-service",
                "type": "oma-task"
            }, {
                "namespace": "kube-system",
                "serviceName": "terminal-service",
                "servicePort": [{
                    "port": 8888.0,
                    "protocol": "TCP",
                    "type": "api"
                }],
                "type": "terminal"
            }]
        },
        "status": {
            "conditions": [{
                "status": true,
                "type": "Ready"
            }]
        }
    }],
    "kind": "ClusterList",
    "metadata": {
        "continue": "",
        "resourceVersion": "7758294",
        "selfLink": "/apis/flycloud.cn/v1/clusters"
    }
}

Namespaces增删改查

使用 CoreV1Api apiInstance = new CoreV1Api(); 操作

Node增删改查

使用 CoreV1Api apiInstance = new CoreV1Api(); 操作

Pod增删改查

使用 CoreV1Api apiInstance = new CoreV1Api(); 操作

优先级增删改查

Services增删改查

使用 CoreV1Api apiInstance = new CoreV1Api(); 操作

操作示例

@Test
    public void CoreV1ApiTest(){

        CoreV1Api apiInstance = new CoreV1Api();
        String pretty = "true"; 
        String _continue = "_continue_example"; 
        String fieldSelector = "fieldSelector_example"; 
        String labelSelector = "labelSelector_example"; 
        Integer limit = 56; 
        String resourceVersion = "resourceVersion_example"; 
        Integer timeoutSeconds = 56; 
        Boolean watch = true; 
        try {

            // Namespace列表
            V1NamespaceList result = apiInstance.listNamespace(null,pretty,null,null,null,null,null,null,null);

            // Node列表
            // V1NodeList result = apiInstance.listNode(null,pretty,null,null,null,null,null,null,null);

           // Service列表
            // V1ServiceList result = apiInstance.listNamespacedService("kube-system", null, null, null, null, null, null, null, null, null);

            // Service 详情
            // /api/v1/namespaces/kube-system/services/webapi-service
            // V1Service result = apiInstance.readNamespacedService("flyapi-service", "kube-system", null, null, null);

            System.out.println(result);

            // JSON
            Gson gsnotallow=new Gson();
            String s = gson.toJson(result);
            System.out.println(s);

        } catch (ApiException e) {
            System.err.println("Exception when calling CoreV1Api#listNode");
            e.printStackTrace();
        }
    }

Deployment增删改查

使用 ExtensionsV1beta1Api apiInstance = new ExtensionsV1beta1Api(); 操作

参考链接：

​​https://github.com/kubernetes-client/java/tree/master/kubernetes​​​​https://k8smeetup.github.io/docs/reference/client-libraries​​

​