nerdctl+buildkit+containerd实现容器镜像打包

背景

众所周知，K8s1.24版本开始已经不支持docker作为默认的容器运行时，前段时间测试了将容器运行时换成containerd的，今天再测试将容器镜像管理客户端工具替换成nerdctl,而nerdctl本身是不能直接进行镜像构建的，需要buildkit作为构建工具，下面记录实现步骤

客户端安装

安装nerdctl

下载地址

https://github.com/containerd/nerdctl/releases/download/v0.20.0/nerdctl-0.20.0-linux-amd64.tar.gz

解压后得到nerdctl可执行文件，直接复制到/usr/bin,并给予执行权限chmod +x /usr/bin/nerdctl

安装buildkt

下载地址

https://github.com/moby/buildkit/releases/download/v0.10.3/buildkit-v0.10.3.linux-amd64.tar.gz

解压后得到一系列build开头的进进制文件，分别将他们移到/usr/bin/并添加可执行权限

配置允许私人仓库登陆

vi /etc/containerd/config.toml
添加

        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."harbor.xxx.com"]
      [plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.xxx.com".tls]
          insecure_skip_verify = true

添加后重启containerd

systemctl restart containerd

构建测试demo镜像

编写Dockerfile

FROM java:8-jre-alpine
ENV TZ=Asia/Shanghai
RUN set -eux;     ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime;     echo Asia/Shanghai > /etc/timezone ;     sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories;    apk update;     apk add --no-cache libuuid libstdc++;
RUN mkdir /home/logs
COPY xxx /home/xxx
COPY start.xxx.sh /home/start.xxx.sh
VOLUME /home/logs
EXPOSE 5512
WORKDIR /home
ENTRYPOINT ["/bin/sh", "-c", "/home/start.xxx.sh"]

构建镜像

nerdctl -t harbor.xxx.com/xxx/xxx:20220616

上传到harbor镜像仓库

nerdctl login -u admin -p "xxx" harbor.xxx.com
nerdctl push harbor.xxx.com/xxx/xxx:2022016

PS

这里面有一个大坑，使用nginx代理harbor的话始终没法用nerdctl登陆harbor仓库，即使按网上说的将harbor里面nginx的配置注释也没有用

vi common/config/nginx/nginx.conf

vi common/config/registry/config.yml

这个改了也没有用，如果有大佬遇到过还望不惜赐教

nerdctl in containerd

在容器中使用nerdctl需要将本地相关可执行文件或sock文件映射给容器使用，这里配置在jenkins在K8s环境里面使用nerdctl和buildkit，给出相关配置

        volumeMounts:
        - mountPath: /var/jenkins_home
          name: data
        - mountPath: /etc/localtime
          name: localtime
        - mountPath: /usr/bin/kubectl
          name: kubectl
        - mountPath: /usr/bin/buildctl
          name: buildctl
        - mountPath: /usr/bin/nerdctl
          name: nerdctl
        - mountPath: /var/run/buildkit/buildkitd.sock
          name: buildkitd-sock
        - mountPath: /run/containerd/containerd.sock
          name: containerd-sock
      volumes:
      - hostPath:
          path: /deploy/sorts/jenkins/data
          type: ""
        name: data
      - hostPath:
          path: /etc/localtime
          type: ""
        name: localtime
      - hostPath:
          path: /usr/bin/kubectl
          type: ""
        name: kubectl
      - hostPath:
          path: /usr/bin/buildctl
          type: ""
        name: buildctl
      - hostPath:
          path: /usr/bin/nerdctl
          type: ""
        name: nerdctl
      - hostPath:
          path: /var/run/buildkit/buildkitd.sock
          type: ""
        name: buildkitd-sock
      - hostPath:
          path: /run/containerd/containerd.sock
          type: ""
        name: containerd-sock