kubernetes环境从docker迁移到containerd
1.实验环境
本实验共两台节点，vms61是master，vms62是worker
kubernetes版本是v1.23.2
系统版本是 centos7.4
具体如下：
[root@vms61 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vms61.rhce.cc Ready control-plane,master 234d v1.23.2
vms62.rhce.cc Ready <none> 234d v1.23.2
[root@vms61 ~]#
[root@vms61 ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
vms61.rhce.cc Ready control-plane,master 234d v1.23.2 192.168.26.61 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 docker://20.10.9
vms62.rhce.cc Ready <none> 234d v1.23.2 192.168.26.62 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 docker://20.10.9
[root@vms61 ~]# 上面运行了一个测试用的deploy，有两个副本。
[root@vms61 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
web1-665f6b46cb-cqj4x 1/1 Running 0 2m59s
web1-665f6b46cb-nr9hc 1/1 Running 0 2m59s
[root@vms61 ~]# 2.迁移master
1.先对master（vms61）执行drain操作。
[root@vms61 ~]# kubectl drain vms61.rhce.cc --ignore-daemonsets
node/vms61.rhce.cc cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/calico-node-lf6t2, kube-system/kube-proxy-mhr5g
evicting pod kube-system/coredns-6d8c4cb4d-vcbjl
evicting pod kube-system/calico-kube-controllers-78d6f96c7b-cfxvw
evicting pod kube-system/coredns-6d8c4cb4d-5wblx
pod/calico-kube-controllers-78d6f96c7b-cfxvw evicted
pod/coredns-6d8c4cb4d-5wblx evicted
pod/coredns-6d8c4cb4d-vcbjl evicted
node/vms61.rhce.cc drained
[root@vms61 ~]#2.关闭并卸载docker
[root@vms61 ~]# systemctl disable docker --now
Removed symlink /etc/systemd/system/multi-user.target.wants/docker.service.
Warning: Stopping docker.service, but it can still be activated by:
docker.socket
[root@vms61 ~]# systemctl disable docker.socket --now
[root@vms61 ~]#卸载docker
[root@vms61 ~]# yum remove docker-ce docker-ce-cli -y
已加载插件：fastestmirror
...输出...
[root@vms61 ~]# 3.安装并配置containerd
在vms61上安装containerd
[root@vms61 ~]# yum install containerd.io cri-tools -y
[root@vms61~]# crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock步骤1：先生成配置文件/etc/containerd/config.toml。
[root@vms61 ~]# containerd config default > /etc/containerd/config.toml步骤2：使用vim编辑器打开/etc/containerd/config.toml。
第一：搜索mirrors，把
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
改成
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["https://frz7i079.mirror.aliyuncs.com"]第二：搜索sandbox，把
sandbox_image = "k8s.gcr.io/pause:3.6"
改为
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.7"第三：搜索SystemdCgroup，把
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
BinaryName = ""
CriuImagePath = ""
CriuPath = ""
CriuWorkPath = ""
IoGid = 0
IoUid = 0
NoNewKeyring = false
NoPivotRoot = false
Root = ""
ShimCgroup = ""
SystemdCgroup = false
改成（注意这中间的行是被删除了并非忽略不写的意思）
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true保存退出。
4.加载模块及修改参数
在vms61上加载模块
[root@vms61 ~]# modprobe overlay ; modprobe br_netfilter在所有机器上执行下面的命令，目的是系统重启时模块能自动加载。
cat > /etc/modules-load.d/containerd.conf <<EOF
overlay
br_netfilter
EOF在vms61上执行下面的命令，目的是实现重启系统后，参数也能继续生效。
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF让上述参数立即生效。
[root@vms61 ~]# sysctl -p /etc/sysctl.d/k8s.conf
[root@vms61 ~]#重启containerd服务，并设置开机自动启动。
[root@vms61 ~]# systemctl enable containerd ; systemctl restart containerd5.配置并启动kubelet
设置kubelet启动参数
[root@vms61 ~]# cat /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
[root@vms61 ~]# 重启kubelet服务。
[root@vms61 ~]# systemctl restart kubelet6.对master执行uncordon操作
[root@vms61 ~]# kubectl uncordon vms61.rhce.cc
node/vms61.rhce.cc uncordoned
[root@vms61 ~]#7.验证
[root@vms61 ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
vms61.rhce.cc Ready control-plane,master 234d v1.23.2 192.168.26.61 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 containerd://1.6.6
vms62.rhce.cc Ready <none> 234d v1.23.2 192.168.26.62 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 docker://20.10.9
[root@vms61 ~]#
[root@vms61 ~]# 3.迁移worker
1.先对worker（vms62）执行drain操作。
[root@vms61 ~]# kubectl drain vms62.rhce.cc --ignore-daemonsets
node/vms62.rhce.cc cordoned
WARNING: ignoring DaemonSet-managed Pods: kube-system/calico-node-hth7g, kube-system/kube-proxy-24z5b
evicting pod kube-system/coredns-6d8c4cb4d-dxqld
evicting pod default/web1-665f6b46cb-cqj4x
evicting pod default/web1-665f6b46cb-nr9hc
evicting pod kube-system/calico-kube-controllers-78d6f96c7b-bckd8
evicting pod kube-system/coredns-6d8c4cb4d-bfshv
pod/web1-665f6b46cb-cqj4x evicted
pod/web1-665f6b46cb-nr9hc evicted
pod/calico-kube-controllers-78d6f96c7b-bckd8 evicted
pod/coredns-6d8c4cb4d-dxqld evicted
pod/coredns-6d8c4cb4d-bfshv evicted
node/vms62.rhce.cc drained
[root@vms61 ~]#2.关闭并卸载docker
[root@vms62 ~]# systemctl disable docker --now
Removed symlink /etc/systemd/system/multi-user.target.wants/docker.service.
Warning: Stopping docker.service, but it can still be activated by:
docker.socket
[root@vms62 ~]# systemctl disable docker.socket --now
[root@vms62 ~]#卸载docker
[root@vms62 ~]# yum remove docker-ce docker-ce-cli -y
已加载插件：fastestmirror
...输出...
[root@vms62 ~]# 3.安装containerd
在vms62上安装containerd
[root@vms62 ~]# yum install containerd.io cri-tools -y
[root@vms62~]# crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock4.把vms61上的文件同步到vms62
[root@vms61 ~]# scp /etc/containerd/config.toml vms62:/etc/containerd/
root@vms62's password:
config.toml 100% 6880 3.8MB/s 00:00
[root@vms61 ~]# scp /etc/modules-load.d/containerd.conf vms62:/etc/modules-load.d/
root@vms62's password:
containerd.conf 100% 21 16.4KB/s 00:00
[root@vms61 ~]# scp /etc/sysctl.d/k8s.conf vms62:/etc/sysctl.d/
root@vms62's password:
k8s.conf 100% 103 219.4KB/s 00:00
[root@vms61 ~]# scp /etc/sysconfig/kubelet vms62:/etc/sysconfig/
root@vms62's password:
kubelet 100% 146 76.4KB/s 00:00
[root@vms61 ~]#5.加载模块并修改参数，并启动kubelet和containerd
[root@vms62 ~]# modprobe overlay ; modprobe br_netfilter
[root@vms62 ~]# sysctl -p /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
[root@vms62 ~]#
[root@vms62 ~]# systemctl enable containerd ; systemctl restart containerd
Created symlink from /etc/systemd/system/multi-user.target.wants/containerd.service to /usr/lib/systemd/system/containerd.service.
[root@vms62 ~]#重启kubelet服务，并设置开机自动启动。
[root@vms62 ~]# systemctl restart kubelet6.对worker执行uncordon操作
[root@vms61 ~]# kubectl uncordon vms62.rhce.cc
node/vms62.rhce.cc uncordoned
[root@vms61 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
vms61.rhce.cc Ready control-plane,master 234d v1.23.2
vms62.rhce.cc Ready <none> 234d v1.23.2
[root@vms61 ~]#7.验证
[root@vms61 ~]# kubectl get nodes -owide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
vms61.rhce.cc Ready control-plane,master 234d v1.23.2 192.168.26.61 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 containerd://1.6.6
vms62.rhce.cc Ready <none> 234d v1.23.2 192.168.26.62 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 containerd://1.6.6
[root@vms61 ~]# 现在可以看到所有节点的运行时都是contianerd了
[root@vms61 ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
web1-665f6b46cb-fl9b4 1/1 Running 0 13m
web1-665f6b46cb-jgzp8 1/1 Running 0 13m