1、初始化系统

原文地址：【云原生】k8s节点系统初始化+内核优化 - (sreok.cn)

2、安装容器运行时

containerd原文地址：【笔记】使用Containerd作为Kubernetes Runtime - (sreok.cn)

docker原文地址：【笔记】使用Docker作为Kubernetes Runtime - (sreok.cn)

3、安装k8s工具

1
2
# 指定安装的k8s工具版本，版本查看：https://mirrors.aliyun.com/kubernetes-new/core/stable/
export k8sVersion=v1.30
1
2
3
4
5
6
7
8
cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/$k8sVersion/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/$k8sVersion/rpm/repodata/repomd.xml.key
EOF
1
2
yum install -y kubelet kubeadm kubectl
systemctl enable kubelet --now

4、高可用负载入口

单主集群跳过此步骤

方式一： kube-vip

原文地址：【云原生】高可用负载方案：使用kube-vip作为控制平面负载入口 - (sreok.cn)

方式二：Keepalived + HAProxy

原文地址：【云原生】高可用负载方案：使用Keepalived + HAProxy作为控制平面负载入口 - (sreok.cn)

5、初始化集群

可选一：单节点-不支持ipv6双协议栈

不支持新增控制平面，因为控制平面的IP已经固定

仅支持外部解析ipv4地址访问业务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
cat > kubeadm.yml << EOF
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.20.13.10 # 修改自己的ip
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: k8s-master01 # 本机的主机名
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/k8s-master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
# 控制平面高可用入口，所有的高可用操作，最终都是为了这个位置的ip
# controlPlaneEndpoint: 10.20.13.100:6443
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.30.0 # 版本必须与kubeadm版本一致
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
---
# 配置ipvs
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
# 指定cgroup为systemd
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
EOF

可选二：单节点-支持ipv4/ipv6双协议栈

不支持新增控制平面，因为控制平面的IP已经固定

支持外部解析ipv4/ipv6地址访问业务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
cat > kubeadm.yml << EOF
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.20.13.10 # 修改自己的ip
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  kubeletExtraArgs:
    # 这里使用maser01的IP 
    node-ip: 10.20.13.10,2031:0:130c::10
  name: k8s-master01 # 本机的主机名
  taints:
  - effect: PreferNoSchedule
    key: node-role.kubernetes.io/k8s-master
---
apiServer:
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
# 控制平面高可用入口，所有的高可用操作，最终都是为了这个位置的ip
# controlPlaneEndpoint: 10.20.13.100:6443
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.30.0 # 版本必须与kubeadm版本一致
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16,2000::/64
  serviceSubnet: 10.96.0.0/12,3000::/112
scheduler: {}
---
# 配置ipvs
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
# 指定cgroup为systemd
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
EOF

可选三：高可用-不支持ipv6双协议栈

支持新增控制平面，api-server通过负载ip访问，控制平面高可用

仅支持外部解析ipv4地址访问业务

1
kubeadm config print init-defaults > kubeadm.yml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
cat > kubeadm.yml << EOF
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.20.13.10 # 修改自己的ip
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: k8s-master01 # 本机的主机名
  taints:
  - effect: NoSchedule
    key: node-role.kubernetes.io/k8s-master
---
apiServer:
  certSANs:
  - 127.0.0.1
  - 10.20.13.100 # vip
  - 10.20.13.10 # master01
  - 10.20.13.11 # master02
  - 10.20.13.12 # master03
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
# 控制平面高可用入口，所有的高可用操作，最终都是为了这个位置的ip
controlPlaneEndpoint: 10.20.13.100:6443
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.30.0 # 版本必须与kubeadm版本一致
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
---
# 配置ipvs
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
# 指定cgroup为systemd
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
EOF
1
kubeadm init --config=kubeadm.yml --upload-certs

**可选四：高可用-**支持ipv4/ipv6双协议栈

支持新增控制平面，api-server通过负载ip访问，控制平面高可用

支持外部解析ipv4/ipv6地址访问业务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
cat > kubeadm.yml << EOF
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 24h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 10.20.13.10 # 修改自己的ip
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  kubeletExtraArgs:
    # 这里使用maser01的IP 
    node-ip: 10.20.13.10,2031:0:130c::10
  name: k8s-master01 # 本机的主机名
  taints:
  - effect: PreferNoSchedule
    key: node-role.kubernetes.io/k8s-master
---
apiServer:
  certSANs:
  - 127.0.0.1
  - 10.20.13.100 # vip
  - 10.20.13.10 # master01
  - 10.20.13.11 # master02
  - 10.20.13.12 # master03
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
# 控制平面高可用入口，所有的高可用操作，最终都是为了这个位置的ip
controlPlaneEndpoint: 10.20.13.100:6443
controllerManager: {}
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.30.0 # 版本必须与kubeadm版本一致
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16,2000::/64
  serviceSubnet: 10.96.0.0/12,3000::/112
scheduler: {}
---
# 配置ipvs
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: ipvs
---
# 指定cgroup为systemd
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
EOF
1
kubeadm init --config=kubeadm.yml --upload-certs

需要注意：上面无论是单节点还是高可用，初始化后都是单节点，高可用可以在后面操作新增控制节点。

配置命令行

1
2
3
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

新增工作节点

1
2
# 生成加入节点命令
kubeadm token create --print-join-command

新增控制节点

1
2
# 生成加入节点命令
kubeadm token create --print-join-command
1
2
# 生成控制节点certificate-key 
kubeadm init phase upload-certs --upload-certs

通过--control-plane --certificate-key拼接命令

kubeadm join 10.20.13.100:6443 –token abcdef.0123456789abcdef –discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxx –control-plane –certificate-key xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

安装命令自动补齐

1
2
3
4
yum install bash-completion -y
source /usr/share/bash-completion/bash_completion
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc

6、安装CNI插件

原文地址：【云原生】Kubernetes 安装CNI插件calico并设置双协议栈- (sreok.cn)

卸载集群

1
2
3
4
5
6
7
8
kubeadm reset -f
rm -rf /etc/kubernetes/
rm -rf /var/lib/kubelet/
rm -rf /var/lib/etcd/
rm -rf $HOME/.kube/
rm -rf /opt/cni/
rm -rf /etc/cni/net.d/
systemctl restart network