需要 3台 云服务器,每台机器装上 Docker、Kubelet、Kubectl、Kubeadm。
青云 - 官网,自己注册个账号,个人认证后,会给你 优惠券
别担心,不要钱,可以白嫖一段时间(服务器 可用 25元的优惠券)
释放了服务器, 公网IP 这里,如果有 就删除。
我的配额 这里,要确保 公网IP 有3个 以上。如果没有,在 工单 -> 创建工单,联系客服 要。
专属私有网络 VPC 帮助您在公有云上构建出一个专属隔离的网络环境。在 VPC 网络内,您可以自定义 IP 地址范围,创建子网,并在子网内创建云服务器、数据库、大数据等各种云资源。您也可以通过管理路由器自定义更多高级管理功能。
创建完毕vpc后,开始购买云服务器。
购买3台 2核4G 的服务器
选择服务器数量为3。
点击 立即购买,刷新一下 即可看到下面的 公网IPv4,我这里不就 开放了哈。
互相 ping,都可以 通。(其他不在 这个 VPC 网段的服务器,是ping 不通的)
这里我使用xshell连接工具连接三台青云云服务器:
注意 Docker 版本,要和 Kubernetes 版本 有对应关系
# 三台机器上执行
# 安装/更新 yum-utils
yum install -y yum-utils
# 配置 yum源
yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
# 安装 docker 20.10.7 版本
yum -y install docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io
# 查看 docker 版本
docker -v
# 启动 docker
systemctl start docker
# 查看 docker 是否成功, 有 Client 和 Server 即成功
docker version
分别查看 3台服务器 docker 状态
https://82m9ar63.mirror.aliyuncs.com,这个镜像加速 是 雷神的(个人具体可以在 阿里云 配置)
# 三台机子都执行
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{
"registry-mirrors": ["https://82m9ar63.mirror.aliyuncs.com"],
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
# 查看是否配置成功
docker info
# 查看主机名
hostname
# 设置主机名
hostnamectl set-hostname k8s-master
hostnamectl set-hostname k8s-node-1
hostnamectl set-hostname k8s-node-2
# 更新
bash
下面是一些 安全设置
每台机子都执行!!!
# 查看 交换分区
free -m
# 将 SELinux 设置为 permissive 模式(相当于将其禁用) 第一行是临时禁用,第二行是永久禁用
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
# 关闭swap;第一行是临时禁用,第二行是永久禁用
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab
# 允许 iptables 检查桥接流量 (K8s 官方要求)
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# 让配置生效
sysctl --system
如果是本地虚拟机的话 ,虚拟机关闭交换区,三台机器上执行
# 禁用 selinux
vim /etc/selinux/config
SELINUX=disabled
# 关闭swap
swapoff -a # 临时
vim /etc/fstab # 永久
# 注释下面这行
#/dev/mapper/centos-swap swap swap defaults 0 0
# 查看是否成功
free -m
安装 kubelet、kebeadm、kubectl;注意版本 (1.20.9)
三台机器上执行
# 配置 k8s 的 yum 源地址
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF
三台机器上执行
# 安装 kubelet、kubeadm、kubectl
yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9 --disableexcludes=kubernetes
# 启动kubelet
systemctl enable --now kubelet
# 查看 kubelet 状态:一会停止 一会运行。 这个状态是对的,kubelet 等待 kubeadm 发号指令。
systemctl status kubelet
# 配置镜像,生成 images.sh
sudo tee ./images.sh <<-'EOF'
#!/bin/bash
images=(
kube-apiserver:v1.20.9
kube-proxy:v1.20.9
kube-controller-manager:v1.20.9
kube-scheduler:v1.20.9
coredns:1.7.0
etcd:3.4.13-0
pause:3.2
)
for imageName in ${images[@]} ; do
docker pull registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/$imageName
done
EOF
# 拉取镜像
chmod +x ./images.sh && ./images.sh
报这个错,没启动 docker
网段小知识
可以看到 docker 的 ip 是 172.17.x.x,所以 vps 设置 网段时候,不选择 172.17.x.x 的。
注:初始化 主节点命令,只在 master 服务执行。其他 node 服务器 不用。
# 所有机器添加 master 域名映射,以下 IP 为 master 的 IP;
# 访问 cluster-endpoint 即 访问 192.168.100.3
echo "192.168.100.3 cluster-endpoint" >> /etc/hosts
# 主节点初始化 (只在 master 服务器执行, 其他 node 不用)
# --apiserver-advertise-address: master 的 IP
# --control-plane-endpoint: master 的域名
# --service-cidr 和 --pod-network-cidr 是网络范围,雷神 建议不要改。要改的话 2 个cidr 和 vps(172.31.x.x) 的,3 个网络互相不能重叠;还要修改 calico.yaml的 IP(下图有写)。
kubeadm init \
--apiserver-advertise-address=192.168.100.3 \
--control-plane-endpoint=cluster-endpoint \
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images \
--kubernetes-version v1.20.9 \
--service-cidr=10.96.0.0/16 \
--pod-network-cidr=192.168.0.0/16
保证三台机子都能ping通
复制上面 输出的语句,这些命令 需要用到
Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:
kubeadm join cluster-endpoint:6443 --token euztag.6yjbhn94ehg6nhn7 \
--discovery-token-ca-cert-hash sha256:a6e917b751e424c7a761c2276ae02db95d87ba5a3cc6b374182519ce89aba28b \
--control-plane
Then you can join any number of worker nodes by running the following on each as root:
kubeadm join cluster-endpoint:6443 --token euztag.6yjbhn94ehg6nhn7 \
--discovery-token-ca-cert-hash sha256:a6e917b751e424c7a761c2276ae02db95d87ba5a3cc6b374182519ce89aba28b
注:只在 master 服务执行。其他 node 服务器 不用。
在 master 执行
# 下载 calico.yaml
curl https://docs.projectcalico.org/manifests/calico.yaml -O
# 加载配置
kubectl apply -f calico.yaml
如果上面链接下载不到calico,试一下下面的链接:
curl https://docs.projectcalico.org/v3.20/manifests/calico.yaml -O
Worker 节点加入到集群中。
注:此处要复制自己的,不要复制 下面的命令。
# 加入到 集群中,只在 2 个 node 服务器运行。
kubeadm join cluster-endpoint:6443 --token euztag.6yjbhn94ehg6nhn7 \
--discovery-token-ca-cert-hash sha256:a6e917b751e424c7a761c2276ae02db95d87ba5a3cc6b374182519ce89aba28b
回到 master 中 查看
监控一下 服务器的状态,可以发现 master 服务器 消耗比较大。
令牌过期
如果令牌过期了,重新 获取令牌。(在 master 服务器中 执行)
# 重新获取令牌
kubeadm token create --print-join-command
集群自我修复能力测试 (需要 docker 启动的前提下,systemctl start docker)
3台 服务器 分别重启
# 重启
reboot
集群 还是好好的。 nodes 的 STATUS 都是 Ready,查看 pods STATUS 过一会 都是 Running。
部署 dashboard(可视化页面),kubernetes 官方提供的可视化界面
在 master 执行
# 下载 yaml文件
curl https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml -O
修改recommended.yaml 修改两处nodeName。
vi recommended.yaml
然后运行
kubectl apply -f recommended.yaml
附上我的yaml文件:
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: Namespace
metadata:
name: kubernetes-dashboard
---
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
ports:
- port: 443
targetPort: 8443
selector:
k8s-app: kubernetes-dashboard
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kubernetes-dashboard
type: Opaque
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-csrf
namespace: kubernetes-dashboard
type: Opaque
data:
csrf: ""
---
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-key-holder
namespace: kubernetes-dashboard
type: Opaque
---
kind: ConfigMap
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-settings
namespace: kubernetes-dashboard
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
rules:
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster", "dashboard-metrics-scraper"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
verbs: ["get"]
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
rules:
# Allow Metrics Scraper to get metrics from the Metrics server
- apiGroups: ["metrics.k8s.io"]
resources: ["pods", "nodes"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubernetes-dashboard
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: kubernetes-dashboard
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kubernetes-dashboard
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
nodeName: k8s-master
containers:
- name: kubernetes-dashboard
image: kubernetesui/dashboard:v2.3.1
imagePullPolicy: Always
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
- --namespace=kubernetes-dashboard
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsUser: 1001
runAsGroup: 2001
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
nodeSelector:
"kubernetes.io/os": linux
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: dashboard-metrics-scraper
name: dashboard-metrics-scraper
namespace: kubernetes-dashboard
spec:
ports:
- port: 8000
targetPort: 8000
selector:
k8s-app: dashboard-metrics-scraper
---
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
k8s-app: dashboard-metrics-scraper
name: dashboard-metrics-scraper
namespace: kubernetes-dashboard
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: dashboard-metrics-scraper
template:
metadata:
labels:
k8s-app: dashboard-metrics-scraper
annotations:
seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'
spec:
nodeName: k8s-master
containers:
- name: dashboard-metrics-scraper
image: kubernetesui/metrics-scraper:v1.0.6
ports:
- containerPort: 8000
protocol: TCP
livenessProbe:
httpGet:
scheme: HTTP
path: /
port: 8000
initialDelaySeconds: 30
timeoutSeconds: 30
volumeMounts:
- mountPath: /tmp
name: tmp-volume
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsUser: 1001
runAsGroup: 2001
serviceAccountName: kubernetes-dashboard
nodeSelector:
"kubernetes.io/os": linux
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
volumes:
- name: tmp-volume
emptyDir: {}
等到 Running
# 修改配置文件 找到 type,将 ClusterIP 改成 NodePort
kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
# 找到端口,在安全组放行
kubectl get svc -A |grep kubernetes-dashboard
输入 /type,找到 type,将 ClusterIP 改成 NodePort
在 ID 右键 -> 解绑,找到 用的哪个资源。
添加完后,在点击 应用修改。
https://xxxxxxx:32229 (要注意是 https,port 是映射的端口,在配置文件查看)
这里需要用火狐游览器进行访问,使用chrome游览器有问题。
#创建访问账号,准备一个yaml文件; vi dash.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
kubectl apply -f dash.yaml
#获取访问令牌
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"
eyJhbGciOiJSUzI1NiIsImtpZCI6InpXSkU0TjhCUmVKQzBJaC03Nk9ES2NMZ1daRTRmQ1FMZU9rRUJ3VXRnM3MifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLXgyczhmIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiIzOTZmYjdlNS0wMjA2LTQxMjctOGQzYS0xMzRlODVmYjU0MDAiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.Hf5mhl35_R0iBfBW7fF198h_klEnN6pRKfk_roAzOtAN-Aq21E4804PUhe9Rr9e_uFzLfoFDXacjJrHCuhiML8lpHIfJLK_vSD2pZNaYc2NWZq2Mso-BMGpObxGA23hW0nLQ5gCxlnxIAcyE76aYTAB6U8PxpvtVdgUknBVrwXG8UC_D8kHm9PTwa9jgbZfSYAfhOHWmZxNYo7CF2sHH-AT_WmIE8xLmB7J11vDzaunv92xoUoI0ju7OBA2WRr61bOmSd8WJgLCDcyBblxz4Wa-3zghfKlp0Rgb8l56AAI7ML_snF59X6JqaCuAcCJjIu0FUTS5DuyIObEeXY-z-Rw
命令 | 内容 |
---|---|
kubectl get nodex | 查看 k8s 集群 所有节点 |
kubectl get ns | 查看命名空间 |
kubectl get pods -A | 查看 k8s 集群 部署了 哪些应用 |
kubectl get pods | 查看 默认命名空间(default)部署的应用 |
kubectl get pod -A -owide | 查看 完整的信息,有 部署的 节点、节点IP 等。 |
kubectl get pod -w | 打印 整个状态变化过程 |
Namespace 命名空间(-n) | |
kubectl get pod -n kubernetes-dashboard | 查看 某个命名空间(kubernetes-dashboard)部署的应用 |
kubectl create ns hello | 创建命名空间 hello |
kubectl delete ns hello | 删除命名空间 hello |
集群资源 | |
kubectl apply -f xxxx.yaml | 根据配置文件,给 k8s 集群创建资源 |
kubectl delete -f hello-yaml.yaml | 删除 hello-yaml.yaml 配置文件创建的资源 |
Pod | |
kubectl run mynginx-k8s --image=nginx | 创建名为 mynginx-k8s 的 Pod (默认命名空间) |
kubectl exec -it Pod名字 – /bin/bash | 进入到 Pod 里面(和 docker exec -it 一样的) |
kubectl describe pod mynginx-k8s | 查看 Pod mynginx-k8s 的描述 |
kubectl delete pod mynginx-k8s | 删除 在默认命名空间的 mynginx-k8s |
kubectl delete pod mynginx-k8s -n xxx | 删除 在 xxx 命名空间的 mynginx-k8s |
kubectl logs mynginx-k8s | 查看 Pod mynginx-k8s 的运行日志(默认的命名空间) |
kubectl logs -f -n xxx命名空间 xxxPodName | 查看 Pod 运行日志(-n 就是 namespace 命名空间) |
Deployment | |
kubectl create deployment mytomcat --image=tomcat:8.5.68 | 用 Deployment 部署 Pod,deploy 名为 mytomcat 使用镜像 tomcat:8.5.68 |
kubectl get deploy | 查看 Deployment 创建的资源 |
kubectl delete deploy mytomcat | 删除 Deployment 创建的资源,用 delete 是删不掉的 |
kubectl create deployment my-dep --image=nginx --replicas=3 --replicas=3, | 部署 3个 Pod my-dep |
kubectl scale deploy/my-dep --relicas=5 | 对 Deployment my-dep 扩容 成 5 个 Pod |
kubectl scale deploy/my-dep --relicas=2 | 对 Deployment my-dep 缩容 成 2 个 Pod |
kubectl get deploy/my-dep -oyaml | 查看之前用的镜像(spec.container.image) |
kubectl get deploy/my-dep -oyaml | grep image |
kubectl set image deployment/my-dep nginx=nginx:1.16.1 --record kubectl rollout status deployment/my-dep | 改变 my-dep 中 nginx 版本(nginx 最新版 -> 1.16.1)滚动更新 |
kubectl rollout history deployment/my-dep | 历史记录 |
kubectl rollout histroy deployment/my-dep --revision=2 | 查看 my-dep 某个历史详情 |
kubectl rollout undo deployment/my-dep | my-dep 回滚(回到上个版本) |
kubectl rollout undo deployment/my-dep --to-revision=2 | my-dep 回滚到指定版本 |