备注:安装mysql只是测试vault加密的数据能否测试登录mysql,并不是把mysql作为vault数据库存储使用
$ kubectl create namespace vault
$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
$ helm install mysql bitnami/mysql -n vault 【启动较慢等2分钟】
$ kubectl get pods -n vault
NAME READY STATUS RESTARTS AGE
mysql-0 1/1 Running 0 2m58s
$ kubectl get services -n vault
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.100.68.110 <none> 3306/TCP 15m
mysql-headless ClusterIP None <none> 3306/TCP 15m
$ ROOT_PASSWORD=$(kubectl get secret --namespace vault mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
$ helm version
$ helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
$ helm install vault hashicorp/vault --namespace vault --dry-run
$ helm search repo hashicorp/vault --versions
$ helm install vault hashicorp/vault --namespace vault --version 0.27.0 --set='server.ha.enabled=true' --set='server.ha.raft.enabled=true'
$ kubectl get pods -n vault
NAME READY STATUS RESTARTS AGE
vault-0 0/1 Running 0 30s
vault-1 0/1 Running 0 30s
vault-2 0/1 Running 0 30s
mysql-0 1/1 Running 0 8m35s
$ kubectl exec -n vault vault-0 -- vault status
Key Value
--- -----
Seal Type shamir
Initialized false
Sealed true
Total Shares 0
Threshold 0
Unseal Progress 0/0
Unseal Nonce n/a
Version 0.27.0
Storage Type raft
HA Enabled true
command terminated with exit code 2
备注:为什么要进行vault初始化,如果不初始化,vault服务pod的READY会一直处于0/1状态,只有把vault的pod进行初始化生成初始token服务才会正常启动运行
cluster-keys.json提供给后续pod使用
$ kubectl exec -n vault vault-0 -- vault operator init \
-key-shares=1 \
-key-threshold=1 \
-format=json > cluster-keys.json
cluster-keys.json文件里的初始密钥
$ cat cluster-keys.json | jq -r ".unseal_keys_b64[]"
rrUtT32GztRy/pVWmcH0ZQLCCXon/TxCgi40FL1Zzus=
VAULT_UNSEAL_KEY
捕获 Vault 初始密钥变量$ VAULT_UNSEAL_KEY=$(cat cluster-keys.json | jq -r ".unseal_keys_b64[]")
vault-0
?上运行的 Vault , 就是把上面的初始化密钥倒进vault-0
$ kubectl exec -n vault vault-0 -- vault operator unseal $VAULT_UNSEAL_KEY
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 0.27.0
Storage Type raft
Cluster Name vault-cluster-16efc511
Cluster ID 649c814a-a505-421d-e4bb-d9175c7e6b38
HA Enabled true
HA Cluster n/a
HA Mode standby
Active Node Address <none>
Raft Committed Index 31
Raft Applied Index 31
vault-0
$ kubectl exec -n vault vault-0 -- vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 0.27.0
Storage Type raft
Cluster Name vault-cluster-16efc511
Cluster ID 649c814a-a505-421d-e4bb-d9175c7e6b38
HA Enabled true
HA Cluster https://vault-0.vault-internal:8201
HA Mode active
Active Since 2022-05-19T17:41:07.226862254Z
Raft Committed Index 36
Raft Applied Index 36
$ cat cluster-keys.json | jq -r ".root_token"
hvs.3VYhJODbhlQPeW5zspVvBCzD
CLUSTER_ROOT_TOKEN?
捕获 Vault的 root密钥/根令牌$ CLUSTER_ROOT_TOKEN=$(cat cluster-keys.json | jq -r ".root_token")
vault-0
$ kubectl exec -n vault vault-0 -- vault login $CLUSTER_ROOT_TOKEN
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token hvs.3VYhJODbhlQPeW5zspVvBCzD
token_accessor 5sy3tZm3qCQ1ai7wTDOS97XG
token_duration ∞
token_renewable false
token_policies ["root"]
identity_policies []
policies ["root"]
vault-0
$ kubectl exec -n vault vault-0 internal-app -- vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
09d9b35d-0336-7de7-cc94-90a1f3a0aff8 vault-0.vault-internal:8201 leader true
vault-1?
Vault 集群$ kubectl exec -n vault vault-1 -- vault operator raft join http://vault-0.vault-internal:8200
Key Value
--- -----
Joined true
vault-1
使用解封密钥解封 Vault 服务器$ kubectl exec -n vault vault-1 -- vault operator unseal $VAULT_UNSEAL_KEY
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 0.27.0
Storage Type raft
Cluster Name vault-cluster-16efc511
Cluster ID 649c814a-a505-421d-e4bb-d9175c7e6b38
HA Enabled true
HA Cluster https://vault-0.vault-internal:8201
HA Mode standby
Active Node Address http://192.168.58.131:8200
Raft Committed Index 76
Raft Applied Index 76
vault-2
Vault 集群$ kubectl exec -n vault vault-2 -- vault operator raft join http://vault-0.vault-internal:8200
Key Value
--- -----
Joined true
vault-2
使用解封密钥解封 Vault 服务器?$ kubectl exec -n vault vault-2 -- vault operator unseal $VAULT_UNSEAL_KEY
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 1
Threshold 1
Version 0.27.0
Storage Type raft
Cluster Name vault-cluster-16efc511
Cluster ID 649c814a-a505-421d-e4bb-d9175c7e6b38
HA Enabled true
HA Cluster https://vault-0.vault-internal:8201
HA Mode standby
Active Node Address http://192.168.58.131:8200
Raft Committed Index 76
Raft Applied Index 76
$ kubectl exec -n vault vault-0 -- vault operator raft list-peers
Node Address State Voter
---- ------- ----- -----
09d9b35d-0336-7de7-cc94-90a1f3a0aff8 vault-0.vault-internal:8201 leader true
7078a8b7-7948-c224-a97f-af64771ad999 vault-1.vault-internal:8201 follower true
aaf46893-0a93-17ce-115e-f57033d7f41d vault-2.vault-internal:8201 follower true
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vault-0 1/1 Running 0 5m49s
vault-1 1/1 Running 0 5m48s
vault-2 1/1 Running 0 5m47s
vault-agent-injector-5945fb98b5-vzbqv 1/1 Running 0 5m50s
mysql-0 1/1 Running 0 13m54s
database
$ kubectl exec -n vault vault-0 -- vault secrets enable database
Success! Enabled the database secrets engine at: database/
$ kubectl exec -n vault vault-0 -- vault write database/config/mysql \
plugin_name=mysql-database-plugin \
connection_url="{{username}}:{{password}}@tcp(mysql.vault.svc.cluster.local:3306)/" \
allowed_roles="readonly" \
username="root" \
password="$ROOT_PASSWORD"
Success! Data written to: database/config/mysql
readonly 【max_ttl令牌最长过期时间】
$ kubectl exec -n vault vault-0 -- vault write database/roles/readonly \
db_name=mysql \
creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';GRANT SELECT ON *.* TO '{{name}}'@'%';" \
default_ttl="1h" \
max_ttl="24h"
Success! Data written to: database/roles/readonly
readonly
$ kubectl exec -n vault vault-0 -- vault read database/creds/readonly
Key Value
--- -----
lease_id database/creds/readonly/qtWlgBT1YTQEPKiXe7CrotsT
lease_duration 1h
lease_renewable true
password WLESe5T-RLkTj-h-lDbT
username v-root-readonly-pk168KvLS8sc80Of
$ kubectl exec -n vault --stdin=true --tty=true vault-0 -- /bin/sh
/ $
$ vault auth enable kubernetes
Success! Enabled kubernetes auth method at: kubernetes/
$ vault write auth/kubernetes/config \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
Success! Data written to: auth/kubernetes/config
$ vault policy write devwebapp - <<EOF
path "database/creds/readonly" {
capabilities = ["read"]
}
EOF
devweb-app【bound_service_account_names 表示RBAC用户名称,bound_service_account_namespaces表示RBAC用户所在的名称空间,policies表示使用的策略名称,ttl表示
?Kubernetes 身份验证角色超时时间】
$ vault write auth/kubernetes/role/devweb-app \
bound_service_account_names=internal-app \
bound_service_account_namespaces=vault \
policies=devwebapp \
ttl=24h
Success! Data written to: auth/kubernetes/role/devweb-app
RBAC用户
$ cat > internal-app.yaml <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
name: internal-app
namespace: vault
EOF
$ kubectl apply -f internal-app.yaml
cat > devwebapp.yaml <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: devwebapp
namespace: vault
labels:
app: devwebapp
spec:
replicas: 1
selector:
matchLabels:
app: devwebapp
template:
metadata:
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-cache-enable: "true"
vault.hashicorp.com/role: "devweb-app" #【Kubernetes 身份验证角色名称】
vault.hashicorp.com/agent-inject-secret-database-connect.sh: "database/creds/readonly" #【设置密码保存的脚本名称database-connect.sh,以及要访问的密钥文件路径】
vault.hashicorp.com/agent-inject-template-database-connect.sh: |
{{- with secret "database/creds/readonly" -}}
mysql -h my-release-mysql.vault.svc.cluster.local --user={{ .Data.username }} --password={{ .Data.password }} my_database
{{- end -}} 【给database-connect.sh脚本传入特定的值】
labels:
app: devwebapp
spec:
serviceAccountName: internal-app
containers:
- name: devwebapp
image: nginx:latest
ports:
- containerPort: 80
EOF
$ kubectl apply -f devwebapp.yaml
deployment/devwebapp created
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
devwebapp 2/2 Running 0 36s
mysql-0 1/1 Running 0 25m32s
vault-0 1/1 Running 0 17m40s
vault-1 1/1 Running 0 17m40s
vault-2 1/1 Running 0 17m40s
vault-agent-injector-76fff8f7c6-lk6gz 1/1 Running 0 17m40s
/vault/secrets/database-connect.sh 的
机密值kubectl exec -n vault --stdin=true \
--tty=true devwebapp \
--container devwebapp \
-- cat /vault/secrets/database-connect.sh
出现如下所示表示获取密钥成功
mysql -h my-release-mysql.default.svc.cluster.local --user=v-kubernetes-readonly-zpqRzAee2b --password=Jb4epAXSirS2s-pnrI9- my_database
以上完成了vault部署到测试的全部流程
官方文档参考:Vault installation to Amazon Elastic Kubernetes Service via Helm | Vault | HashiCorp Developer
vault 中文参考文档: