【架构】docker实现集群主从扩容【案例3/4】

实现集群主从扩容

当整个集群扛不住流量的情况时，需要给集群扩容增加设备，由3主3从，扩为4主4从。实现：

示意图如下：

第一步：新创建两个节点（redis-node-7，端口6387和 redis-node-8，端口6388），并启动。
尽可能和之前创建节点的命名规则一致

docker run -d --name redis-node-7 --net host --privileged=true -v /data/redis/share/redis-node-7:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6387

docker run -d --name redis-node-8 --net host --privileged=true -v /data/redis/share/redis-node-8:/data redis:6.0.8 --cluster-enabled yes --appendonly yes --port 6388

第二步：进入其中任意一个容器实例内部

docker exec -it redis-node-7 /bin/bash

第三步：将新的redis-node-7节点（空槽位），作为master加入集群 【--cluster add-node】

redis-cli --cluster add-node  12.114.161.16:6387  12.114.161.16:6381

root@localhost:/data# redis-cli --cluster add-node  12.114.161.16:6387    12.114.161.16:6381
>>> Adding node 12.114.161.16:6387 to cluster 12.114.161.16:6381
>>> Performing Cluster Check (using node 12.114.161.16:6381)
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 12.114.161.16:6387 to make it join the cluster.
[OK] New node added correctly.

第四步：查看集群情况（第一次）

redis-cli --cluster check 127.0.0.1:6387

root@localhost:/data# redis-cli --cluster check 127.0.0.1:6387
127.0.0.1:6387 (bf73145e...) -> 0 keys | 0 slots | 0 slaves.      # 新增加Master主机 包含0槽位，0台从机器
12.114.161.16:6381 (6e961a47...) -> 2 keys | 5461 slots | 1 slaves.
12.114.161.16:6383 (78ac7be1...) -> 1 keys | 5461 slots | 1 slaves.
12.114.161.16:6382 (f097fec9...) -> 1 keys | 5462 slots | 1 slaves.
[OK] 4 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:6387)
M: bf73145e2eaaaa08084c9c2dad0d8c757faa48e1 127.0.0.1:6387
   slots: (0 slots) master                               # 新增加Master主机，但是哈希槽个数为 0    
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

第五步：给该主机，重新分派槽号【--cluster reshard】

redis-cli --cluster reshard 12.114.161.16:6381

root@localhost:/data# redis-cli --cluster reshard 12.114.161.16:6381
>>> Performing Cluster Check (using node 12.114.161.16:6381)
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
M: bf73145e2eaaaa08084c9c2dad0d8c757faa48e1 12.114.161.16:6387
   slots: (0 slots) master
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096                # 分配槽位个数为：16384 / Master主机个数 
What is the receiving node ID? bf73145e2eaaaa08084c9c2dad0d8c757faa48e1   # 选择新加的Master主机id号
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1: all    # 选用之前的Master主机作为 哈希槽源
...
...

第六步：查看集群情况（第二次）

redis-cli --cluster check 127.0.0.1:6387

root@localhost:/data# redis-cli --cluster check 12.114.161.16:6381
12.114.161.16:6381 (6e961a47...) -> 1 keys | 4096 slots | 1 slaves.
12.114.161.16:6387 (bf73145e...) -> 1 keys | 4096 slots | 0 slaves.   # 注意： 新加Master主机分得槽位，0个从机
12.114.161.16:6382 (f097fec9...) -> 1 keys | 4096 slots | 1 slaves.
12.114.161.16:6383 (78ac7be1...) -> 1 keys | 4096 slots | 1 slaves.
[OK] 4 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 12.114.161.16:6381)
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
M: bf73145e2eaaaa08084c9c2dad0d8c757faa48e1 12.114.161.16:6387
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master    # 特别注意：新主机的4096槽位，分三个段，从原主机瓜分而来
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

第七步：增加第四台Slave主机（端口6388，redis-node-8）到新的Master主机

redis-cli --cluster add-node ip:新slave端口 ip:新master端口 --cluster-slave --cluster-master-id 新主机节点ID

redis-cli --cluster add-node 12.114.161.16:6388 12.114.161.16:6387 --cluster-slave --cluster-master-id   bf73145e2eaaaa08084c9c2dad0d8c757faa48e1

root@localhost:/data# redis-cli --cluster add-node 12.114.161.16:6388   12.114.161.16:6387 --cluster-slave --cluster-master-id bf73145e2eaaaa08084c9c2dad0d8c757faa48e1
>>> Adding node 12.114.161.16:6388 to cluster 12.114.161.16:6387
>>> Performing Cluster Check (using node 12.114.161.16:6387)
M: bf73145e2eaaaa08084c9c2dad0d8c757faa48e1 12.114.161.16:6387
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 12.114.161.16:6388 to make it join the cluster.   # 端口6388，redis-node-8 新的从主机，加入集群
Waiting for the cluster to join

>>> Configure node as replica of 12.114.161.16:6387.
[OK] New node added correctly.

第八步：查看集群情况（第三次）

redis-cli --cluster check 127.0.0.1:6387

root@localhost:/data# redis-cli --cluster check 127.0.0.1:6387
127.0.0.1:6387 (bf73145e...) -> 1 keys | 4096 slots | 1 slaves.   # 新加入1个从主机
12.114.161.16:6381 (6e961a47...) -> 1 keys | 4096 slots | 1 slaves.
12.114.161.16:6383 (78ac7be1...) -> 1 keys | 4096 slots | 1 slaves.
12.114.161.16:6382 (f097fec9...) -> 1 keys | 4096 slots | 1 slaves.
[OK] 4 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 127.0.0.1:6387)
M: bf73145e2eaaaa08084c9c2dad0d8c757faa48e1 127.0.0.1:6387
   slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
   1 additional replica(s)
S: 4ed46e0368698cd9d5af2ee84631c878b8ebc4d0 12.114.161.16:6386
   slots: (0 slots) slave
   replicates 6e961a4765b555189708bebb69badf7dfad25cd5
S: 664e456ae7d5cf27ea6583ba2f437dac11d4e767 12.114.161.16:6388   # 从主机redis-node-8,成功加入master主机
   slots: (0 slots) slave
   replicates bf73145e2eaaaa08084c9c2dad0d8c757faa48e1
S: 9bc2417a9cd6545ef2445eb4aa0610d586acd73b 12.114.161.16:6385
   slots: (0 slots) slave
   replicates 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2
M: 6e961a4765b555189708bebb69badf7dfad25cd5 12.114.161.16:6381
   slots:[1365-5460] (4096 slots) master
   1 additional replica(s)
S: 0e34147ce2544cd90f5cce78d5493ae9e8625dfe 12.114.161.16:6384
   slots: (0 slots) slave
   replicates f097fec937f54d147d316c1c62e26cb67c9fd059
M: 78ac7be13522bd4ffd6fcf900c6c149c6938ecc2 12.114.161.16:6383
   slots:[12288-16383] (4096 slots) master
   1 additional replica(s)
M: f097fec937f54d147d316c1c62e26cb67c9fd059 12.114.161.16:6382
   slots:[6827-10922] (4096 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

- 前3个Master机器，每个机器“切”出一部分槽位，分别给第四台新加的Master的槽位，好处：
1、避免从新分配，资源分配浪费；
2、原来每个Master槽位的数据，不用重新分配；