使用docker搭建redis-cluster

在Linux上使用docker搭建redis-cluster集群,三主三从,并在Spring Boot中使用该集群


搭载集群

下面的集群搭建参考:https://juejin.im/post/5c9ca08f5188252d5a14a31b

拉取Docker镜像

使用pull命令拉取镜像,也可以不拉,待后面新建容器时自动添加。

1
docker pull redis

新建虚拟网卡

1
docker network create redis-net

配置redis.conf文件

我才用的文件目录是:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
/redis-cluster
/redis-7000
/conf
/redis.conf
/data
/redis-7001
/conf
/redis.conf
/data
/redis-7002
/conf
/redis.conf
/data
/redis-7003
/conf
/redis.conf
/data
/redis-7004
/conf
/redis.conf
/data
/redis-7005
/conf
/redis.conf
/data

redis-7000,为例

1
2
3
4
5
6
7
8
9
port 7000
protected-mode no
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 0.0.0.0
cluster-announce-port 7000
cluster-announce-bus-port 17000
appendonly yes

要注意的是cluster-announce-ip应该要天自己所在网卡的IP,也就是上面新建的虚拟网卡redis-net分配给该redis的IP,由于目前还不知道具体IP是多少,暂时留空

新建容器

1
2
3
4
5
6
7
8
9
10
11
12
for port in `seq 7000 7005`
do
docker run -d -p ${port}:${port} -p 1${port}:1${port} \
-v /home/ogic/docker-container/redis-cluster/redis-${port}/conf/redis.conf:/usr/local/etc/redis/redis.conf \
-v /home/ogic/docker-container/redis-cluster/redis-${port}/data:/data \
--restart always \
--name redis-${port} \
--net redis-net \
--sysctl net.core.somaxconn=1024 \
redis \
redis-server /usr/local/etc/redis/redis.conf
done

docker run [options] image [command] [arg...]

option meaning
-d 后台运行(守护进程运行)
-p 暴露容器端口并与主机做端口映射
-v 磁盘绑定
–restart 容器重启选项
–name 容器名
–net 容器使用网卡
–sysctl net.core.somaxconn=1024 linux中listen backlog 的上限,默认值是128
command meaning
redis-server /usr/local/etc/redis/redis.conf 以该路径下的redis.conf为配置文件启动redis

修改容器IP

使用docker network inspect redis-net查看容器IP

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
$ docker network inspect redis-net 
[
{
"Name": "redis-net",
"Id": "239656759a677fbd80f3705037312ff1658ed7f8872337271769537f2d3e2be8",
"Created": "2019-09-03T14:10:22.890513968+08:00",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.20.0.0/16",
"Gateway": "172.20.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"2f666cbafca26d5f479110890e59db322f90eeda5b675eb341b705b265b6d79e": {
"Name": "redis-7005",
"EndpointID": "7f92f0e8616625627007a66ad34f99d199522be3e9c82d4a569405e53c913b66",
"MacAddress": "02:42:ac:14:00:03",
"IPv4Address": "172.20.0.3/16",
"IPv6Address": ""
},
"6c453e97d3fdd9ec5339aa7b02b6fdc169005d4f0f61b66aabb5a4c04165b7aa": {
"Name": "redis-7000",
"EndpointID": "d708b37e94c3475a196a8465385ffa7663a7ea240087c6f7f7043dfa58ebc83b",
"MacAddress": "02:42:ac:14:00:05",
"IPv4Address": "172.20.0.5/16",
"IPv6Address": ""
},
"74b9643d5601b113fd720378afd03c63d71902ae64adbb386913535d3c8c8484": {
"Name": "redis-7004",
"EndpointID": "3f921d04a5edc16408a8f04dd4813fd8924a51415c46083bc8bcb2367ba8a07a",
"MacAddress": "02:42:ac:14:00:04",
"IPv4Address": "172.20.0.4/16",
"IPv6Address": ""
},
"78c3b79881c7cc5afb0466c325bdde11d17bde322b7d34c584cf2c62e3f2e90f": {
"Name": "redis-7001",
"EndpointID": "ea687be382e26cf2efff36dd7ebf66c25d8addcc96fdc7d92211e247889ed9c2",
"MacAddress": "02:42:ac:14:00:07",
"IPv4Address": "172.20.0.7/16",
"IPv6Address": ""
},
"8bd20b1db64d3fd6e4b914616fb2399c06b4470916ab9aa7b33ea098b844f8db": {
"Name": "redis-7003",
"EndpointID": "c85240fba023bc7efb706ab620489df197581e215d3aa567817f203af67cd59b",
"MacAddress": "02:42:ac:14:00:02",
"IPv4Address": "172.20.0.2/16",
"IPv6Address": ""
},
"ac1e36d16f50525aced7bfe6522f88a49bfffa7cfa97cdabbd2e2c9a61a73180": {
"Name": "redis-7002",
"EndpointID": "90bafa5d912506b2bbb4e5f5b1d1067a336f5be29f67c4f6fca9c92542420caa",
"MacAddress": "02:42:ac:14:00:06",
"IPv4Address": "172.20.0.6/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]

然后在对应redis.conf中修改cluster-announce-ip项,根据上面虚拟网卡分配的IP进行修改

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# redis-7000
cluster-announce-ip 172.20.0.2
---
# redis-7001
cluster-announce-ip 172.20.0.3
---
# redis-7002
cluster-announce-ip 172.20.0.4
---
# redis-7003
cluster-announce-ip 172.20.0.5
---
# redis-7004
cluster-announce-ip 172.20.0.6
---
# redis-7005
cluster-announce-ip 172.20.0.7

修改完全部6个redis.conf后,执行

1
2
3
for port in `seq 7000 7005`
do docker restart redis-${port}
done

运行集群

进入容器redis-7000

1
docoker exec -it redis-7000 bash

执行命令redis-cli --cluster create {ip}:{port} --cluster-replicas 1

在本例中即:

1
redis-cli --cluster create 172.20.0.2:7000 172.20.0.3:7001 172.20.0.4:7002 172.20.0.5:7003 172.20.0.6:7004 172.20.0.7:7005 --cluster-replicas 1

结果如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
$ redis-cli --cluster create 172.20.0.2:7000 172.20.0.3:7001 172.20.0.4:7002 172.20.0.5:7003 172.20.0.6:7004 172.20.0.7:7005 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.20.0.6:7004 to 172.20.0.2:7000
Adding replica 172.20.0.7:7005 to 172.20.0.3:7001
Adding replica 172.20.0.5:7003 to 172.20.0.4:7002
M: 351c958bd4c3b36423f3cc7de016b045e9e2f185 172.20.0.2:7000
slots:[0-5460] (5461 slots) master
M: 4b3656b1abcfbc2bc87ec6ad679c43ad951eedde 172.20.0.3:7001
slots:[5461-10922] (5462 slots) master
M: 1f3fceb3b823c5ba7320b855511bdbec350e141a 172.20.0.4:7002
slots:[10923-16383] (5461 slots) master
S: 6c9d306c45c3453143e05158cf44e6d4bff93013 172.20.0.5:7003
replicates 1f3fceb3b823c5ba7320b855511bdbec350e141a
S: 7ef99b3dac7d3175b52ad9ac6f5e9df9a466900d 172.20.0.6:7004
replicates 351c958bd4c3b36423f3cc7de016b045e9e2f185
S: 2c892465a513946f037cdee5439c2f866b356471 172.20.0.7:7005
replicates 4b3656b1abcfbc2bc87ec6ad679c43ad951eedde
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 172.20.0.2:7000)
M: 351c958bd4c3b36423f3cc7de016b045e9e2f185 172.20.0.2:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 2c892465a513946f037cdee5439c2f866b356471 172.20.0.7:7005
slots: (0 slots) slave
replicates 4b3656b1abcfbc2bc87ec6ad679c43ad951eedde
S: 7ef99b3dac7d3175b52ad9ac6f5e9df9a466900d 172.20.0.6:7004
slots: (0 slots) slave
replicates 351c958bd4c3b36423f3cc7de016b045e9e2f185
M: 4b3656b1abcfbc2bc87ec6ad679c43ad951eedde 172.20.0.3:7001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 6c9d306c45c3453143e05158cf44e6d4bff93013 172.20.0.5:7003
slots: (0 slots) slave
replicates 1f3fceb3b823c5ba7320b855511bdbec350e141a
M: 1f3fceb3b823c5ba7320b855511bdbec350e141a 172.20.0.4:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

在Spring Boot中使用该集群

引入依赖

1
2
3
4
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

修改application.yml

application.yml中添加以下内容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
spring:
redis:
jedis:
pool:
max-wait: 5000
max-idle: 50
min-idle: 5
cluster:
nodes:
- 127.0.0.1:7000
- 127.0.0.1:7001
- 127.0.0.1:7002
- 127.0.0.1:7003
- 127.0.0.1:7004
- 127.0.0.1:7005
timeout: 500

自定义RedisTemplate

这一步可以不用做,我只是针对自己使用的情况(使用的key和value的种类)来配置而已,不配置只是可能会出错(例如类型转换异常或空指针异常),且这一步与redis是不是集群无关。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
@Configuration
public class RedisConfig {

private Logger logger = LoggerFactory.getLogger(this.getClass());

/**
* 配置redisTemplate并默认将对象序列化成json格式,然后将bean添加到IOC容器
*
* @param redisConnectionFactory redis链接工厂
* @return redis模板
*/
@Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {

RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();

redisTemplate.setConnectionFactory(redisConnectionFactory);

StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();
redisTemplate.setKeySerializer(stringRedisSerializer);

Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class);
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
jackson2JsonRedisSerializer.setObjectMapper(objectMapper);

redisTemplate.setHashKeySerializer(jackson2JsonRedisSerializer);
redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
}