Python: ubuntu安装配置redis3.0的cluster集群模式
May 29, 2015
今天，天气不错…. 出去逛了下，在咖啡店看了会美女…. 休闲时，看到携程的一朋友在github关注了一个python redis cluster集群模块…. 立马警醒，不能在看妹子了…. 人家大周末也学习，我也要学习….
标注下，关于redis cluster配置安装的文章是在 http://xiaorui.cc/?p=1443 http://xiaorui.cc
公司的业务大量的依靠redis做支撑，每个业务线的redis使用量都不小，像我们buzz业务就有好几个128G内存的redis服务器，在程序里面做的一致性hash。也尝试过用keepalived的ha方式来保证redis的高可用，但是keepalived也只能保证两个master的高可行，再就是用了keepalived之后，只能是一个读，一个做写，在某种程度上来说，在我们这种写入远比读业务要多的场景下，他是在是浪费…. 单点问题一直是我们关注的，我自己也尝试过豌豆荚的codis，也听过codis作者的演讲….. 但现在redis3.0正式版也已经出来了，问了下周围的圈子里面的人，也是网易有道云在用，貌似在rc的版本，他们就在使用了…. 当然不能 人云亦云 ，我自己也要测试下redis codis….
同事说，公司的针对redis cluster的机器已经分配下来了，今天就先在单机简单的测试下，后期可能在部署上采用docker redis的方式….. 现在同事们已经很享受用docker来快速的部署业务了。 我会在今天下午的时候，把redis cluster的dockerfile分享出来。
我的测试机器是ubuntu14:10,本来是打算用centos的，结果起不来…. 首先下载并安装redis3
Python
wget http://download.redis.io/releases/redis-3.0.1.tar.gz tar zxvf redis-3.0.1.tar.gz cd redis-3.0.1/ make make install
在单机测试的话，我们用redis-server分别启动多个文件就可以了，我这里采用的是6个端口，在redis cluster 里是三主、三从…. 当时也可以按照自己的情况设立主从…. 需要注意的是redis.conf要开发cluster的选项。 我简单写了一个批量修改redis.conf的脚本.
Python
for i in `seq -w 0 5`;do
cp redis.conf 700$i.conf
sed -i "s/6379/700$i/g>" 700$i.conf
sed -i 's/daemonize no/daemonize yes/g' 700$i.conf
sed -i 's/# cluster-enabled yes/cluster-enabled yes/g' 700$i.conf
sed -i 's/# cluster-node-timeout 15000/cluster-node-timeout 15000/g' 700$i.conf
sed -i "s/# cluster-config-file nodes-700*/cluster-config-file nodes-700$i.conf/g>" 700$i.conf
redis-server 700$i.conf
done
如果你想自己手动修改的话，注意这么几个点. 端口一定不要重复，然后就是cluster-enabled yes
Python
port xxxx appendonly yes cluster-enabled yes cluster-config-file nodes.conf cluster-node-timeout 5000
如果只是更改了端口，会造成一个无法创建集群的问题。
Python
Installing RDoc documentation for redis-3.2.1... root@ubuntu:~/redis-3.0.1# ./src/redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 >>> Creating cluster Connecting to node 127.0.0.1:7000: OK [ERR] Node 127.0.0.1:7000 is not configured as a cluster node. root@ubuntu:~/redis-3.0.1#
启动后，我们就可以在linux 进城管理里面看到redis的进程. 注意后面的端口是刚才咱们指定的。
Python
root@ubuntu:~/redis-3.0.1/src# ps aux|grep redis root 4974 0.1 0.9 40416 9388 ? Ssl 12:11 0:03 redis-server *:7000 [cluster] root 4983 0.1 0.9 40416 9452 ? Ssl 12:11 0:04 redis-server *:7001 [cluster] root 4992 0.1 0.9 40416 9440 ? Ssl 12:11 0:03 redis-server *:7002 [cluster] root 5001 0.1 0.9 40416 9420 ? Ssl 12:11 0:04 redis-server *:7003 [cluster] root 5010 0.1 0.7 40416 7392 ? Ssl 12:11 0:04 redis-server *:7004 [cluster] root 5019 0.1 0.9 40416 9424 ? Ssl 12:11 0:04 redis-server *:7005 [cluster]
redis cluster的配置是用的ruby脚本写得，那么就需要你最少安装了ruby (apt-get install ruby )和gem。更主要的是你还要用安装ruby所需要的redis模块。
Python
root@ubuntu:~/redis-3.0.1# ./src/redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- redis (LoadError)
from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from ./src/redis-trib.rb:25:in `<main>'
root@ubuntu:~/redis-3.0.1#
安装下ruby的redis驱动。
Python
root@ubuntu:~/redis-3.0.1# gem install redis Fetching: redis-3.2.1.gem (100%) Successfully installed redis-3.2.1 1 gem installed Installing ri documentation for redis-3.2.1... Installing RDoc documentation for redis-3.2.1... root@ubuntu:~/redis-3.0.1#
然后我们用redis-trib.rb来创建redis cluster集群，下面是成功的提示。
Python
root@ubuntu:~/redis-3.0.1# cd src root@ubuntu:~/redis-3.0.1/src# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 >>> Creating cluster Connecting to node 127.0.0.1:7000: OK Connecting to node 127.0.0.1:7001: OK Connecting to node 127.0.0.1:7002: OK Connecting to node 127.0.0.1:7003: OK Connecting to node 127.0.0.1:7004: OK Connecting to node 127.0.0.1:7005: OK >>> Performing hash slots allocation on 6 nodes... Using 3 masters: 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 Adding replica 127.0.0.1:7003 to 127.0.0.1:7000 Adding replica 127.0.0.1:7004 to 127.0.0.1:7001 Adding replica 127.0.0.1:7005 to 127.0.0.1:7002 M: bde32811cbc7a55886413c52b8444d3353b9da86 127.0.0.1:7000 slots:0-5460 (5461 slots) master M: d8815ad0a69436196908ae458f94704ff7325a53 127.0.0.1:7001 slots:5461-10922 (5462 slots) master M: ee05942ee38a56421a07eea01bc6072fe5e23bfd 127.0.0.1:7002 slots:10923-16383 (5461 slots) master S: 25f13acb0707a29cd2952dd0c6bdffa13721b001 127.0.0.1:7003 replicates bde32811cbc7a55886413c52b8444d3353b9da86 S: d6dd1200fe377dc3aee2fd7d11a57504a6bd7529 127.0.0.1:7004 replicates d8815ad0a69436196908ae458f94704ff7325a53 S: 2282e7f24bacf1b6b316382eb8f6448bc41bd89d 127.0.0.1:7005 replicates ee05942ee38a56421a07eea01bc6072fe5e23bfd Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join.... >>> Performing Cluster Check (using node 127.0.0.1:7000) M: bde32811cbc7a55886413c52b8444d3353b9da86 127.0.0.1:7000 slots:0-5460 (5461 slots) master M: d8815ad0a69436196908ae458f94704ff7325a53 127.0.0.1:7001 slots:5461-10922 (5462 slots) master M: ee05942ee38a56421a07eea01bc6072fe5e23bfd 127.0.0.1:7002 slots:10923-16383 (5461 slots) master M: 25f13acb0707a29cd2952dd0c6bdffa13721b001 127.0.0.1:7003 slots: (0 slots) master replicates bde32811cbc7a55886413c52b8444d3353b9da86 M: d6dd1200fe377dc3aee2fd7d11a57504a6bd7529 127.0.0.1:7004 slots: (0 slots) master replicates d8815ad0a69436196908ae458f94704ff7325a53 M: 2282e7f24bacf1b6b316382eb8f6448bc41bd89d 127.0.0.1:7005 slots: (0 slots) master replicates ee05942ee38a56421a07eea01bc6072fe5e23bfd [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. root@ubuntu:~/redis-3.0.1/src#
这就说明，集群已经OK了， redis-cli客户端默认是不支持集群模式的…. 如果像以前那么用redis-cli的话，会出现下面的情况….. 各种moved… error
Python
root@ubuntu:~/redis-3.0.1/src# redis-cli -p 7000 127.0.0.1:7000> 127.0.0.1:7000> 127.0.0.1:7000> set a 11 (error) MOVED 15495 127.0.0.1:7002 127.0.0.1:7000> set b 22 OK 127.0.0.1:7000> set c 33 (error) MOVED 7365 127.0.0.1:7001 127.0.0.1:7000> set d 33 (error) MOVED 11298 127.0.0.1:7002 127.0.0.1:7000> set e 33 (error) MOVED 15363 127.0.0.1:7002 127.0.0.1:7000> set f 33 OK 127.0.0.1:7000> set g 33 (error) MOVED 7233 127.0.0.1:7001 127.0.0.1:7000> get g (error) MOVED 7233 127.0.0.1:7001
只有加了-c参数，才能开启redis cluster 模式…. 所以一定要好好看文档，就是因为这没有加-c 参数，导致了花费些时间来我排除问题 。
Python
root@ubuntu:~/redis-3.0.1/src# redis-cli -h
redis-cli 3.0.1
Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]
-a <password> Password to use when connecting to the server.
-r <repeat> Execute specified command N times.
-i <interval> When -r is used, waits <interval> seconds per command.
It is possible to specify sub-second times like -i 0.1.
-n <db> Database number.
-x Read last argument from STDIN.
-d <delimiter> Multi-bulk delimiter in for raw formatting (default: n).
-c Enable cluster mode (follow -ASK and -MOVED redirections).
我们最后来测试下….
Python
root@ubuntu:~/redis-3.0.1/src# redis-cli -c -p 7000 127.0.0.1:7002> set blog xiaorui.cc -> Redirected to slot [7653] located at 127.0.0.1:7001 OK 127.0.0.1:7001> set news xiaorui.cc -> Redirected to slot [5161] located at 127.0.0.1:7000 OK 127.0.0.1:7000> set nima xiaorui.cc -> Redirected to slot [16259] located at 127.0.0.1:7002 OK 127.0.0.1:7002> set ganni xiaorui.cc -> Redirected to slot [1198] located at 127.0.0.1:7000 OK 127.0.0.1:7000> set love xiaorui.cc -> Redirected to slot [16198] located at 127.0.0.1:7002 OK 127.0.0.1:7002> set satlstack xiaorui.cc -> Redirected to slot [2455] located at 127.0.0.1:7000 OK
看来用以前的python redis模块也是不能直接用集群模式了，需要替换成支持redis cluster的python模块….
原文:http://xiaorui.cc/2015/05/16/ubuntu安装测试redis3-0的cluster集群模式/
0 Comments