KeepAlived+LVS+Nginx
架构：
|
角色 |
安装软件 |
IP |
|
Keepalived 主+LVS |
Keepalived+ipvsadm |
192.168.2.221 |
|
Keepalived 从+LVS |
Keepalived+ipvsadm |
192.168.2.222 |
|
Web 集群1 |
Nginx |
192.168.2.187 |
|
Web 集群2 |
Nginx |
192.168.2.188 |
|
虚拟IP |
/ |
192.168.2.223 |
一.Keepalived 主(221) 、Keepalived 从(222) 分别安装ipvsadm 和Keepalived ；
二. 配置keepalived.conf ；
三. Web 集群1(187) 、Web 集群2(188) 分别更改arp 内核参数等；
四. Web 集群1(187) 、Web 集群2(188) 分别安装配置nginx ；
五. 测试负载均衡和故障转移http://192.168.2.223
一.Keepalived 主(221) 、Keepalived 从(222) 分别安装ipvsadm 和Keepalived
1.1 安装ipvsadm
--- Keepalived 主(221) 、Keepalived 从(222) 分别执行
[root@jumplinux01 ~]# rpm -qa|grep ipvsadm
[root@jumplinux01 ~]# yum install ipvsadm -y
1.2 安装Keepalived
--- Keepalived 主(221) 、Keepalived 从(222) 分别安装keepalived
[root@jumplinux01 src]# cd /usr/src/
[root@jumplinux01 src]# rpm -qa|grep openssl-devel
[root@jumplinux01 src]# yum install openssl-devel -y
[root@jumplinux01 src]# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz
[root@jumplinux01 src]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/popt-static-1.13-7.el6.x86_64.rpm
[root@jumplinux01 src]# pwd
/usr/src
[root@jumplinux01 src]# ll -rth
total 316K
-rw-r--r-- 1 root root 22K Jul 3 2011 popt-static-1.13-7.el6.x86_64.rpm
drwxr-xr-x. 2 root root 4.0K Sep 23 2011 kernels
drwxr-xr-x. 2 root root 4.0K Sep 23 2011 debug
-rw-r--r-- 1 root root 283K Dec 1 2016 keepalived-1.2.7.tar.gz
[root@jumplinux01 src]# yum -y install popt-static-1.13-7.el6.x86_64.rpm
[root@jumplinux01 src]# yum -y install kernel-devel make gcc openssl-devel libnl* popt*
[root@jumplinux01 src]# ln -s /usr/src/kernels/2.6.32-220.13.1.el6.x86_64/ /usr/src/linux
[root@jumplinux01 src]# tar zxvf keepalived-1.2.7.tar.gz
[root@jumplinux01 src]# cd keepalived-1.2.7
[root@jumplinux01 src]# ./configure --with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/
[root@jumplinux01 keepalived-1.2.7]# make && make install
[root@jumplinux01 keepalived-1.2.7]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
[root@jumplinux01 keepalived-1.2.7]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
[root@jumplinux01 keepalived-1.2.7]# mkdir /etc/keepalived
[root@jumplinux01 keepalived-1.2.7]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
[root@jumplinux01 keepalived-1.2.7]# cp /usr/local/sbin/keepalived /usr/sbin/
二. 配置keepalived.conf
2.1 Keepalived 主(221) 、Keepalived 从(222) 分别打开IP Forward 功能
[root@jumplinux01 keepalived-1.2.7]# vi /etc/sysctl.conf
......
# Controls IP packet forwarding
#net.ipv4.ip_forward = 0
net.ipv4.ip_forward = 1
......
2.2 Keepalived 主(221) 配置keepalived.conf
[root@jumplinux01 keepalived]# pwd
/etc/keepalived
[root@jumplinux01 keepalived]# cp keepalived.conf /root/keepalived.conf.bak
[root@jumplinux01 keepalived-1.2.7]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
chenjch001@test.com
}
notification_email_from chenjch002@test.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_MASTER
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 60
priority 150
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.223
}
}
virtual_server 192.168.2.223 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 2
protocol TCP
real_server 192.168.2.187 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.2.188 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
2.3 Keepalived 从(222) 配置keepalived.conf
[root@jumplinux02 keepalived]# pwd
/etc/keepalived
[root@jumplinux02 keepalived]# cp keepalived.conf /root/keepalived.conf.bak
[root@jumplinux02 keepalived-1.2.7]# vi /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
chenjch001@test.com
}
notification_email_from chenjch002@test.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_ BACKUP
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 60
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.2.223
}
}
virtual_server 192.168.2.223 80 {
delay_loop 6
lb_algo rr
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 2
protocol TCP
real_server 192.168.2.187 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
real_server 192.168.2.188 80 {
weight 1
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
}
}
}
三. Web 集群1(187) 、Web 集群2(188) 分别更改arp 内核参数等
Web 集群1(187) 、Web 集群1(188) 分别执行：
在keepalived 的配置文件中定义的LVS 模式为DR 模式，还需要在两台rs 上执行lvs_dr_rs.sh 脚本。
[root@rac1 ~]# vi /usr/local/sbin/lvs_dr_rs.sh
#/bin/bash
vip=192.168.2.223
# 把vip 绑定在lo 上，是为了实现rs 直接把结果返回给客户端
ifconfig lo:0 $vip broadcast $vip netmask 255.255.255.255 up
route add -host $vip lo:0
# 以下操作为更改arp 内核参数，目的是为了让rs 顺利发送mac 地址给客户端
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
[root@rac1 ~]# chmod a+x /usr/local/sbin/lvs_dr_rs.sh
[root@rac1 ~]# sh /usr/local/sbin/lvs_dr_rs.sh
[root@rac1 ~]# ip addr|grep 192.168.2.223
inet 192.168.2.223/32 brd 192.168.2.223 scope global lo:0
[root@rac2 ~]# ip addr|grep 192.168.2.223
inet 192.168.2.223/32 brd 192.168.2.223 scope global lo:0
四. Web 集群1(187) 、Web 集群2(188) 分别安装配置nginx
4.1 Web 集群1(187) 、Web 集群2(188) 分别安装nginx
[root@rac1 local]# cd /usr/local/
[root@rac1 local]# wget http://nginx.org/download/nginx-1.11.5.tar.gz
[root@rac1 local]# yum install gcc-c++ pcre pcre-devel zlib zlib-devel openssl openssl-devel -y
[root@rac1 local]# tar -zxvf nginx-1.11.5.tar.gz
[root@rac1 local]# mv /usr/local/nginx-1.11.5 /usr/local/nginx
[root@rac1 local]# cd /usr/local/nginx
[root@rac1 nginx-1.11.5]# ./configure
[root@rac1 nginx-1.11.5]# make
[root@rac1 nginx-1.11.5]# make install
[root@rac1 system]# /usr/local/nginx/sbin/nginx
[root@rac2 nginx-1.11.5]# /usr/local/nginx/sbin/nginx
[root@rac1 system]# ps -ef|grep nginx
root 11866 1 0 17:35 ? 00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
nobody 11867 11866 0 17:35 ? 00:00:00 nginx: worker process
root 11886 3579 0 17:35 pts/1 00:00:00 grep --color=auto nginx
4.2 Web 集群1(187) 、Web 集群2(188) 分别配置nginx
Web 集群1(187) 、Web 集群2(188) 分别替换不同的html ，方便轮询测试；
[root@rac2 html]# pwd
/usr/local/nginx/html/index.html 替换成awr.html 和gpr.html
五. 测试负载均衡和故障转移
5.1 测试负载均衡
负载方式：rr 轮询
浏览器输入 http://192.168.2.223/
[root@jumplinux01 ~]# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
TCP 00:01 NONE 192.168.90.105:0 192.168.2.223:80 192.168.2.188:80
TCP 00:00 ESTABLISHED 192.168.90.105:63625 192.168.2.223:80 192.168.2.188:80
两秒后刷新页面
页面不同了，轮询成功
5.2 测试故障转移
Web 集群2(188) 杀掉nginx 服务
[root@rac2 ~]# ps -ef|grep nginx
root 11046 1 0 Oct29 ? 00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
nobody 11047 11046 0 Oct29 ? 00:00:04 nginx: worker process
root 19625 19401 0 15:36 pts/0 00:00:00 grep --color=auto nginx
[root@rac2 ~]# kill -9 11046
[root@rac2 ~]# kill -9 11047
[root@rac2 ~]# ps -ef|grep nginx
root 19665 19401 0 15:36 pts/0 00:00:00 grep --color=auto nginx
浏览器输入 http://192.168.2.223/
间隔一段时间多长刷新，页面不变，也没有报错
[root@jumplinux01 ~]# ipvsadm -lnc
IPVS connection entries
pro expire state source virtual destination
TCP 00:00 NONE 192.168.90.105:0 192.168.2.223:80 192.168.2.187:80
启动nginx 后，可以刷新到另一个页面了
[root@rac2 ~]# /usr/local/nginx/sbin/nginx
欢迎关注我的微信公众号"IT小Chen"，共同学习，共同成长！！！
