跟着ArthurXF学习——FREEBSD的负载均衡

007

alexa2008年2月4日大事件

大 | 中 | 小

2008/01/29 12:35

ArthurXF

　　很多Linux的朋友也许都非常熟悉LVS负载均衡软件了，但是在FREEBSD下如何做负载均衡呢？
　　其实FREEBSD也有很多负载均衡的方法。本文由ArthurXF倾情奉献，谢谢支持！
　　负载均衡就是要把高流量合理的分配给不同的服务器，让每台服务器在最大承受能力范围内，最好的为用户服务。负载均衡其实就是如何把流量合理分配的工作。
　　当然负载均衡的办法很多，我ArthurXF这里只给出两个方法，以供大家学习参考。
1.构建流量控制防火墙（桥接模式）
概述
利用FreeBSD内核支持的BRIDGE、IPFIREWALL以及DUMMYNET选项，可以建立基于FreeBSD的透明流量控制防火墙（桥接模式），起到限制流量和包过滤的功能。

引用

准备
可以在任何FreeBSD的兼容硬件上构建流量控制防火墙，但是基于性能和管理上的考虑，建议：
使用Intel PII450以上的处理器
使用至少128MB RAM
使用高性能10/100Mbps自适应网络适配器
如果多于一组桥接设备，建议使用双处理器系统
另外准备一块单独的网络适配器用于管理
--------------------------------------------------------------------------------
实例
联想万全1300 PC服务器，具有一颗Intel PII300处理器，配置有128MB RAM，主板集成Intel 82557网络适配器，另外安装了4块3Com 3C905B 10/100Mbps自适应PCI网络适配器。

引用

配置
完成操作系统安装以后，必须重新编译系统内核使其支持桥接流量控制防火墙选项。必须在系统内核编译配置文件中添加以下内容：

options BRIDGE
options DUMMYNET
options IPFIREWALL
options IPFIREWALL_DEFAULT_TO_ACCEPT

重新编译完系统内核以后必须重新启动计算机。

--------------------------------------------------------------------------------
实例

# cd /usr/src/sys/i386/conf
# cp GENERIC BRGFW
# echo "options BRIDGE" >;>; BRGFW
# echo "options DUMMYNET" >;>; BRGFW
# echo "options IPFIREWALL" >;>; BRGFW
# echo "options IPFIREWALL_DEFAULT_TO_ACCEPT" >;>; BRGFW
# config
# cd ../../compile/BRGFW
# make depend all install
# reboot

引用

激活桥接
重新启动计算机以后，使用以下命令激活桥接流量控制防火墙选项：
sysctl -w net.link.ether.bridge_ipfw=1
sysctl -w net.link.ether.bridge_cfg=""
sysctl -w net.link.ether.bridge=1

其中bridge_cfg参数用于设置多组桥接设备，如果仅考虑单组桥接，可以忽略。实例使用集成的Intel 82557网络适配器作为管理网络，其他3Com 3C905B网络适配器分为两组网桥使用。

--------------------------------------------------------------------------------
实例

# sysctl -w net.link.ether.bridge_ipfw=1
# sysctl -w net.link.ether.bridge_cfg="xl0:0,xl1:0,xl2:1,xl3:1"
# sysctl -w net.link.ether.bridge=1

引用

使用
使用ipfw命令来控制流量和防火墙策略。其中流量控制是作为一条防火墙策略实现的，因此ipfw是唯一的管理界面。通过实例来说明ipfw的使用。

在实例中，网段192.168.254.0/24经过第一组网桥，网段192.168.250.0/24经过第二组网桥，并建立以下策略：

允许所有的ICMP连接，限制总流量为10Kbit/s

允许所有的UDP链接，限制总流量为100Kbit/s

允许TCP到网段192.168.254.0/24的所有连接，限制流量为5Mbit/s

允许TCP到主机192.168.250.222的HTTP连接，限制流量为2Mbit/s

允许TCP到主机192.168.250.0/24的所有其他连接，限制流量为1Mbit/s

禁止其他所有连接

--------------------------------------------------------------------------------
实例

# ipfw -flush
# ipfw add 100 pipe 1 icmp from any to any
# ipfw pipe 1 config bw 10Kbit/s
# ipfw add 200 pipe 2 udp from any to any
# ipfw pipe 2 config bw 100Kbit/s
# ipfw add 300 pipe 3 tcp from 192.168.254.0/24 to any
# ipfw pipe 3 config bw 5Mbit/s
# ipfw add 400 pipe 4 tcp from any to 192.168.254.0/24
# ipfw pipe 4 config bw 5Mbit/s
# ipfw add 500 pipe 5 tcp from any to 192.168.250.222 80
# ipfw pipe 5 config bw 2Mbit/s
# ipfw add 600 pipe 6 tcp from 192.168.250.222 80 to any
# ipfw pipe 6 config bw 2Mbit/s
# ipfw add 700 pipe 7 tcp from 192.168.250.0/24 to any
# ipfw pipe 7 config bw 1Mbit/s
# ipfw add 800 pipe 8 tcp from any to 192.168.250.0/24
# ipfw pipe 8 config bw 1Mbit/s
# ipfw add 60000 deny ip from any to any
# ipfw -a l

2.SG Cluster（一个简单实现集群的方法）
官方网址：http://turtle.ee.ncku.edu.tw/sgcluster/
SG Cluster是基于FREEBSD的NAT执行和SG软件分配建立的负载均衡工具。
具有如下特点：
易管理 - 有非常友好和简单安装的web用户管理界面。

图形化 - 系统将运行在不同系统上的集群后台服务高亮图形化显示在一个客户端。

可升级 - 系统负载能力可随着加入新的集群服务器而得到提升。

负载平衡 - 系统自动发送请求到最小负载的服务器上，以达到系统的最佳性能。

容错性 - SG负载均衡系统监控所有的服务器，仅仅将请求发送给有效的服务器，并可在整个负载均衡系统中设置多个负载均衡服务器，以避免系统中的单点故障。

高负载 - SG负载均衡可以标记系统中的过多冗余，系统还能在系统升级的时候维持正常服务。

配置
Config document from offical site..

http://turtle.ee.ncku.edu....

CONFIGURATION
edit system config files

There is an editor 'ee' bundled in SG system disk, you can use 'ee' to edit the configuration files under /etc

/etc/rc.conf.local

this file contains hostname information, see below for example

1 hostname="ds211.ee.ncku.edu.tw" # Set this!
2
3 gateway_enable="YES" # Set to YES if this host will be a gateway.
4 firewall_enable="YES" # firewall (see /etc/rc.firewall) or NO

line 1: set the hostname of the SG load balancer

/etc/ sg.conf

this file contains the configuration information of SG system, see below for example

1 # SG CLUSTER CONFIGURATION, CHANGE FOR YOUR NEED!!!
2
3 sgpath="/stand" # where sg related program is
4 temporary_ip="10.0.0.1" # ip before start sg
5 temporary_gw="10.0.0.253" # gateway before start sg
6
7 public_interface="fxp0"
8 private_interface="fxp1"
9
10 default_gw="140.116.72.253"
11
12 public_ip="140.116.72.136"
13 public_netmask="255.255.255.0"
14 private_ip="192.168.1.253"
15 private_netmask="255.255.255.0"
16
17 group_ip="140.116.72.137 140.116.72.138"
18
19 # map real servers to server group
20 natd_parameter="
21 -redirect_address 192.168.1.1 140.116.72.137
22 -redirect_address 192.168.1.2 140.116.72.137
23 -redirect_address 192.168.1.3 140.116.72.137
24 -redirect_port tcp 192.168.1.2:23 140.116.72.138:23
25 -redirect_port tcp 192.168.1.3:23 140.116.72.138:23
26 -redirect_port tcp 192.168.1.4:23 140.116.72.138:23
27 "
28
29 # username/passwd used by sgcmd to connect sgctrld
30 username="dslab"
31 password="dslab"
32
33 # init command sent to sgctrld
34 init_command="
35 set g 140.116.72.137:0 keyport_list 0
36 set g 140.116.72.137:0 s 192.168.1.1:0 weight 2
37 set g 140.116.72.137:0 s 192.168.1.2:0 weight 1
38 set g 140.116.72.137:0 s 192.168.1.3:0 weight 1
39 set g 140.116.72.138:23 keep_same_server 1
40 set g 140.116.72.138:23 keyport_list 0 23/tcp
41 set g 140.116.72.138:23 s 192.168.1.2:23 weight 1
42 set g 140.116.72.138:23 s 192.168.1.3:23 weight 1
43 set g 140.116.72.138:23 s 192.168.1.4:23 weight 2
44 "
45
46 # THERE SHOULD NO NEED TO CHANGE FROM BELOW ########################################
47
48 bidd_ip="234.5.6.7" # multicast group ip for bidd
49 bidd_port="3456" # port for bidd
50 bidd_master_heartbeat_interval="8"
51 bidd_master_timeout="10"
52 bidd_bid_timeout="3"
53 bidd_start_script="$sgpath/sgstart.sh"
54 bidd_stop_script="$sgpath/sgstop.sh"
55 bidd_continue_script="$sgpath/sgcontinue.sh"
56
57 sgmon_calc_status_interval="10"
58 sgmon_port_test_interval="60"
59 sgmon_host_timeout="2"
60
61 sgctrld_passwd_file="/etc/sgctrld.passwd"

line 7,8: set the name of public interface and private interface

Ethernet card supported by SG load balancer:

ed - NE2000, 3com 3C503 (ed0 port 0x300 irq10, ed1 port 0x320 irq11)
ie - 3com 30507, Intel Ether Express (ie0 port 0x280 irq5, ie1 port 0x340 irq 7)
de - DEC 21040/21140 based PCI card
fxp - Intel Ether Express pro
vr - DLink DFE530tx
xl - 3Com 3C900/3C905/3C905B

line 10: the default gateway for public interface

line 12-15: set the ip and netmask of the public interface and private interface

line 17: the ip of server groups (separated by space)

line 20-27: define servers in each server group

In this example, 2 server groups, 140.116.72.137:0 and 140.116.72.138:23, are defined, echo of them has 3 member server in the group.

line 30,31: the username/passwd used by web interface to login sgctrld

line 34-44: initial command sent to sgctrld to set SG properties,

property setting syntax:

set group [group_ip] [group_property] [value]
set group [group_ip] server [server_ip] [server_property] [value]

group property name description
name group name
active_flag 0=off, 1=on
keyport_list port list, 0 means icmp, NNN/udp=udp port, NNN/tcp=tcp port, where NNN=1..65535
select_method 0=roundrobin, 1=by_conn, 2=by_pkt, 3=by_clntip, 4=by_ext
keep_same_server 0=off, 1=on
failure_detect_by_packet_snoop 0=off, 1=on
recovery_detect_by_packet_snoop 0=off, 1=on
packet_delta_threshold pkt lost upper limit for each keyport
packet_timeout_threshold timeout upper limit for each keyport, unit:second
failure_detect_by_porttest 0=off, 1=on
recovery_detect_by_porttest 0=off, 1=on
porttest_error_threshold 0-65535
failure_detect_by_heartbeat 0=off, 1=on
recovery_detect_by_heartbeat 0=off, 1=on
heartbeat_timeout_threshold 0-65535
mcast_mode 0=deny, 1=bypass, 2=readwrite, 3=rdonly
multicast_addr multicast address for service program
mcast_error_threshold 0-65535
deny_interval deny interval for evil client
connection_count_limit 1-65535, limit per client, 0 no limit
connection_rate_limit 1-65535, limit per client, 0 no limit
finwait_tcp_limit 1-65535, limit per client, 0 no limit

server property name value
name server name
ac_list server access control list, ex: "140.116.72/24 !140.116.49.0/24"
weight 0-255
external_count a counter representing server load defined server program
status 0=dead, 1=pending, 2=alive

/etc/resolv.conf

set the dns server

/etc/hosts

local host table

set root password

use 'passwd root' to change root password of SG system

set sgctrld password

The password file of sgctrld is /etc/sgctrld.passwd, you can use

echo 'your_name:`makepwd your_passwd`'>;>;/etc/sgctrld.passwd

to add your_name/your_passwd to /etc/sgctrld.passwd

Or you can also 'ee' to edit the /etc/sgctrld.passwd directly to remove old account

synchronize all modification into floppy disk

When SG system boot up, all things are loaded into the ram disk, and your modification is also on the ram disk. You have to use 'update' to synchronize the change to the floppy disk.

reboot the SG system disk

If everything is right, you would see some message like 'bidd...BID->;MASTER' appears on the console. Try to telnet to the SG load balancer from remote host and browse the web page of 'http://your.load.balancer.hostname.or.ip'. If everything is right, congratulations!