模拟linux 双网卡双ip，实现multipath多路径连接存储

模拟linux 双网卡双ip，实现multipath多路径连接存储
系统(Centos6.5:oracle-1)：
eth0:192.168.21.23/24
eth1:172.16.112.17/24
存储Ip: 192.168.21.230
首先要配置iscsi存储，再其次一下操作，我这里配置了两个ISCIS存储
网卡路由配置
由于只能有一个网关，所以要添加静态路由实现双网卡通信

1、编辑网卡eth0,eth1
设置好配置文件，不添加默认网关
# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=192.168.21.23
NETMASK=255.255.255.0

# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=yes
BOOTPROTO=static
IPADDR=172.16.112.17
NETMASK=255.255.255.0

2、配置默认网关为任意一个，这个默认网关，决定了系统主动去使用网络时，使用的接口
修改配置文件
# vi /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=oracle-1.localdomain
#NTPSERVERARGS=iburst
GATEWAYDEV=eth0

3、添加路由表
# echo “21 net_21” >>/etc/iproute2/rt_tables
# echo “112 net_112” >>/etc/iproute2/rt_tables

4、添加原路返回路由
# ip route flush table net_21
# ip route add default via 192.168.21.1 deveth0 src 192.168.21.23 table net_21
# ip rule add from 192.168.21.23 tablenet_21

# ip route flush table net_112
# ip route add default via 172.16.112.1 deveth1 src 172.16.112.17 table net_112
# ip rule add from 172.16.112.17 tablenet_112

5、在其他机验证是否可以ping通这两个ip
# ping 192.168.21.23
# ping 172.16.112.17

6、实现开机自动添加路由
# cat >> /etc/rc.local << Eof
>ip route flush table net_21
>ip route add default via 192.168.21.1dev eth0 src 192.168.21.23 table net_21
>ip rule add from 192.168.21.23 tablenet_21
>ip route flush table net_112
>ip route add default via 172.16.112.1dev eth1 src 172.16.112.17 table net_112
>ip rule add from 172.16.112.17 tablenet_112
>Eof

安装ISCSI存储发起程序
1、yum安装iscsi
# yum -y install iscsi-initiator-utils
# chkconfig iscsi on
2、建立两个iscsi发起文件
# iscsiadm -m iface -I iscsi_eth0 --op=new
# iscsiadm -m iface -I iscsi_eth1 --op=new
3、建立网卡映射关系
# iscsiadm -m iface -I iscsi_eth0--op=update -n iface.net_ifacename -v eth0
# iscsiadm -m iface -I iscsi_eth1--op=update -n iface.net_ifacename -v eth1
4、Iscsi存储发现
# iscsiadm -m discovery -t sendtargets -p192.168.21.230:3260
5、重启iscsi服务，之后查看磁盘连接情况(可以看到已经成功连接上了)
# service iscsi restart
# iscsiadm -m session

安装multipath多路径挂载磁盘
1、yum安装multipath# yum install -y device-mapper-persistent-data device-mapper-multipathdevice-mapper-libs device-mapper-event-libs device-mapper-multipath-libsdevice-mapper device-mapper-event2、加载multipath模块# modprobe dm-multipath# modprobe dm-round-robin3、创建multipath配置文件# cat > /etc/multipath.conf << Eof>defaults {> user_friendly_names yes>}>Eof
4、查看挂载存储的uid号（因为是多路径，所以会出现两个相同的uid号）# multipath -v3 | grep uidJan 18 16:29:59 | sdb: getuid = /lib/udev/scsi_id --whitelisted--device=/dev/%n (config file default)Jan 18 16:29:59 | sdb: uid = 36001405a75df6ded0277d3bf3dae7ddb(callout)Jan 18 16:29:59 | sdc: getuid = /lib/udev/scsi_id --whitelisted--device=/dev/%n (config file default)Jan 18 16:29:59 | sdc: uid = 36001405a75df6ded0277d3bf3dae7ddb(callout)Jan 18 16:29:59 | sdd: getuid = /lib/udev/scsi_id --whitelisted--device=/dev/%n (config file default)Jan 18 16:29:59 | sdd: uid = 36001405b363aaa9d92bad39c9dbd7bdd (callout)Jan 18 16:29:59 | sde: getuid = /lib/udev/scsi_id --whitelisted--device=/dev/%n (config file default)Jan 18 16:29:59 | sde: uid = 36001405b363aaa9d92bad39c9dbd7bdd (callout)uuid hcil dev dev_t pri dm_st chk_st vend/pro
5、将uid号添加进配置文件中，实现多路径访问# cat >> /etc/multipath.conf << Eof >blacklist { >devnode "^sda" >devnode "^sdb" > devnode "^sdc" >} >multipaths{ > multipath{ > wwid36001405a75df6ded0277d3bf3dae7ddb > alias mpath1 >} >multipath{ >wwid36001405b363aaa9d92bad39c9dbd7bdd > alias mpath2 > } >}>Eof
6、启动服务# /etc/init.d/multipathd start# chkconfig multipathd on# fdisk –l (能够查看到挂载了两个multipath磁盘/dev/mapper/mpath1/dev/mapper/mpath2}

附：
创建用户逻辑卷。
# pvcreate /dev/mapper/mpath1
# vgcreate -s 256m datavg /dev/mapper/mpath1
# lvcreate --size 2000G --name lv_src vg_data
# mkfs.ext4 /dev/mapper/vg_data-lv_src
# mkdir /src
# mount -t ext4 /dev/mapper/vg_data-lv_src /src

负载均衡测试。
# cd /src
# dd if=/dev/zero of=test.img
# iostat 1
Device: tps Blk_read/s Blk_wrtn/s
sda 2020.00 8.00 466272.00 # 两条线路都有数据传输，主主模式起作用。
sdb 2024.00 0.00 368952.00 # 如果只有一条线路传输数据，就是主备模式。
sdc 0.00 0.00 0.00
dm-0 4044.00 8.00 834776.00 # 合并后的路径，数据等于两条。

oracle挂载ASM
# /etc/init.d/oracleasm configure
# /etc/init.d/oracleasm enable
# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath1
# /etc/init.d/oracleasm scandisks
# /etc/init.d/oracleasm listdisks
# oracleasm querydisk -p VOL1