使用openfiler测试rhel中的device-mapper-multipath多路径软件

http://candon123.blog.51cto.com/704299/297011/

一台机器装openfiler充当存储，另外一台当作客户端,主机名为server.xzxj.edu.cn.
openfile的安装这里不做具体介绍，完成多路径在openfiler端必须存在两块网卡，这里如下：
eth0:192.168.1.5
eth1:192.168.1.6
openfiler中的iscsi target配置如图所示：




 
server.xzxj.edu.cn端的配置如下:
启动iscsi进程：service iscsi start;chkconfig iscsi on
首先发现openfiler端的iscsi目标，其次是登录到iscsi目标上，如图所示：








因为要测试linux自带的多路径软件，所以这里要登录两次。两次的IP是不同的,但iqn名称是一致的。这里是我写的一个脚本，所以显示是这样子的。
现在已经登录上去了，可以使用fdisk -l命令查看是否已经识别到磁盘了：
[root@server ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sda1   *           1          13      104391   83  Linux

/dev/sda2              14        2349    18763920   8e  Linux LVM

/dev/sda3            2350        2610     2096482+  82  Linux swap / Solaris
Disk /dev/sdj: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdj doesn't contain a valid partition table
Disk /dev/sdk: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdk doesn't contain a valid partition table
Disk /dev/sdl: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdl doesn't contain a valid partition table
Disk /dev/sdm: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdm doesn't contain a valid partition table
Disk /dev/sdn: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdn doesn't contain a valid partition table
Disk /dev/sdo: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdo doesn't contain a valid partition table
Disk /dev/sdp: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdp doesn't contain a valid partition table
Disk /dev/sdq: 10.7 GB, 10737418240 bytes

64 heads, 32 sectors/track, 10240 cylinders

Units = cylinders of 2048 * 512 = 1048576 bytes
Disk /dev/sdq doesn't contain a valid partition table
看到了吧，这里多出了8块盘。在openfiler中只map了四次，为什么这里是8块而不是4块呢？因为在上面提过，使用两个IP登录两次iscsi target，所以这里有两块是重复的。
下面是多路径软件device-mapper-multipath的配置文件具体配置，如下所示：








完了后，启动multipathd进程：
service multipathd start
chkconfig multipathd on
启动之后，使用multipath -ll可以查看是否成功：




看到了吧，重复的磁盘现在绑定到一起了。
给这几块盘分区，创建文件系统，使用fdisk /dev/dm-1或者fdisk /dev/mapper/openfiler_vol01即可。如下所示：
[root@server45 ~]# fdisk /dev/dm-3

Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel. Changes will remain in memory only,

until you decide to write them. After that, of course, the previous

content won't be recoverable.

The number of cylinders for this disk is set to 1305.

There is nothing wrong with that, but this is larger than 1024,

and could in certain setups cause problems with:

1) software that runs at boot time (e.g., old versions of LILO)

2) booting and partitioning software from other OSs

   (e.g., DOS FDISK, OS/2 FDISK)

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Command (m for help): p
Disk /dev/dm-3: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
     Device Boot      Start         End      Blocks   Id  System
Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 1

First cylinder (1-1305, default 1):

Using default value 1

Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305): +5120M
Command (m for help): t

Selected partition 1

Hex code (type L to list codes): fd

Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): n

Command action

   e   extended

   p   primary partition (1-4)

p

Partition number (1-4): 2

First cylinder (624-1305, default 624):

Using default value 624

Last cylinder or +size or +sizeM or +sizeK (624-1305, default 1305):

Using default value 1305
Command (m for help): t

Partition number (1-4): 2

Hex code (type L to list codes): fd

Changed system type of partition 2 to fd (Linux raid autodetect)
Command (m for help): w

The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 22: Invalid argument.

The kernel still uses the old table.

The new table will be used at the next reboot.

Syncing disks.
分区完结果如下：
[root@server45 ~]# fdisk -l
Disk /dev/sda: 21.4 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sda1   *           1          13      104391   83  Linux

/dev/sda2              14        2349    18763920   8e  Linux LVM

/dev/sda3            2350        2610     2096482+  82  Linux swap / Solaris
Disk /dev/sdb: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1               1         623     5004216   fd  Linux raid autodetect

/dev/sdb2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdc: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdc1               1         623     5004216   fd  Linux raid autodetect

/dev/sdc2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdd: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdd1               1         623     5004216   fd  Linux raid autodetect

/dev/sdd2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sde: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sde1               1         623     5004216   fd  Linux raid autodetect

/dev/sde2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdf: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdf1               1         623     5004216   fd  Linux raid autodetect

/dev/sdf2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdg: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdg1               1         623     5004216   fd  Linux raid autodetect

/dev/sdg2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdh: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdh1               1         623     5004216   fd  Linux raid autodetect

/dev/sdh2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/sdi: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
   Device Boot      Start         End      Blocks   Id  System

/dev/sdi1               1         623     5004216   fd  Linux raid autodetect

/dev/sdi2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/dm-1: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
     Device Boot      Start         End      Blocks   Id  System

/dev/dm-1p1               1         623     5004216   fd  Linux raid autodetect

/dev/dm-1p2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/dm-2: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
     Device Boot      Start         End      Blocks   Id  System

/dev/dm-2p1               1         623     5004216   fd  Linux raid autodetect

/dev/dm-2p2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/dm-3: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
     Device Boot      Start         End      Blocks   Id  System

/dev/dm-3p1               1         623     5004216   fd  Linux raid autodetect

/dev/dm-3p2             624        1305     5478165   fd  Linux raid autodetect
Disk /dev/dm-4: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
     Device Boot      Start         End      Blocks   Id  System

/dev/dm-4p1               1         623     5004216   fd  Linux raid autodetect

/dev/dm-4p2             624        1305     5478165   fd  Linux raid autodetect
这里的/dev/dm-[1,2,3,4]p[1,2]就是分区1和2。
查看下/dev/mapper目录下有何不同：

[root@server45 ~]# ll /dev/mapper/

total 0

crw------- 1 root root  10, 63 Apr 17  2010 control

brw-rw---- 1 root disk 253,  1 Apr 17 10:05 openfiler_vol01

brw-rw---- 1 root disk 253,  2 Apr 17 09:58 openfiler_vol02

brw-rw---- 1 root disk 253,  3 Apr 17 09:58 openfiler_vol03

brw-rw---- 1 root disk 253,  4 Apr 17 09:58 openfiler_vol04

brw-rw---- 1 root disk 253,  0 Apr 17 09:43 vg-lv01

[root@server45 ~]# fdisk -l /dev/mapper/openfiler_vol01
Disk /dev/mapper/openfiler_vol01: 10.7 GB, 10737418240 bytes

255 heads, 63 sectors/track, 1305 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes
                       Device Boot      Start         End      Blocks   Id  System

/dev/mapper/openfiler_vol01p1               1         623     5004216   fd  Linux raid autodetect

/dev/mapper/openfiler_vol01p2             624        1305     5478165   fd  Linux raid autodetect
[root@server45 ~]# ll /dev/dm-*

brw-rw---- 1 root root 253, 1 Apr 17 09:58 /dev/dm-1

brw-rw---- 1 root root 253, 2 Apr 17 10:06 /dev/dm-2

brw-rw---- 1 root root 253, 3 Apr 17 10:07 /dev/dm-3

brw-rw---- 1 root root 253, 4 Apr 17 10:07 /dev/dm-4
如果要了解iscsi多路径的访问机制，可以参考我的另外一篇文章：深入了解iSCSI的2种多路径访问机制