2016-3-22 from—http://forum.huawei.com/enterprise/thread-184781.html
问题描述

华为存储映射多个lun到linux主机,重启后lun盘符漂移

告警信息

重启前存储lun盘符是/dev/sdb,重启后变成/dev/sdc或者其他,影响用户体验和应用

处理过程

1.获取lun的详细信息,此信息在linux操作系统中是唯一的

[[email protected] ~]# fdisk -l

Disk /dev/sda: 42.9 GB, 42949672960 bytes
64 heads, 32 sectors/track, 40960 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009793e

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           2         201      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2             202        4297     4194304   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3            4298       40960    37542912   83  Linux
Partition 3 does not end on cylinder boundary.

Disk /dev/sdb: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xdc10e190

Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        3916    31454208    7  HPFS/NTFS

Disk /dev/sdc: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xdc10e194

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1        3916    31454208    7  HPFS/NTFS
[[email protected] ~]# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
[3:0:0:0]    disk    HUAWEI   S2600            1     /dev/sdb
[3:0:0:1]    disk    HUAWEI   S2600            1     /dev/sdc
[[email protected] ~]#

[[email protected] ~]# udevadm info -q path -n /dev/sdb
/devices/platform/host3/session1/target3:0:0/3:0:0:0/block/sdb

[[email protected] ~]# udevadm info -a -p /devices/platform/host3/session1/target3:0:0/3:0:0:0/block/sdb

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

looking at device ‘/devices/platform/host3/session1/target3:0:0/3:0:0:0/block/sdb’:
KERNEL==”sdb”
SUBSYSTEM==”block”
DRIVER==””
ATTR{range}==”16″
ATTR{ext_range}==”256″
ATTR{removable}==”0″
ATTR{ro}==”0″
ATTR{size}==”62914560″
ATTR{alignment_offset}==”0″
ATTR{discard_alignment}==”0″
ATTR{capability}==”52″
ATTR{stat}==”     339       16     2840     2009        0        0        0        0        0     2009     2009″
ATTR{inflight}==”       0        0″

looking at parent device ‘/devices/platform/host3/session1/target3:0:0/3:0:0:0’:
KERNELS==”3:0:0:0″
SUBSYSTEMS==”scsi”
DRIVERS==”sd”
ATTRS{device_blocked}==”0″
ATTRS{type}==”0″
ATTRS{scsi_level}==”5″
ATTRS{vendor}==”HUAWEI  ”
ATTRS{model}==”S2600           ”
ATTRS{rev}==”1   ”
ATTRS{state}==”running”
ATTRS{timeout}==”30″
ATTRS{iocounterbits}==”32″
ATTRS{iorequest_cnt}==”0x171″
ATTRS{iodone_cnt}==”0x171″
ATTRS{ioerr_cnt}==”0x1″
ATTRS{modalias}==”scsi:t-0x00″
ATTRS{evt_media_change}==”0″
ATTRS{dh_state}==”detached”
ATTRS{queue_depth}==”32″
ATTRS{queue_ramp_up_period}==”120000″
ATTRS{queue_type}==”none”

looking at parent device ‘/devices/platform/host3/session1/target3:0:0’:
KERNELS==”target3:0:0″
SUBSYSTEMS==”scsi”
DRIVERS==””

looking at parent device ‘/devices/platform/host3/session1’:
KERNELS==”session1″
SUBSYSTEMS==””
DRIVERS==””

looking at parent device ‘/devices/platform/host3’:
KERNELS==”host3″
SUBSYSTEMS==”scsi”
DRIVERS==””

looking at parent device ‘/devices/platform’:
KERNELS==”platform”
SUBSYSTEMS==””
DRIVERS==””

[[email protected] ~]# udevadm info -q path -n /dev/sdc
/devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdc
[[email protected] ~]# udevadm info -a -p /devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdc

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

looking at device ‘/devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdc’:
KERNEL==”sdc”
SUBSYSTEM==”block”
DRIVER==””
ATTR{range}==”16″
ATTR{ext_range}==”256″
ATTR{removable}==”0″
ATTR{ro}==”0″
ATTR{size}==”62914560″
ATTR{alignment_offset}==”0″
ATTR{discard_alignment}==”0″
ATTR{capability}==”52″
ATTR{stat}==”     168        3     1368      363        0        0        0        0        0      363      363″
ATTR{inflight}==”       0        0″

looking at parent device ‘/devices/platform/host3/session1/target3:0:0/3:0:0:1’:
KERNELS==”3:0:0:1″
SUBSYSTEMS==”scsi”
DRIVERS==”sd”
ATTRS{device_blocked}==”0″
ATTRS{type}==”0″
ATTRS{scsi_level}==”5″
ATTRS{vendor}==”HUAWEI  ”
ATTRS{model}==”S2600           ”
ATTRS{rev}==”1   ”
ATTRS{state}==”running”
ATTRS{timeout}==”30″
ATTRS{iocounterbits}==”32″
ATTRS{iorequest_cnt}==”0xbd”
ATTRS{iodone_cnt}==”0xbd”
ATTRS{ioerr_cnt}==”0x0″
ATTRS{modalias}==”scsi:t-0x00″
ATTRS{evt_media_change}==”0″
ATTRS{dh_state}==”detached”
ATTRS{queue_depth}==”32″
ATTRS{queue_ramp_up_period}==”120000″
ATTRS{queue_type}==”none”

looking at parent device ‘/devices/platform/host3/session1/target3:0:0’:
KERNELS==”target3:0:0″
SUBSYSTEMS==”scsi”
DRIVERS==””

looking at parent device ‘/devices/platform/host3/session1’:
KERNELS==”session1″
SUBSYSTEMS==””
DRIVERS==””

looking at parent device ‘/devices/platform/host3’:
KERNELS==”host3″
SUBSYSTEMS==”scsi”
DRIVERS==””

looking at parent device ‘/devices/platform’:
KERNELS==”platform”
SUBSYSTEMS==””
DRIVERS==””

2.写udev策略,是lun在操作系统中的盘符固定下来

[[email protected] ~]# cd /etc/udev/rules.d/
[[email protected] rules.d]# touch 92-s2600.rules

[[email protected] rules.d]# cat 92-s2600.rules
SUBSYSTEMS==”scsi”,SUBSYSTEM==”block”,KERNELS==”3:0:0:1″,NAME=”/sdz%n”
SUBSYSTEMS==”scsi”,SUBSYSTEM==”block”,KERNELS==”3:0:0:0″,NAME=”/sdy%n”

[[email protected] rules.d]#
此策略的目的是将SUBSYSTEMS如果等于”scsi”,SUBSYSTEM如果等于”block”,KERNELS如果等于”3:0:0:1″,则命名为sdz,同理KERNELS如果等于”3:0:0:0″,则命名为sdy

KERNELS==”3:0:0:0是sdb,KERNELS==”3:0:0:1是sdc

3.重启linux服务器查看,fdisk -l命令中查不到sdb和sdc,ll /dev/sdy则有结果,并且能够看到分区

[[email protected] rules.d]# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
[3:0:0:0]    disk    HUAWEI   S2600            1     /dev/sdb
[3:0:0:1]    disk    HUAWEI   S2600            1     /dev/sdc
[[email protected] rules.d]#
[[email protected] rules.d]# reboot

Broadcast message from [email protected]
(/dev/pts/0) at 9:15 …

The system is going down for reboot NOW!
[[email protected] rules.d]# Last login: Thu Mar 12 09:09:45 2015 from 192.168.199.243
[[email protected] ~]#
[[email protected] ~]# lsscsi
[1:0:0:0]    cd/dvd  NECVMWar VMware IDE CDR10 1.00  /dev/sr0
[2:0:0:0]    disk    VMware   Virtual disk     1.0   /dev/sda
[3:0:0:0]    disk    HUAWEI   S2600            1     /dev/sdy
[3:0:0:1]    disk    HUAWEI   S2600            1     /dev/sdz
[[email protected] ~]#
[[email protected] ~]#
[[email protected] ~]# fdisk -l

Disk /dev/sda: 42.9 GB, 42949672960 bytes
64 heads, 32 sectors/track, 40960 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0009793e

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           2         201      204800   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2             202        4297     4194304   82  Linux swap / Solaris
Partition 2 does not end on cylinder boundary.
/dev/sda3            4298       40960    37542912   83  Linux
Partition 3 does not end on cylinder boundary.
[[email protected] ~]#
[[email protected] ~]#
[[email protected] ~]# ll /dev/sdy
brw-rw—-. 1 root disk 8, 16 Mar 12 09:16 /dev/sdy
[[email protected] ~]# ll /dev/sdz
brw-rw—-. 1 root disk 8, 32 Mar 12 09:16 /dev/sdz

[[email protected] ~]# ll /dev/sdy
sdy   sdy1

根因

linux重启盘符会有一定几率漂移,和驱动加载顺序有关系

解决方案

通过Udev策略,执行lun在操作系统中的名称或者盘符
[[email protected] rules.d]# cat 92-s2600.rules
SUBSYSTEMS==”scsi”,SUBSYSTEM==”block”,KERNELS==”3:0:0:1″,NAME=”/sdz%n”
SUBSYSTEMS==”scsi”,SUBSYSTEM==”block”,KERNELS==”3:0:0:0″,NAME=”/sdy%n”