Hi Matt
I'm still trying to fully understand the new functionailty
of the ESX3.5 SW iSCSI stack, but it seems they've made some improvements in MCS
(Multi Connection Sessions) or some other technology that allows the iSCSI SW
stack to be aware of and fail over between paths.
I'm 90% sure of this so far, but ...
I would leave the paths as they are. You can
use tpgroups (documented in the man pages and Block Access Guide) on
the filer to restrict iSCSI traffic to certain ports for certain igroups, but I
think you're better off leaving the other path as a failover. In ESX, set
the preferred path of the LUN(s) to go through the most direct network or the
back-end / storage network if that's what you have.
esxcfg-mpath -l on the CLI should show you similar info to
what you see in the VI Client. vmkiscsi-ls (one word / no space) will list
all the target portals that iSCSI knows about. They should look like
this: (I've snipped some other junk)
[root@esx1 mcs_test]# esxcfg-mpath
-l
Disk vmhba32:2:105 /dev/sdg (10240MB) has 2 paths and
policy of Fixed
iScsi sw
iqn.1998-01.com.vmware:esx1-2e0f4af9<->iqn.1992-08.com.netapp:sn.118042901
vmhba32:2:105 On active preferred
iScsi sw
iqn.1998-01.com.vmware:esx1-2e0f4af9<->iqn.1992-08.com.netapp:sn.118042901
vmhba32:4:105 On
[root@esx1 mcs_test]#
vmkiscsi-ls
*******************************************************************************
SFNet iSCSI Driver Version ... 3.6.3 (27-Jun-2005
)
*******************************************************************************
TARGET
NAME :
iqn.1992-08.com.netapp:sn.118042901
TARGET
ALIAS
:
HOST
NO
: 3
BUS
NO
: 0
TARGET
ID
: 2
TARGET ADDRESS :
192.168.42.61:3260
SESSION
STATUS : ESTABLISHED AT
Thu Dec 20 19:43:21 2007
NO. OF
PORTALS : 1
PORTAL
ADDRESS 1 :
192.168.42.61:3260,1001
SESSION
ID
: ISID 00023d000001 TSIH
0e
*******************************************************************************
TARGET
NAME :
iqn.1992-08.com.netapp:sn.118042901
TARGET
ALIAS
:
HOST
NO
: 3
BUS
NO
: 0
TARGET
ID
: 3
TARGET ADDRESS :
10.41.87.61:3260
SESSION
STATUS : NOT
ESTABLISHED
NO. OF
PORTALS : 1
PORTAL
ADDRESS 1 :
10.41.87.61:3260,1000
SESSION
ID
: ISID 00023d000002 TSIH
00
*******************************************************************************
TARGET
NAME :
iqn.1992-08.com.netapp:sn.118042901
TARGET
ALIAS
:
HOST
NO
: 3
BUS
NO
: 0
TARGET
ID
: 4
TARGET ADDRESS :
192.168.42.62:3260
SESSION
STATUS : ESTABLISHED AT
Thu Dec 20 19:47:54 2007
NO. OF
PORTALS : 1
PORTAL
ADDRESS 1 :
192.168.42.62:3260,1003
SESSION
ID
: ISID 00023d000003 TSIH
11
*******************************************************************************
Note
the TARGET ID for the portals listed as ESTABLISHED will match the target ID in
the vmhba LUN name, in my case, 2 and 4. So from the TARGET ID, you can
get the corresponding IP address, and that gives you the preferred target.
You use esxcfg-mpath to set the preferred path:
[root@esx1 mcs_test]# esxcfg-mpath -L vmhba32:2:105 -P vmhba32:4:105 -f
-p fixed
Setting vmhba32:2:105 -- vmhba32:4:105 as preferred
path
[root@esx1 mcs_test]# esxcfg-mpath -q -L vmhba32:2:105
Disk
vmhba32:2:105 /dev/sdg (10240MB) has 2 paths and policy of Fixed
iScsi
sw
iqn.1998-01.com.vmware:esx1-2e0f4af9<->iqn.1992-08.com.netapp:sn.118042901
vmhba32:2:105 On
iScsi sw
iqn.1998-01.com.vmware:esx1-2e0f4af9<->iqn.1992-08.com.netapp:sn.118042901
vmhba32:4:105 On active preferred
Your
iSCSI traffic will use the storage network, until it breaks, at which time I/O
(and possibly more) will hang for ~30 seconds while ESX figures out that the
path failed and fails over to the other target on the other network. I
tested that by downing the preferred interface, and the service console hung for
a few seconds, then picked back up, and esxcfg-mapth showed the pref path dead,
and active on the other path. vmkiscsi-ls showed the session status as
"Dropped" with a timestamp.
I have
not yet played with the new path policies, so there may be a better option than
preferred and fixed. I'm also wondering if this might lead to load
balancing in iSCSI...
Please
let us know how this goes for you!
Thanks
Peter
Hello all,
Having decided to upgrade our VMware ESX servers to 3.5 from
3.0.2, we have come across a strange problem.
On each ESX server we have 3 VSwitches, 2 of them each
contain a service console port and VMkernel port. One of Vswitches we have been
using for Iscsi in ESX 3.0.2 without any problems. (The first VMkernel port is
enabled for Vmotion and is routable)
Once we upgraded to 3.5 we have noticed that the filer now
shows 2 sessions from the esx host, one from each of the VMkernel ports, however
one of the paths is routed and one of the paths is switched. On the ESX host we
see each LUN twice, with different paths but with the same canonical
path.
Does anyone know how to restrict what targets are being sent
by the filer based on the interface ? We cannot disable Iscsi on any of the
interfaces on the filer.
Or does anyone have any other bright ideas ?
Thanks
Matt
____________________________
Matt
Davies
Director of
International IT Operations
General
Atlantic
83 Pall
Mall
London
SW1Y
5ES
Tel: +44 207 484
3203
Fax: +44 207 484
2803
Mobile: +44 777
559 4265
____________________________
____________________________________________________________
This e-mail (including all attachments) is confidential and may be
privileged.
It is for the exclusive use of the addressee only.
If you are not the addressee,
you are hereby notified that any dissemination
of this communication is strictly
prohibited. If you have received this
communication in error, please erase all
copies of the message and its
attachments and notify us immediately at
help@generalatlantic.com. Thank
You.