Greetings,
I have a SAS stack (plus other FC ds14s) in a FAS60xx cluster running 7.3.5.1. It has 2 quad port SAS HBAs per filer. The stack currently uses ports 6a and 7d on each filer.
I need to add a couple more stacks. To make the cabling consistent I would like to move the current 7d connection to port 7a. When I moved the cable from 7d to 7a on one of the heads, it never brought the port online and never saw the disks on that port. I tried the storage enable adapter 7a but it still showed it was offline. I ran various sysconfig, storage show, disk show, etc, type commands to get the system to go find the disks on the new path. Nothing seemed to help. I waited 5 minutes after doing this just in case there was some type of discovery lag, but they never showed up. When I plugged back into 7d, everything is fine.
I thought moving to a new port should work. Is there something else I'm missing to make this work?
Thanks,
Jeff
Changing source ports (SAS or FC initiators) without taking the node down is dangerous and can cause a system panic and WAFL inconsistency (then downtime to wafliron/wafl_check). During a flush to disk there may not be enough time to update the disk identity which has the port in use... if you get lucky and catch it between CPs it works.
The supported method is to take the node down to maintenance mode...recable, then check connections (sasadmin expander_map, sasadmin shelf, storage show disk -p, etc.). You can do this with a cf takeover then boot the down node to maintenance mode, confirm all is working, then reboot and cf giveback.
Since there may some issues after moving ports around outside of maintenance mode with the node serving data, I would open a case and confirm next steps... likely will be to recable in maintenance mode.
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Jeff Cleverley Sent: Monday, May 07, 2012 3:40 PM To: toasters@teaparty.net Subject: SAS port swap.
Greetings,
I have a SAS stack (plus other FC ds14s) in a FAS60xx cluster running 7.3.5.1. It has 2 quad port SAS HBAs per filer. The stack currently uses ports 6a and 7d on each filer.
I need to add a couple more stacks. To make the cabling consistent I would like to move the current 7d connection to port 7a. When I moved the cable from 7d to 7a on one of the heads, it never brought the port online and never saw the disks on that port. I tried the storage enable adapter 7a but it still showed it was offline. I ran various sysconfig, storage show, disk show, etc, type commands to get the system to go find the disks on the new path. Nothing seemed to help. I waited 5 minutes after doing this just in case there was some type of discovery lag, but they never showed up. When I plugged back into 7d, everything is fine.
I thought moving to a new port should work. Is there something else I'm missing to make this work?
Thanks,
Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Scott,
Is this really a problem even in the multi-path HA cabling configuration? I know I've done this with FC ports on multiple occasions. We've had the optics fail on both onboard ports and HBAs. Instead of leaving them with a single path while we waited for parts and a maintenance window, we just plugged them into a spare port. Everything failed over just fine. Once we got the parts replaced, we put the cable back and everything was happy.
If worst comes to worst I'll do a non-symmetric cabling configuration and clean it up whenever we get a downtime window.
Jeff
On Mon, May 7, 2012 at 4:59 PM, Gelb, Scott sgelb@insightinvestments.com wrote:
Changing source ports (SAS or FC initiators) without taking the node down is dangerous and can cause a system panic and WAFL inconsistency (then downtime to wafliron/wafl_check). During a flush to disk there may not be enough time to update the disk identity which has the port in use... if you get lucky and catch it between CPs it works.
The supported method is to take the node down to maintenance mode...recable, then check connections (sasadmin expander_map, sasadmin shelf, storage show disk -p, etc.). You can do this with a cf takeover then boot the down node to maintenance mode, confirm all is working, then reboot and cf giveback.
Since there may some issues after moving ports around outside of maintenance mode with the node serving data, I would open a case and confirm next steps... likely will be to recable in maintenance mode.
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Jeff Cleverley Sent: Monday, May 07, 2012 3:40 PM To: toasters@teaparty.net Subject: SAS port swap.
Greetings,
I have a SAS stack (plus other FC ds14s) in a FAS60xx cluster running 7.3.5.1. It has 2 quad port SAS HBAs per filer. The stack currently uses ports 6a and 7d on each filer.
I need to add a couple more stacks. To make the cabling consistent I would like to move the current 7d connection to port 7a. When I moved the cable from 7d to 7a on one of the heads, it never brought the port online and never saw the disks on that port. I tried the storage enable adapter 7a but it still showed it was offline. I ran various sysconfig, storage show, disk show, etc, type commands to get the system to go find the disks on the new path. Nothing seemed to help. I waited 5 minutes after doing this just in case there was some type of discovery lag, but they never showed up. When I plugged back into 7d, everything is fine.
I thought moving to a new port should work. Is there something else I'm missing to make this work?
Thanks,
Jeff
Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611 _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Gelb, Scott -
Jeff Cleverley