Hi everybody Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
Of course the fc loops are on fc ports forced at 4 Gbps while the at to negotiate at 8. Can be this the.issue?
Should I.have to.use a separate hba just for sas and another one for fc disk loops? Reading docs this don't seem a requisite. Regards
Inviato da Samsung Mobile
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx,
Maybe I've not well explained using my smartphone to write. I know how the connections must be made, this is not my first FMC. My question is related to this one using at the same time existing DS14MK4FC shelves and a new set of SAS ones bridged to FC switches using the Atto.
In this way on the switch some zoning must be made and at the end we will have zone for FC loops and controller initiators and zone for FC datapath coming from SAS and the same initiators. Suppose that you have an FC disk loop coming to port 7 and the 4 HBA initiators fm NetApp controllers on a side of the MC we will have:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,7" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,7"
now we plug the fibers coming fm the two Atto on port 8 on each switch, so:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,8" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,8"
In this way the initiator ports on ports 1 and 2 we'll receive traffic at 4 Gbps from the FC shelves and traffic at 8 Gbps from Atto...
With this configurazion our 8040 crashed after few minutes from Atto insertion with a lot of messages like this:
[localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L9: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 -----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bert Kiers Inviato: domenica 15 giugno 2014 19:39 A: toasters@teaparty.net Oggetto: Re: Fabrica metro cluster with ds14mk4 and ds2246
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx, -- Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters- bounces@teaparty.net] On Behalf Of Milazzo Giacomo Sent: Monday, June 16, 2014 1:03 AM To: Bert Kiers; toasters@teaparty.net Subject: R: Fabrica metro cluster with ds14mk4 and ds2246
Maybe I've not well explained using my smartphone to write. I know how the connections must be made, this is not my first FMC. My question is related to this one using at the same time existing DS14MK4FC shelves and a new set of SAS ones bridged to FC switches using the Atto.
In this way on the switch some zoning must be made and at the end we will have zone for FC loops and controller initiators and zone for FC datapath coming from SAS and the same initiators. Suppose that you have an FC disk loop coming to port 7 and the 4 HBA initiators fm NetApp controllers on a side of the MC we will have:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,7" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,7"
now we plug the fibers coming fm the two Atto on port 8 on each switch, so:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,8" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,8"
In this way the initiator ports on ports 1 and 2 we'll receive traffic at 4 Gbps from the FC shelves and traffic at 8 Gbps from Atto...
To be sure - ports with DS14 must be set as loop port, while ports with ATTO *must* be set as locked G-Port. Did you verify switch port configuration? Could you paste output of "portcfgshow 8" on both switches?
With this configurazion our 8040 crashed after few minutes from Atto insertion with a lot of messages like this:
[localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L9: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd! .selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:deb! ug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 -----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bert Kiers Inviato: domenica 15 giugno 2014 19:39 A: toasters@teaparty.net Oggetto: Re: Fabrica metro cluster with ds14mk4 and ds2246
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx,
Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hi Andrei,
I think you've given me the right indication. It seems that port 8 (dedicated to Atto fiber) has been configured as the other "stor" port. There're a lot of misconfiguration I think Assuming is an existing for FC disck and 8 to SAS. Do you think that with 8040 controller and their FCVI up to 16 Gbps I have to change the spees of port 1 (for FCVI) to 8 Gbps? (8 is the limit of my switches)
Regards,
NAPPCAM1:admin> portcfgshow 9 Area Number: 9 Speed Level: 4G Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfgshow 8 Area Number: 8 Speed Level: AUTO(HW) Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfglport Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ----------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+--+-- Locked L_Port .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Loop Fairness .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. ..
Ports of Slot 0 16 17 18 19 20 21 22 23 -----------------+--+--+--+--+----+--+--+-- Locked L_Port .. .. .. .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. Loop Fairness .. .. .. .. .. .. .. ..
Bert Kiers kiersb@xs4all.net; toasters@teaparty.net
-----Messaggio originale----- Da: Borzenkov, Andrei [mailto:andrei.borzenkov@ts.fujitsu.com] Inviato: lunedì 16 giugno 2014 08:41 A: Milazzo Giacomo; Bert Kiers; toasters@teaparty.net Oggetto: RE: Fabrica metro cluster with ds14mk4 and ds2246
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters- bounces@teaparty.net] On Behalf Of Milazzo Giacomo Sent: Monday, June 16, 2014 1:03 AM To: Bert Kiers; toasters@teaparty.net Subject: R: Fabrica metro cluster with ds14mk4 and ds2246
Maybe I've not well explained using my smartphone to write. I know how the connections must be made, this is not my first FMC. My question is related to this one using at the same time existing DS14MK4FC shelves and a new set of SAS ones bridged to FC switches using the Atto.
In this way on the switch some zoning must be made and at the end we will have zone for FC loops and controller initiators and zone for FC datapath coming from SAS and the same initiators. Suppose that you have an FC disk loop coming to port 7 and the 4 HBA initiators fm NetApp controllers on a side of the MC we will have:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,7" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,7"
now we plug the fibers coming fm the two Atto on port 8 on each switch, so:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,8" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,8"
In this way the initiator ports on ports 1 and 2 we'll receive traffic at 4 Gbps from the FC shelves and traffic at 8 Gbps from Atto...
To be sure - ports with DS14 must be set as loop port, while ports with ATTO *must* be set as locked G-Port. Did you verify switch port configuration? Could you paste output of "portcfgshow 8" on both switches?
With this configurazion our 8040 crashed after few minutes from Atto insertion with a lot of messages like this:
[localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L9: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd! .selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:deb! ug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 -----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bert Kiers Inviato: domenica 15 giugno 2014 19:39 A: toasters@teaparty.net Oggetto: Re: Fabrica metro cluster with ds14mk4 and ds2246
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx,
Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hi Andrei,
I think you've given me the right indication. It seems that port 8 (dedicated to Atto fiber) has been configured as the other "stor" port.
Please, see actual "Brocade® Switch Configuration Guide For Fabric-attached MetroCluster™ 7-Mode Systems" for details on port configuration. The first thing you must do is to clear Locked L_Port and configure it as Locked G_Port. It's usually recommended to set fixed speed as well.
Do not forget to check and update ATTO firmware as well. In my experience they come with old firmware.
There're a lot of misconfiguration I think Assuming is an existing for FC disck and 8 to SAS. Do you think that with 8040 controller and their FCVI up to 16 Gbps I have to change the spees of port 1 (for FCVI) to 8 Gbps? (8 is the limit of my switches)
I'd say, connection cannot be faster than ISL anyway, so yes, it probably makes sense to avoid extra contention.
Regards,
NAPPCAM1:admin> portcfgshow 9 Area Number: 9 Speed Level: 4G Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfgshow 8 Area Number: 8 Speed Level: AUTO(HW) Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfglport Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ----------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+- -+-- Locked L_Port .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Loop Fairness .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. ..
Ports of Slot 0 16 17 18 19 20 21 22 23 -----------------+--+--+--+--+----+--+--+-- Locked L_Port .. .. .. .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. Loop Fairness .. .. .. .. .. .. .. ..
Bert Kiers kiersb@xs4all.net; toasters@teaparty.net
-----Messaggio originale----- Da: Borzenkov, Andrei [mailto:andrei.borzenkov@ts.fujitsu.com] Inviato: lunedì 16 giugno 2014 08:41 A: Milazzo Giacomo; Bert Kiers; toasters@teaparty.net Oggetto: RE: Fabrica metro cluster with ds14mk4 and ds2246
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters- bounces@teaparty.net] On Behalf Of Milazzo Giacomo Sent: Monday, June 16, 2014 1:03 AM To: Bert Kiers; toasters@teaparty.net Subject: R: Fabrica metro cluster with ds14mk4 and ds2246
Maybe I've not well explained using my smartphone to write. I know how the connections must be made, this is not my first FMC. My question is related to this one using at the same time existing DS14MK4FC shelves and a new set of SAS ones bridged to FC switches using the Atto.
In this way on the switch some zoning must be made and at the end we will have zone for FC loops and controller initiators and zone for FC datapath coming from SAS and the same initiators. Suppose that you have an FC disk loop coming to port 7 and the 4 HBA initiators fm NetApp controllers on a side of the MC we will have:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,7" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,7"
now we plug the fibers coming fm the two Atto on port 8 on each switch, so:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,8" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,8"
In this way the initiator ports on ports 1 and 2 we'll receive traffic at 4 Gbps from the FC shelves and traffic at 8 Gbps from Atto...
To be sure - ports with DS14 must be set as loop port, while ports with ATTO *must* be set as locked G-Port. Did you verify switch port configuration? Could you paste output of "portcfgshow 8" on both switches?
With this configurazion our 8040 crashed after few minutes from Atto insertion with a lot of messages like this:
[localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L9: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd! .selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:deb! ug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 -----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bert Kiers Inviato: domenica 15 giugno 2014 19:39 A: toasters@teaparty.net Oggetto: Re: Fabrica metro cluster with ds14mk4 and ds2246
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx,
Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
You're right again Andrei. I've just double checked the manual from FMC preparation from Brocade/NetApp. My collegue did not set the right parameters and I did not verify them...things that happen working in the night :-) (my first worries was to restart with old FC disks, the SAS are new and empty...)
I've just performed those commands and the port #8 now should be ok.
portcfglport 8 0 portcfggport 8 1 portcfgspeed 8 8
I'm agree with you about "not faster than ISL". I've asked to the provider there to have info on ISL capacity.
Thank you so much! Regards,
-----Messaggio originale----- Da: Borzenkov, Andrei [mailto:andrei.borzenkov@ts.fujitsu.com] Inviato: lunedì 16 giugno 2014 13:00 A: Milazzo Giacomo Cc: Bert Kiers; toasters@teaparty.net Oggetto: RE: Fabrica metro cluster with ds14mk4 and ds2246
Hi Andrei,
I think you've given me the right indication. It seems that port 8 (dedicated to Atto fiber) has been configured as the other "stor" port.
Please, see actual "Brocade® Switch Configuration Guide For Fabric-attached MetroClusterT 7-Mode Systems" for details on port configuration. The first thing you must do is to clear Locked L_Port and configure it as Locked G_Port. It's usually recommended to set fixed speed as well.
Do not forget to check and update ATTO firmware as well. In my experience they come with old firmware.
There're a lot of misconfiguration I think Assuming is an existing for FC disck and 8 to SAS. Do you think that with 8040 controller and their FCVI up to 16 Gbps I have to change the spees of port 1 (for FCVI) to 8 Gbps? (8 is the limit of my switches)
I'd say, connection cannot be faster than ISL anyway, so yes, it probably makes sense to avoid extra contention.
Regards,
NAPPCAM1:admin> portcfgshow 9 Area Number: 9 Speed Level: 4G Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfgshow 8 Area Number: 8 Speed Level: AUTO(HW) Fill Word(On Active) 3(A-A then SW I-A) Fill Word(Current) 3(A-A then SW I-A) AL_PA Offset 13: OFF Trunk Port ON Long Distance OFF VC Link Init OFF Locked L_Port ON Locked G_Port OFF Disabled E_Port OFF Locked E_Port OFF ISL R_RDY Mode OFF RSCN Suppressed OFF Persistent Disable OFF LOS TOV enable OFF NPIV capability ON QOS E_Port AE Port Auto Disable: OFF Rate Limit OFF Mirror Port OFF Credit Recovery ON F_Port Buffers OFF Fault Delay: 0(R_A_TOV) NPIV PP Limit: 126 CSCTL mode: OFF
NAPPCAM1:admin> portcfglport Ports of Slot 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ----------------------+--+--+--+--+----+--+--+--+----+--+--+--+----+--+- -+-- Locked L_Port .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. Loop Fairness .. .. .. ON .. ON ON ON ON ON ON .. .. .. .. ..
Ports of Slot 0 16 17 18 19 20 21 22 23 -----------------+--+--+--+--+----+--+--+-- Locked L_Port .. .. .. .. .. .. .. .. Private L_Port .. .. .. .. .. .. .. .. Locked Loop HD .. .. .. .. .. .. .. .. Loop Fairness .. .. .. .. .. .. .. ..
Bert Kiers kiersb@xs4all.net; toasters@teaparty.net
-----Messaggio originale----- Da: Borzenkov, Andrei [mailto:andrei.borzenkov@ts.fujitsu.com] Inviato: lunedì 16 giugno 2014 08:41 A: Milazzo Giacomo; Bert Kiers; toasters@teaparty.net Oggetto: RE: Fabrica metro cluster with ds14mk4 and ds2246
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters- bounces@teaparty.net] On Behalf Of Milazzo Giacomo Sent: Monday, June 16, 2014 1:03 AM To: Bert Kiers; toasters@teaparty.net Subject: R: Fabrica metro cluster with ds14mk4 and ds2246
Maybe I've not well explained using my smartphone to write. I know how the connections must be made, this is not my first FMC. My question is related to this one using at the same time existing DS14MK4FC shelves and a new set of SAS ones bridged to FC switches using the Atto.
In this way on the switch some zoning must be made and at the end we will have zone for FC loops and controller initiators and zone for FC datapath coming from SAS and the same initiators. Suppose that you have an FC disk loop coming to port 7 and the 4 HBA initiators fm NetApp controllers on a side of the MC we will have:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,7" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,7"
now we plug the fibers coming fm the two Atto on port 8 on each switch, so:
zonecreate "STOR_1_8", "1,1; 1,2; 3,1; 3,2; 1,8" zonecreate "STOR_3_8", "1,1; 1,2; 3,1; 3,2; 3,8"
In this way the initiator ports on ports 1 and 2 we'll receive traffic at 4 Gbps from the FC shelves and traffic at 8 Gbps from Atto...
To be sure - ports with DS14 must be set as loop port, while ports with ATTO *must* be set as locked G-Port. Did you verify switch port configuration? Could you paste output of "portcfgshow 8" on both switches?
With this configurazion our 8040 crashed after few minutes from Atto insertion with a lot of messages like this:
[localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L9: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L5: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L7: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd! .selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L12: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L6: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:deb! ug]: Disk device NAPPFAB1:8.125L22: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L13: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 [localhost:scsi.cmd.selectionTimeout:debug]: Disk device NAPPFAB1:8.125L3: Adapter/target error: HA status 0x7: cdb 0x12. Targeted device did not respond to requested I/O. I/O w Jun 14 23:08:17 -----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Bert Kiers Inviato: domenica 15 giugno 2014 19:39 A: toasters@teaparty.net Oggetto: Re: Fabrica metro cluster with ds14mk4 and ds2246
On 6/15/14 11:15 AM, Milazzo Giacomo wrote:
Hi everybody
Hi,
Can fc path coming fm atto bridges share the same iinitiators on the fas used also by the fc disk loop?
Following an head swap fm 3160 to 8040 with 8.2.1 I've zoned a store zone with the fc port used by atto and the 4 initiators on 8040. Exactly as done with port coming from fc loops
The fas starts to message a lot of timeouts and errors related to the sas disks then panic
In an FMC, all shelves are attached via an FC-switch, right? I expect the ATTO-brides are attached like a disk shelf, so connected to the switch, not to the head. We're going to do that in July / August.
Grtnx,
Bert Kiers XS4ALL UNIX systeembeheerder, suspected terrorist 1984 was not meant as a manual _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Bert Kiers -
Borzenkov, Andrei -
Milazzo Giacomo