Hi Mike,
Thanks, I had a similar issue with 1G copper SFPs in the past – but that doesn’t seem to apply here since the 40G ports are on an expansion slot and do not seem to be CNA capable:
*> cna_flash update all
NOTE: The flash update will take up to 30 minutes per update. (Current 2 chip(s) in the system) Do not interrupt power during the update, or you risk flash image corruption.
Version (5.0.2) is the same in flash and file No update of flash is needed for port 0e. Version (5.0.2) is the same in flash and file No update of flash is needed for port 0g.
-> While the expansion ports are in slot 2.
Also, the upgrade to Ontap 9.1 did not solve the issue – will check the cables now when I’m in the datacenter.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Mike Gossett [mailto:cmgossett@gmail.com] Gesendet: Samstag, 28. Januar 2017 00:58 An: Alexander Griesser AGriesser@anexia-it.com Cc: tmac tmacmd@gmail.com; toasters@teaparty.net Betreff: Re: X1144A - No link
Just a thought- I noticed that on a FAS2554, when trying to use 1G SFPs (copper) in the UTA ports, I had similar issues. I ended up having to apply a firmware update to the UTA2 system, IIRC, from maintenance mode. I have nothing that leads me to believe that is the case here, but it sounds somewhat close.
The updated firmware was actually on the system with the code it shipped with, but needed to be manually applied.
If all else fails, there's always a support ticket :)
HTH
Mike
On Fri, Jan 27, 2017 at 7:39 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Those are optical cables, that got shipped with the card and the transceivers. According to HWU, those are the transceivers:
<< pic removed >>
They look exactly like these and HWU calls them X65402. The connector looks exactly like the MPO connector which can also be seen in HWU and hast he cable number X66200-2 – so this is what I got:
X1144A Adapter X65402 Transceivers X66200-2 Cables
Should all play well with 9.1RC2 on the AFF-A300.
Since 9.1 has already been released, I’ll upgrade this system to 9.1 and see if that changes anything – maybe there’s a reason why they call it RC ☺
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Freitag, 27. Januar 2017 14:34
An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: X1144A - No link
OK, a little confused now. Are you trying to use copper or optical?
According to HWU, the X1144A as QSFP+ slots and the cable ends look identical to the SAS cables. The NetApp end should be any supported copper cable with a QSFP+ end or an optical transciever (X65402A, mfg 332-00389). The optical transciver will have the QSFP+ going into the filer and exposing the MPO end for your optical cable
Since it looks like you are trying to use Optical, are you *sure* the optical transcivers are the right ones for that card?
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Fri, Jan 27, 2017 at 8:14 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: That’s the 2-port 40G card with QSFPs and those funny new cables, no breakout cables.
http://cdn3.bigcommerce.com/s-uiigcj2/product_images/uploaded_images/mtp-cab...
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Freitag, 27. Januar 2017 13:40
An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: X1144A - No link
Curious...are you using it as a two-port 40Gb card or using the breakout cable?
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Fri, Jan 27, 2017 at 7:13 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Good question – I’m using the QSFPs and the cables that came with the cards, so I’m positive that those should be supported. But I’m trying to find out.
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Freitag, 27. Januar 2017 13:04 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: X1144A - No link
what is the cable that you are using? Anything supported? [Inline image 1][Inline image 2]
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/ On Fri, Jan 27, 2017 at 5:16 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Hi,
I’ve just installed those fancy X1144A cards in one of our AFF300 systems and it seems they’re getting recognized properly, but I’m not able to get a link on them.
sysconfig says:
slot 2: 40 Gigabit Ethernet Controller XL710 QSFP+ e2a MAC Address: 3c:fd:fe:a2:90:c8 (auto-unknown-down) e2e MAC Address: 3c:fd:fe:a2:90:c9 (auto-unknown-down)
SAN01::> net port show -port e2* (network port show)
Node: SAN01-01 Speed(Mbps) Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status --------- ------------ ---------------- ---- ---- ----------- -------- e2a Default - down 1500 auto/10 - e2e Default - down 1500 auto/10 -
Node: SAN01-02 Speed(Mbps) Health Port IPspace Broadcast Domain Link MTU Admin/Oper Status --------- ------------ ---------------- ---- ---- ----------- -------- e2a Default - down 1500 auto/10 - e2e Default - down 1500 auto/10 - 4 entries were displayed.
For testing purposes, I did connect e2a on node1 to e2a on node 2 and I already tried exchanging the QSFPs and cables, but still no dice. Any idea what could be wrong here?
According to HWU, the minimum required OS version is 9.1RC2, which is exactly what is installed currently:
SAN01::> version NetApp Release 9.1RC2: Thu Nov 24 08:59:46 UTC 2016
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hey Alexander,
Curious if you solved this issue.
I'm running into the same thing, but using copper QSFP NetApp cables.
I have several FAS9000s with X91440A-C dual-port 40GbE cards, connecting to X190001 (Cisco 3132Q-V cluster net switches) via 3 meter copper X66100-3 40GbE QSFP cables.
No link on either the filers, or the switches.
The switches see the cables are inserted, but have status 'notconnec'. The ISL trunk connections between switches (ports 31/32 listed here) are working though:
Ports 7 and 8 listed here are going back to the filers, and that's where I'm hoping to get link, but no luck.
pd-cluster-switch-2# show interface status
-------------------------------------------------------------------------------- Port Name Status Vlan Duplex Speed Type -------------------------------------------------------------------------------- mgmt0 -- connected routed full 1000 -- Eth1/1/1 10GbE Node Port xcvrAbsen 1 auto auto -- Eth1/1/2 10GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/7 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/8 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/9 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/10 40GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/29 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/30 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/31 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Eth1/32 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Po1 -- connected trunk full 40G --
Sysconfig for the card in question:
slot 4: 40 Gigabit Ethernet Controller XL710 QSFP+ e4a MAC Address: 00:a0:98:c5:e3:3c (auto-unknown-down) e4e MAC Address: 00:a0:98:c5:e3:40 (auto-unknown-down) Device Type: XL710 B1 Firmware Version: nvm 5.04 etid 8000250c Part Number: 111-02590 Hardware Revision: A2 Serial Number: 031648001359
Just upgraded from 9.1 to 9.1p2 in hopes that might have some effect, but no change.
I'm stumped.
On Mon, Jan 30, 2017 at 1:26 AM, Alexander Griesser <AGriesser@anexia-it.com
wrote:
Hi Mike,
Thanks, I had a similar issue with 1G copper SFPs in the past – but that doesn’t seem to apply here since the 40G ports are on an expansion slot and do not seem to be CNA capable:
*> cna_flash update all
NOTE: The flash update will take up to 30 minutes per update.
(Current 2 chip(s) in the system)Do not interrupt power during the update, or you risk flash image corruption.
Version (5.0.2) is the same in flash and file
No update of flash is needed for port 0e.
Version (5.0.2) is the same in flash and file
No update of flash is needed for port 0g.
-> While the expansion ports are in slot 2.
Also, the upgrade to Ontap 9.1 did not solve the issue – will check the cables now when I’m in the datacenter.
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* Mike Gossett [mailto:cmgossett@gmail.com] *Gesendet:* Samstag, 28. Januar 2017 00:58 *An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* tmac tmacmd@gmail.com; toasters@teaparty.net *Betreff:* Re: X1144A - No link
Just a thought-
I noticed that on a FAS2554, when trying to use 1G SFPs (copper) in the UTA ports, I had similar issues. I ended up having to apply a firmware update to the UTA2 system, IIRC, from maintenance mode. I have nothing that leads me to believe that is the case here, but it sounds somewhat close.
The updated firmware was actually on the system with the code it shipped with, but needed to be manually applied.
If all else fails, there's always a support ticket :)
HTH
Mike
[image: http://t.sidekickopen68.com/e1t/o/5/f18dQhb0S7ks8dDMPbW2n0x6l2B9gXrN7sKj6v5d...]
On Fri, Jan 27, 2017 at 7:39 AM, Alexander Griesser < AGriesser@anexia-it.com> wrote:
Those are optical cables, that got shipped with the card and the transceivers.
According to HWU, those are the transceivers:
<< pic removed >>
They look exactly like these and HWU calls them X65402.
The connector looks exactly like the MPO connector which can also be seen in HWU and hast he cable number X66200-2 – so this is what I got:
X1144A Adapter
X65402 Transceivers
X66200-2 Cables
Should all play well with 9.1RC2 on the AFF-A300.
Since 9.1 has already been released, I’ll upgrade this system to 9.1 and see if that changes anything – maybe there’s a reason why they call it RC J
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* tmac [mailto:tmacmd@gmail.com] *Gesendet:* Freitag, 27. Januar 2017 14:34
*An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* toasters@teaparty.net *Betreff:* Re: X1144A - No link
OK, a little confused now. Are you trying to use copper or optical?
According to HWU, the X1144A as QSFP+ slots and the cable ends look identical to the SAS cables.
The NetApp end should be any supported copper cable with a QSFP+ end or an optical transciever (X65402A, mfg 332-00389).
The optical transciver will have the QSFP+ going into the filer and exposing the MPO end for your optical cable
Since it looks like you are trying to use Optical, are you *sure* the optical transcivers are the right ones for that card?
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the **#NetAppATeam* https://twitter.com/NetAppATeam
*I Blog at **TMACsRack* https://tmacsrack.wordpress.com/
On Fri, Jan 27, 2017 at 8:14 AM, Alexander Griesser < AGriesser@anexia-it.com> wrote:
That’s the 2-port 40G card with QSFPs and those funny new cables, no breakout cables.
http://cdn3.bigcommerce.com/s-uiigcj2/product_images/ uploaded_images/mtp-cable.jpg?t=1424456134
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* tmac [mailto:tmacmd@gmail.com] *Gesendet:* Freitag, 27. Januar 2017 13:40
*An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* toasters@teaparty.net *Betreff:* Re: X1144A - No link
Curious...are you using it as a two-port 40Gb card or using the breakout cable?
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the **#NetAppATeam* https://twitter.com/NetAppATeam
*I Blog at **TMACsRack* https://tmacsrack.wordpress.com/
On Fri, Jan 27, 2017 at 7:13 AM, Alexander Griesser < AGriesser@anexia-it.com> wrote:
Good question – I’m using the QSFPs and the cables that came with the cards, so I’m positive that those should be supported.
But I’m trying to find out.
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* tmac [mailto:tmacmd@gmail.com] *Gesendet:* Freitag, 27. Januar 2017 13:04 *An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* toasters@teaparty.net *Betreff:* Re: X1144A - No link
what is the cable that you are using? Anything supported?
[image: Inline image 1][image: Inline image 2]
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the **#NetAppATeam* https://twitter.com/NetAppATeam
*I Blog at **TMACsRack* https://tmacsrack.wordpress.com/
On Fri, Jan 27, 2017 at 5:16 AM, Alexander Griesser < AGriesser@anexia-it.com> wrote:
Hi,
I’ve just installed those fancy X1144A cards in one of our AFF300 systems and it seems they’re getting recognized properly, but I’m not able to get a link on them.
sysconfig says:
slot 2: 40 Gigabit Ethernet Controller XL710 QSFP+ e2a MAC Address: 3c:fd:fe:a2:90:c8 (auto-unknown-down) e2e MAC Address: 3c:fd:fe:a2:90:c9 (auto-unknown-down)SAN01::> net port show -port e2*
(network port show)
Node: SAN01-01
Speed(Mbps) HealthPort IPspace Broadcast Domain Link MTU Admin/Oper Status
e2a Default - down 1500 auto/10 -
e2e Default - down 1500 auto/10 -
Node: SAN01-02
Speed(Mbps) HealthPort IPspace Broadcast Domain Link MTU Admin/Oper Status
e2a Default - down 1500 auto/10 -
e2e Default - down 1500 auto/10 -
4 entries were displayed.
For testing purposes, I did connect e2a on node1 to e2a on node 2 and I already tried exchanging the QSFPs and cables, but still no dice.
Any idea what could be wrong here?
According to HWU, the minimum required OS version is 9.1RC2, which is exactly what is installed currently:
SAN01::> version
NetApp Release 9.1RC2: Thu Nov 24 08:59:46 UTC 2016
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Hi Mike,
yes, I got it solved and wanted to reply back earlier but somehow always forgot about it :-/ I’m not really sure what finally was the fix, since I was having the filer in my lab when I encountered the issues and time ran out on the project to further debug this, so we shipped it to its final destination, powered it back on and everything was working. So I’m not sure i fit was the full halt of both filers to finally accept the NICs or i fit was a stronger guy than me who pushed in the QSFP+ modules (since the datacenter guy said that compared to the other modules in the servers f.ex., those had to be pushed in harder, whatever that means) and from then on, I was having full connectivity on all four modules in my gear.
I think I’ll never find out if it was me being to weak, or if the full powercycle helped it.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Mike Thompson [mailto:mike.thompson@gmail.com] Gesendet: Sonntag, 2. April 2017 10:28 An: Alexander Griesser AGriesser@anexia-it.com Cc: toasters@teaparty.net Betreff: Re: X1144A - No link
Hey Alexander, Curious if you solved this issue.
I'm running into the same thing, but using copper QSFP NetApp cables. I have several FAS9000s with X91440A-C dual-port 40GbE cards, connecting to X190001 (Cisco 3132Q-V cluster net switches) via 3 meter copper X66100-3 40GbE QSFP cables. No link on either the filers, or the switches. The switches see the cables are inserted, but have status 'notconnec'. The ISL trunk connections between switches (ports 31/32 listed here) are working though: Ports 7 and 8 listed here are going back to the filers, and that's where I'm hoping to get link, but no luck.
pd-cluster-switch-2# show interface status -------------------------------------------------------------------------------- Port Name Status Vlan Duplex Speed Type -------------------------------------------------------------------------------- mgmt0 -- connected routed full 1000 -- Eth1/1/1 10GbE Node Port xcvrAbsen 1 auto auto -- Eth1/1/2 10GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/7 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/8 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/9 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/10 40GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/29 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/30 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/31 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Eth1/32 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Po1 -- connected trunk full 40G -- Sysconfig for the card in question:
slot 4: 40 Gigabit Ethernet Controller XL710 QSFP+ e4a MAC Address: 00:a0:98:c5:e3:3c (auto-unknown-down) e4e MAC Address: 00:a0:98:c5:e3:40 (auto-unknown-down) Device Type: XL710 B1 Firmware Version: nvm 5.04 etid 8000250c Part Number: 111-02590 Hardware Revision: A2 Serial Number: 031648001359 Just upgraded from 9.1 to 9.1p2 in hopes that might have some effect, but no change. I'm stumped.
If I recall, I remember reading when changing the personality of those cards (i.e. 40G to 4x10G or vice-versa), it requires going into maintenance mode, making the change, doing a halt and then a power cycle of the head to fully reset the NIC. a reboot or halt/boot is not good enough.
--tmac
*Tim McCarthy, **Principal Consultant*
*Proud Member of the #NetAppATeam https://twitter.com/NetAppATeam*
*I Blog at TMACsRack https://tmacsrack.wordpress.com/*
On Sun, Apr 2, 2017 at 11:39 AM, Alexander Griesser <AGriesser@anexia-it.com
wrote:
Hi Mike,
yes, I got it solved and wanted to reply back earlier but somehow always forgot about it :-/
I’m not really sure what finally was the fix, since I was having the filer in my lab when I encountered the issues and time ran out on the project to further debug this, so we shipped it to its final destination, powered it back on and everything was working.
So I’m not sure i fit was the full halt of both filers to finally accept the NICs or i fit was a stronger guy than me who pushed in the QSFP+ modules (since the datacenter guy said that compared to the other modules in the servers f.ex., those had to be pushed in harder, whatever that means) and from then on, I was having full connectivity on all four modules in my gear.
I think I’ll never find out if it was me being to weak, or if the full powercycle helped it.
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* Mike Thompson [mailto:mike.thompson@gmail.com] *Gesendet:* Sonntag, 2. April 2017 10:28 *An:* Alexander Griesser AGriesser@anexia-it.com *Cc:* toasters@teaparty.net *Betreff:* Re: X1144A - No link
Hey Alexander,
Curious if you solved this issue.
I'm running into the same thing, but using copper QSFP NetApp cables.
I have several FAS9000s with X91440A-C dual-port 40GbE cards, connecting to X190001 (Cisco 3132Q-V cluster net switches) via 3 meter copper X66100-3 40GbE QSFP cables.
No link on either the filers, or the switches.
The switches see the cables are inserted, but have status 'notconnec'. The ISL trunk connections between switches (ports 31/32 listed here) are working though:
Ports 7 and 8 listed here are going back to the filers, and that's where I'm hoping to get link, but no luck.
pd-cluster-switch-2# show interface status
Port Name Status Vlan Duplex Speed Type
mgmt0 -- connected routed full 1000 -- Eth1/1/1 10GbE Node Port xcvrAbsen 1 auto auto -- Eth1/1/2 10GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/7 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/8 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/9 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/10 40GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/29 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/30 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/31 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Eth1/32 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Po1 -- connected trunk full 40G --
Sysconfig for the card in question:
slot 4: 40 Gigabit Ethernet Controller XL710 QSFP+ e4a MAC Address: 00:a0:98:c5:e3:3c (auto-unknown-down) e4e MAC Address: 00:a0:98:c5:e3:40 (auto-unknown-down) Device Type: XL710 B1 Firmware Version: nvm 5.04 etid 8000250c Part Number: 111-02590 Hardware Revision: A2 Serial Number: 031648001359Just upgraded from 9.1 to 9.1p2 in hopes that might have some effect, but no change.
I'm stumped.
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
All –
I am a little late to the party here, but FYI changing these from 1x40GbE to 4x10GbE or vice versa requires that the controller hosting the card you’re doing this on requires a restart to be functional (just like changing a CNA card personality from FC to Ethernet or an FC interface to target/initiator).
Cheers & Good Luck!
Anthony Bar | BERKCOM.COM
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of tmac Sent: Sunday, April 02, 2017 8:46 AM To: Alexander Griesser AGriesser@anexia-it.com Cc: toasters@teaparty.net Subject: Re: X1144A - No link
If I recall, I remember reading when changing the personality of those cards (i.e. 40G to 4x10G or vice-versa), it requires going into maintenance mode, making the change, doing a halt and then a power cycle of the head to fully reset the NIC. a reboot or halt/boot is not good enough.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Sun, Apr 2, 2017 at 11:39 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Hi Mike,
yes, I got it solved and wanted to reply back earlier but somehow always forgot about it :-/ I’m not really sure what finally was the fix, since I was having the filer in my lab when I encountered the issues and time ran out on the project to further debug this, so we shipped it to its final destination, powered it back on and everything was working. So I’m not sure i fit was the full halt of both filers to finally accept the NICs or i fit was a stronger guy than me who pushed in the QSFP+ modules (since the datacenter guy said that compared to the other modules in the servers f.ex., those had to be pushed in harder, whatever that means) and from then on, I was having full connectivity on all four modules in my gear.
I think I’ll never find out if it was me being to weak, or if the full powercycle helped it.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Mike Thompson [mailto:mike.thompson@gmail.commailto:mike.thompson@gmail.com] Gesendet: Sonntag, 2. April 2017 10:28 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: X1144A - No link
Hey Alexander, Curious if you solved this issue.
I'm running into the same thing, but using copper QSFP NetApp cables. I have several FAS9000s with X91440A-C dual-port 40GbE cards, connecting to X190001 (Cisco 3132Q-V cluster net switches) via 3 meter copper X66100-3 40GbE QSFP cables. No link on either the filers, or the switches. The switches see the cables are inserted, but have status 'notconnec'. The ISL trunk connections between switches (ports 31/32 listed here) are working though: Ports 7 and 8 listed here are going back to the filers, and that's where I'm hoping to get link, but no luck.
pd-cluster-switch-2# show interface status -------------------------------------------------------------------------------- Port Name Status Vlan Duplex Speed Type -------------------------------------------------------------------------------- mgmt0 -- connected routed full 1000 -- Eth1/1/1 10GbE Node Port xcvrAbsen 1 auto auto -- Eth1/1/2 10GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/7 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/8 40GbE Node Port notconnec 1 auto auto QSFP-40G-CR4 Eth1/9 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/10 40GbE Node Port xcvrAbsen 1 auto auto -- ... Eth1/29 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/30 40GbE Node Port xcvrAbsen 1 auto auto -- Eth1/31 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Eth1/32 Inter-Cluster Swit connected trunk full 40G QSFP-40G-CR4 Po1 -- connected trunk full 40G -- Sysconfig for the card in question:
slot 4: 40 Gigabit Ethernet Controller XL710 QSFP+ e4a MAC Address: 00:a0:98:c5:e3:3c (auto-unknown-down) e4e MAC Address: 00:a0:98:c5:e3:40 (auto-unknown-down) Device Type: XL710 B1 Firmware Version: nvm 5.04 etid 8000250c Part Number: 111-02590 Hardware Revision: A2 Serial Number: 031648001359 Just upgraded from 9.1 to 9.1p2 in hopes that might have some effect, but no change. I'm stumped.
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Alexander Griesser -
Mike Thompson -
tmac -
Tony Bar