Piece of cake, actually. You _CAN_ add ports with different portspeed to an already existing ifgrp. The filer will write the following into the log:
nodeX ERROR pvif.speedDuplexErr: ifgrp: a0a: Link e0c detected speed or duplex mismatch with ifgrp - link (speed: 40000, duplex: 1), ifgrp (speed: 10000, duplex: 1)
And it will disable them from partitipating in the ifgrp, of course. So what I did now was: * remove all off-board 10G adapters from the interface group, so that only the onboard NICs were still in there * 2x TO/GB to replace the PCI card with the 40G cards * add the 40G adapters to the ifgrp – both 40G ports will show as „not participating“ in the ifgrp, but that’s OK. * remove both 10G ports from the ifgrp -> ifgrp will immediately switch to 40Gbit/s linkspeed afterwards and you’re back up and running.
Since I’m a coward, I have migrated off all LIFs prior to that, but I think this should also work quite well (with just a short interruption) if you don’t do that, since the switch from one LACP channel to the other after the last 10G port was removed took only 1-2 seconds.
-> Topic done ☺
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser Gesendet: Dienstag, 18. April 2017 16:04 An: tmac tmacmd@gmail.com Cc: toasters@teaparty.net Betreff: AW: AW: Add new ports to interface groups
Alright, will do some testing here and let you guys know ☺ Thanks so far for the comments, let’s hope it works the easy way without having to recreate all the logical interfaces…
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.com] Gesendet: Dienstag, 18. April 2017 16:02 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: Scott M Gelb <scottgelb@yahoo.commailto:scottgelb@yahoo.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: AW: Add new ports to interface groups
Not sure. Figured if you can't ONTAP would let you know ;) It also might make a huge difference if all the ifgrp ports are DOWN i.e. migrate all working LIFS off the node, then take all ifgrp ports down: set adv; net port modify -node home-01 -port a0a* -up-admin false -> try the add/remove ports here.
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Tue, Apr 18, 2017 at 9:50 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Hmm… Can I mix different speeds in the same portgroup? I thought this was not possible if the portgroup is LACP? If I can mix them, then this will be easy actually, because I can just add in one of the unused Gigabit onboard Ports to the portgroup first and remove all the 10G ports ☺ But I recall that mixing different port speeds is not possible in one ifgrp – will try that later this evening on a test system.
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Dienstag, 18. April 2017 15:46 An: Scott M Gelb <scottgelb@yahoo.commailto:scottgelb@yahoo.com> Cc: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: AW: Add new ports to interface groups
Thinking a little crazy here....might just work:
Scott is correct. The same thing does happen in the SIM.
Do you have any unused ports at all? if you do, maybe you could do something like this:
net int migrate (any LIF on a0a)) to any other node. --> or modify the home ports to another node and then "net int revert" Here is where an extended query could work: net int modify { -home-node home-01 } -home-node home-02 Find any LIFs on home-01, change to home-02 net int revert * You may need to modify a bit to localize to a vserver or the like -> would not need to worry about auto-revert! net int show -fields auto-revert -> make sure auto-revert is off for all those LIFs
ifgrp add-port -node home-01 -ifgrp a0a -port e0x -> add the unused port, if available ifgrp remove-port -node home-01 -ifgrp a0a -port e0a -> remove 10Gb ifgrp remove-port -node home-01 -ifgrp a0a -port e0b -> remove 10Gb
failover node, halt node, remove 10GB, add 40GB, boot
(add in the 40Gb) ifgrp add-port -node home-01 -ifgrp a0a -port e0a ifgrp add-port -node home-01 -ifgrp a0a -port e0b (remove the unused port) ifgrp add-port -node home-01 -ifgrp a0a -port e0x
Make sure all broadcast domains are right. check your LIF failover: net int show -failover
Start migrating your LIFs back and make sure they work faster now
Some steps may be missed, but this overview may just do the trick!
--tmac
Tim McCarthy, Principal Consultant
Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam
I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Tue, Apr 18, 2017 at 9:05 AM, Scott M Gelb <scottgelb@yahoo.commailto:scottgelb@yahoo.com> wrote: You will get an error trying to remove the last 10Gb port on the ifgrp if VLANs are assigned "Error: command failed: Can't remove the last port of an ifgrp while the ifgrp is referenced by a vlan". However, if you leave the ifgrp and port before you reconfigure, if the ports are the same you may not need to delete the ports from the ifgrp. Of if different, you may be able to add the new port before removing the old port with all LIFs re-homed temporarily to another node. Need hardware to test on.. Since you are using the same PCIe slot this might work well after testing this first.
________________________________ From: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> To: tmac <tmacmd@gmail.commailto:tmacmd@gmail.com> Cc: "toasters@teaparty.netmailto:toasters@teaparty.net" <toasters@teaparty.netmailto:toasters@teaparty.net> Sent: Tuesday, April 18, 2017 5:56 AM Subject: AW: Add new ports to interface groups
I’m never sure how much you can trust the simulator with hardware related things like these and I do not have spare gear at hand here to try it myself :-/ I’ll see if I can dig something up and will try it on a physical system too, then.
Thanks,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: tmac [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Dienstag, 18. April 2017 14:41 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Cc: toasters@teaparty.netmailto:toasters@teaparty.net Betreff: Re: Add new ports to interface groups
That might work.
If I get a chance, I will try on a simulator today.
--tmac
Tim McCarthy, Principal Consultant Proud Member of the #NetAppATeamhttps://twitter.com/NetAppATeam I Blog at TMACsRackhttps://tmacsrack.wordpress.com/
On Tue, Apr 18, 2017 at 7:02 AM, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> wrote: Yah, that’s what I thought needs to be done, but that’s a bit cumbersome given the fact that there are like 100 VLANs configured on a0a and I need to update all broadcast domain memberships, etc. too – and I can only remove the VLAN ports if they’re not configured as home ports on any lifs I guess, to quite some fiddeling would be involved here.
What I’m looking for basically is something like disable a0a, remove all 10g ports, so I will end up with an empty a0a but all the VLANs on top of it will still be there, then halt the node, add the 40G adapter, add its ports to a0a -> done. Would that work or am I overlooking something obvious?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: Tim McCarthy [mailto:tmacmd@gmail.commailto:tmacmd@gmail.com] Gesendet: Dienstag, 18. April 2017 12:59 An: toasters@teaparty.netmailto:toasters@teaparty.net; Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Betreff: Re: Add new ports to interface groups
If you fail the a0a over to another node, making sure auto fall back is disabled, then destroy the a0a on the node, halt it, remove the 10g card, replace with the 40g card and recreate a0a with its vlans. Then you should be able to fall back.
Spelling errors courtesy of Swype for iOS...
Tim McCarthy, Consultant at Presidio ________________________________ From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net <toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net> on behalf of Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> Sent: Tuesday, April 18, 2017 6:50:31 AM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Add new ports to interface groups
Hey there,
I’m currently thinking about upgrading some of my filers from 10G to 40G interfaces. This would involve pulling out the 10G card and inserting the 40G card, so one halt per node is necessary, which isn’t really the problem. I’m just a bit unsure of the best way to move my a0a ifgrp to the new ports – I remember that in the past when you wanted to switch the ifgrp to a different portspeed, you had to recreate it completely which is a bit cumbersome due to all the VLANs built on top of it – has that changed?
If I issue a takeover of the first system and replace the card, it will probably boot up with a broken ifgrp configuration since it cannot find the physical ports in the ifgrp anymore and I’m looking for the best way to do a physical port upgrade like this now, so if anyone has gone through that already, feedback would be greatly appreciated.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters