Hi all
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris
What I do is just disable the management port, and use the RLM, or SP for management. It's an additional hop, but is your saving grace to entry if the filer panics..
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Picton Sent: Thursday, March 21, 2013 8:31 AM To: toasters@teaparty.net Subject: Separating management and data traffic (e0M vs replication interfaces)
Hi all
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Same here.
On Thu, Mar 21, 2013 at 10:34 AM, Klise, Steve klises@sutterhealth.orgwrote:
What I do is just disable the management port, and use the RLM, or SP for management. It's an additional hop, but is your saving grace to entry if the filer panics..
-----Original Message----- From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Picton Sent: Thursday, March 21, 2013 8:31 AM To: toasters@teaparty.net Subject: Separating management and data traffic (e0M vs replication interfaces)
Hi all
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
I had seen the suggestion elsewhere. Is there any real point to the e0M interface then, as is it not truly out of band.
Do you then do other remote management (system manager/snmp monitoring/etc) via the data interfaces?
On 2013/03/21 5:37 PM, Brian Beaulieu wrote:
Same here.
On Thu, Mar 21, 2013 at 10:34 AM, Klise, Steve <klises@sutterhealth.org mailto:klises@sutterhealth.org> wrote:
What I do is just disable the management port, and use the RLM, or SP for management. It's an additional hop, but is your saving grace to entry if the filer panics.. -----Original Message----- From: toasters-bounces@teaparty.net <mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net <mailto:toasters-bounces@teaparty.net>] On Behalf Of Chris Picton Sent: Thursday, March 21, 2013 8:31 AM To: toasters@teaparty.net <mailto:toasters@teaparty.net> Subject: Separating management and data traffic (e0M vs replication interfaces) Hi all I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces. However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive. I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler. Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net <mailto:Toasters@teaparty.net> http://www.teaparty.net/mailman/listinfo/toasters _______________________________________________ Toasters mailing list Toasters@teaparty.net <mailto:Toasters@teaparty.net> http://www.teaparty.net/mailman/listinfo/toasters
I do.. Lots of stuff you can do with the networking pieces with these filers, but I try to keep mine simple. Learned a lot from these folks that post stuff..
I run everything across the 10gb interface. I don't even come close to saturation. Other mileage may vary (if you are facebook for example..) Some folks like to setup snapmirror down a set of interfaces, iSCSI down one, and maybe CIFS on its own.. Sometimes doing protocols on there own interfaces helps with troubleshooting issues, that about it.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Picton Sent: Thursday, March 21, 2013 8:43 AM Cc: toasters@teaparty.net Subject: Re: Separating management and data traffic (e0M vs replication interfaces)
I had seen the suggestion elsewhere. Is there any real point to the e0M interface then, as is it not truly out of band.
Do you then do other remote management (system manager/snmp monitoring/etc) via the data interfaces?
On 2013/03/21 5:37 PM, Brian Beaulieu wrote: Same here. On Thu, Mar 21, 2013 at 10:34 AM, Klise, Steve <klises@sutterhealth.orgmailto:klises@sutterhealth.org> wrote: What I do is just disable the management port, and use the RLM, or SP for management. It's an additional hop, but is your saving grace to entry if the filer panics..
-----Original Message----- From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] On Behalf Of Chris Picton Sent: Thursday, March 21, 2013 8:31 AM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Separating management and data traffic (e0M vs replication interfaces)
Hi all
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris _______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
On 2013-3-21 16:30 , Chris Picton wrote:
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system?
There's nothing special about snapmirror, it just uses the standard routing table like it would on a regular unix host. If you want to send your snapmirror traffic over a specific interface, then make sure that the target filer is reachable via that interface, for example by making sure that the target is in the same VLAN as the source.
What we do is send all production (NFS) traffic over private IP space in vlan's that aren't routed (no default gateway). Our vif interfaces are divided into multiple different vlan interfaces. If necessary, you can send the snapmirror traffic over such a vlan interface by making sure that the destination is in the right vlan.
I had the same issue years ago. First of all best practice is to avoid to put e0M/SP on the same subnet of other LAN/VLANs and they should be on a separate one. But I can understand this is not always possibile. There are a couple of options you could use to avoid the SM use the slow e0M.
interface.snapmirror.blocked The option is set to a comma-separated list of interface names for which snapmirror is blocked. The default is the empty list, "", which means that snapmirror is not blocked on any interface. The interface list cannot include TOE-enabled interfaces or iSCSI HBAs. See the NMG for details
ip.match_any_ifaddr If the option is on, the filer will accept any packet that is addressed to it even if that packet came in on the wrong interface. If you are concerned about security, you should turn this off. Valid values for this option are on or off. The default value for this option is on.
.
Regards,
-----Messaggio originale----- Da: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Jan-Pieter Cornet Inviato: giovedì 21 marzo 2013 22.36 A: Chris Picton Cc: toasters@teaparty.net Oggetto: Re: Separating management and data traffic (e0M vs replication interfaces)
On 2013-3-21 16:30 , Chris Picton wrote:
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system?
There's nothing special about snapmirror, it just uses the standard routing table like it would on a regular unix host. If you want to send your snapmirror traffic over a specific interface, then make sure that the target filer is reachable via that interface, for example by making sure that the target is in the same VLAN as the source.
What we do is send all production (NFS) traffic over private IP space in vlan's that aren't routed (no default gateway). Our vif interfaces are divided into multiple different vlan interfaces. If necessary, you can send the snapmirror traffic over such a vlan interface by making sure that the destination is in the right vlan.
This isn't the way that e0M is meant to be used.
Your default route needs to be on a VLAN which the production interfaces are a member of, otherwise any non-local traffic of any protocol will be routed via the management interface. And your e0M interface must be in a completely separate VLAN from any production interface.
I believe the e0M interface is only really useful if any machine that you would use to manage the system is also on the same VLAN as the e0M interface. NetApps have a clean separation of management protocols from data access protocols, so you can disable management protocols (e.g. ssh) except on the e0M interface, and use "options interface.blocked.mgmt_data_traffic on" to disable data protocols on the management interface.
The one thing which breaks this model is OnCommand Unified Manager (DFM). It sends both ssh (management) and NDMP (data, but used for management) requests to a single IP. I got around that by having an access rule as follows, though using the other interface.blocked options might work as well:
options ssh.access if=e0M OR netgroup=dfmserver.example.com
All of this is only any use at all if you care about securing your management access in a different way (e.g. via switch ACLs) from your data access. Otherwise, just don't use the e0M port at all. You don't need it.
As others have said, enabling the RLM/SP is important for out of band management (we do). But even then, having the serial console accessible via a serial terminal server will get you out of a situation where the RLM/SP loses its IP number (this has happened to us while recovering a filer).
HTH, Jeremy
On 22/03/2013, at 2:30 AM, Chris Picton wrote:
Hi all
I have two pairs of 3210s. The default gateway of the systems are via the management interface IP range, so that they are reachable remotely on the e0M interfaces.
However, this is causing snapmirror replication to use those interfaces as well, which is undesirable from a speed/data path persepctive.
I have considered putting e0M into its own ipspace, but then how would I manipulate its routing table as it would not be in a vfiler.
Any other ideas about having e0M reachable from anywhere, but still use a different vif as the default gateway for generic traffic on the system? Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Brian Beaulieu -
Chris Picton -
Jan-Pieter Cornet -
Jeremy Webber -
Klise, Steve -
Milazzo Giacomo