toasters

toasters@lists.teaparty.net

13529 discussions

Re: pNFS not working on vserver when several lifs with different Vlans available
by Florian Schmid 17 Aug '21

17 Aug '21

Good morning, thank you very much for all those replys. I really haven't thought that :) Yes, our network wasn't designed for pNFS at all, because we have never used it. We have started now to use clients, which are using this by default and we have encountered this problem. Our storage network was ever only switched, so no routing is/was possible and we always had to choose the right IP range for the client and node. We also normally don't have huge clusters, so no volume movement either. But I have really thought, that the pNFS implementation is at least broadcast domain aware, but it seems it isn't. This means, that pNFS is not working in our environment. That's ok for us. Thanks a lot for the deeper insight you all have given me. Best regards Florian Schmid

1 0

Re: pNFS not working on vserver when several lifs with different Vlans available
by Michael Bergman 16 Aug '21

16 Aug '21

On 2021-08-16 18:41, Justin Parisi wrote: > I think it was "working as designed." RFE is several years old; filed it > when I was still in support. It is. Working as designed. That's my take on this. Meaning: if you misunderstand things a bit and design your network around the cluster intended to run and leverage pNFS in such a way that doesn't make sense really, you'll have to turn pNFS off and then it'll work again for the clients. Back to the drawing board, re-design the network e.g. move your LIFs around so that they're all in the same VLAN (subnet). We don't run pNFS anywhere, but that's the customary way we do it for our very large clusters (our biggest is a 20-node now, in refresh state b/w 5 y old FAS8080 and new A800) in any case, since many years. And *everything*, all NFS traffic, always goes across at least one L3 hop. Nothing else, no NFS clients!, are allowed in the VLANs where our LIFs sit. That way we "own" that VLAN completely and can control it in a good fashion. BTW: Wow. A several years old RFE from *you* who hasn't been implemented yet. Not much demand for it then I'd say /M -------- Original Message -------- Subject: Re: pNFS not working on vserver when several lifs with different Vlans available Date: Mon, 16 Aug 2021 12:17:45 -0400 From: tmac <tmacmd(a)gmail.com> To: Parisi, Justin <Justin.Parisi(a)netapp.com> CC: Michael Bergman <michael.bergman(a)ericsson.com>, Toasters <toasters(a)teaparty.net> Wow...Thanks Justin. I guess that was possibly an oversight when developed (and hence the RFE)? --tmac *Tim McCarthy, */Principal Consultant/ *Proud Member of the #NetAppATeam *I Blog at TMACsRack

1 0

Re:
by tmac 16 Aug '21

16 Aug '21

Wow...Thanks Justin. I guess that was possibly an oversight when developed (and hence the RFE)? --tmac *Tim McCarthy, **Principal Consultant* *Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>* *I Blog at TMACsRack <https://tmacsrack.wordpress.com/>* On Mon, Aug 16, 2021 at 12:09 PM Parisi, Justin via Toasters < toasters(a)teaparty.net> wrote: > > > > ---------- Forwarded message ---------- > From: "Parisi, Justin" <Justin.Parisi(a)netapp.com> > To: Michael Bergman <michael.bergman(a)ericsson.com>, Toasters < > toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 16:06:37 +0000 > Subject: Re: > FYI TR-4067 calls out this exact issue as well. There’s currently no way > to “blacklist” inelegible interfaces for pnfs. If it’s in the SVM, it will > be used with pNFS, regardless of if it’s reachable. > > We have an open RFE to allow you to specify LIFs for pNFS to avoid this > problem. In the meantime you can blacklist devices from the client as per > page 54 of the tr. > > https://www.netapp.com/pdf.html?item=/media/10720-tr-4067.pdf > ------------------------------ > *From:* Toasters <toasters-bounces(a)teaparty.net> on behalf of Michael > Bergman via Toasters <toasters(a)teaparty.net> > *Sent:* Monday, August 16, 2021 11:10:12 AM > *To:* Toasters <toasters(a)teaparty.net> > *Subject:* > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is > safe. > > > > > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters > > > > ---------- Forwarded message ---------- > From: "Parisi, Justin via Toasters" <toasters(a)teaparty.net> > To: Michael Bergman <michael.bergman(a)ericsson.com>, Toasters < > toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 17:06:46 +0100 > Subject: > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters

2 1

Re:
by Parisi, Justin 16 Aug '21

16 Aug '21

FYI TR-4067 calls out this exact issue as well. There’s currently no way to “blacklist” inelegible interfaces for pnfs. If it’s in the SVM, it will be used with pNFS, regardless of if it’s reachable. We have an open RFE to allow you to specify LIFs for pNFS to avoid this problem. In the meantime you can blacklist devices from the client as per page 54 of the tr. https://www.netapp.com/pdf.html?item=/media/10720-tr-4067.pdf ________________________________ From: Toasters <toasters-bounces(a)teaparty.net> on behalf of Michael Bergman via Toasters <toasters(a)teaparty.net> Sent: Monday, August 16, 2021 11:10:12 AM To: Toasters <toasters(a)teaparty.net> Subject: NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. _______________________________________________ Toasters mailing list Toasters(a)teaparty.net https://www.teaparty.net/mailman/listinfo/toasters

1 0

Re: pNFS not working on vserver when several lifs with different Vlans available
by Michael Bergman 16 Aug '21

16 Aug '21

Exactly. I was too terse in my reply. Plus a blurry distinction b/w "striped" and "distributed". Where one can argue (as you did) that striping is a special case of distribution. Which it is, I agree As for the metadata server concept in pNFS see some details here: <https://datatracker.ietf.org/doc/html/rfc8434#section-12> There is, conceptually, one (1) metadata server. It has to keep track of a whole bunch of things, the clients *must* talk NFSv4.1 to it and it has to control the so called layouts that pNFS clients are given. Or revoke layouts when and if applicable. The ONTAP implementation may "stripe" the metadata server (!) ;-) somehow, but still. I really don't know -- pNFS is cumbersome to set up and I have no incentive (or interest) to do it in any strong enough use case we have for our internal Storage Service An excerpt from the above doc: data server (DS): a pNFS server that provides the file's data when the file system object is accessed over a file-based protocol. Note that this usage differs from that in [RFC5661], which applies the term in some cases even when other sorts of protocols are being used. Depending on the layout, there might be one or more data servers over which the data is striped. While the metadata server is strictly accessed over the NFSv4.1 protocol, the data server could be accessed via any file access protocol that meets the pNFS requirements. metadata server (MDS): the pNFS server that provides metadata information for a file system object. It is also responsible for generating, recalling, and revoking layouts for file system objects, for performing directory operations, and for performing I/O operations to regular files when the clients direct these to the metadata server itself. See also <http://www.pnfs.com/> Again: how ONTAP implements the metadata server, I'm not sure. Presumably in a scale-out way, as cleverly as possible with the technology available to NetApp already in ONTAP. There has been (still is..?) ideas on how to in a fully standardised way stripe the metadata server function across many devices as well, but I'm not clear on the current status of this. I only find old references now (really old, 5+ y): <https://datatracker.ietf.org/doc/html/draft-mbenjamin-nfsv4-pnfs-metastripe…> Maybe I'm missing something, comments welcome. I also stumbled across this [URL below], it's 10+ y old but I've never seen this before. It was an interesting read; so many things have taken place and become mature in the past decade of the stuff which is mentioned in this slide pack. Not bad, NetApp :-) It took a wee bit longer than we all wanted, sure, but it works and one can trust it in a VERY BIG system (cluster). <http://www.nfsv4bat.org/Documents/ConnectAThon/2010/wafl-unstriped.pdf> /M On 2021-08-16 16:22, tmac wrote: Just out of curiosity, what is the output of "network interface show -vserver pnfs-svm" The answer might lie there. I doubt it is a VLAN issue and more a network design issue. I found this: https://mysupport.netapp.com/site/article?lang=en&page=%2FAdvice_and_Troubl… Issue * When NFS client attempts to Read/Write to the SVM with pNFS enabled, the operation hangs, with "nfs: [LIF address / hostname] server not responding" in /var/log/messages. Cause * This issue occurs in situations where pNFS is enabled on an SVM and there are LIFs created in different subnets, which are non-routable by the client * Data ONTAP may provide any LIF which belong's to the same SVM as the referral to a client * The Read/Write operation hangs because the data LIF IP is not reachable by the client Solution * Make sure clients can reach all data LIFs of the SVM -or- * Disable support for pNFS on the SVM ::> nfs modify -v4.1-pnfs disabled --tmac Tim McCarthy, Principal Consultant Proud Member of the #NetAppATeam I Blog at TMACsRack

1 0

Re:
by tmac 16 Aug '21

16 Aug '21

Just out of curiosity, what is the output of "network interface show -vserver pnfs-svm" The answer might lie there. I doubt it is a VLAN issue and more a network design issue. I found this: https://mysupport.netapp.com/site/article?lang=en&page=%2FAdvice_and_Troubl… Issue - When NFS client attempts to Read/Write to the SVM with pNFS enabled, the operation hangs, with "nfs: [LIF address / hostname] server not responding" in /var/log/messages. Cause - This issue occurs in situations where pNFS is enabled on an SVM and there are LIFs created in different subnets, which are non-routable by the client - Data ONTAP may provide any LIF which belong's to the same SVM as the referral to a client - The Read/Write operation hangs because the data LIF IP is not reachable by the client Solution - Make sure clients can reach all data LIFs of the SVM -or- - Disable support for pNFS on the SVM ::> nfs modify -v4.1-pnfs disabled --tmac *Tim McCarthy, **Principal Consultant* *Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>* *I Blog at TMACsRack <https://tmacsrack.wordpress.com/>* On Mon, Aug 16, 2021 at 9:16 AM Florian Schmid via Toasters < toasters(a)teaparty.net> wrote: > > > > ---------- Forwarded message ---------- > From: Florian Schmid <fschmid(a)ubimet.com> > To: toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 13:13:15 +0000 (UTC) > Subject: pNFS not working on vserver when several lifs with different > Vlans available > Hi, > > I had a very strange incident some weeks ago. > > We are on 9.7 branch. > > In one of our vServers, where we had NFSv4 and 4.1 enabled, clients with > NFSv4.1 and pNFS where unable to read or write data. > Directory listing was possible. > > On the vServer, we have several different Vlans and each Vlans has an > interface for each node there. Our storage network is completely switched, > so no routing between Vlans possible. > > After creating a NetApp case, they told me, that pNFS doesn't handle Vlans > and the client will get any IP on that node, even when the IP is from a > different Vlan and therefor not accessible from the client. > > Is this really the case? Is the pNFS process really not Vlan aware and > sends any IP from that node where volume is located, even when the IP is in > a different network than the client IP? > > As suggested from NetApp, I have disabled pNFS and now everything is > working. > > What is your experience here? > Is anybody using pNFS with NetApp and if yes, do you only have one network > in your vServer or is the client able to access all IPs there through > different Vlans? > > I'm curious about your answers. > > Best regards > Florian Schmid > > > > > ---------- Forwarded message ---------- > From: Florian Schmid via Toasters <toasters(a)teaparty.net> > To: toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 14:13:20 +0100 > Subject: > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters

1 0

Re:
by tmac 16 Aug '21

16 Aug '21

I would respectfully disagree. pNFS does not stripe data across controllers. Data is distributed, not striped. Every node is a metadata server and will redirect the client process to the node directly holding the volume. No striping. Maybe you are confusing pNFS with FlexGroups which distributes files across nodes. A very long time ago in ONTAP GX, there >>was<< an ability to stripe data across controllers. That feature was removed upon going to ONTAP 8.0 --tmac *Tim McCarthy, **Principal Consultant* *Proud Member of the #NetAppATeam <https://twitter.com/NetAppATeam>* *I Blog at TMACsRack <https://tmacsrack.wordpress.com/>* On Mon, Aug 16, 2021 at 9:36 AM Michael Bergman via Toasters < toasters(a)teaparty.net> wrote: > > > > ---------- Forwarded message ---------- > From: Michael Bergman <michael.bergman(a)ericsson.com> > To: Toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 13:34:17 +0000 > Subject: Re: pNFS not working on vserver when several lifs with different > Vlans available > On 2021-08-16 15:13, Florian Schmid wrote: > > Is this really the case? Is the pNFS process really not Vlan aware and > > sends any IP from that node where volume is located, even when the IP is > > in a different network than the client IP? > > Yes. That's how pNFS is supposed to work. That's the whole point one can > say, of pNFS. It's striping of the data traffic (not metadata) across > several nodes, all at once. > > In that respect, the setup you say you have, this: > > "On the vServer, we have several different Vlans and each Vlans has an > interface for each node there. Our storage network is completely switched, > so no routing between Vlans possible." > > is totally wrong for pNFS. In such a set up you definitely do not want > pNFS. > > /M > > > > ---------- Forwarded message ---------- > From: Michael Bergman via Toasters <toasters(a)teaparty.net> > To: Toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Mon, 16 Aug 2021 14:34:25 +0100 > Subject: > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters

1 0

Re: pNFS not working on vserver when several lifs with different Vlans available
by Michael Bergman 16 Aug '21

16 Aug '21

On 2021-08-16 15:13, Florian Schmid wrote: > Is this really the case? Is the pNFS process really not Vlan aware and > sends any IP from that node where volume is located, even when the IP is > in a different network than the client IP? Yes. That's how pNFS is supposed to work. That's the whole point one can say, of pNFS. It's striping of the data traffic (not metadata) across several nodes, all at once. In that respect, the setup you say you have, this: "On the vServer, we have several different Vlans and each Vlans has an interface for each node there. Our storage network is completely switched, so no routing between Vlans possible." is totally wrong for pNFS. In such a set up you definitely do not want pNFS. /M

1 0

pNFS not working on vserver when several lifs with different Vlans available
by Florian Schmid 16 Aug '21

16 Aug '21

Hi, I had a very strange incident some weeks ago. We are on 9.7 branch. In one of our vServers, where we had NFSv4 and 4.1 enabled, clients with NFSv4.1 and pNFS where unable to read or write data. Directory listing was possible. On the vServer, we have several different Vlans and each Vlans has an interface for each node there. Our storage network is completely switched, so no routing between Vlans possible. After creating a NetApp case, they told me, that pNFS doesn't handle Vlans and the client will get any IP on that node, even when the IP is from a different Vlan and therefor not accessible from the client. Is this really the case? Is the pNFS process really not Vlan aware and sends any IP from that node where volume is located, even when the IP is in a different network than the client IP? As suggested from NetApp, I have disabled pNFS and now everything is working. What is your experience here? Is anybody using pNFS with NetApp and if yes, do you only have one network in your vServer or is the client able to access all IPs there through different Vlans? I'm curious about your answers. Best regards Florian Schmid

1 0

Use X382A in DS224C on A200/A220
by Alexander Griesser 16 Aug '21

16 Aug '21

Hey there, Does anyone know if it's safe to use the X382A SSDs from C190 storages in A200/A220 storages in DS224C shelves? HWU is giving me mixed information (or I'm reading it wrong), DS2242C seems to be supported for this SKU but when you check for platform compatibility, only the C190 is listed (for whatever reason). The reason I'm asking is that I could get a few oft hem but they're too small for my use cases, but the SSDs could be useful in other environments. Thanks, Alexander Griesser Head of Systems Operations ANEXIA Internetdienstleistungs GmbH E-Mail: AGriesser(a)anexia-it.com<mailto:AGriesser@anexia-it.com> Web: http://www.anexia-it.com<http://www.anexia-it.com/> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

2 2

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters