Exactly. I was too terse in my reply. Plus a blurry distinction b/w "striped" and "distributed". Where one can argue (as you did) that striping is a special case of distribution. Which it is, I agree

As for the metadata server concept in pNFS see some details here: https://datatracker.ietf.org/doc/html/rfc8434#section-12

There is, conceptually, one (1) metadata server. It has to keep track of a whole bunch of things, the clients *must* talk NFSv4.1 to it and it has to control the so called layouts that pNFS clients are given. Or revoke layouts when and if applicable. The ONTAP implementation may "stripe" the metadata server (!) ;-) somehow, but still. I really don't know -- pNFS is cumbersome to set up and I have no incentive (or interest) to do it in any strong enough use case we have for our internal Storage Service

An excerpt from the above doc:

data server (DS): a pNFS server that provides the file's data when the file system object is accessed over a file-based protocol. Note that this usage differs from that in [RFC5661], which applies the term in some cases even when other sorts of protocols are being used. Depending on the layout, there might be one or more data servers over which the data is striped. While the metadata server is strictly accessed over the NFSv4.1 protocol, the data server could be accessed via any file access protocol that meets the pNFS requirements.

metadata server (MDS): the pNFS server that provides metadata information for a file system object. It is also responsible for generating, recalling, and revoking layouts for file system objects, for performing directory operations, and for performing I/O operations to regular files when the clients direct these to the metadata server itself.

See also http://www.pnfs.com/

Again: how ONTAP implements the metadata server, I'm not sure. Presumably in a scale-out way, as cleverly as possible with the technology available to NetApp already in ONTAP.

There has been (still is..?) ideas on how to in a fully standardised way stripe the metadata server function across many devices as well, but I'm not clear on the current status of this. I only find old references now (really old, 5+ y):

https://datatracker.ietf.org/doc/html/draft-mbenjamin-nfsv4-pnfs-metastripe-03

Maybe I'm missing something, comments welcome.

I also stumbled across this [URL below], it's 10+ y old but I've never seen this before. It was an interesting read; so many things have taken place and become mature in the past decade of the stuff which is mentioned in this slide pack. Not bad, NetApp :-) It took a wee bit longer than we all wanted, sure, but it works and one can trust it in a VERY BIG system (cluster).

http://www.nfsv4bat.org/Documents/ConnectAThon/2010/wafl-unstriped.pdf

/M

On 2021-08-16 16:22, tmac wrote: Just out of curiosity, what is the output of "network interface show -vserver pnfs-svm" The answer might lie there. I doubt it is a VLAN issue and more a network design issue.

I found this:

https://mysupport.netapp.com/site/article?lang=en&page=%2FAdvice_and_Tro...

Issue

* When NFS client attempts to Read/Write to the SVM with pNFS enabled, the operation hangs, with "nfs: [LIF address / hostname] server not responding" in /var/log/messages.

Cause

* This issue occurs in situations where pNFS is enabled on an SVM and there are LIFs created in different subnets, which are non-routable by the client * Data ONTAP may provide any LIF which belong's to the same SVM as the referral to a client * The Read/Write operation hangs because the data LIF IP is not reachable by the client

Solution

* Make sure clients can reach all data LIFs of the SVM

-or-

* Disable support for pNFS on the SVM

::> nfs modify -v4.1-pnfs disabled

--tmac

Tim McCarthy, Principal Consultant Proud Member of the #NetAppATeam I Blog at TMACsRack