This issues re-occurred on already running (ie it's not freshly provisioned servers) CentOS 6.6, and also on CentOS 7.2.1511. What I observed is that NFS clients could not mount the share on ONTAP 9 SVM after "lif migrate .." command.
This time I was able to collect tcpdump while running mount command. It was as follows:
1. Client sends (decoded by wireshark):
Remote Procedure Call, Type:Call XID:0x9fc37eb7 Network File System, Ops(3): SETCLIENTID_CONFIRM, PUTROOTFH, GETATTR [Program Version: 4] [V4 Procedure: COMPOUND (1)] Tag: <EMPTY> minorversion: 0 Operations (count: 3): SETCLIENTID_CONFIRM, PUTROOTFH, GETATTR Opcode: SETCLIENTID_CONFIRM (36) clientid: 0x36c1330000000076 verifier: 0x0000000000000000 Opcode: PUTROOTFH (24) Opcode: GETATTR (9) Attr mask: 0x00000400 (Lease_Time) reqd_attr: Lease_Time (10) [Main Opcode: SETCLIENTID_CONFIRM (36)]
2. SVM responds with:
Network File System, Ops(2): SETCLIENTID_CONFIRM PUTROOTFH(NFS4ERR_DELAY) [Program Version: 4] [V4 Procedure: COMPOUND (1)] Status: NFS4ERR_DELAY (10008) Tag: <EMPTY> length: 0 contents: <EMPTY> Operations (count: 2) Opcode: SETCLIENTID_CONFIRM (36) Status: NFS4_OK (0) Opcode: PUTROOTFH (24) Status: NFS4ERR_DELAY (10008) [Main Opcode: SETCLIENTID_CONFIRM (36)]
IMO, it was in an endless loop until I CTRL+C'ed the mount command.
On Tue, Nov 8, 2016 at 5:23 PM, Momonth momonth@gmail.com wrote:
I'm not 100% sure what has happened, most likely in-house "server provisioning tool" failed in such a way that some NFS related packages were not installed. As after re-installing the server in question from scratch everything runs fine. Unfortunately I was not able to collect enough debug info to have better conclusions.
On Wed, Nov 2, 2016 at 11:06 AM, Momonth momonth@gmail.com wrote:
Hi All,
We upgraded our 6 nodes cluster to ONTAP9 yesterday. Today I discovered some freshly installed CentOS 6.8 hosts not being able to mount file shares via NFS v4 protocol. The mount command just kind of hangs. I verified it's not a connectivity issue, ie connections to the filer's port 2049 are possible.
Also NFS v3 works just fine.
Did anyone experience anything like that?
Cheers, Vladimir