Well as is always the way with these random issues it is the simplest thing to fix the problem. After a test to mount the storage on one of the SLES servers which worked fine we have changed the redhat server mounts to tcp as that was what we used for the SLES mounts and the file system is fine and all databases have started up and been running for a number of hours with no problems.
I have chased down the person who set the servers up and he was trying different options to see what gave the best performance and left udp in the options as while he wasn't sure that it was any better it hadn't got any worse.
Thank you to everyone with the quick replies as if I had been waiting on the ticket I logged I would be no further forward that Tuesday morning.
-----Original Message----- From: John Stoffel [mailto:john@stoffel.org] Sent: 24 January 2018 17:20 To: Steiner, Jeffrey Cc: Mark Saunders; Parisi, Justin; Fenn, Michael; toasters@teaparty.net Subject: RE: NFS issue after upgrading filers to 9.2P2
Jeffrey> Could this be TCP slot tables? Flow control capabilities on Jeffrey> ONTAP continue to improve. If you don't have TCP slot tables Jeffrey> capped at 128 you could see quasi-hangs like this.
But the problem was with UDP NFS traffic, right?
I've run into wierd problems in the past. I think we had a problem where the volumes holding the oracle tablespaces were mounted with "forcedirectio", but if we had any executables on there, they just wouldn't work and we'd have all kinds of problems.
Maybe it's something like that?
And just to confirm, other clients running newer RHEL versions, or SLES using the exact same interfaceces/IPs/mountpoint from the Netapp cluster don't show the problem?
Just because VMWare and SLES 11/12 aren't seeing the problem, doesn't mean you don't have some wierd configuration issue somewhere. What does "config advisor" say when run against your cluster?
Remember, change just one thing at atime, otherwise you're going to go mad. Of course I'm sure the business is jumping up and down screaming, which makes it hard to be methodical.
Good luck and let us know what you find please!
John