Hello All,
We run a VMWare farm that mounts its datastores via NFS on two different NetApp clusters, one a 3020 HA pair running 7.2.4 and the other a 3170 running 7.3.5P5.
We also mount those datastore exports to two linux (CentOS 5.x) hosts, which have the CommVault/Simpana backup agents installed. We effectively back up a snapshot of those datastores.
We've been testing restores of files back to the datastores on the 3170 and they've been failing. We get a linux Error 13, basically "permission denied," in spite of the fact that the exports are configured for root access to both the ESX hosts and the backup hosts. We've confirmed that the backup hosts are mounting as root.
This works on the 3020 pair. Can't spot any meaningful differences in "options NFS" between the two NetApp systems, or in any other setting.
Thought it might be a posix permissions issue as a CV restore runs under the local linux group "simpana." The files are written with ownership of root:simpana and then permissions are changed when the data is all there.
But if we try to rsync any test data to the 3170 datastores we get the same error, even if we're logged in as root and copying from a local directory on the backup host. Weirder, we CAN write to the datastore folder using cp, rm, mkdir, rmdir, but NOT using rsync.
Help?
Hi Randy In your exports, are you using root=<hosts> or anon=0? I've seen cases where anon=0 didn't work 100%.
Peter
-----Original Message----- From: Randy Rue [mailto:rrue@fhcrc.org] Sent: Thursday, December 01, 2011 9:14 AM To: toasters@teaparty.net Subject: 3170 NFS weirdness
Hello All,
We run a VMWare farm that mounts its datastores via NFS on two different NetApp clusters, one a 3020 HA pair running 7.2.4 and the other a 3170 running 7.3.5P5.
We also mount those datastore exports to two linux (CentOS 5.x) hosts, which have the CommVault/Simpana backup agents installed. We effectively back up a snapshot of those datastores.
We've been testing restores of files back to the datastores on the 3170 and they've been failing. We get a linux Error 13, basically "permission denied," in spite of the fact that the exports are configured for root access to both the ESX hosts and the backup hosts. We've confirmed that the backup hosts are mounting as root.
This works on the 3020 pair. Can't spot any meaningful differences in "options NFS" between the two NetApp systems, or in any other setting.
Thought it might be a posix permissions issue as a CV restore runs under the local linux group "simpana." The files are written with ownership of root:simpana and then permissions are changed when the data is all there.
But if we try to rsync any test data to the 3170 datastores we get the same error, even if we're logged in as root and copying from a local directory on the backup host. Weirder, we CAN write to the datastore folder using cp, rm, mkdir, rmdir, but NOT using rsync.
Help?
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
A couple of things struck me off the bat:
you say, "effectively back up the snapshot of the datastore", what does effectively, and snapshot mean in this context? You also say, "Can't spot any _meaningful_ differences in "options NFS" between the two NetApp systems, or in any other setting."
Can you verify that all the nfs options on the machine that isn't working are identical to the ones that is? That's where I'd start. Secondly, I would simplify the situation, and then add complexity. Mimic what the backup software is doing by hand. By copying a file there are root and seeing if it works, and gradually adding complexity back in until it breaks.
Good luck!
On Thu, 1 Dec 2011 09:36:34 -0800, "Learmonth, Peter" Peter.Learmonth@netapp.com wrote:
Hi Randy In your exports, are you using root=<hosts> or anon=0? I've seen cases where anon=0 didn't work 100%.
Peter
-----Original Message----- From: Randy Rue [mailto:rrue@fhcrc.org] Sent: Thursday, December 01, 2011 9:14 AM To: toasters@teaparty.net Subject: 3170 NFS weirdness
Hello All,
We run a VMWare farm that mounts its datastores via NFS on two different NetApp clusters, one a 3020 HA pair running 7.2.4 and the other a 3170 running 7.3.5P5.
We also mount those datastore exports to two linux (CentOS 5.x) hosts, which have the CommVault/Simpana backup agents installed. We effectively back up a snapshot of those datastores.
We've been testing restores of files back to the datastores on the 3170 and they've been failing. We get a linux Error 13, basically "permission denied," in spite of the fact that the exports are configured for root access to both the ESX hosts and the backup hosts. We've confirmed that the backup hosts are mounting as root.
This works on the 3020 pair. Can't spot any meaningful differences in "options NFS" between the two NetApp systems, or in any other setting.
Thought it might be a posix permissions issue as a CV restore runs under the local linux group "simpana." The files are written with ownership of root:simpana and then permissions are changed when the data is all there.
But if we try to rsync any test data to the 3170 datastores we get the same error, even if we're logged in as root and copying from a local directory on the backup host. Weirder, we CAN write to the datastore folder using cp, rm, mkdir, rmdir, but NOT using rsync.
Help?
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Am 01.12.2011 18:14, schrieb Randy Rue:
Weirder, we CAN write to the datastore folder using cp, rm, mkdir, rmdir, but NOT using rsync.
Could be a time problem. Do all filers and client hosts sync against a time source (ntp?) and did you check that syncing is enabled? Is rsync -v (or more -vvvvv) reporting any errors?
-Stefan
-
Learmonth, Peter -
Nick Bernstein -
Randy Rue -
Stefan Funke