We have experienced the following problem with the --delete option when using rsync to mirror data...
When a directory on the source file system is replaced by a symbolic link with the same name, the directory is deleted from the destination file system by rsync by unlinking the directory without having recursively removed all files/directories under the parent (deleted) directory.
The net effect on our local destination file system seems to be that this leaves unreferenced files and directories that must be cleaned up by fsck at some future date. We noticed this by accident simply because we decided to take an rsync-mirrored disk offline and fsck it to check its integrity.
Our question is...What will happen to a wafl file system if rsync deletes a directory in this fashion? Does NFS deal with this properly or is it a potential issue?
Thanks in advance.
Greg Kitch UCLA SEASnet Computing Facility
The unlink will fail. How rsync then handles that case is up to rsync.
Bruce
PS - It should fail on your UNIX system as well unless rsync is running as root.
Greg Kitch writes:
We have experienced the following problem with the --delete option when using rsync to mirror data...
When a directory on the source file system is replaced by a symbolic link with the same name, the directory is deleted from the destination file system by rsync by unlinking the directory without having recursively removed all files/directories under the parent (deleted) directory.
The net effect on our local destination file system seems to be that this leaves unreferenced files and directories that must be cleaned up by fsck at some future date. We noticed this by accident simply because we decided to take an rsync-mirrored disk offline and fsck it to check its integrity.
Our question is...What will happen to a wafl file system if rsync deletes a directory in this fashion? Does NFS deal with this properly or is it a potential issue?
Apparantly this is an issue with Solaris (root can unlink() an non empty directory).
There was a recent commit message on the rsync mailing lists which specifically ranted against this behaviour (with an appropriate workaround). I believe rsync 2.4.1 has just been released which may solve this issue.
Luke.
PS: I don't think the filers are affected by it.
2000-01-30-20:02:02 Bruce Sterling Woodcock:
Apparantly this is an issue with Solaris (root can unlink() an non empty directory).
It's not Solaris per se; it's been a standard UNIX-ism for a long time.
I agree with that, the ability to unlink a non-empty directory (requiring fixes at fsck time) has been a documented feature of Unix everywhere I've seen.
I think it was deemed necessary for fsck to work properly?
I don't think it is necessary for fsck, rather fsck was necessary to repair the damage after doing it.
In all my years I've only seen one lone case where unlinking a non-empty directory was a good idea; it was the way to undo damage cause by another bug (since fixed). Once upon a time there was an NFS server implementation (in SunOS 3.x in the mid-'80s, since fixed I'm sure) which allowed an NFS client to create a file with a "/" in the middle of a filename. The NFS client that tickled this was a Macintosh gateway, a gizmo called a Cayman Systems Gatorbox, that hooked up to an Appleshare net on one side and an NFS/IP/Ethernet on the other, and did high-level protocol conversion. In that setting, Mac users could create, delete, and otherwise correctly manipulate files whose names contained slashes on a Unix server. But on the server such a file couldn't be touched. The fix was to move everything else out of the directory, unlink the directory, recreate the directory, and move everything else back. And to be on hand to hand-hold fsck through the ugly repairs next time you rebooted.
I just occurred to me, maybe unlink-nonempty-directory was required for "mv dir1 dir2", back before rename(2). When was rename(2) added anyway?
-Bennett
On Fri, 28 Jan 2000, Greg Kitch wrote:
When a directory on the source file system is replaced by a symbolic link with the same name, the directory is deleted from the destination file system by rsync by unlinking the directory without having recursively removed all files/directories under the parent (deleted) directory.
Just as a followup for others using rsync; this "problem" has been fixed in rsync 2.4.x (nee 2.3.3):
| **************************************** | Date: Monday January 24, 2000 @ 20:13 | Author: tridge | | Update of /data/cvs/rsync | In directory samba:/tmp/cvs-serv18128 | | Modified Files: | rsync.c | | Log Message: | I can't believe it - solaris allows unlink of a non-empty directory as | root, leaving a corrupt filesystem. Are those guys on drugs? | | try to avoid the problem as best we can.
| Log Message: | I can't believe it - solaris allows unlink of a non-empty directory as | root, leaving a corrupt filesystem. Are those guys on drugs?
I can't believe it - people are coding what are rapidly becoming standard UNIX utilities without any knowledge of UNIX history. Are those guys on drugs?
Bruce
On Mon, 21 Feb 2000, Bruce Sterling Woodcock wrote:
| Log Message: | I can't believe it - solaris allows unlink of a non-empty directory as | root, leaving a corrupt filesystem. Are those guys on drugs?
I can't believe it - people are coding what are rapidly becoming standard UNIX utilities without any knowledge of UNIX history. Are those guys on drugs?
Corrupt is such a strong word. Deleting non-empty directories is not such a big deal. Sure, if you do this often pretty soon you'll have a bunch of disk that is used up, but not accounted for. However, once you fsck the system well, you'll get all the lost inodes back along with their blocks. Annoying, but no biggie. :)
Tom
-
Bennett Todd -
Brian Tao -
Bruce Sterling Woodcock -
Greg Kitch -
Luke Mewburn -
tkaczma＠gryf.net