Stephen,

Thank you for your reply; and thanks also to others from NetApp who sent me information about these files but were too shy to cc toasters ...

...

We upgraded one of our F740's from 5.3.7R1 to 6.0.1R3 a week ago. It went very smoothly, but there are a few things still puzzling me. Here is one of them...

Every time a dump command is run, at completion a new file appears in the filer's /etc, with a name like

/etc/oldmaps/bkp/A_home1/3abbe75d | | | \ This is the time(2) of the dump in hex | \ This is the "n" key from the dump command

Peeking at /etc/tmp while dump is running, and comparing inode numbers, it seems that these files are what was /etc/tmp/dump_all.bitmap.[n]. (There are several other files in /etc/tmp, but this seems to be the only one that is saved.) Its size appears to be 1 bit per inode in the volume being dumped, plus a little.

Are these files going to accumulate for ever? None have disappeared as yet, but maybe they will in due course... And what is their purpose in life?

Hey Chris -- Thses files do describe the state of the all the volume's inodes with respect to the tree you're dumping (allocated in this tree or not allocated in this tree). So we need one per tree you dump.

They accumulate -- one for each non-level 0 dump that you do. A level 0 dump will clear them out and leave just the newest one. If you don't use the 'u' option, we don't store the file (since you have declared that you won't be doing an incremental dump based on this dump)

We had taken level 0 dumps shortly before upgrading to 6.0.1R3 [of course, we trust NetApp implicitly, but ... :-) ] so that for some time after we were doing only incemental dumps. I can confirm now that the old files do go away when a level 0 dump is taken.

But it seems to me that many of them could be cleared out earlier. If I understand this correctly, the only ones needed are those that correspond to current entries in /etc/dumpdates. We use a fairly aggressively incremental dump scheme, roughly

0 3 6 9 9 6 9 9 3 6 9 9 6 9 9 0 ...

for each tree dumped. That leaves 15 of these /etc/oldmaps files to be removed when the next level 0 is taken, of which only 3 could ever be useful. [In fact, one could argue that those corresponding to level 9 dumps are not worth keeping, even if "u" is specified, couldn't one? One can't take an incremental dump relative to them because one can't use levels greater than 9!]

When we base an incremental dump off this base, the bitmaps enable us to find "new files" and "deleted" files better. Now obviously, the file doesn't get you ALL of that information, since an inode can be de-allocated and re-allocated and the map wouldn't know the difference. So we still retain our old code paths as well.

But, basically, we're consuming a little bit of disk space for some performance gain.

As I understand it, this copes with cases like the following:

Suppose one has a quota tree /home, divided into two for dumping purpose, with directories such as

/home/part1/alpha /home/part1/bravo /home/part1/charlie /home/part2/delta /home/part2/echo /home/part2/foxtrot

Sometimes users are moved from one part to another to rebalance the size of the dumps. Now suppose:

1. Level N dumps of /home/part1 and /home/part2 have been taken. 2. /home/part1/juliet is rename(2)'d as /home/part2/juliet. 3. Level N+1 dumps are now taken.

Pre-6.0, the level N+1 dump of /home/part2 would know to dump /home/part2/juliet itself, because its mtime & ctime have changed, but would not know to dump its contents. Hence a subsequent restore using this dump would cause all juliet's files to disappear - except the ones she had modified between (1) & (3).

It's problems like this which cause Solaris ufsdump (and other BSD dump derivatives) to only allow incremental dumps for whole filing systems, not subtrees. [The logical equivalent for NetApp would be qtrees: a boundary across which you can't move files or directories without altering *all* their ctime's.]

The above scenario is actually not unlike what we do with our /home quota tree, and cases like juliet's have been a lurking worry at the back of my mind! Now to test whether the 6.0 solution using the oldmaps actually works... who shall I pick on as the victim to lose all their files if it doesn't? :-)

Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.