Steve (not NetApp one :-),
What do u mean by CIFS hidden attributes - is it ACL with some permissions ? I think what you're looking for is maybe rsh to NT server and running CACLS or something...
----------------------------------------------------------------- eTraitel - I'm the new eBuzzword around !!!
Filer Escalation Engineer CNA, MCSE, CSA, LCA, NetApp CA
Network Appliance BV Hoofddorp, The Netherlands Office: +31 23 567 9685 Cellular: +31 6 5497 2568
Get answers NOW! - NetApp On the Web - http://now.netapp.com -----------------------------------------------------------------
-----Original Message----- From: Stephen Manley [mailto:stephen@netapp.com] Sent: Wednesday, August 08, 2001 09:19 To: scl@sasha.acc.virginia.edu Cc: toasters@mathworks.com Subject: re: restore is very expensive
Hey Steve --
The extra dump space and restore temp file space is consumed because of the number of inodes on the dumped volume.
Restore creates (and zeroes) a couple of files that map between each source inode and destination inode.
To minimize time/space, you want to make your dump from a volume with a tiny "maxfiles" setting.
One way to do this would be to: 1) create a 2 disk volume (don't touch the default maxfiles) 2) restore newuser.dump into that volume 3) dump the resulting new directory to a newuser.dump.small 4) Use newuser.dump.small for your student population scripts.
There will still be zeroed temp files, etc. But the size of the files will be significantly smaller, since the 2 disk volume will have fewer inodes. (remember it is available inodes, not used inodes)
Anyway, just one thought, Stephen Manley DAM and NDMP D*mned Dirty Human
We are in the midst of our annual bulk account creation for incoming students. We use a perl script running on a NFS client.
We store home directories on our filers and these directories are accessible from both NFS and CIFS.
We have a template directory for new accounts that contains the following. You will notice that there is only one file with any data and it is only 711 bytes.
newuser: drwx--s--x 4 root usr 4096 Aug 13 1999 . drwx------ 15 accounts staff 4096 Jul 25 15:15 .. -rw-r--r-- 1 root usr 0 Aug 5 1998 .forward -rw-r--r-- 1 root usr 0 Aug 13 1999 .pinerc drwx--S--- 2 root usr 4096 Aug 5 1998 private drwxr-sr-x 2 root usr 4096 Aug 9 2000 public_html
newuser/private: total 16 drwx--S--- 2 root usr 4096 Aug 5 1998 . drwx--s--x 4 root usr 4096 Aug 13 1999 ..
newuser/public_html: total 24 drwxr-sr-x 2 root usr 4096 Aug 9 2000 . drwx--s--x 4 root usr 4096 Aug 13 1999 .. -rw-r--r-- 1 root usr 711 Aug 9 2000 home.html
We want the CIFS hidden attribute set on the .forward and .pinerc files so that CIFS users are less likely to see them and remove them.
We have a method that works, but it is *extremely* expensive.
From CIFS we set the hidden attributes in our template directory.
We used the netapp dump command to dump the template directory and we saved the output in a file:
rsh filer dump 0Af - /vol/vol0/template-dir > newuser.dump
Whenever we create a new account, we run the netapp restore:
rsh filer restore xfD - /vol/vol0/new-user-dir < newuser.dump
This nicely preserves the hidden attributes and works fine, but it takes somewhere between 20 and 30 seconds to run the restore on a F630. Furthermore, the CPU on the filer is pegged at near 100% for many seconds, and disk reads and writes go through the roof.
We are adding over 5,000 new accounts, so this is taking a good long while.
First of all, the dump file that we created from this nearly empty template is a whopping 1.6M. I have since determined that most of this is a bit map of the inodes in the filesystem. But even still, 1.6M doesn't take 30 seconds to transfer across the net to restore, so what else is slowing us down? I also noticed that on the filer in /etc/tmp that restore creates several temp files and two of them are rather large. One is 26M and the other is 53M. I suppose that these are some sort of representation of the inode table in the netapp filesystem that we are restoring into. This filesystem is 243G with 172G used, 3657389 inodes used, 5776187 free.
Obviously, if we bundled up the template directory into a tar file and restored it with tar, it would happen in a flash, but we would lose the hidden attribute.
We are running DOT 6.0.1R3. Can anyone think of a better way to do this? Perhaps there is some option to dump to create a smaller dump file or perhaps an option to restore that just does a simple restore without all the other jazz it seems to be doing.
I suppose we could use tar and then write some sort of NT script (I only know unix) that would run through and set the hidden attributes after we add all the accounts.
Are there any magic DOT commands that might help us get the hidden attribute set?
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support
Steve (not NetApp one :-),
What do u mean by CIFS hidden attributes - is it ACL with some permissions ? I think what you're looking for is maybe rsh to NT server and running CACLS or something...
I think Steve is referring to what we have historically called a DOS bit -- the "hidden bit".
If you right click a file via Windows and check out the "General" tab, you will see at the bottom
Attributes: Read-only Hidden
with the Standard Msft checkbox next to them.
Dump/restore preserves these attributes, as well as the ACLs, NT streams, 8.3 names, etc.
Stephen Manley DAM and NDMP Supreme Court Justice
-----Original Message----- From: Stephen Manley [mailto:stephen@netapp.com] Sent: Wednesday, August 08, 2001 09:19 To: scl@sasha.acc.virginia.edu Cc: toasters@mathworks.com Subject: re: restore is very expensive
Hey Steve --
The extra dump space and restore temp file space is consumed because of the number of inodes on the dumped volume.
Restore creates (and zeroes) a couple of files that map between each source inode and destination inode.
To minimize time/space, you want to make your dump from a volume with a tiny "maxfiles" setting.
One way to do this would be to:
- create a 2 disk volume (don't touch the default maxfiles)
- restore newuser.dump into that volume
- dump the resulting new directory to a newuser.dump.small
- Use newuser.dump.small for your student population scripts.
There will still be zeroed temp files, etc. But the size of the files will be significantly smaller, since the 2 disk volume will have fewer inodes. (remember it is available inodes, not used inodes)
Anyway, just one thought, Stephen Manley DAM and NDMP D*mned Dirty Human
We are in the midst of our annual bulk account creation for incoming students. We use a perl script running on a NFS client.
We store home directories on our filers and these directories are accessible from both NFS and CIFS.
We have a template directory for new accounts that contains the following. You will notice that there is only one file with any data and it is only 711 bytes.
newuser: drwx--s--x 4 root usr 4096 Aug 13 1999 . drwx------ 15 accounts staff 4096 Jul 25 15:15 .. -rw-r--r-- 1 root usr 0 Aug 5 1998 .forward -rw-r--r-- 1 root usr 0 Aug 13 1999 .pinerc drwx--S--- 2 root usr 4096 Aug 5 1998 private drwxr-sr-x 2 root usr 4096 Aug 9 2000 public_html
newuser/private: total 16 drwx--S--- 2 root usr 4096 Aug 5 1998 . drwx--s--x 4 root usr 4096 Aug 13 1999 ..
newuser/public_html: total 24 drwxr-sr-x 2 root usr 4096 Aug 9 2000 . drwx--s--x 4 root usr 4096 Aug 13 1999 .. -rw-r--r-- 1 root usr 711 Aug 9 2000 home.html
We want the CIFS hidden attribute set on the .forward and .pinerc files so that CIFS users are less likely to see them and remove them.
We have a method that works, but it is *extremely* expensive.
From CIFS we set the hidden attributes in our template directory.
We used the netapp dump command to dump the template directory and we saved the output in a file:
rsh filer dump 0Af - /vol/vol0/template-dir > newuser.dump
Whenever we create a new account, we run the netapp restore:
rsh filer restore xfD - /vol/vol0/new-user-dir < newuser.dump
This nicely preserves the hidden attributes and works fine, but it takes somewhere between 20 and 30 seconds to run the restore on a F630. Furthermore, the CPU on the filer is pegged at near 100% for many seconds, and disk reads and writes go through the roof.
We are adding over 5,000 new accounts, so this is taking a good long while.
First of all, the dump file that we created from this nearly empty template is a whopping 1.6M. I have since determined that most of this is a bit map of the inodes in the filesystem. But even still, 1.6M doesn't take 30 seconds to transfer across the net to restore, so what else is slowing us down? I also noticed that on the filer in /etc/tmp that restore creates several temp files and two of them are rather large. One is 26M and the other is 53M. I suppose that these are some sort of representation of the inode table in the netapp filesystem that we are restoring into. This filesystem is 243G with 172G used, 3657389 inodes used, 5776187 free.
Obviously, if we bundled up the template directory into a tar file and restored it with tar, it would happen in a flash, but we would lose the hidden attribute.
We are running DOT 6.0.1R3. Can anyone think of a better way to do this? Perhaps there is some option to dump to create a smaller dump file or perhaps an option to restore that just does a simple restore without all the other jazz it seems to be doing.
I suppose we could use tar and then write some sort of NT script (I only know unix) that would run through and set the hidden attributes after we add all the accounts.
Are there any magic DOT commands that might help us get the hidden attribute set?
Steve Losen scl@virginia.edu phone: 434-924-0640
University of Virginia ITC Unix Support
-
Stephen Manley -
Traitel, Eyal