toasters August 2001

toasters@lists.teaparty.net

122 participants
161 discussions

RE: restore is very expensive
by Mike Sphar 08 Aug '01

08 Aug '01

Maybe I'm confusing some netapp settings with samba settings, but isn't it possible to configure the netapp to automatically hide files that begin with "." from CIFS clients? I don't have my docs handy at the moment, so I can't double-check that. If not, that would be a good enhancement request, perhaps even to have the netapp automatically translate a file beginning with a dot to have the DOS hidden attribute set. (In which case, the client could have the option of displaying hidden files to see them.) This would mimic the UNIX behavior in the CIFS environment. -----Original Message----- From: Stephen Manley [mailto:stephen@netapp.com] Sent: Wednesday, August 08, 2001 10:42 AM To: eyal(a)netapp.com Cc: stephen(a)netapp.com; scl(a)sasha.acc.virginia.edu; toasters(a)mathworks.com Subject: Re: restore is very expensive > Steve (not NetApp one :-), > > What do u mean by CIFS hidden attributes - is it ACL with some permissions ? > I think what you're looking for is maybe rsh to NT server and running CACLS or something... I think Steve is referring to what we have historically called a DOS bit -- the "hidden bit". If you right click a file via Windows and check out the "General" tab, you will see at the bottom Attributes: Read-only Hidden with the Standard Msft checkbox next to them. Dump/restore preserves these attributes, as well as the ACLs, NT streams, 8.3 names, etc. Stephen Manley DAM and NDMP Supreme Court Justice > -----Original Message----- > From: Stephen Manley [mailto:stephen@netapp.com] > Sent: Wednesday, August 08, 2001 09:19 > To: scl(a)sasha.acc.virginia.edu > Cc: toasters(a)mathworks.com > Subject: re: restore is very expensive > > > Hey Steve -- > > The extra dump space and restore temp file space is consumed because of the number of inodes on the dumped volume. > > Restore creates (and zeroes) a couple of files that map between each source inode and destination inode. > > To minimize time/space, you want to make your dump from a volume with a tiny "maxfiles" setting. > > One way to do this would be to: > 1) create a 2 disk volume (don't touch the default maxfiles) > 2) restore newuser.dump into that volume > 3) dump the resulting new directory to a newuser.dump.small > 4) Use newuser.dump.small for your student population scripts. > > > There will still be zeroed temp files, etc. But the size of the files will be significantly smaller, since the 2 disk volume will have fewer inodes. (remember it is available inodes, not used inodes) > > > Anyway, just one thought, > Stephen Manley > DAM and NDMP D*mned Dirty Human > > > > > > We are in the midst of our annual bulk account creation for incoming > > students. We use a perl script running on a NFS client. > > > > We store home directories on our filers and these directories are > > accessible from both NFS and CIFS. > > > > We have a template directory for new accounts that contains the > > following. You will notice that there is only one file with any data > > and it is only 711 bytes. > > > > newuser: > > drwx--s--x 4 root usr 4096 Aug 13 1999 . > > drwx------ 15 accounts staff 4096 Jul 25 15:15 .. > > -rw-r--r-- 1 root usr 0 Aug 5 1998 .forward > > -rw-r--r-- 1 root usr 0 Aug 13 1999 .pinerc > > drwx--S--- 2 root usr 4096 Aug 5 1998 private > > drwxr-sr-x 2 root usr 4096 Aug 9 2000 public_html > > > > newuser/private: > > total 16 > > drwx--S--- 2 root usr 4096 Aug 5 1998 . > > drwx--s--x 4 root usr 4096 Aug 13 1999 .. > > > > newuser/public_html: > > total 24 > > drwxr-sr-x 2 root usr 4096 Aug 9 2000 . > > drwx--s--x 4 root usr 4096 Aug 13 1999 .. > > -rw-r--r-- 1 root usr 711 Aug 9 2000 home.html > > > > We want the CIFS hidden attribute set on the .forward and .pinerc > > files so that CIFS users are less likely to see them and remove them. > > > > We have a method that works, but it is *extremely* expensive. > > > > >From CIFS we set the hidden attributes in our template directory. > > We used the netapp dump command to dump the template directory and we > > saved the output in a file: > > > > rsh filer dump 0Af - /vol/vol0/template-dir > newuser.dump > > > > Whenever we create a new account, we run the netapp restore: > > > > rsh filer restore xfD - /vol/vol0/new-user-dir < newuser.dump > > > > This nicely preserves the hidden attributes and works fine, but it > > takes somewhere between 20 and 30 seconds to run the restore on a > > F630. Furthermore, the CPU on the filer is pegged at near 100% for > > many seconds, and disk reads and writes go through the roof. > > > > We are adding over 5,000 new accounts, so this is taking a good long > > while. > > > > First of all, the dump file that we created from this nearly empty > > template is a whopping 1.6M. I have since determined that most of > > this is a bit map of the inodes in the filesystem. But even still, > > 1.6M doesn't take 30 seconds to transfer across the net to restore, so > > what else is slowing us down? I also noticed that on the filer in > > /etc/tmp that restore creates several temp files and two of them are > > rather large. One is 26M and the other is 53M. I suppose that these > > are some sort of representation of the inode table in the netapp > > filesystem that we are restoring into. This filesystem is 243G with > > 172G used, 3657389 inodes used, 5776187 free. > > > > Obviously, if we bundled up the template directory into a tar file and > > restored it with tar, it would happen in a flash, but we would lose > > the hidden attribute. > > > > We are running DOT 6.0.1R3. Can anyone think of a better way to do > > this? Perhaps there is some option to dump to create a smaller dump > > file or perhaps an option to restore that just does a simple restore > > without all the other jazz it seems to be doing. > > > > I suppose we could use tar and then write some sort of NT script (I > > only know unix) that would run through and set the hidden attributes > > after we add all the accounts. > > > > Are there any magic DOT commands that might help us get the hidden > > attribute set? > > > > > > Steve Losen scl(a)virginia.edu phone: 434-924-0640 > > > > University of Virginia ITC Unix Support > > > >

1 0

restore is very expensive
by Steve Losen 08 Aug '01

08 Aug '01

We are in the midst of our annual bulk account creation for incoming students. We use a perl script running on a NFS client. We store home directories on our filers and these directories are accessible from both NFS and CIFS. We have a template directory for new accounts that contains the following. You will notice that there is only one file with any data and it is only 711 bytes. newuser: drwx--s--x 4 root usr 4096 Aug 13 1999 . drwx------ 15 accounts staff 4096 Jul 25 15:15 .. -rw-r--r-- 1 root usr 0 Aug 5 1998 .forward -rw-r--r-- 1 root usr 0 Aug 13 1999 .pinerc drwx--S--- 2 root usr 4096 Aug 5 1998 private drwxr-sr-x 2 root usr 4096 Aug 9 2000 public_html newuser/private: total 16 drwx--S--- 2 root usr 4096 Aug 5 1998 . drwx--s--x 4 root usr 4096 Aug 13 1999 .. newuser/public_html: total 24 drwxr-sr-x 2 root usr 4096 Aug 9 2000 . drwx--s--x 4 root usr 4096 Aug 13 1999 .. -rw-r--r-- 1 root usr 711 Aug 9 2000 home.html We want the CIFS hidden attribute set on the .forward and .pinerc files so that CIFS users are less likely to see them and remove them. We have a method that works, but it is *extremely* expensive. From CIFS we set the hidden attributes in our template directory. We used the netapp dump command to dump the template directory and we saved the output in a file: rsh filer dump 0Af - /vol/vol0/template-dir > newuser.dump Whenever we create a new account, we run the netapp restore: rsh filer restore xfD - /vol/vol0/new-user-dir < newuser.dump This nicely preserves the hidden attributes and works fine, but it takes somewhere between 20 and 30 seconds to run the restore on a F630. Furthermore, the CPU on the filer is pegged at near 100% for many seconds, and disk reads and writes go through the roof. We are adding over 5,000 new accounts, so this is taking a good long while. First of all, the dump file that we created from this nearly empty template is a whopping 1.6M. I have since determined that most of this is a bit map of the inodes in the filesystem. But even still, 1.6M doesn't take 30 seconds to transfer across the net to restore, so what else is slowing us down? I also noticed that on the filer in /etc/tmp that restore creates several temp files and two of them are rather large. One is 26M and the other is 53M. I suppose that these are some sort of representation of the inode table in the netapp filesystem that we are restoring into. This filesystem is 243G with 172G used, 3657389 inodes used, 5776187 free. Obviously, if we bundled up the template directory into a tar file and restored it with tar, it would happen in a flash, but we would lose the hidden attribute. We are running DOT 6.0.1R3. Can anyone think of a better way to do this? Perhaps there is some option to dump to create a smaller dump file or perhaps an option to restore that just does a simple restore without all the other jazz it seems to be doing. I suppose we could use tar and then write some sort of NT script (I only know unix) that would run through and set the hidden attributes after we add all the accounts. Are there any magic DOT commands that might help us get the hidden attribute set? Steve Losen scl(a)virginia.edu phone: 434-924-0640 University of Virginia ITC Unix Support

3 2

RE: restore is very expensive
by Traitel, Eyal 08 Aug '01

08 Aug '01

Steve (not NetApp one :-), What do u mean by CIFS hidden attributes - is it ACL with some permissions ? I think what you're looking for is maybe rsh to NT server and running CACLS or something... ----------------------------------------------------------------- eTraitel - I'm the new eBuzzword around !!! Filer Escalation Engineer CNA, MCSE, CSA, LCA, NetApp CA Network Appliance BV Hoofddorp, The Netherlands Office: +31 23 567 9685 Cellular: +31 6 5497 2568 Get answers NOW! - NetApp On the Web - http://now.netapp.com ----------------------------------------------------------------- -----Original Message----- From: Stephen Manley [mailto:stephen@netapp.com] Sent: Wednesday, August 08, 2001 09:19 To: scl(a)sasha.acc.virginia.edu Cc: toasters(a)mathworks.com Subject: re: restore is very expensive Hey Steve -- The extra dump space and restore temp file space is consumed because of the number of inodes on the dumped volume. Restore creates (and zeroes) a couple of files that map between each source inode and destination inode. To minimize time/space, you want to make your dump from a volume with a tiny "maxfiles" setting. One way to do this would be to: 1) create a 2 disk volume (don't touch the default maxfiles) 2) restore newuser.dump into that volume 3) dump the resulting new directory to a newuser.dump.small 4) Use newuser.dump.small for your student population scripts. There will still be zeroed temp files, etc. But the size of the files will be significantly smaller, since the 2 disk volume will have fewer inodes. (remember it is available inodes, not used inodes) Anyway, just one thought, Stephen Manley DAM and NDMP D*mned Dirty Human > > We are in the midst of our annual bulk account creation for incoming > students. We use a perl script running on a NFS client. > > We store home directories on our filers and these directories are > accessible from both NFS and CIFS. > > We have a template directory for new accounts that contains the > following. You will notice that there is only one file with any data > and it is only 711 bytes. > > newuser: > drwx--s--x 4 root usr 4096 Aug 13 1999 . > drwx------ 15 accounts staff 4096 Jul 25 15:15 .. > -rw-r--r-- 1 root usr 0 Aug 5 1998 .forward > -rw-r--r-- 1 root usr 0 Aug 13 1999 .pinerc > drwx--S--- 2 root usr 4096 Aug 5 1998 private > drwxr-sr-x 2 root usr 4096 Aug 9 2000 public_html > > newuser/private: > total 16 > drwx--S--- 2 root usr 4096 Aug 5 1998 . > drwx--s--x 4 root usr 4096 Aug 13 1999 .. > > newuser/public_html: > total 24 > drwxr-sr-x 2 root usr 4096 Aug 9 2000 . > drwx--s--x 4 root usr 4096 Aug 13 1999 .. > -rw-r--r-- 1 root usr 711 Aug 9 2000 home.html > > We want the CIFS hidden attribute set on the .forward and .pinerc > files so that CIFS users are less likely to see them and remove them. > > We have a method that works, but it is *extremely* expensive. > > >From CIFS we set the hidden attributes in our template directory. > We used the netapp dump command to dump the template directory and we > saved the output in a file: > > rsh filer dump 0Af - /vol/vol0/template-dir > newuser.dump > > Whenever we create a new account, we run the netapp restore: > > rsh filer restore xfD - /vol/vol0/new-user-dir < newuser.dump > > This nicely preserves the hidden attributes and works fine, but it > takes somewhere between 20 and 30 seconds to run the restore on a > F630. Furthermore, the CPU on the filer is pegged at near 100% for > many seconds, and disk reads and writes go through the roof. > > We are adding over 5,000 new accounts, so this is taking a good long > while. > > First of all, the dump file that we created from this nearly empty > template is a whopping 1.6M. I have since determined that most of > this is a bit map of the inodes in the filesystem. But even still, > 1.6M doesn't take 30 seconds to transfer across the net to restore, so > what else is slowing us down? I also noticed that on the filer in > /etc/tmp that restore creates several temp files and two of them are > rather large. One is 26M and the other is 53M. I suppose that these > are some sort of representation of the inode table in the netapp > filesystem that we are restoring into. This filesystem is 243G with > 172G used, 3657389 inodes used, 5776187 free. > > Obviously, if we bundled up the template directory into a tar file and > restored it with tar, it would happen in a flash, but we would lose > the hidden attribute. > > We are running DOT 6.0.1R3. Can anyone think of a better way to do > this? Perhaps there is some option to dump to create a smaller dump > file or perhaps an option to restore that just does a simple restore > without all the other jazz it seems to be doing. > > I suppose we could use tar and then write some sort of NT script (I > only know unix) that would run through and set the hidden attributes > after we add all the accounts. > > Are there any magic DOT commands that might help us get the hidden > attribute set? > > > Steve Losen scl(a)virginia.edu phone: 434-924-0640 > > University of Virginia ITC Unix Support > >

2 1

Configuring Veritas Business Server with NDMP option on a HP9000/ J6000
by Schepers, Jan 08 Aug '01

08 Aug '01

Hello all, I have problems configuring 0 Veritas Netbackup Business Server 3.4GA with an 0 NDMP option running on a 0 HP9000/J6000 (HPUX 11.00) as management server to backup a 0 F850 running Ontap 6.0.1 that has a 0 HP C7145-8000 DLT8K library connected to it. When I try to do a "robot inventory", I get the error message "robot inventory failed: Unable to open robotic path (201)" At the same time the filer console shows the following messages: Wed Aug 8 18:20:09 CES [Java Thread:warning]: Ndmpd25: Top level IOException Wed Aug 8 18:20:09 CES [Java Thread:warning]: Ndmpd25: IO exception: Socket closed. What simple thing did I do wrong ? regards, YS

1 0

Re: 3rd party drives for Netapp F820 (fwd)
by don_maloy＠closures.intier.com 08 Aug '01

08 Aug '01

To the best of my knowledge, it is a drive firmware issue. I had taken several Seagate drives (ST39103FC) from various Sun machines and put them into our FC9 shelf attached to a 740. The drives work, but it does complain about them. I tried modifying the etc/qual_devices file which contained what firmware level was expected by the filer, but that didn't work either. I tried the disk_fw_update on the disks hoping that it would just put the Netapp firmware that I downloaded from the NOW site on top of the Sun firmware, but it would not perform the operation. If you figure out a way to perform this, please post it to the list. Donald J. Maloy Sr. CAD Administrator Intier Automotive - Latching Systems 19888 Haggerty Road Livonia, MI 48152 734.432.4418 734.591.6463 (Fax) Daniel Feenberg To: toasters(a)mathworks.com <feenberg@nber cc: (bcc: Don Maloy/PDD/DORTEC) .org> Subject: 3rd party drives for Netapp F820 (fwd) 08/08/01 07:34 AM Sorry if this is a FAQ, but I am new to the list and can't seem to browse http://teaparty.mathworks.com:1999/toasters. We have an F820 filer from Network Appliance with a single DS14 drive shelf. The shelf came with 14 cartriges, of which 7 had 36 gigabyte ST336704FC drives, and 7 were empty. We would like to fill the empty drive cartridges with Seagate ST373405FC (72 gigabyte) drives, but the F820 complains "Drive not supported" when we insert the drives in the slots. Is the error message because we didn't buy the drive from Netapp, didn't buy the right drives, or some firmware issue, or do we just need to update the F820 OS software? Does anyone have experience with using non-Netapp drives in an F820? We have heard that it works, but this is our first attempt. Daniel Feenberg feenberg(a)nber.org

1 0

3rd party drives for Netapp F820 (fwd)
by Daniel Feenberg 08 Aug '01

08 Aug '01

Sorry if this is a FAQ, but I am new to the list and can't seem to browse http://teaparty.mathworks.com:1999/toasters. We have an F820 filer from Network Appliance with a single DS14 drive shelf. The shelf came with 14 cartriges, of which 7 had 36 gigabyte ST336704FC drives, and 7 were empty. We would like to fill the empty drive cartridges with Seagate ST373405FC (72 gigabyte) drives, but the F820 complains "Drive not supported" when we insert the drives in the slots. Is the error message because we didn't buy the drive from Netapp, didn't buy the right drives, or some firmware issue, or do we just need to update the F820 OS software? Does anyone have experience with using non-Netapp drives in an F820? We have heard that it works, but this is our first attempt. Daniel Feenberg feenberg(a)nber.org

1 0

SSAM Hangs
by NicholosC 08 Aug '01

08 Aug '01

Hi, Has anyone experienced a hung session of the SSAM while converting a domain of 10,000 records? Any experience to share would be greatly appreciated. Regards, Nic

1 0

RE: qpopper file locks
by Sims, Aaron 07 Aug '01

07 Aug '01

You don't mention whether or not you'll have more than one HP-UX system accessing the mail spool. If it's just one, then no problem, stop reading here. If more than one system can access the same mailbox (i.e. load balancing), read on. There are two mailbox formats commonly used. One is 'mbox' the other is 'maildir'. 'mbox' format mailboxes are the most common. They use one file for multiple messages. File locking is always an issue with this format. 'maildir' format is one file per message. This format is generally headache-free in terms of file locking. My experience is that relying on lockd for file locking works 99.9...% of the time. Your milage can and will vary here. Not all NFS clients are made the same. Some UNIX vendors have better NFS and NLM (Network Lock Manager) clients than others. While I don't have direct experience with HP-UX, in small environments, lockd should work fine. (Although in a past life, I've run a F630 cluster with at least 8 Solaris based clients and 250,000 mailboxes relying on lockd with no problems.) If you run into problems, then you should convert to the maildir format. This will require that you patch or change your local delivery agent (the program that sendmail calls to actually write the email to a file, probably mail.local) and your qpopper. Aaron > -----Original Message----- > From: Todd Watson [mailto:tkw@southwestern.edu] > Sent: Tuesday, August 07, 2001 3:34 PM > To: toasters(a)mathworks.com > Subject: qpopper file locks > > > > My apologies if this question has been hashed over before on the list, > but the archives seem inaccessible at the moment. > > We are planning on migrating our mail spool files from an HP-UX server > to our 740 filer and then mounting them back via NFS. This HP system > runs sendmail and pop (qpopper specifically). We are concerned about > file-locking issues which may arise or any other "gotchas" that we > should be aware of. > > Thanks! > > Todd > -- > Todd K. Watson > Senior System & Network Administrator > Southwestern University, Georgetown, TX > tkw(a)southwestern.edu || TEL:512.863.1508 || FAX:512.863.1605 >

1 0

qpopper file locks
by Todd Watson 07 Aug '01

07 Aug '01

My apologies if this question has been hashed over before on the list, but the archives seem inaccessible at the moment. We are planning on migrating our mail spool files from an HP-UX server to our 740 filer and then mounting them back via NFS. This HP system runs sendmail and pop (qpopper specifically). We are concerned about file-locking issues which may arise or any other "gotchas" that we should be aware of. Thanks! Todd -- Todd K. Watson Senior System & Network Administrator Southwestern University, Georgetown, TX tkw(a)southwestern.edu || TEL:512.863.1508 || FAX:512.863.1605

1 0

RE: HP surestore dlt1
by Joe Luchtenberg 07 Aug '01

07 Aug '01

If you need support for LVD devices, P/N X2023B-C is an LVD SCSI backup adapter for tape. Check the Sysconfig guide for slots and supported tape drives: http://now.netapp.com/NOW/knowledge/docs/hardware/syscfg/scdot611/index.htm Joe Luchtenberg Dataline, Inc. 757-858-0600 757-285-1223 (mobile) 757-858-0606 (fax) joe.luchtenberg(a)data-line.com www.data-line.com > -----Original Message----- > From: Jim Davis [mailto:jdavis@cs.arizona.edu] > Sent: Tuesday, August 07, 2001 12:31 PM > To: > Subject: Re: HP surestore dlt1 > > > Thanks to the people who reminded me the onboard adapter > really is for HVD > devices, not LVD devices. Of course the drive I was trying was an LVD > device... >

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters August 2001