toasters

toasters@lists.teaparty.net

13520 discussions

Re: wafl unicode options and conversion times
by Todd C. Merrill 17 Feb '00

17 Feb '00

Bruce Sterling Woodcock wrote: > From: Joan Pearson <jpearson(a)netapp.com> > > Yes, while a directory is being converted, all access to it should be > > locked out. Perhaps the nfs client was using information it had > > already cached before the conversion. > > Yeah, but doesn't the cache check something on the directory before > it makes assumptions? Access or modify time? > > Maybe the filer isn't updating that at the start of the conversion > process. I dunno. There's gotta be a way to defeat the client > cacheing other than mounting "noac". This was all "first contact" stuff, from both a freshly rebooted filer and client. No caching as far as I understand it was involved. In our debugging, we tried the "noac" option; it didn't work (for obvious reasons looking back on it now). The first contact initiates the conversion; unfortunately for us, it was the dirents() from the `rm -rf` returning an incomplete directory listing. Joan, the call was 83902. On Wed, 16 Feb 2000, Hal Siegel wrote: > This conversion took close to a day and a half on a 630 full of > hundreds of thousands of files of gzipped archive data. Interesting. I interpreted the docs as saying a single directory with 100,000 files took 9 minutes. I never expected a directory with 20-30 files to be converted in anything more than a negligible time. My expectations were just out of line with the behaviour of the filer. NetApp folks, perhaps this could be expounded upon in future docs with more warnings and other examples. > Note that you > can monitor the conversion taking place by watching sysstat - there's a > steady 3-400 disk reads per second going on while the conversion is > being done. Looking at our MRTG data a few days later, I figured that out. Doh! Eyal Traitel wrote: > Can one of you explain the need for this anyway ? It's really only a concern in a mixed CIFS/NFS environment, which we are heavily. The docs can explain the wafl ucode options much better than I can. But, one problem with not having the unicode information in the directory structure is this, which we ran into all the time: o create a directory via an NFS client o do not access this directory from a CIFS client o create a snapshot o try to access this directory via a CIFS client via the snapshot You will get "permission denied." The CIFS access tries to create the unicode information in the directory upon creation or access, but, since it's in a read-only snapshot, it can't. The wafl ucode options allow you to create directories with the unicode information already there, even if created via an NFS client, or to convert the directories upon any access, NFS or CIFS. It is the latter case that takes a non-trivial amount of time, enough to impact normal behavior. Until next time... The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill(a)mathworks.com http://www.mathworks.com ---

1 0

wafl unicode options and conversion times
by Todd C. Merrill 16 Feb '00

16 Feb '00

Just thought I'd share a caveat that we ran into in the recent past in upgrading from 5.2.x to 5.3.x. It took two separate cases and 50+ emails and phone calls to figure out, so hopefully this info will save somebody else quite a bit of time. (Many thanks to Mike Smith in tech support for his patience and perseverance in resolving it.) 5.3 (?) introduced two new options, wafl.convert_ucode and wafl.create_ucode, to convert directories upon access and create directories from scratch with unicode bits, respectively, even if accessed/created via an NFS client. We updated our rc files with these options and proceeded with business as usual after the upgrade. Hours later, our entire automated testing environment came to a screeching halt with "permission denied" and other bizarre errors. For instance, an `rm -rf` of a directory would fail, leaving files around, when all the file and directory permissions would have allowed the removal: % rm -rf build.removeme rm: Unable to remove directory build.removeme/test/toolbox/map/map: File exists rm: Unable to remove directory build.removeme/test/toolbox/map: File exists ... Cut to the chase: after getting packet snoops and trying various things on different filers, and the usual troubleshooting, NetApp was able to reproduce the "problem." It wasn't a problem. The docs state, regarding unicode conversion of directories: "For example, converting a directory of 100,000 files takes the filer about 9 minutes on a netapp f540." I read this, and said to myself, "Self, we don't have any directories that big," so I dismissed as trivial the time it would take to convert our directories. We had maximum 2,000 files in our largest directories. Well, even with a semi-honkin' F740, two generations distant from the aforementioned F540, we were seeing `rm -rf` failures on small directories, with only tens of files. And, the clients doing the `rm -rf` weren't all that speedy, a 200 MHz Sun Ultra2, a 270 MHz Ultra5, and even a SPARC20, if I recall correctly (among other types of UNIX boxes, like 200 MHz Pentium Linux boxes). Turns out, this unicode conversion takes a non-trivial amount of time for *any* directory. System call traces showed directory entry calls returning not all files in that directory. Apparently, during the conversion, the "new" directory list can be incomplete. Hence, an `rm -rf` first gets an (incomplete) directory listing, then removes all those files, does a `cd ..` and then tries an rmdir(), at which point, since there are still files in the directory, it spits out an error like the above. Once we figured this out, we tried to forcibly convert all the directories by doing a `find . -print > /dev/null` to traverse the entire filer directory structure. Well, this apparently didn't work (my notes get sparse here) in all cases. Our final workaround was to go to a CIFS client (an NT box), right-click on each top level folder on the filer and select "properties," and let it finish calculating the folder size, thus doing the unicode conversion as usual (the way it had to be done before the wafl options in 5.3.x). So much for the interpretation of "wafl.convert_ucode" working via NFS access to the directory. At the conclusion of the call, I requested the docs to be expounded upon and made clearer regarding unicode conversions for future releases. To date, we have not had any further problems. The "access denied" messages when trying to access non-unicode directories (those created via an NFS client pre-5.3 or with the wafl.create_ucode option unset) from read-only snapshots via CIFS clients has been eliminated. Yea! Hope this helps others. Until next time... The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill(a)mathworks.com http://www.mathworks.com ---

4 3

New Gigabit Ethernet cards in cluster?
by Michael Rogan 16 Feb '00

16 Feb '00

Hello, Has anyone tried to use the new Gigabit Ethernet II (Intel) cards using a single vif in a cluster? I have tried these cards individually and failed the cluster over and everything works just fine but if I put them in a single/multi vif and fail the cluster, I can ping the machine, I can telnet to it and it states that it has failed over, I can http to it; however when I try NFS connections to it, it does not respond? Any suggestions? Config file (Partner rc is nearly identical): ## # RC, file san1 -- MJR # Network 172.17.1.0 to be changed after testing -- MJR # Modified for testing -- MJR ## hostname san1 options dns.domainname blackberry.net options dns.enable on options nis.enable off # Setup Virtual Interfaces ifconfig e6 flowcontrol none ifconfig e7 flowcontrol none vif create single sanvif e6 e7 vif favor e6 # configure interfaces ifconfig e0 192.168.200.110 netmask 255.255.255.0 mediatype 100tx-fd partner 192.168.200.111 ifconfig sanvif 172.17.1.110 netmask 255.255.255.0 trusted partner sanvif # Test, works if vif not configured - MJR #ifconfig e6 172.17.1.110 netmask 255.255.255.0 partner e6 route add default 192.168.200.1 1 savecore exportfs -a nfs on vol snapmirror on ## Michael Michael J. Rogan BlackBerry Sys. Adm. Research In Motion 519-888-7465 www.blackberry.net www.rim.net

1 0

Please add me to your list
by WAY,PETER (HP-Loveland,ex1) 16 Feb '00

16 Feb '00

Peter Way

1 0

RE: Which spare 9 or 18 or 36
by Barrett,Eric 16 Feb '00

16 Feb '00

Yup. When a disk is removed from a RAID group, for whatever reason, the filer will use the smallest possible hot spare disk that it can. (It cannot, of course, reconstruct onto a disk that was smaller than the original.) If a hot spare of the same size isn't available, it will pick the next largest one. -- Eric Barrett, TSE, Network Appliance ebarrett(a)netapp.com / 408-822-4779 (office) NetApp Tech Support: 1-888-4NETAPP > -----Original Message----- > From: Jim Harm [mailto:harm1@llnl.gov] > Sent: Tuesday, February 15, 2000 3:24 PM > To: toasters(a)mathworks.com > Subject: Which spare 9 or 18 or 36 > > > Dear super informative list; > > Let's say there is a volume of 9 GB drives in a 760 > and another volume of 18 GB(or 36 GB). > Now I have a 9 GB spare and an 18 GB spare in the box also. > When a drive dies, does the box pick the spare of the same size > as the failed disk to rebuild? > > }}}===============>> LLNL > James E. Harm (Jim); jharm(a)llnl.gov > (925) 422-4018 Page: 423-7705x57152

1 0

RE: Backing up a filer
by Kelly Wyatt 16 Feb '00

16 Feb '00

Hmmm... I just saw in the technote that Veritas NetBackup does not support that (a little table in that same technote indicates that)... I will be checking with Veritas on that because they know we plan to test this and have given no indication that this would not work. Kelly -- Kelly Wyatt, Kelly.Wyatt(a)SAS.com Systems Programmer Integrated Solutions Consulting SAS Institute Inc. / SAS Campus Drive / Cary, NC 27513 http://www.sas.com -----Original Message----- From: [ISS] Veritas-Users Sent: Wednesday, February 09, 2000 9:54 AM To: 'Roth, William'; 'gwong(a)futureshop.com' Cc: 'toasters(a)mathworks.com' Subject: RE: Backing up a filer I understand that you can do NDMP dump across the network to a media server... it is even mentioned in technote 3066, section 4.3 (# 4 below): ---- The NDMP protocol specification allows the following backup configurations: 1. Local backup from a filer to a SCSI-attached tape device 2. Three-way backup from a filer via the network to another filer with a local tape device 3. Backup from a UNIX or Windows NT server via the network to a NetApp filer with a local tape device 4. Backup from a NetApp filer via the network to a UNIX or Windows NT backup server with a local tape device ---- We currently backup filers via dedicated directly attached tape drives (using Veritas NetBackup)... but, this is a costly solution because the tape drives are idle probably 75% of the time. We will be testing NDMP across a dedicated GB ethernet network in the near future. Even if we see some decrease in backup throughput, it will good if we can get those 20 dedicated DLT7000 tape drives in S'Tek 9710's doing more work for us... Is anyone already using a private network and NDMP to backup a filer? Kelly -- Kelly Wyatt, Kelly.Wyatt(a)SAS.com Systems Programmer Integrated Solutions Consulting SAS Institute Inc. / SAS Campus Drive / Cary, NC 27513 http://www.sas.com -----Original Message----- From: Roth, William [mailto:William.Roth@netapp.com] Sent: Friday, January 07, 2000 11:21 AM To: 'gwong(a)futureshop.com' Cc: 'toasters(a)mathworks.com' Subject: RE: Backing up a filer Greg, If you want to use NetBackup with NDMP to back up the filer, you need to have at least one direct attached tape drive connected to the filer(not considering a three way NDMP backup using two filers with a tape device direct connected to the remote filer). You can dedicate one or more drives from an existing library controlled by NetBackup, or you can attach a new/different library to the filer(yes, the filer does support a locally attached library). Either the NetBackup host (NDMP client) or the filer can be direct connected to the robotic portion of the library(assuming the library has individual scsi ports for the connection). In the end, NetBackup ends up controlling the library either way. If the robot is direct connected to the NetBackup host via scsi, NetBackup controls it. If the robot is connected to the filer via scsi, the NetBackup host sends scsi cdb's via NDMP to the robot through the filer, thus the robot is still controller by NetBackup. What other information or assistance do you require? Bill Roth Technical Marketing Engineer Network Appliance, Inc. 495 East Java Drive Sunnyvale, CA 94089 Telephone 408-822-3308 -----Original Message----- From: Greg Wong [mailto:gwong@futureshop.com] Sent: Thursday, January 06, 2000 11:45 AM To: toasters(a)mathworks.com Subject: Backing up a filer Hi all, I'm kind of new to the this filer business and I have a question about backing it up. We have an F720 on site that isn't doing much since we aren't backing it up. I need some options on how to do this. We currently back up our other systems vith Veritas Netbackup and a tape library. Now, my question(s) is if I want to use Veritas to back up the filer using NDMP, do I have to dedicate a drive on the remote tape library to the filer? Will the filer support a locally attached library, and if it does, does Veritas control the library or does the filer? Thanks for your help Greg

6 7

RE: max file system size
by Jason Middlebrooks 16 Feb '00

16 Feb '00

So on a F760 I can have 56 32GB drives on one fibre loop? Is any performance lost here? Thanks Jason "Bond, Andrew" <abond(a)netapp.com> on 01/28/2000 09:48:33 AM To: Jason Middlebrooks/Datalink cc: toasters(a)mathworks.com Subject: RE: max file system size This question should really be addressed to your local NetApp sales person or SE, however: > Is the maximum storage per fibre loop still 464.4GB? No. The maximum storage per F760 is presently approx. 1.4TB raw disk space. This can be on one loop or more than one loop, in whatever config you require, subject to a maximum of 56 devices per loop. Regards, Andrew

4 3

Which spare 9 or 18 or 36
by Jim Harm 15 Feb '00

15 Feb '00

Dear super informative list; Let's say there is a volume of 9 GB drives in a 760 and another volume of 18 GB(or 36 GB). Now I have a 9 GB spare and an 18 GB spare in the box also. When a drive dies, does the box pick the spare of the same size as the failed disk to rebuild? }}}===============>> LLNL James E. Harm (Jim); jharm(a)llnl.gov (925) 422-4018 Page: 423-7705x57152

1 0

Re: Backing up a filer
by Steve Kappel 15 Feb '00

15 Feb '00

mark wrote: >> NetBackup NDMP does NOT use multiplexing. Also note that for NDMP >> the tape format is determined by the NDMP server vendor. > >NDMP uses dump format as well doesn't it? So all the propositions above >get a little messed around here anyhoo. No, NDMP is format neutral. NetApp's implementation of NDMP uses dump. tar is used by some others. __________________________________________________________________________ Steve Kappel steve.kappel(a)veritas.com VERITAS Software steve.kappel(a)iname.com (Personal)

1 0

Re: Backing up a filer
by Steve Kappel 15 Feb '00

15 Feb '00

Eyal, >Are there any plans / ideas from Veritas on improving backup/restore times of >filers ? The only time NetBackup has a performance impact on backup is when there are a very large number of files in the backup (on the order of millions). Currently we get all the NDMP history in DIR & NODE messages which we then have to convert into full paths for our catalog. This can be CPU, RAM, and disk intensive. We are investigating options for how we might address this issue. As for restore time there isn't much we can do besides add support for 'direct access restore'. Sounds easy but it isn't. Even when NetApp supports it, it may not be what you want - it will have limitations. It is a hard thing to do with a dump-style backup. Because of the limitations and the NetBackup model, we may not be able to support it reliably. We always take the conservative approach of doing everything we can to make sure you get the correct data and any new optimizations have to be bullet-proof. In the end, all we can do is support what NDMP and each NDMP server implementation provides to us. >It seems that the tape/backup s/w is not keeping up well with the filers' huge >sizes ? I don't think it is just filers - the whole industry is seeing an explosion of data. Tape drives certainly haven't kept up with disk. Of course NetBackup has other features that are not available in the NDMP world simply because 1) we have raw access to the disk, and 2) we control the tape format. >What is the best configuration that can give us best performance ? From a NetBackup perspective the only thing you can do is try to keep the number of files per backup from getting very large. Trouble is that breaking a volume up into different backups has an impact on filer efficiency (unless you use quota trees). __________________________________________________________________________ Steve Kappel steve.kappel(a)veritas.com VERITAS Software steve.kappel(a)iname.com (Personal)

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters