toasters

toasters@lists.teaparty.net

13515 discussions

pegged filer....write bound?
by Todd C. Merrill 20 Jul '01

20 Jul '01

Scenario: clustered F840's, ONTAP 5.3.7R2 one is a busy filer of mostly home directories other is fine, no problems, much less loaded middle of the day, response takes a nosedive on the busy one CPU is pegged at 100% very little NFS, CIFS, or network traffic no backups or restores going on no snapshots in progress (that we can tell) As a user, response is *extremely* slow; sometimes a stat of a known populated directory returns empty. Effectively the filer is not serving data. (Worse, in my opinion, that it returns the *wrong* data.) We turn off NFS to see if that is the culprit. Still CPU is pegged. We terminate CIFS to see if that is. CPU drops down, but not to zero. We have only NFS, CIFS, and cluster licensed (i.e., no HTTP). Here's what we saw: home> sysstat 2 CPU NFS CIFS HTTP Net kB/s Disk kB/s Tape kB/s Cache in out read write read write age 10% 0 0 0 10 6 12074 0 0 0 24 47% 0 0 0 8 5 6369 7 0 0 24 47% 0 0 0 13 7 16479 20258 0 0 24 33% 0 0 0 4 3 11858 13180 0 0 24 34% 0 0 0 8 4 12698 14703 0 0 24 9% 0 0 0 9 5 11392 8 0 0 24 9% 0 0 0 7 4 11097 0 0 0 24 58% 0 0 0 9 3 6453 2415 0 0 24 39% 0 0 0 7 2 15218 17034 0 0 24 39% 0 0 0 5 2 13560 16924 0 0 24 31% 0 0 0 9 5 11633 11593 0 0 24 8% 0 0 0 8 4 10634 8 0 0 24 10% 0 0 0 9 5 12992 0 0 0 23 62% 0 0 0 8 3 9828 8030 0 0 23 44% 0 0 0 9 4 17156 19024 0 0 23 37% 0 0 0 6 3 15229 17994 0 0 23 9% 0 0 0 6 2 12204 8 0 0 23 10% 0 0 0 11 5 13574 8 0 0 23 66% 0 0 0 5 3 9354 11421 0 0 23 Pardon my French, but WTF is this filer doing? It looks and smells like snapshot behaviour, but we weren't even near to the time it should be doing a snapshot via the schedule. No external scripts would initiate one, either. Our solution was, unfortunately, a reboot. The second time this happened, we grabbed some output from `wafl_susp` to check on the consistency points, since we are suspecting this poor filer is write-bound (insufficient NVRAM cache--half is "lost" to the partner for clustering). The counts of all the cp_* parameters show *less* than the minimum number of consistency points expected (uptime times 6 per minute, i.e., a minimum of once every 10 seconds). And, of course, lots of cp_from_log_full and cp_from_cp. Anybody seen anything like this? Any idea what is going on? Are we just beating the crap out of this thing, and it gives up the ghost by pretending it is busy to avoid doing anything else? (We've already got another clustered pair of F840's in house, in testing, soon to be deployed. Not soon enough. Figures.) Until next time... The Mathworks, Inc. 508-647-7000 x7792 3 Apple Hill Drive, Natick, MA 01760-2098 508-647-7001 FAX tmerrill(a)mathworks.com http://www.mathworks.com ---

3 4

Re: general filer gripe
by Jamey Maze 20 Jul '01

20 Jul '01

Roger, Assuming you can get to the filer's console, you could list the rc file using "rdfile /etc/rc" and then re-create it using "wrfile /etc/rc". Be warned, however, wrfile completely wipes out the existing file and you will need to re-enter every line. It's as if you did "cat > /etc/rc". But if you are using a terminal where you can do cut & paste, paste the original text up to the point where you need to make the change, make the change, and then paste the rest of the file. Remember to hit Control-C on a blank line to exit wrfile. wrfile is very basic, but effective in situations like the one you described. At 10:19 AM 7/20/2001 -0400, Leonard, Roger wrote: >This is probably covered in NetApp 101 but never having taken the classes >(yet) I need something clarified. > >Environment: Sun solaris with automounter and different flavors of filers >running ATM and LANE. > >i mucked up the networking portion of the rc file and the filer was >essentially hung trying to export filesystems. it could not resolve the >addresses to these hosts because it could not get dns or nis or any name >resolution or even a default route for that matter. It hung while >complaining for over an hour. so i could not get in through my normal "cd >/net/filername/etc" my final recourse was to create boot floppies, boot >from them, selecting "boot without rc" bringing up networking with >ifconfigs and elconfigs, then going in through "cd /net/filername/etc" and >editing the rc file accordingly and rebooting. very slow and tedious. > >My basic question is: was this the proper way to fix a screwed up rc? > >boot floppy >manually config network >cd /net/filername/etc >edit rc > >Or is there a way to edit the rc file from a terminal session? or any other >of the /etc files? > >what is the quickest way to recover from this type of problem?? > >can i boot the filer kernel in single user mode without networking? > >What is the best way to get around this problem in the future? Can i make >the exports fail and just come up to a prompt or do i have to boot floppy >everytime? > >if i hardcoded my export hosts in the /etc/hosts would the filer come up >even without networking and NIS/DNS, etc? > >thanks > >Roger D. Leonard -- James N. (Jamey) Maze Network Appliance SE (615) 496-4799

1 0

RE: general filer gripe
by Mike Sphar 20 Jul '01

20 Jul '01

I've had similar situations where networking info was messed up after an ontap upgrade, but it only hung for a few minutes before I got a login prompt on the console, at which point I just logged in and re-ran "setup" which prompts for all the networking info. Then after the networking was fixed I restored some of my original settings (like the exports file) from snapshots. -----Original Message----- From: Leonard, Roger [mailto:Roger.Leonard@marconi.com] Sent: Friday, July 20, 2001 7:19 AM To: 'toasters(a)mathworks.com' Subject: general filer gripe This is probably covered in NetApp 101 but never having taken the classes (yet) I need something clarified. Environment: Sun solaris with automounter and different flavors of filers running ATM and LANE. i mucked up the networking portion of the rc file and the filer was essentially hung trying to export filesystems. it could not resolve the addresses to these hosts because it could not get dns or nis or any name resolution or even a default route for that matter. It hung while complaining for over an hour. so i could not get in through my normal "cd /net/filername/etc" my final recourse was to create boot floppies, boot from them, selecting "boot without rc" bringing up networking with ifconfigs and elconfigs, then going in through "cd /net/filername/etc" and editing the rc file accordingly and rebooting. very slow and tedious. My basic question is: was this the proper way to fix a screwed up rc? boot floppy manually config network cd /net/filername/etc edit rc Or is there a way to edit the rc file from a terminal session? or any other of the /etc files? what is the quickest way to recover from this type of problem?? can i boot the filer kernel in single user mode without networking? What is the best way to get around this problem in the future? Can i make the exports fail and just come up to a prompt or do i have to boot floppy everytime? if i hardcoded my export hosts in the /etc/hosts would the filer come up even without networking and NIS/DNS, etc? thanks Roger D. Leonard

1 0

RE: Corruption in data
by Muhlestein, Mark 19 Jul '01

19 Jul '01

This is bug 8075. It's fixed in 6.0 and later releases. Mark -----Original Message----- From: Clawson, Simon [mailto:simon_clawson@mentorg.com] Sent: Thursday, July 19, 2001 3:41 AM To: 'toasters(a)mathworks.com' Subject: Corruption in data Hi, I have a user who is experiencing data corruption. He opens a file in textpad, saves it then uses the file in an application and finds it has null characters in it. I checked through and found he had run out of quota. I increased this and all appears to be well. My question is - can low or no disk quota cause data corruption? Simon Simon Clawson HDS Team Systems Administrator Mentor Graphics Uk Rivergate London Road Newbury Berkshire RG14 2QB <<Clawson, Simon.vcf>>

1 0

Novell on NetApp, is it workable longterm?
by Robin Croley 19 Jul '01

19 Jul '01

Does anyone have any experience integrating Netware 5.1, NDS 7 and Groupwise 5.2 with a Filer? I sell NetApps in Dallas and need war stories, feedback, testimonials, and any other info for a potential client that has roughly 20 Novell servers, 36 NT boxes, and 9 or 10 Unix machines. They have no plans to retire Novell anytime soon, so need a long-term workable solution instead of a planned migration scenario. Thanks, Robin Croley Axis Technologies Ph: 972.588.1875 Fax: 972.588.1873 www.axistec.com <http://www.axistec.com/> robin(a)axistec.com

1 0

72GB BCS disk as hot spare
by Moshe Linzer 19 Jul '01

19 Jul '01

Assume I have an F760 filer running 6.x with disk shelves as follows: 2xFC8 (18GB disks), 4xFC9 (36GB disks), and 1xDS14 (72GB disks). The 18 and 36GB disks (2 volumes) use Zone checksums, while the 72GB disks (1 volume) use Block checksums. My question: can I keep a single 72GB BCS disk as hot spare for the whole filer? If a failure occurs on a smaller disk, the filer will reconstruct on the 72GB spare. Then I can replace the broken disk with one the correct size, and fail the rebuilt 72GB disk over to the new disk. The 72GB disk is then reinserted as hot spare. The only question is: Will it work? NOW says that a BCS disk can be a hot spare for a ZCS volume, but they don't relate to DS14 shelves. And any unforseen gotchas. Thanks, Moshe -- ----------------------------------------------------------------------------- Moshe Linzer | On the Internet, IT Manager | National Semiconductor, Israel | nobody knows you're a moron. Phone: 972-9-970-2247 | Fax: 972-9-970-2001 | - Network Magazine Email: moshel(a)nsc.com | -----------------------------------------------------------------------------

1 0

Corruption in data
by Clawson, Simon 19 Jul '01

19 Jul '01

Hi, I have a user who is experiencing data corruption. He opens a file in textpad, saves it then uses the file in an application and finds it has null characters in it. I checked through and found he had run out of quota. I increased this and all appears to be well. My question is - can low or no disk quota cause data corruption? Simon Simon Clawson HDS Team Systems Administrator Mentor Graphics Uk Rivergate London Road Newbury Berkshire RG14 2QB <<Clawson, Simon.vcf>>

1 0

vol2 appears before vol0
by Moshe Linzer 19 Jul '01

19 Jul '01

I recently installed a new filer, and physically moved one of my volumes (vol2) to it. Since this volume was composed of FC8 shelves, while the root volume (vol0) on the new filer had FC9 shelves, the technician installed the FC8 shelves first on the loop, then the FC9's. The result is, whatever command I issue (df, sysconfig -r), it displays vol2 first, and then vol0. Frankly, this drives me crazy, and I am forever rechecking which volume I am looking at. Is there a reason for this, and any way to fix it? Thanks, Moshe -- ----------------------------------------------------------------------------- Moshe Linzer | On the Internet, IT Manager | National Semiconductor, Israel | nobody knows you're a moron. Phone: 972-9-970-2247 | Fax: 972-9-970-2001 | - Network Magazine Email: moshel(a)nsc.com | -----------------------------------------------------------------------------

2 1

UNSCRIBE remove
by Puneet Gulati 18 Jul '01

18 Jul '01

UNSCRIBE remove

1 0

Re: Consecutive count
by Sam Cramer 18 Jul '01

18 Jul '01

> However, if I do a 'vif status', I get a very high number for "consecutive": > > toaster> vif status > default: transmit 'multi', fail 'log' > redundo: 2 links, transmit 'single', fail 'default' > up: > e8: state up, since 18Jul2001 12:37:34 (01:00:17) > mediatype: auto-1000sx-fd-up > flags: enabled next-link favored address-set-1 > input packets 11626, input bytes 729869 > output packets 528, output bytes 83621 > up indications 1, broken indications 1 > drops (if) 0, drops (link) 0 > indication: up at 18Jul2001 12:36:41 > consecutive 0, transitions 2 > down: > e0: state down, since 18Jul2001 12:37:34 (01:00:17) > mediatype: auto-100tx-fd-up > flags: disabled > input packets 286, input bytes 25148 > output packets 136, output bytes 17952 > up indications 3669, broken indications 0 > drops (if) 0, drops (link) 0 > indication: up at 18Jul2001 13:37:51 > consecutive 3669, transitions 0 > The "up indications" seems to be the cause of it. Can anyone tell me > what that is from? It looks like there is about 1 every second or so. > Is this a normal thing? I searched on NOW and could only find one hit > for "consecutive" that both had to do with virtual interfaces and wasn't > in the documentation. The vif man page (na_vif.1) explains Links constantly issue either up or broken indications based on their interaction with the switch. The consecu tive count indicates the number of consecutively received indications with the same value (in this example, up). The transitions count indicates how many times the indica tion has gone from up to down or from down to up. So, you don't need to worry about this. Sam

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters