toasters

toasters@lists.teaparty.net

13514 discussions

Re: Possible software cause for total data loss?
by sirbruce＠ix.netcom.com 07 Nov '97

07 Nov '97

On 11/04/97 10:48:56 you wrote: >> During the pre-production testing of one of our F230 filers, we >discovered a problem with one of them that we wereonly able to fix by >rebuilding the RAID (and thus losing whatever OS and data was on the >filer). Be careful with precise words. "Rebuilding the RAID" is something that happens when a disk fails, and it doesn't cause you to lose your data. If you mean, say, re-initialize the filesystem, then that makes more sense. > Part of the tests consisted of filling up the filesystem via NFS >and NDMP copies from a host Ultrasparc. Three other F230's of >identical configuration survived the tests, but the remaining F230 >experienced the following panic four times: > >PANIC: ../common/wafl/nvlog.c: 1088: Assertion failure > > > I will be running the NVRAM diagnostics later today to see if they >turn up anything. However, more distressing is the behaviour of the >Netapp upon reboot: You are correct that it's probably a software bug, although running the NVRAM diagnostics (and perhaps re-seating the card) is certainly something you should try. >[... other boot messages deleted...] >Loading filesystem. >Recomputing parity in NVRAM > >PANIC: ../driver/disk/disk.c:2633: Assertion failure. > >version: NetApp Release 4.2a: Fri Sep 5 09:36:36 PDT 1997 >cc flags: 3 >dumping core: .......... Old core present on disk --- not dumped. >Program terminated >ok > > > At this point the filer is inaccessible, and I can't find a way to >get it up and running. Why was it "innaccessible"? Just reboot it again. >Is there a way to flush the NVRAM or ignore an >existing dump... some way to turn NFS back on so the data can be >retrieved. Yes... just keep rebooting and evetually it will throw away the NVRAM. As for "ignore an existing dump", no I don't think so, but that's okay... the fact that the filer can't dump core isn't preventing you from rebooting (although I think it does prevent the auto-reboot). >Booting the kernel off floppy doesn't help because it >tries to replay the WAFL logs too, and another panic occurs. When NVRAM is corrupt, you have to keep rebooting several times. The sequence is usually like this. 1. Filer crashes while running - Reboot 2. Filer crashes replaying NVRAM - Reboot 3. Filer crahses again while replaying NVRAM - Reboot 4. Filer realizes it's failed replaying NVRAM twice in a row, so it flags it as bad, dumps the NVRAM, and - Reboot 5. Filer comes back up, probably in degraded mode, and is thus reconstructing. If there has been filesystem damage it may crash here again, and reboot again. If you still can't get it up (it may say "Filesystem may be scrambled) or you can't get it up for any length of time, you should call Netapp support and have them help you with the procedure for fixing the filesystem (wack) from floppy. The real kickers in this are you have to "know" that it'll do 2 and 3, and won't just keep rebooting forever. I think I've seen cases where it takes more than that for it to jettison NVRAM, but I can't be positive. This made since from a design point of you, to only give up if you fail to replay the NVRAM twice in a row, but in reality it seems that with most bugs (not all) if it fails once, it'll fail again. Furthermore, once it decides NVRAM is corrupt, it tries to dumb core and reboot *AGAIN*. The design thought here was again a sound one - get a core dump so we can look at the corrupt NVRAM and figure out what's wrong. However, in reality, if you've gotten to this point you've probably already crashed, and dumped core once, so you'll never be able to see this faulty NVRAM core... at least not until Netapp starts supporting multiple cores. The other bad thing about this sequence is you have several crashes stemming from the original crash, and possibly even several different bugs, but you'll never be able to get the cores from anything but the first one. I think there is a way to bypass some of this by booting off floppy and jettisoning the NVRAM manually, but given the time involved you are probably better off just rebooting the filer again. >The only >way around I've found is to wipe out the filesystem and start over >again (obviously not the optimal solution). Ideas? The above should help. Bruce

2 2

netapp NFS server crash by FreeBSD client [w/patch] (fwd)
by Tom Yates 07 Nov '97

07 Nov '97

this just in from the bugtraq mailing list. i figure *some* of us probably aren't on that. i also don't expect thousands of people to be trying this exploit, but given that it's just gone out on bugtraq, if your toaster panics anytime soon, you might want to bear this in mind. needless to say, i haven't tried it. anyone want to have a go? Tom Yates - Unix Chap - The Mathworks, Inc. - +1 (508) 647 7561 MAG#65061 DoD#0135 AMA#461546 1024/CFDFDE39 0C E7 46 60 BB 96 87 05 04 BD FB F8 BB 20 C1 8C ---------- Forwarded message ---------- Date: Wed, 5 Nov 1997 21:34:00 -0800 From: "Dmitry Kohmanyuk [KOI8-R] �� " <dk(a)GENESYSLAB.COM> To: BUGTRAQ(a)NETSPACE.ORG Subject: netapp NFS server crash by FreeBSD client [w/patch] this is only relevant to those using NetApp NFS file servers. background: there are 2 versions of NFS in town, v2 and v3. In v3, one of things introduced was ability to read names of files in directory with stat(2)ing them at the same time; the procedure name is readdirplus. It can be used to speed up programs like ls(1). Apparently, NFS code in FreeBSD 2.2 (derived from 4.4BSD code, so perhaps this applies to all of modern BSD systems here) allow client to specify this in mount options without using NFS v3. This meaningless set of options panics NetApp file server. the following simple patch (attached) for /usr/src/sys/nfs/nfs_bio.c fixes this problem. --- nfs_bio.c.ok Wed Nov 5 20:11:17 1997 +++ nfs_bio.c Wed Nov 5 20:14:06 1997 @@ -1031,6 +1031,8 @@ case VDIR: nfsstats.readdir_bios++; uiop->uio_offset = ((u_quad_t)bp->b_lblkno) * NFS_DIRBLKSIZ; + if (!(nmp->nm_flag & NFSMNT_NFSV3)) + nmp->nm_flag &= ~NFSMNT_RDIRPLUS; /* dk(a)farm.org */ if (nmp->nm_flag & NFSMNT_RDIRPLUS) { error = nfs_readdirplusrpc(vp, uiop, cr); if (error == NFSERR_NOTSUPP)

3 2

archive
by Tom Yates 07 Nov '97

07 Nov '97

ok, i did some work. the list archive is better now, it automatically rebuilds every time someone sends a message to the list, so i'ts a lot more up-to-date than it had got any time recently. it moved a little, too... go to http://teaparty.mathworks.com:1999/toasters/ to find it. hope this is useful for people. Tom Yates - Unix Chap - The Mathworks, Inc. - +1 (508) 647 7561 MAG#65061 DoD#0135 AMA#461546 1024/CFDFDE39 0C E7 46 60 BB 96 87 05 04 BD FB F8 BB 20 C1 8C

1 0

NetApp Release 4.2a bug?
by Steve Losen 07 Nov '97

07 Nov '97

We just got a F630 so I joined this list. I was wondering if anyone else has seen a problem with non-root users on NFS clients being unable to set the set-group bit. We are running Data ONTAP 4.2a on a F630. On a NFS client the following commands have no effect when run by a non-root user on files owned by the user and served by the F630. chmod g+s file chmod 2755 file Neither of these set the set-group bit. If superuser runs these commands on a NFS client with root access, they work. Once the set-group bit is set, a non-root user can unset it with chmod g-s file or chmod 755 file This has been reported to Netapp. Steve Losen scl(a)virginia.edu phone: 804-924-0640 University of Virginia ITC Unix Support

2 1

Experiences with CIFS
by Garrett Burke 05 Nov '97

05 Nov '97

Hi, I'm considering a F230 with CIFS support to replace several creaking NT file servers which are being used in a SW development environment. I would be interested in any experience that people have of using a filer with CIFS. I'm interested in primarily a CIFS only use, but may start using NFS in the future as well (as UNIX development grows) How do people handle backups of data , especially data that NT apps have locked. Anyone running Lotus Notes with the data on a filer? (I currently use ARCServe with their various agents to handle Notes & MS Exchange data) Being able to point ARCServe at the filer and get a consistent backup would be nice. I note from the netapp site that Windows Networking Software 2.0 is out, but I quote from the FAQ: " The next release of Windows Networking Software will also provide NTFS-style file and directory permissions for Windows users." This is what was said about version 1.0 Q to Netapp people on the list (yes, I have asked directly a week or 2 ago and am still awaiting a reply ) When can file level ACLs be expected? I know having file level ACLs is more complex to manage, but it does allow us to have multiple groups with different ranges of access (which stops the Analysts & Business people deleting/changing the developers work and project teams to have read access to other teams work) Even directory level ACLs would go a long way to meeting our requirements, but I think that if netapp is serious about the CIFS market then they need to duplicate the NT ACLs on the filers. Thanks, GB -- Garrett Burke, Network Admin,Managed Solutions Corp. 32 Upper Mount St., Dublin, Ireland. burkeg(a)msc.ie +353-1-661-4840 A jug of wine, a loaf of bread and thou.

1 0

Re: Redundant filer power supplies
by Robert Hamilton 04 Nov '97

04 Nov '97

At 06:52 AM 11/4/97 -0500, Brian Tao wrote: > How do these work? Do they replace the existing power supply with >two separate modules? Does one supply full power while the other is >on standby, or do both supply half power during normal operation? >-- The filer power supply is replaced by a RPS adapter module. This adapter module is connected via an "umbilical" cord to the separate rack-mount RPS unit. Inside this RPS unit are two power supplies, either of which is capable of powering the filer. In normal operations, these power supplies share the load. This load-sharing feature extends the life of the power supplies over a standby implementation. Robert

1 0

Redundant filer power supplies
by Brian Tao 04 Nov '97

04 Nov '97

How do these work? Do they replace the existing power supply with two separate modules? Does one supply full power while the other is on standby, or do both supply half power during normal operation? -- Brian Tao (BT300, taob(a)netcom.ca) "Though this be madness, yet there is method in't"

2 1

Possible software cause for total data loss?
by Brian Tao 04 Nov '97

04 Nov '97

During the pre-production testing of one of our F230 filers, we discovered a problem with one of them that we wereonly able to fix by rebuilding the RAID (and thus losing whatever OS and data was on the filer). The hardware configuration is as follows: NetApp Release 4.2a: Fri Sep 5 09:36:36 PDT 1997 System ID: 0016783805 slot 0: System Board 90 MHz (NetApp System Board Ia rev-b) Firmware release: 1.6_i Memory Size: 192 MB slot 0: SCSI Host Adapter 0 (QLogic ISP 1040B) Firmware Version 2.26 Clock Rate 40MHz. slot 0: Ethernet Controller e0 MAC Address: 00:a0:98:00:0f:1b (100tx) slot 6: NVRAM (NetApp NVRAM I) Revision: D2 Serial Number: 6589 Memory Size: 4 MB Battery 1 Status: 100% (3.07v) Battery 2 Status: 100% (3.07v) slot 7: Ethernet Controller e7 (Znyx) MAC Address: 00:c0:95:f8:49:c9 (Twisted pair) slot 9: Dual SCSI Host Adapter (NetApp SCSI Adapter II) SCSI Host Adapter 9a (Qlogic ISP 1040B) Firmware Version 2.26 Clock Rate 40MHz. 0: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) 1: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) 2: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) 3: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) 5: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) 4: SEAGATE ST15150W 9107 Size=3.9GB (8388315 blocks) SCSI Host Adapter 9b (Qlogic ISP 1040B) Firmware Version 2.26 Clock Rate 40MHz. Part of the tests consisted of filling up the filesystem via NFS and NDMP copies from a host Ultrasparc. Three other F230's of identical configuration survived the tests, but the remaining F230 experienced the following panic four times: PANIC: ../common/wafl/nvlog.c: 1088: Assertion failure I will be running the NVRAM diagnostics later today to see if they turn up anything. However, more distressing is the behaviour of the Netapp upon reboot: [... other boot messages deleted...] Loading filesystem. Recomputing parity in NVRAM PANIC: ../driver/disk/disk.c:2633: Assertion failure. version: NetApp Release 4.2a: Fri Sep 5 09:36:36 PDT 1997 cc flags: 3 dumping core: .......... Old core present on disk --- not dumped. Program terminated ok At this point the filer is inaccessible, and I can't find a way to get it up and running. Is there a way to flush the NVRAM or ignore an existing dump... some way to turn NFS back on so the data can be retrieved. Booting the kernel off floppy doesn't help because it tries to replay the WAFL logs too, and another panic occurs. The only way around I've found is to wipe out the filesystem and start over again (obviously not the optimal solution). Ideas? -- Brian Tao (BT300, taob(a)netcom.ca) "Though this be madness, yet there is method in't"

1 0

Re: raid swap question
by sirbruce＠ix.netcom.com 04 Nov '97

04 Nov '97

On 10/08/97 09:10:10 you wrote: >We have "raid swap"ed almost three shelves worth of drives into >an F330 at one time. The AE recommended this method as >being faster than installing all the drives at config >time and building that large file system from scratch. >This worked fine, but still takes a significant amount >of time. I think by "at one time" you still mean one at a time (a raid swap for each drive). >We later added a fourth shelf and "raid swap"ed all those >drives in at once. The filer was only off the net for >the length of time it took to halt/connect the SCSI cable/ >reboot. A few hours later, after the raid swap completes, >all the additional disk space shows up - it does not show >up one drive at a time. I don't quite understand this... if you added them while the filer was down, you didn't "raid swap" anything. And they show up immediately, not a few hours later. I think perhaps you are talking about *raid add*, i.e. actually adding the drives to the filesystem. In that case you can add them individually one drive at a time (and the filer would get the space on drive at a time), or you can raid add them all at once, in which case they'll all finish at about the same time. >Some comments regarding DLT drives - I have heard that you >want to keep them "streaming" (feed with data) or the >drive has to stop, wait for data, backup, reseek to the >stop point, and start again. This applies to both 4000/7000 >series. You pay a significant speed penalty for dropping >the drive out of streaming mode. Our word from Netapp is >that for an F230, they sell the FW/DIff card and it works, but >it is not officially QA'd in the system, but soon should be. Yes, streaming is important, but my point was I don't think a 7000 non-streaming is going to be much slower than a 4000 streaming. But, I could be wrong on this. In any case, there are *plenty* are start-stop behavior in a Netapp dump even locally, so I don't think you can avoid it. Bruce

2 1

Re: Dumping Clearcase Storage Pools (fwd)i
by Dave Compton 31 Oct '97

31 Oct '97

Gil Smith wrote: > Hey Dave, > > Did you know about the "toasters" mail list at mathworks? > One of our customers started this mailing list for owners > of netapp boxes to share information with each other. > > Anyway below is a question that was on the list just recently > related to clearcase. Thought you might be interrested in > the list and this question. > > --Gil > > ----- Forwarded message from Scott MacDonald ----- > > >From scott(a)msi-uk.com Fri Oct 31 06:51:22 1997 > Sender: scott(a)msi-uk.com > Message-ID: <3459EC09.38A8(a)msi-uk.com> > Date: Fri, 31 Oct 1997 14:32:41 +0000 > From: Scott MacDonald <scott(a)msi-uk.com> > Organization: Mobile Systems International Plc, UK > X-Mailer: Mozilla 3.01Gold (X11; I; SunOS 5.5 sun4m) > To: toasters(a)mathworks.com > Subject: Dumping Clearcase Storage Pools > > Does anyone have any experience using NetApp filers to hold > Clearcase storage pools, and if so, how did you manage the backups? > We're just about to move to them, and we think we should be able > to lock the VOBs, copy the db to the filer, then do a local dump > (from a snapshot) to DLT. > If anyone has any experience they wouldn't mind sharing, I'd > love to hear it. > TIA, > > Scott. > -- > _______________________________________________________________ > > Scott MacDonald Mobile Systems International Plc > Tel: 0171 971 8798 > System Administrator Fax: 0171 971 8712 > E-Mail: scott(a)msi-uk.com > _______________________________________________________________ > > ----- End of forwarded message from Scott MacDonald ----- Scott, We have been doing this for 7 or 8 months now. We have approximately 50 VOBs to backup on a daily basis. Our backups have dropped from around 7 hours to 1 hour and 1/2 hours. Basically this is what we do. Snapshots are set so that there is a snapshot at midnight. At 2357 we lock all the VOBs and wait for the snapshot. At around 0004 we copy the database files from every VOB to a backup directory on the NetApps, create links between this backup area and the source pools in the snapshot. Unlock the VOB and start copying the VOB data to tape. The tape process takes about 7-8 hours. But the most important thing is that our VOB are only locked for around 1 hour. The process has worked very well for us. Hope this helps. Dave

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters