We have the exact same setup here and saw the same error. its apparently a documented bug with 5.0.1 and netapp released 5.0.2 to fix this fc-specific problem.
Apparently involves the FC controller and certain raid operations. Yours may have failed during a raid scrub, but I cant tell from the logs.
Reboot the shelf and the failed disk will be marked as 'spare' again. And upgrade to 5.0.2 as soon as possible. Make sure you let your rep know what happened, and if it makes you more comfortable, demand a new disk. =)
I can't suggest 5.1, as trying to upgrade from 5.0.1 to 5.1 on our 630/FC caused 2 days of catastrophic downtime. 5.1 has the 5.0.2 FC bug fix rolled in, though.
matto
On Tue, 28 Jul 1998, Daniel Quinlan wrote:
Does something about the timing of this disk failure strike anyone as odd or is it just me? I'm curious to know whether some checks only done once an hour, once a day, or if something else is at play.
This happened on an 18 day old F630/FiberChannel running 5.0.1D1.
Dan
------- start of cut text -------------- Mon Jul 27 23:00:00 PDT [statd]: 11:00pm up 17 days, 18 mins, 319161196 NFS ops, 0 CIFS ops, 0 HTTP ops Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008e2330): request failed after retry #20 Mon Jul 27 23:59:59 PDT [raid_stripe_owner]: Read on data disk 8.29 in volume vol0, RAID group 0, failed; reverting to degraded mode. Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008de230): opcode=0x2a sector 0 not ready (2 4, 1) Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008de230): request failed after retry #20 Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008da610): opcode=0x2a sector 0 not ready (2 4, 1) Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008da610): request failed after retry #20 Mon Jul 27 23:59:59 PDT [raid_stripe_owner]: Read on data disk 8.29 in volume vol0, RAID group 0, failed; entered degraded mode. Mon Jul 27 23:59:59 PDT [raid_stripe_owner]: One disk is missing from volume vol0, RAID group 0. A "hot spare" disk is available and the missing disk will be reconstructed on the spare disk. Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008d9910): opcode=0x28 sector 0 not ready (2 4, 1) Mon Jul 27 23:59:59 PDT [isp2100_main]: Disk 8.29(0xfffffc00008d9910): request failed after retry #20 Tue Jul 28 00:00:00 PDT [statd]: 12:00am up 17 days, 1:18 319612667 NFS ops, 0 CIFS ops, 0 HTTP ops Tue Jul 28 00:00:00 PDT [asup_main]: System Notification mail sent ------- end ----------------------------
--matt@bikkle.interq.or.jp------------------------------------------- Matt Ghali MG406/GM023JP - System Administrator, interQ, Inc AS7506 "Sub-optimal is a state of mind." -Dave Rand, dlr@bungi.com