Hey,
this did work, indeed – at least the disk could be unfailed with `disk unfail –s` here on 8.1.4 in a different filer; I’m currently zeroing them and will then remove_ownership and bring them back to the older filer.
Thanks for your help!
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com] Gesendet: Samstag, 24. Oktober 2015 10:08 An: Alexander Griesser AGriesser@anexia-it.com Cc: toasters@teaparty.net Betreff: Re: AW: Used Replacement disk not recognized
Assuming it is indeed too new label (it does not look exactly as one) your options are
- update to 8.0.2 or above and use "disk unfail -s"
- put disk in filer with newer version and make it zeroed unowned spare
Messing with labels directly is unlikely going to help.
https://kb.netapp.com/support/index?page=content&id=2011366&locale=e...
Отправлено с iPhone
23 окт. 2015 г., в 21:45, Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com> написал(а): Alright, but now I do have a label issue I guess ☺
An even older filer (FAS2040, OnTap 8.0.1P2 7-Mode) is writing this to the log when I insert one of the refurbished disks:
Fri Oct 23 20:24:52 CEST [megatron: raid.disk.missing:info]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA00] S/N [JXV0TL2J] is missing from the system Fri Oct 23 20:26:52 CEST [megatron: diskown.changingOwner:info]: changing ownership for disk 0d.01.5 (S/N JXXDSRMJ) from unowned (ID 4294967295) to megatron (ID 142227074) Fri Oct 23 20:26:52 CEST [megatron: raid.assim.disk.nolabels:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has no valid labels. It will be taken out of service to prevent possible data loss. Fri Oct 23 20:26:52 CEST [megatron: raid.config.disk.bad.label:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has bad label. Fri Oct 23 20:26:52 CEST [megatron: callhome.dsk.label:CRITICAL]: Call home for DISK BAD LABEL Fri Oct 23 20:27:00 CEST [megatron: monitor.globalStatus.nonCritical:warning]: Disk on adapter 0d, shelf 1, bay 5, failed.
I guess this means that I will have to erase the label here, correct?
aggr status -r Broken disks
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks) --------- ------ ------------- ---- ---- ---- ----- -------------- -------------- bad label 0d.01.5 0d 1 5 SA:B 0 SAS 15000 272000/557056000 274845/562884296
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser Gesendet: Donnerstag, 22. Oktober 2015 16:55 An: andrei.borzenkov@ts.fujitsu.commailto:andrei.borzenkov@ts.fujitsu.com; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: AW: Used Replacement disk not recognized
Many thanks!
riddler> vol status -s
Pool1 spare disks (empty)
Pool0 spare disks
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks) --------- ------ ------------- ---- ---- ---- ----- -------------- -------------- Spare disks for block checksum spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816 spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
I’m up and running ☺
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: andrei.borzenkov@ts.fujitsu.commailto:andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com] Gesendet: Donnerstag, 22. Oktober 2015 16:52 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: Used Replacement disk not recognized
disk assign 0d.01.13 -f
--- With best regards
Andrei Borzenkov Senior system engineer FTS WEMEAI RUC RU SC TMS FOS <image001.gif> FUJITSU Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation Tel.: +7 495 730 62 20 ( reception) Mob.: +7 916 678 7208 Fax: +7 495 730 62 14 E-mail: Andrei.Borzenkov@ts.fujitsu.commailto:Andrei.Borzenkov@ts.fujitsu.com Web: ru.fujitsu.comhttp://ts.fujitsu.com/ Company details: ts.fujitsu.com/imprinthttp://ts.fujitsu.com/imprint.html This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation. Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.
From: Alexander Griesser [mailto:AGriesser@anexia-it.com] Sent: Thursday, October 22, 2015 5:50 PM To: Borzenkov, Andrei; toasters@teaparty.netmailto:toasters@teaparty.net Subject: AW: Used Replacement disk not recognized
Bingo:
riddler> disk show -v DISK OWNER POOL SERIAL NUMBER HOME CHKSUM ------------ ------------- ----- ------------- ------------- ------ 0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block
„w-nas-04“ is none of my filers – so I need to remove the ownership here now, I guess.
riddler*> disk remove_ownership 0d.01.13 -f disk remove_ownership: Disk 0d.01.13 is not owned by this node. riddler*> disk reassign -o w-nas-04 -n riddler disk reassign: Can only be run in maintenance mode or during takeover in advanced mode riddler*>
So, I guess I have to – again – boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: andrei.borzenkov@ts.fujitsu.commailto:andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com] Gesendet: Donnerstag, 22. Oktober 2015 16:47 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: Used Replacement disk not recognized
Does it appear in “disk show -v”?
--- With best regards
Andrei Borzenkov Senior system engineer FTS WEMEAI RUC RU SC TMS FOS <image001.gif> FUJITSU Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation Tel.: +7 495 730 62 20 ( reception) Mob.: +7 916 678 7208 Fax: +7 495 730 62 14 E-mail: Andrei.Borzenkov@ts.fujitsu.commailto:Andrei.Borzenkov@ts.fujitsu.com Web: ru.fujitsu.comhttp://ts.fujitsu.com/ Company details: ts.fujitsu.com/imprinthttp://ts.fujitsu.com/imprint.html This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation. Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.
From: Alexander Griesser [mailto:AGriesser@anexia-it.com] Sent: Thursday, October 22, 2015 5:44 PM To: Borzenkov, Andrei; toasters@teaparty.netmailto:toasters@teaparty.net Subject: AW: Used Replacement disk not recognized
Hi Andrei,
the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool… The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: andrei.borzenkov@ts.fujitsu.commailto:andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com] Gesendet: Donnerstag, 22. Oktober 2015 16:41 An: Alexander Griesser <AGriesser@anexia-it.commailto:AGriesser@anexia-it.com>; toasters@teaparty.netmailto:toasters@teaparty.net Betreff: RE: Used Replacement disk not recognized
Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.
KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=e... implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.
Last command you pasted suggests that everything is OK now; or what is remaining problem?
--- With best regards
Andrei Borzenkov Senior system engineer FTS WEMEAI RUC RU SC TMS FOS <image001.gif> FUJITSU Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation Tel.: +7 495 730 62 20 ( reception) Mob.: +7 916 678 7208 Fax: +7 495 730 62 14 E-mail: Andrei.Borzenkov@ts.fujitsu.commailto:Andrei.Borzenkov@ts.fujitsu.com Web: ru.fujitsu.comhttp://ts.fujitsu.com/ Company details: ts.fujitsu.com/imprinthttp://ts.fujitsu.com/imprint.html This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation. Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser Sent: Thursday, October 22, 2015 5:06 PM To: toasters@teaparty.netmailto:toasters@teaparty.net Subject: Used Replacement disk not recognized
Hey everybody,
I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market. Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.
Here are the facts: FAS2040 NetApp Release 8.1.4P8 7-Mode
The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:
Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).
The interesting thing, I can see the disk by using `storage show disk`:
riddler> storage show disk DISK SHELF BAY SERIAL VENDOR MODEL REV --------------------- --------- ---------------- -------- ---------- ---- 0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04
But I cannot see it in disk show.
riddler> disk show 0d.01.13 DISK OWNER POOL SERIAL NUMBER HOME ------------ ------------- ----- ------------- -------------
Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:
01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect) 01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)
So I first tried to downgrade firmware using:
disk_fw_update –R 0d.01.13
but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs. So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.
So I booted into maintenance mode and have run the following command:
*> label makespare 0d.01.13 ********************************************************* Label editing is highly dangerous and should be attempted ONLY by NetApp Customer Support Engineers. Misuse can cause ALL data to be permanently lost. ********************************************************* 'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15. Label save: 1 of 35 disks could not save Label 1. 'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15. Label save: 1 of 35 disks could not save Label 2. The current set of labels on the disks have been saved. You can use the 'label restore' command to revert to these labels, the 'label keep' command to erase these labels, or 'label save' to preserve the current set of labels. ********************************************************* *** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***
Disk 0d.01.13 does not exist.
The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:
riddler> disk show 0d.03.22 DISK OWNER POOL SERIAL NUMBER HOME ------------ ------------- ----- ------------- ------------- 0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)
I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.com
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters