The other method that seems to work every time that will work for you is a cluster takeover and giveback.
I doubt rebooting the IOM will fix it.
The two methods I have used that seem to work every time:
1. Pull the IOM for 4 minutes and re-insert (more details earlier in thread) 2. Takeover/Giveback.
The short pull of the IOM was unreliable. Sometimes it worked, sometimes it did not and if it did not, it would usually cause other IOMS to misbehave for some reason.
--tmac
*Tim McCarthy* *Principal Consultant*
On Sat, May 9, 2015 at 5:59 AM, Borzenkov, Andrei < andrei.borzenkov@ts.fujitsu.com> wrote:
ACP is not necessary for FW update. There is also separate ACP firmware - here I am not sure whether it is update over SAS or Ethernet.
There is also possibility to reboot IOM remotely, but you need to ask support how to do it.
Отправлено с iPhone
9 мая 2015 г., в 11:25, Alexander Griesser ag@anexia.at написал(а):
Thanks everyone,
since the filer is a few hundred kilometers away ATM I was hoping for a remote fix J System is MPHA of course and I verified that with Config Advisor as well as on the CLI, so that shouldn’t be a problem.
Does it make sense to open a Case to make them aware of the fact? Could that also be a firmware issue on the IOMs or ACP that might be fixed by applying a new firmware? I’m not sure how firmware updates are applied to the IOMs – is that happening through the SAS cables or is ACP necessary for that? My idea was to start a firmware update on the shelves to see if that fixes the issue, since a firmware update also reboots the IOMs one at a time – but if ACP connection is necessary for the FW update to complete I’m out of luck here and will have to dispatch remote hands.
Best,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: ag@anexia.at
Web: http://www.anexia.at
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
*Von:* Clark, André M. [mailto:Andre.Clark@redeight.com Andre.Clark@redeight.com] *Gesendet:* Freitag, 08. Mai 2015 23:18 *An:* tmac; Alexander Griesser *Cc:* toasters@teaparty.net *Betreff:* RE: ACP - missing IOMs
I’ve seen this issues on numerous installs now, including brand new systems. This is something that we should not have to be doing all the time. There has to be something else going on here.
Regards,
André M. Clark Andre.Clark@redeight.com | Sr. Solutions Architect | 917.388.8236
*Tell me I will forget... Show me I may remember... Involve me I WILL UNDERSTAND!!! Start by doing what's necessary, then what's possible, and suddenly you are doing the impossible!!!*
*Red8*, An Insight Investments Company
www.redeight.com
*From:* toasters-bounces@teaparty.net [ mailto:toasters-bounces@teaparty.net toasters-bounces@teaparty.net] *On Behalf Of *tmac *Sent:* Friday, 8 May, 2015 15:12 *To:* Alexander Griesser *Cc:* toasters@teaparty.net *Subject:* Re: ACP - missing IOMs
Make sure your Multipath HA is working.
go to shelf 23 and disconnect the B IOM from the backplane (no need to fully remove, just electrical disconnect)
go to shelf 15 and disconnect the B IOM
WAIT AT LEAST 4 MINUTES!!!
Insert them back in to the shelves.
Re-check after 15-20 minutes.
That should do it. If there are still some missing, pull and wait at least 4 minutes. Make sure to only do one side (a or b) at a time!
--tmac
*Tim McCarthy*
*Principal Consultant*
On Fri, May 8, 2015 at 3:02 PM, Alexander Griesser ag@anexia.at wrote:
Hey there,
I was adding two shelves to one of my FAS8020s (cDOT, if that matters) recently and one IOM on each of the newly added shelves is missing in `storage show acp` or `acpadmin list_all` directly on the nodeshell.
smokescreen> acpadmin list_all
IP MAC Reset Last Contact Protocol Assigner Shelf Current Inband IOM
Address Address Cnt (seconds ago) Version ACPA ID S/N State ID Type
192.168.0.1 00:a0:98:71:a0:00 000 163 1.2.2.8 536878789 SHFMS1414001364 0x5 0a.14.B IOM6
192.168.0.73 00:a0:98:72:d0:48 000 309 1.2.2.8 536878789 SHJMS1416001181 0x5 2a.21.B IOM6
192.168.0.101 00:a0:98:72:d0:64 000 373 1.2.2.8 536878789 SHJMS1416001174 0x5 2a.22.B IOM6
192.168.0.105 00:a0:98:72:d0:68 000 407 1.2.2.8 536878620 SHJMS1416001174 0x5 0b.22.A IOM6
192.168.0.191 00:a0:98:71:a0:be 000 154 1.2.2.8 536878789 SHFMS1414001364 0x5 2b.14.A IOM6
192.168.0.225 00:a0:98:83:14:e1 000 252 1.2.2.8 536878789 SHJHU1515000294 0x4 ------- IOM6
192.168.1.29 00:a0:98:72:d1:1c 000 335 1.2.2.8 536878789 SHJMS1416001181 0x5 0b.21.A IOM6
192.168.2.103 00:a0:98:83:12:67 000 251 1.2.2.8 536878789 SHJHU1515000294 0x5 0b.23.A IOM6
192.168.2.173 00:a0:98:72:ba:ac 000 392 1.2.2.8 536878789 SHFMS1417000556 0x5 2b.11.A IOM6
192.168.3.7 00:a0:98:72:cb:06 000 337 1.2.2.8 536878620 SHFMS1417000567 0x5 0a.12.B IOM6
192.168.3.13 00:a0:98:81:87:0c 000 384 1.2.2.8 536878789 SHFHU1513000260 0x4 ------- IOM6
192.168.3.19 00:a0:98:81:87:12 000 285 1.2.2.8 536878620 SHFHU1513000260 0x5 2b.15.A IOM6
192.168.3.21 00:a0:98:72:cb:14 000 407 1.2.2.8 536878789 SHFMS1417000567 0x5 2b.12.A IOM6
192.168.3.25 00:a0:98:72:2f:18 000 305 1.2.2.8 536878789 SHFMS1416000221 0x5 2b.13.A IOM6
192.168.3.95 00:a0:98:72:cb:5e 000 471 1.2.2.8 536878620 SHFMS1417000556 0x5 0a.11.B IOM6
192.168.3.131 00:a0:98:72:2f:82 000 386 1.2.2.8 536878789 SHFMS1416000221 0x5 0a.13.B IOM6
smokescreen> storage show acp
Alternate Control Path: Enabled
Ethernet Interface: e0P
ACP Status: Active
ACP IP Address: 192.168.2.215
ACP Subnet: 192.168.0.0
ACP Netmask: 255.255.252.0
ACP Connectivity Status: Additional Connectivity
ACP Partner Connectivity Status: Additional Connectivity
Shelf Module Reset Cnt IP Address FW Version Module Type Status
0b.23.A 000 192.168.2.103 02.08 IOM6 active
0b.21.A 000 192.168.1.29 02.08 IOM6 active
0b.22.A 000 192.168.0.105 02.08 IOM6 active
2b.15.A 000 192.168.3.19 02.08 IOM6 active
2b.12.A 000 192.168.3.21 02.08 IOM6 active
2b.14.A 000 192.168.0.191 02.08 IOM6 active
2b.11.A 000 192.168.2.173 02.08 IOM6 active
2b.13.A 000 192.168.3.25 02.08 IOM6 active
2a.21.B 000 192.168.0.73 02.08 IOM6 active
2a.22.B 000 192.168.0.101 02.08 IOM6 active
0a.14.B 000 192.168.0.1 02.08 IOM6 active
0a.13.B 000 192.168.3.131 02.08 IOM6 active
0a.12.B 000 192.168.3.7 02.08 IOM6 active
0a.11.B 000 192.168.3.95 02.08 IOM6 active
NA 000 192.168.0.225 02.08 IOM6 inactive (no in-band connectivity)
NA 000 192.168.3.13 02.08 IOM6 inactive (no in-band connectivity)
I’ve done some research and found that usually a TO/GB helps in this situation, but I’m – of course – trying to avoid that.
So, any ideas how I can try to make those pesky IOMs respond on the ACP? I tried to disable and re-enable ACP, but that didn’t help (maybe I did it wrong, have never fully disabled ACP on a cDOT system), so any pointers are welcome.
Also, config advisor suggests to install missing shelf firmware, so I was wondering if it is safe to install the shelf firmware while an IOM is missing on the ACP?
Thanks,
*Alexander Griesser*
Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: ag@anexia.at
Web: http://www.anexia.at
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters