I have seen this! Support was unable to reproduce for me. I will dig up a case number later if you like.

I will give you two options:
1. Open a support case with netapp and refer to the BURT 1050364 / Case 2006580487 
2. The ultimate fix that I did was this:
-> Downgrade *all* IOM firmware to 2.09. Wait for it to stabilize.
-> Convert to in-band. Wait for it to stabilize and confirm IN-Band
-> Convert to out-of-band and upgrade to 2.10 firmware.
-> Wait for it to stabilize and then convert to out-of-band

I got to believe there may be some race condition where the 2.08 -> 2.10 firmware is not completing 100%.
If you use the out-of-band, all looks ok, but switch to in-band and not so much.

The thought is something special happens on 2.09 that fixes everything allowing the upgrade to 2.10

If possible, please open a case and refer to the data above. I think it is in everyones best interest NetApp gets to the bottom of this to fix.

Thanks


--tmac

Tim McCarthy, Principal Consultant

Proud Member of the #NetAppATeam

I Blog at TMACsRack



On Sat, Apr 8, 2017 at 5:55 AM, Alexander Griesser <AGriesser@anexia-it.com> wrote:

Hey Toasters,

 

I’ve just upgraded one of my HA-pairs to 9.1P2 and wanted to switch to ACP inband connectivity following this KB:

https://kb.netapp.com/support/s/article/ka31A00000012NsQAI/in-band-acp-setup-and-support

 

According to this KB, I should be eligible for the switch:

 

::*> storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state

node    module-name protocol-version firmware-version shelf-serial-number iom-type state

------- ----------- ---------------- ---------------- ------------------- -------- ------

nodeA 1.22.B      1.2.02.10        02.10            SHJMS1424000060     iom6     active

nodeA 1.22.A      1.2.02.10        02.10            SHJMS1424000060     iom6     active

nodeA 2.11.A      1.2.02.10        02.10            SHFMS1423000507     iom6     active

nodeA 2.11.B      1.2.02.10        02.10            SHFMS1423000507     iom6     active

nodeA 2.14.B      1.2.02.10        02.10            SHFMS1423000491     iom6     active

nodeA 2.14.A      1.2.02.10        02.10            SHFMS1423000491     iom6     active

nodeA 2.12.A      1.2.02.10        02.10            SHFMS1423000493     iom6     active

nodeA 2.12.B      1.2.02.10        02.10            SHFMS1423000493     iom6     active

nodeA 2.13.A      1.2.02.10        02.10            SHFMS1423000504     iom6     active

nodeA 2.13.B      1.2.02.10        02.10            SHFMS1423000504     iom6     active

nodeA 1.21.B      1.2.02.10        02.10            SHJMS1424000434     iom6     active

nodeA 1.21.A      1.2.02.10        02.10            SHJMS1424000434     iom6     active

nodeB

        1.22.B      1.2.02.10        02.10            SHJMS1424000060     iom6     active

nodeB

        1.22.A      1.2.02.10        02.10            SHJMS1424000060     iom6     active

nodeB

        2.11.A      1.2.02.10        02.10            SHFMS1423000507     iom6     active

nodeB

        2.11.B      1.2.02.10        02.10            SHFMS1423000507     iom6     active

nodeB

        2.14.B      1.2.02.10        02.10            SHFMS1423000491     iom6     active

nodeB

        2.14.A      1.2.02.10        02.10            SHFMS1423000491     iom6     active

nodeB

        2.12.A      1.2.02.10        02.10            SHFMS1423000493     iom6     active

nodeB

        2.12.B      1.2.02.10        02.10            SHFMS1423000493     iom6     active

nodeB

        2.13.A      1.2.02.10        02.10            SHFMS1423000504     iom6     active

nodeB

        2.13.B      1.2.02.10        02.10            SHFMS1423000504     iom6     active

nodeB

        1.21.B      1.2.02.10        02.10            SHJMS1424000434     iom6     active

nodeB

        1.21.A      1.2.02.10        02.10            SHJMS1424000434     iom6     active

24 entries were displayed.

 

 

The requirements state that the IOM6 modules must be upgraded to 2.09 (I have 2.10) and ACP firmware must be at least 1.2.2.9 (I have 1.2.2.10).

So I went ahead and ran this command:

 

storage shelf acp configure -is-enabled true -channel in-band

 

After a few minutes, I saw this:

 

::*> storage shelf acp show

Node                Channel          Connectivity

------------------  ---------------  ----------------------

nodeA             in-band          active

nodeB          in-band          active

2 entries were displayed.

 

But when I ran the initial check command again, I saw this output:

 

::*> storage shelf acp module show -fields module-name,protocol-version,firmware-version,shelf-serial-number,iom-type,state

node    module-name protocol-version firmware-version shelf-serial-number iom-type state

------- ----------- ---------------- ---------------- ------------------- -------- ------------------------

nodeA 1.22.B      1.2.02.10        02.10            SHJMS1424000060     iom6     firmware-update-required

nodeA 1.22.A      1.2.02.10        02.10            SHJMS1424000060     iom6     firmware-update-required

nodeA 2.11.A      1.2.02.10        02.10            SHFMS1423000507     iom6     firmware-update-required

nodeA 2.11.B      1.2.02.10        02.10            SHFMS1423000507     iom6     firmware-update-required

nodeA 2.14.B      1.2.02.10        02.10            SHFMS1423000491     iom6     firmware-update-required

nodeA 2.14.A      1.2.02.10        02.10            SHFMS1423000491     iom6     firmware-update-required

nodeA 2.12.A      1.2.02.10        02.10            SHFMS1423000493     iom6     firmware-update-required

nodeA 2.12.B      1.2.02.10        02.10            SHFMS1423000493     iom6     firmware-update-required

nodeA 2.13.A      1.2.02.10        02.10            SHFMS1423000504     iom6     firmware-update-required

nodeA 2.13.B      1.2.02.10        02.10            SHFMS1423000504     iom6     firmware-update-required

nodeA 1.21.B      1.2.02.10        02.10            SHJMS1424000434     iom6     firmware-update-required

nodeA 1.21.A      1.2.02.10        02.10            SHJMS1424000434     iom6     firmware-update-required

nodeB

        1.22.B      1.2.02.10        02.10            SHJMS1424000060     iom6     firmware-update-required

nodeB

        1.22.A      1.2.02.10        02.10            SHJMS1424000060     iom6     firmware-update-required

nodeB

        2.11.A      1.2.02.10        02.10            SHFMS1423000507     iom6     firmware-update-required

nodeB

        2.11.B      1.2.02.10        02.10            SHFMS1423000507     iom6     firmware-update-required

nodeB

        2.14.B      1.2.02.10        02.10            SHFMS1423000491     iom6     firmware-update-required

nodeB

        2.14.A      1.2.02.10        02.10            SHFMS1423000491     iom6     firmware-update-required

nodeB

        2.12.A      1.2.02.10        02.10            SHFMS1423000493     iom6     firmware-update-required

nodeB

        2.12.B      1.2.02.10        02.10            SHFMS1423000493     iom6     firmware-update-required

nodeB

        2.13.A      1.2.02.10        02.10            SHFMS1423000504     iom6     firmware-update-required

nodeB

        2.13.B      1.2.02.10        02.10            SHFMS1423000504     iom6     firmware-update-required

nodeB

        1.21.B      1.2.02.10        02.10            SHJMS1424000434     iom6     firmware-update-required

nodeB

        1.21.A      1.2.02.10        02.10            SHJMS1424000434     iom6     firmware-update-required

24 entries were displayed.

 

Out of curiosity, I tried to run the ACP update again:

 

nodeB> storage download acp

 

Downloading ACP firmware can take several minutes

and will not disrupt client access during that time.

However, normal ACP recovery capabilities will not be

available while the firmware upgrade is in progress.

 

Are you sure you want to continue with ACP processor firmware update? y

[storage download acp] Firmware will not be downloaded to module 2b.14.B as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0a.14.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0b.21.B as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0a.13.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0a.12.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 2a.21.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 2b.12.B as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 2b.13.B as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0b.22.B as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 2a.22.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 0a.11.A as ACPP state is not active, Current ACPP state is 0x9.

[storage download acp] Firmware will not be downloaded to module 2b.11.B as ACPP state is not active, Current ACPP state is 0x9.

nodeB> exit

 

Eventlog also said, that a FW Upgrade is required:

 

::*> event log show -severity infORMATIONAL

Time                Node             Severity      Event

------------------- ---------------- ------------- ---------------------------

4/8/2017 11:41:15   nodeB       INFORMATIONAL sfu.firmwareUpToDate: Firmware is up-to-date on all disk shelves.

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000507).

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000504).

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000491).

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000493).

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHJMS1424000060).

4/8/2017 11:39:37   nodeA          INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHJMS1424000434).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000493).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000504).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000507).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHFMS1423000491).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHJMS1424000434).

4/8/2017 11:39:37   nodeB       INFORMATIONAL acp.need.firmware.update: ACP module  (NA) needs a firmware upgrade, (disk shelf serial number: SHJMS1424000060).

 

I then switched back to OOB connectivity, and all is good again:

 

::*> storage shelf acp show

Node                Channel          Connectivity

------------------  ---------------  ----------------------

nodeA             out-of-band      full-connectivity

nodeB          out-of-band      full-connectivity

2 entries were displayed.

 

Tried to run the ACP upgrade again, getting this:

 

nodeB> storage download acp

 

Downloading ACP firmware can take several minutes

and will not disrupt client access during that time.

However, normal ACP recovery capabilities will not be

available while the firmware upgrade is in progress.

 

Are you sure you want to continue with ACP processor firmware update? y

[storage download acp] ACP will check 12 modules for firmware download.

[storage download acp] ACP module 192.168.0.124 is not eligible for update.

[storage download acp] ACP module 192.168.0.126 is not eligible for update.

[storage download acp] ACP module 192.168.0.180 is not eligible for update.

[storage download acp] ACP module 192.168.0.202 is not eligible for update.

[storage download acp] ACP module 192.168.0.208 is not eligible for update.

[storage download acp] ACP module 192.168.0.248 is not eligible for update.

[storage download acp] ACP module 192.168.1.112 is not eligible for update.

[storage download acp] ACP module 192.168.1.170 is not eligible for update.

[storage download acp] ACP module 192.168.1.238 is not eligible for update.

[storage download acp] ACP module 192.168.2.66 is not eligible for update.

[storage download acp] ACP module 192.168.3.130 is not eligible for update.

[storage download acp] ACP module 192.168.3.138 is not eligible for update.

nodeB> exit

 

And now I’m back at the starting point.

Any ideas what’s going on here? Why does it tell me „firmware-update-required“ when I’m already on the latest versions?

 

Thanks,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: AGriesser@anexia-it.com

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 


_______________________________________________
Toasters mailing list
Toasters@teaparty.net
http://www.teaparty.net/mailman/listinfo/toasters