Hey all,
I've got a 8.0.2 c-mode cluster that recently had a single node joined to
it, and a few empty aggregates created on it. We had a extended power
outtage that required a lot of gear in the data center to get shut down,
and since this node in the cluster didn't have any live data or VIFs on it,
it got shut down.
A few days later, and we are now powering it up, but get this upon login:
"The contents of the root volume may have changed and the local management
databases may be out of sync with the replicated databases due to
corruption of
NVLOG data during takeover. This node is not fully operational. Contact
support
personnel for the root volume recovery procedures."
The node comes up fine, can see all it's aggregates, and the other nodes in
the cluster can see it via the cluster network, but the node is indeed not
fully functional and part of the cluster again. It's aggregates and other
info are not visible from the other nodes in the cluster.
Did a wafl_check of the root aggr and vol0 and that came back clean. I
seem to recall having been through this before in the past, but can't find
anything in my notes.
This particular cluster is not under support, due to some genius decisions
by management, so I'm on my own with this.
There are a few empty aggregates on this node, no volumes other than the
root vol. Maybe I can force unjoin it from the cluster and rebuild it?
Would rather not try to do that. If there is a way to sync up the dbs on
the rootvol so it will come back into the cluster, that would be ideal.
Any ideas?