Hi Tim,
Thanks for the offer to help.  We will just have to see how long it will take to reunite our cluster pairs, though I’d be happy to just dump the missing pair and handle the data loss.

Which won’t be much if anything at all.

Sent from my iPhone

On May 15, 2021, at 7:04 PM, Timothy Naple <tnaple@berkcom.com> wrote:


John,

If it were me, I would take it one step at a time and first get the systems physically in place, cabled up and clusters switches up and running and once you can login on console or SSH then start looking at the aggregates, LIF's, cluster epsilon/quorum and which volumes are source, etc.  Keep me posted if you hit any major roadblocks and I can try to help out.

Thank you,
Tim


From: Toasters <toasters-bounces@teaparty.net> on behalf of John Stoffel <john@stoffel.org>
Sent: Saturday, May 15, 2021 6:47 PM
To: <Toasters@teaparty.net> <Toasters@teaparty.net>
Subject: 4 node cluster - only two nodes coming up
 


Hi all,

We're in the middle of hell, where our 4-node FAS8060 cluster was
shutdown cleanly for a move, but only one pair made it onto the truck
to the new DC.  Luckily I have all the volumes snapmirrored between
the two pairs of nodes and their aggregates. 

But now I need to bring up the pair that made the trip, figure out
which mirrors are source and which are destination on this pair, and
then break the destination ones so I can promote them to read-write.

This is not something I've practiced, and I wonder that if I have
volume foo, mounted on /foo, and it's snapmirror is volume foo_sm,
when I do the break, will it automatically mount to /foo?  I guess
I'll find out later tonight, and I can just unmount and remount. 

I think this is all good with just a simple 'snapmirror break ...' but
then when we get the chance to rejoin the other two nodes into the
cluster down the line, I would asusme I just have to (maybe) wipe the
old nodes and rejoin them one at a time.  Mostly because by that point
I can't have the original source volumes come up and cause us to lose
all the writes that have happened on the now writeable destination
volumes. 

And of course there's the matter of getting epsilon back up and
working on the two node cluster when I reboot it.  Along with all the
LIFs, etc.  Not going to be a fun time.  Not at all...

And of course we're out of support with Netapp.  Sigh...

And who knows if the pair that came down won't lose some disks and end
up losing one or more aggregates as well.  Stressful times for sure.

So I'm just venting here, but any suggestions or tricks would be
helpful.

And of course I'm not sure if the cluster switches made it down here
yet.

Never put your DC on the second floor if there isn't a second freight
elevator.  Or elevator in general.  Sigh...

John
_______________________________________________
Toasters mailing list
Toasters@teaparty.net
https://www.teaparty.net/mailman/listinfo/toasters