Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris
Are you migrating to a system with new disk? Or are you keeping the current data on the current disk? Do you have cluster switches that came with the 8200?
On Tue, Mar 28, 2017 at 12:57 PM, Chris Goodchris@g2.nu wrote: Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
We're keeping data on the current disk but clearly we have to buy new disk for the root volumes since Netapp won't let you use ADP on the 8200. Converting the FAS2552 to a shelf is the endgame.
Currently we don't have cluster switches but by the sounds of it that is going to be an essential component in the migration. What the steps are for that migration is the bit that nobody seems to be able to figure out :(
On 28 March 2017 at 18:03, Fred Grieco fredgrieco@yahoo.com wrote:
Are you migrating to a system with new disk? Or are you keeping the current data on the current disk?
Do you have cluster switches that came with the 8200?
On Tue, Mar 28, 2017 at 12:57 PM, Chris Good chris@g2.nu wrote: Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
You'll need at least 4 10GB switch ports - the process to convert from switchless to switched isn't that bad: http://www.rejected.io/2014/09/netapp-cluster-mode-converting-switchless-to-...
Assuming you have enough capacity on the new disk purchased with the 8200, your best bet is to move all the volumes to new disk and then swing the other shelves (including the converted 2552).
Disabling ADP on those disks was, IIRC, a pain when I've done it in the past. I think I would get all the data \ other shelves to the 8200, and then before converting the 2552 to a shelf, disable ADP and wipe disks and remove ownership. From loader, I believe the command was: setenv root-uses-shared-disks? false
As ADP is only supported on the on-board disk, I'm assuming that you have one or more aggregates that can be moved. https://library.netapp.com/ecmdocs/ECMP1196986/html/GUID-85C8CF29-1FF6-4DC1-... is the procedure to swing the disk manually if you aren't logically moving the data onto new disks.
On Tue, Mar 28, 2017 at 1:44 PM, Chris Good chris@g2.nu wrote:
We're keeping data on the current disk but clearly we have to buy new disk for the root volumes since Netapp won't let you use ADP on the 8200. Converting the FAS2552 to a shelf is the endgame.
Currently we don't have cluster switches but by the sounds of it that is going to be an essential component in the migration. What the steps are for that migration is the bit that nobody seems to be able to figure out :(
On 28 March 2017 at 18:03, Fred Grieco fredgrieco@yahoo.com wrote:
Are you migrating to a system with new disk? Or are you keeping the current data on the current disk?
Do you have cluster switches that came with the 8200?
On Tue, Mar 28, 2017 at 12:57 PM, Chris Good chris@g2.nu wrote: Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris _______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
If not for the onboard adp, Aggregate relocation (arl) may have been an option to swing the disks over to another system in place - - maybe. But on this case, I think you'll need some swing disk to make this work.
On Tue, Mar 28, 2017 at 3:09 PM, Mike Gossettcmgossett@gmail.com wrote: You'll need at least 4 10GB switch ports - the process to convert from switchless to switched isn't that bad: http://www.rejected.io/2014/09/netapp-cluster-mode-converting-switchless-to-... Assuming you have enough capacity on the new disk purchased with the 8200, your best bet is to move all the volumes to new disk and then swing the other shelves (including the converted 2552). Disabling ADP on those disks was, IIRC, a pain when I've done it in the past. I think I would get all the data \ other shelves to the 8200, and then before converting the 2552 to a shelf, disable ADP and wipe disks and remove ownership. From loader, I believe the command was: setenv root-uses-shared-disks? false As ADP is only supported on the on-board disk, I'm assuming that you have one or more aggregates that can be moved. https://library.netapp.com/ecmdocs/ECMP1196986/html/GUID-85C8CF29-1FF6-4DC1-... is the procedure to swing the disk manually if you aren't logically moving the data onto new disks.
On Tue, Mar 28, 2017 at 1:44 PM, Chris Good chris@g2.nu wrote:
We're keeping data on the current disk but clearly we have to buy new disk for the root volumes since Netapp won't let you use ADP on the 8200. Converting the FAS2552 to a shelf is the endgame.
Currently we don't have cluster switches but by the sounds of it that is going to be an essential component in the migration. What the steps are for that migration is the bit that nobody seems to be able to figure out :(
On 28 March 2017 at 18:03, Fred Grieco fredgrieco@yahoo.com wrote:
Are you migrating to a system with new disk? Or are you keeping the current data on the current disk? Do you have cluster switches that came with the 8200?
On Tue, Mar 28, 2017 at 12:57 PM, Chris Goodchris@g2.nu wrote: Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris ______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailma n/listinfo/toasters
______________________________ _________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/ mailman/listinfo/toasters
On 28 March 2017 at 20:09, Mike Gossett cmgossett@gmail.com wrote:
You'll need at least 4 10GB switch ports - the process to convert from switchless to switched isn't that bad: http://www.rejected.io/20 14/09/netapp-cluster-mode-converting-switchless-to-switched-cluster/
Yep, that bit looks easy enough.
Assuming you have enough capacity on the new disk purchased with the 8200, your best bet is to move all the volumes to new disk and then swing the other shelves (including the converted 2552).
That might be an option. The inbuild drives on the 2552 are 4*SSD + 20*900G SAS (which house root) plus another shelf of bulk SATA storage. With the 8200 we'll have another 24*960GB SSDs, so if we have R-D2 partitioning on there then we should squeek by with enough space on the SSDs to migrate the volumes from the builtin drives - Yay!.
I think I would get all the data \ other shelves to the 8200, and then
before converting the 2552 to a shelf, disable ADP and wipe disks and remove ownership. From loader, I believe the command was: setenv root-uses-shared-disks? false
Gotcha, so we wipe the builtin disks, add them back to the 8200, then we can migrate the volumes back off the SSDs and onto the SAS drives. Migrating the bulk storage shelf and SVMs should be trivial
Thanks for that rundown, seems to make sense to me.
Out of interest would this be significantly simpler if rather than using SSDs and R-D2 we had another shelf of SAS drives for dedicated root drives?
Chris
I don't know if it really makes it any more or less simple. If you are going to end up with a long term SAS disk footprint, then if it were me, i'd rather be using those SAS disks for my root than SSDs. Sounds like you're going to have at least 20x900G SAS disks in your end-state configuration, so you might want to evaluate buying another some more SAS disk to hold that.
I haven't looked at the latest changes, but historically you had the choice of AFF or flash-pool \ hybrid aggregates. Are you sure that you are going to be able to have an all-flash shelf on the same system as hybrid \ spinning-disk shelves? As of 6 months ago, this didn't fly.
On Tue, Mar 28, 2017 at 4:03 PM, Chris Good chris@g2.nu wrote:
On 28 March 2017 at 20:09, Mike Gossett cmgossett@gmail.com wrote:
You'll need at least 4 10GB switch ports - the process to convert from switchless to switched isn't that bad: http://www.rejected.io/20 14/09/netapp-cluster-mode-converting-switchless-to-switched-cluster/
Yep, that bit looks easy enough.
Assuming you have enough capacity on the new disk purchased with the 8200, your best bet is to move all the volumes to new disk and then swing the other shelves (including the converted 2552).
That might be an option. The inbuild drives on the 2552 are 4*SSD + 20*900G SAS (which house root) plus another shelf of bulk SATA storage. With the 8200 we'll have another 24*960GB SSDs, so if we have R-D2 partitioning on there then we should squeek by with enough space on the SSDs to migrate the volumes from the builtin drives - Yay!.
I think I would get all the data \ other shelves to the 8200, and then
before converting the 2552 to a shelf, disable ADP and wipe disks and remove ownership. From loader, I believe the command was: setenv root-uses-shared-disks? false
Gotcha, so we wipe the builtin disks, add them back to the 8200, then we can migrate the volumes back off the SSDs and onto the SAS drives. Migrating the bulk storage shelf and SVMs should be trivial
Thanks for that rundown, seems to make sense to me.
Out of interest would this be significantly simpler if rather than using SSDs and R-D2 we had another shelf of SAS drives for dedicated root drives?
Chris
"Mike" == Mike Gossett cmgossett@gmail.com writes:
Mike> I don't know if it really makes it any more or less simple. If Mike> you are going to end up with a long term SAS disk footprint, Mike> then if it were me, i'd rather be using those SAS disks for my Mike> root than SSDs. Sounds like you're going to have at least Mike> 20x900G SAS disks in your end-state configuration, so you might Mike> want to evaluate buying another some more SAS disk to hold Mike> that.
Mike> I haven't looked at the latest changes, but historically you had Mike> the choice of AFF or flash-pool \ hybrid aggregates. Are you Mike> sure that you are going to be able to have an all-flash shelf on Mike> the same system as hybrid \ spinning-disk shelves? As of 6 Mike> months ago, this didn't fly. *
I don't know why you think so. I've been running an 8060 four node cluster with a shelf of SSDs with most of them in an aggregate, along with a couple or three disks (I forget) were setup in a FlashPool. But since we also had FlashCache, it was explained to us that they didn't really work like we thought/wanted, so we just nuked the FlashPools and added the disks back into the SSD aggregate.
John
Just based on what I was told by a NetApp sales rep. Very glad to hear that this is a supported configuration. It had been explained to me ~6-8 months ago that all-ssd aggregates were only supported by AFF systems, which would not support hybrid or spinning disk shelves, and that if one needed an all-ssd aggregate (flash-cache\pool not sufficient) , adding an AFF HA pair to a cluster was the only way to go
On Tue, Mar 28, 2017 at 6:37 PM, John Stoffel john@stoffel.org wrote:
"Mike" == Mike Gossett cmgossett@gmail.com writes:
Mike> I don't know if it really makes it any more or less simple. If Mike> you are going to end up with a long term SAS disk footprint, Mike> then if it were me, i'd rather be using those SAS disks for my Mike> root than SSDs. Sounds like you're going to have at least Mike> 20x900G SAS disks in your end-state configuration, so you might Mike> want to evaluate buying another some more SAS disk to hold Mike> that.
Mike> I haven't looked at the latest changes, but historically you had Mike> the choice of AFF or flash-pool \ hybrid aggregates. Are you Mike> sure that you are going to be able to have an all-flash shelf on Mike> the same system as hybrid \ spinning-disk shelves? As of 6 Mike> months ago, this didn't fly. *
I don't know why you think so. I've been running an 8060 four node cluster with a shelf of SSDs with most of them in an aggregate, along with a couple or three disks (I forget) were setup in a FlashPool. But since we also had FlashCache, it was explained to us that they didn't really work like we thought/wanted, so we just nuked the FlashPools and added the disks back into the SSD aggregate.
John
I think this is still true; I don’t have the exact details, but you can, of course, add an SSD only aggregate to an existing filer, but you should always try to keep different storage tiers on separate controllers. F.ex. if you want to run SAS, SATA and SSD aggregates, you should have a 4-node cluster with an AFF HA pair, and one pair where one of the controllers is serving SAS primarily and the other is serving SATA.
There are multiple reasons (so I’ve been told) for doing that, one of the most important being that especially on AFF systems, a lot of internal configuration values and timeout settings have been removed which were designed for disk-based systems, because they simply do not apply for all-flash systems. When you mix SAS and SATA aggregates on the same controller, please keep in mind that both are running off the same consistency point, so slow SATA writes will automatically also slow down your SAS write throughput – and that will even be worse when you mix up SATA and SSD aggregates on the same controller.
Personally, I try to avoid mixing storage tiers on one controller because what has been told to me by some of the NetApp Techs on several occasions and what I’ve read about mixing workloads and disk types etc. sounds reasonable enough to just not do it if not absolutely necessary.
Best,
Alexander Griesser Head of Systems Operations
ANEXIA Internetdienstleistungs GmbH
E-Mail: AGriesser@anexia-it.commailto:AGriesser@anexia-it.com Web: http://www.anexia-it.comhttp://www.anexia-it.com/
Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt Geschäftsführer: Alexander Windbichler Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Mike Gossett Gesendet: Mittwoch, 29. März 2017 16:16 An: John Stoffel john@stoffel.org Cc: toasters@teaparty.net Betreff: Re: FAS2552 -> FAS8200 migration
Just based on what I was told by a NetApp sales rep. Very glad to hear that this is a supported configuration. It had been explained to me ~6-8 months ago that all-ssd aggregates were only supported by AFF systems, which would not support hybrid or spinning disk shelves, and that if one needed an all-ssd aggregate (flash-cache\pool not sufficient) , adding an AFF HA pair to a cluster was the only way to go
On Tue, Mar 28, 2017 at 6:37 PM, John Stoffel <john@stoffel.orgmailto:john@stoffel.org> wrote:
"Mike" == Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> writes:
Mike> I don't know if it really makes it any more or less simple. If Mike> you are going to end up with a long term SAS disk footprint, Mike> then if it were me, i'd rather be using those SAS disks for my Mike> root than SSDs. Sounds like you're going to have at least Mike> 20x900G SAS disks in your end-state configuration, so you might Mike> want to evaluate buying another some more SAS disk to hold Mike> that.
Mike> I haven't looked at the latest changes, but historically you had Mike> the choice of AFF or flash-pool \ hybrid aggregates. Are you Mike> sure that you are going to be able to have an all-flash shelf on Mike> the same system as hybrid \ spinning-disk shelves? As of 6 Mike> months ago, this didn't fly. *
I don't know why you think so. I've been running an 8060 four node cluster with a shelf of SSDs with most of them in an aggregate, along with a couple or three disks (I forget) were setup in a FlashPool. But since we also had FlashCache, it was explained to us that they didn't really work like we thought/wanted, so we just nuked the FlashPools and added the disks back into the SSD aggregate.
John
Root-data-(data) partitions are supported only on all-SSD systems; as you have SAS/SATA disks you cannot use them – at least in supported way. Also cluster switches must be supported by NetApp and run validated configuration and be dedicated switches – which means you cannot just use arbitrary 4 10G ports on your existing switches (again, it will technically work, just if you hit a problem you will have hard time to get it through support).
What you can do in – as far as I can tell – supported way is to use ARL to replace controllers. High level steps are
1. Connect new shelf to 2552, move data from internal disks to new shelf. Make sure to leave 4 disks for two root aggregates
2. Move root aggregates to new shelf: https://kb.netapp.com/support/s/article/ka31A00000013ok/How-to-non-disruptiv...
3. Replace controllers: https://library.netapp.com/ecm/ecm_download_file/ECMP1995386
4. Repurpose 2552 as shelf and redistribute volumes as you see fit
All of this can be done without cluster outage.
--- With best regards
Andrei Borzenkov Senior system engineer FTS WEMEAI RUC RU SC TMS FOS [cid:image001.gif@01CBF835.B3FEDA90] FUJITSU Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation Tel.: +7 495 730 62 20 ( reception) Mob.: +7 916 678 7208 Fax: +7 495 730 62 14 E-mail: Andrei.Borzenkov@ts.fujitsu.commailto:Andrei.Borzenkov@ts.fujitsu.com Web: ru.fujitsu.comhttp://ts.fujitsu.com/ Company details: ts.fujitsu.com/imprinthttp://ts.fujitsu.com/imprint.html This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation. Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Chris Good Sent: Wednesday, March 29, 2017 12:04 AM To: Mike Gossett Cc: toasters@teaparty.net Subject: Re: FAS2552 -> FAS8200 migration
On 28 March 2017 at 20:09, Mike Gossett <cmgossett@gmail.commailto:cmgossett@gmail.com> wrote: You'll need at least 4 10GB switch ports - the process to convert from switchless to switched isn't that bad: http://www.rejected.io/2014/09/netapp-cluster-mode-converting-switchless-to-...
Yep, that bit looks easy enough.
Assuming you have enough capacity on the new disk purchased with the 8200, your best bet is to move all the volumes to new disk and then swing the other shelves (including the converted 2552).
That might be an option. The inbuild drives on the 2552 are 4*SSD + 20*900G SAS (which house root) plus another shelf of bulk SATA storage. With the 8200 we'll have another 24*960GB SSDs, so if we have R-D2 partitioning on there then we should squeek by with enough space on the SSDs to migrate the volumes from the builtin drives - Yay!. I think I would get all the data \ other shelves to the 8200, and then before converting the 2552 to a shelf, disable ADP and wipe disks and remove ownership. From loader, I believe the command was: setenv root-uses-shared-disks? false
Gotcha, so we wipe the builtin disks, add them back to the 8200, then we can migrate the volumes back off the SSDs and onto the SAS drives. Migrating the bulk storage shelf and SVMs should be trivial Thanks for that rundown, seems to make sense to me.
Out of interest would this be significantly simpler if rather than using SSDs and R-D2 we had another shelf of SAS drives for dedicated root drives? Chris
On 29 March 2017 at 08:07, andrei.borzenkov@ts.fujitsu.com < andrei.borzenkov@ts.fujitsu.com> wrote:
Root-data-(data) partitions are supported only on all-SSD systems; as you have SAS/SATA disks you cannot use them – at least in supported way.
Thanks for that insight, I really appreciate it. Feeling really let down by Netapp now for them to propose a solution that is unsupported! I will have words :(
Hi Chris
I suggest you work with your Netapp Partner to arrange for some swing gear if possible. A DS2246 shelf or two will allow you to unpick the ADP and setup dedicated root aggregates for the FAS2552 controllers. From there a head swap should hopefully be straight forward.
We're in the initial phases of planning a controller upgrade for our FAS2552 clusters (also upgrading to FAS8200). We were fortunate that we had enough disks at the start to remove ADP. At the time it was very new and we were aware of the problems it would pose for a controller upgrade. I know that's not very helpful but if there's anything we learn in the process that might help you I'll pass it on.
It would also be great if someone from Netapp could chime in and clarify how customers are supposed to upgrade entry level SAS/hybrid systems with ADP, particularly switch-less clusters. I hope that Netapp will support FAS8200s running with R-D ADP setups that are upgrades from entry systems.
Good luck and please let us know if you learn anything from Netapp that could help others.
Steve
On 29 March 2017 at 09:49, Chris Good chris@g2.nu wrote:
On 29 March 2017 at 08:07, andrei.borzenkov@ts.fujitsu.com < andrei.borzenkov@ts.fujitsu.com> wrote:
Root-data-(data) partitions are supported only on all-SSD systems; as you have SAS/SATA disks you cannot use them – at least in supported way.
Thanks for that insight, I really appreciate it. Feeling really let down by Netapp now for them to propose a solution that is unsupported! I will have words :(
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Just to complete the loop I've just had confirmation direct from Netapp that R-D2 is supported on mixed storage system. The "all-flash" requirement _only_ refers to the initial shelf/root volume. As long as that requirement is satisfied you can add whatever type of shelves you like. So FAS with a shelf of R-D2 SSDs can totally have SAS or SATA shelves added to it later. Now why they can't spell this out in a TR or communicate in a way that doesn't feel like pulling teeth is a mystery to me...
On 29 March 2017 at 09:49, Chris Good chris@g2.nu wrote:
On 29 March 2017 at 08:07, andrei.borzenkov@ts.fujitsu.com < andrei.borzenkov@ts.fujitsu.com> wrote:
Root-data-(data) partitions are supported only on all-SSD systems; as you have SAS/SATA disks you cannot use them – at least in supported way.
Thanks for that insight, I really appreciate it. Feeling really let down by Netapp now for them to propose a solution that is unsupported! I will have words :(
Chris,
What is your workload on the existing cluster? You can convert the 2552 to a switched cluster fairly easily ( https://library.netapp.com/ecm/ecm_get_file/ECMP1140536) - I'd personally schedule an outage for this, but it is not purely necessary. Once the systems are in the same cluster, migration of data \ SVMs \ LIFs are a fairly easy task and non-disruptive. If you are buying all new disks, or at least enough capacity temporarily before swinging shelves between heads, this is likely the approach I'd use.
Depending on the workload, given that you are seemingly replacing the 2552, you could also look to host-based tools like storage vmotion to move your datasets. Great for VMware environments; less so for CIFS \ etc.
On Tue, Mar 28, 2017 at 11:47 AM, Chris Good chris@g2.nu wrote:
Folks,
Currently we have a FAS2552 switchless cluster running 9.1. We're planning an upgrade to FAS8200 and considering our migration options. Netapp themselves aren't being terribly forthcoming on the best route to do this.
I had hoped that upgrading from a FAS25xx would be a common thing with a reference workflow to go with it. Maybe things are complicated somewhat because the FAS2552 runs ADP and the FAS8200 doesn't. Has anyone been through a similar process or have any recommendations for a route to achieve this?
Thanks,
Chris
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
-
Alexander Griesser -
andrei.borzenkov＠ts.fujitsu.com -
Chris Good -
Fred Grieco -
John Stoffel -
Mike Gossett -
Stephen Stocke