What are the recommendations for the "best" way to preserve a copy data on a filer for future use?
Background: A company has ceased operations and wants to preserve copies of its data "just in case someone wants it in the future". All the interesting data has been copied to the a single "archive" filer. There are about eight volumes in using about 50TB (with dedupe but no compression). This will be backup copy #1. The other filers that previously held the data are being sold and scheduled to be wiped.
The data is a mixed of virtual machines and file shares.
Now I'm trying to see if if a way to make another copy of the data for safe keeping at a reasonable cost. Speed of restore would not be high the feature list of this secondary archive.
One of the first thoughts simply dump the data onto an AWS volume for "cold storage". Sane? Tips/tricks? Data presumably gets scrubbed to prevent bit rot. Conveniently available just about anywhere. Has a recurring cost but avoids some upfront cost.
Second thought is find a relatively inexpensive array of disks and dump the data to that. Most likely benefit is fastest to write to. Maybe cheapest but in the end all the data is on non-spinning disks without verification.
Tapes? I don't like tapes. Don't know if a working tape system available but if there was is this a good case for it. Tape static storage better than disk?
Thanks.
arnold
Amazon Glacier or maybe a bunch of LTO tapes.
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Arnold de Leon Sent: Friday, June 19, 2015 1:07 PM To: toasters Subject: Recommendations on preserving a filers data?
What are the recommendations for the "best" way to preserve a copy data on a filer for future use?
Background: A company has ceased operations and wants to preserve copies of its data "just in case someone wants it in the future". All the interesting data has been copied to the a single "archive" filer. There are about eight volumes in using about 50TB (with dedupe but no compression). This will be backup copy #1. The other filers that previously held the data are being sold and scheduled to be wiped.
The data is a mixed of virtual machines and file shares.
Now I'm trying to see if if a way to make another copy of the data for safe keeping at a reasonable cost. Speed of restore would not be high the feature list of this secondary archive.
One of the first thoughts simply dump the data onto an AWS volume for "cold storage". Sane? Tips/tricks? Data presumably gets scrubbed to prevent bit rot. Conveniently available just about anywhere. Has a recurring cost but avoids some upfront cost.
Second thought is find a relatively inexpensive array of disks and dump the data to that. Most likely benefit is fastest to write to. Maybe cheapest but in the end all the data is on non-spinning disks without verification.
Tapes? I don't like tapes. Don't know if a working tape system available but if there was is this a good case for it. Tape static storage better than disk?
Thanks.
arnold
"Arnold" == Arnold de Leon a-toasters@deleons.com writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
SMtape or 3rd party backups over NDMP are valid options, but these options preserve that filer at a point in time only so you'll have to keep multiple copies around if you needed to say restore system state to 7 years ago (plus have version compatible hardware to restore to as well).
You could also look at something like SnapVault where you set up custom schedules and retentions to basically never expire snapshots and execute them over cron, you'll need to plan carefully that the destination volume has a large enough aggregate to support potentially years of growth. SnapVault would also let you restore a single file (or VM) through something like Windows explorer or UX shell (mount the SV volume in your namespace and then mount/map on your client). If you use OnCommand Protection Manager to manage the relationships it also allows you to browse the snaps in a friendly interface as well.
Good luck!
________________________________________ From: toasters-bounces@teaparty.net [toasters-bounces@teaparty.net] on behalf of John Stoffel [john@stoffel.org] Sent: Friday, June 19, 2015 11:15 AM To: Arnold de Leon Cc: toasters Subject: Re: Recommendations on preserving a filers data?
"Arnold" == Arnold de Leon a-toasters@deleons.com writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
_______________________________________________ Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
You can always get something from demo depot or a referb filer. I have used qnap fort home office and is a pretty sweet little box. As mentioned glacier is cheap but you need to look at the monthly cost and or recovery costs via amazon.
Sent from my iPhone
On Jun 19, 2015, at 11:34 AM, Tony Bar tbar@BERKCOM.com wrote:
SMtape or 3rd party backups over NDMP are valid options, but these options preserve that filer at a point in time only so you'll have to keep multiple copies around if you needed to say restore system state to 7 years ago (plus have version compatible hardware to restore to as well).
You could also look at something like SnapVault where you set up custom schedules and retentions to basically never expire snapshots and execute them over cron, you'll need to plan carefully that the destination volume has a large enough aggregate to support potentially years of growth. SnapVault would also let you restore a single file (or VM) through something like Windows explorer or UX shell (mount the SV volume in your namespace and then mount/map on your client). If you use OnCommand Protection Manager to manage the relationships it also allows you to browse the snaps in a friendly interface as well.
Good luck!
From: toasters-bounces@teaparty.net [toasters-bounces@teaparty.net] on behalf of John Stoffel [john@stoffel.org] Sent: Friday, June 19, 2015 11:15 AM To: Arnold de Leon Cc: toasters Subject: Re: Recommendations on preserving a filers data?
"Arnold" == Arnold de Leon a-toasters@deleons.com writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Thank you for the replies so far. Some clarification and additional information.
The intent is that current archive filer will be turned off. That is going to be the primary backup and presumably the easiest to restore if some valid reason came about to try to retrieve that data. The second back up is the protection from the just in case the building housing it catches fire or enough disks fail to spin back up when the time comes.
There is probably a 3 to 6 month window where the potential value is high and it then start dropping as time goes by. So $400 to $600 a month on Glacier wouldn't be unreasonable for a year. Then a decision can be made to start dropping archives beyond that.
I don't have a rational case against tapes for this application although the person who may have to do this is not exactly excited about needing to jockey tapes.
arnold
On Fri, Jun 19, 2015 at 11:40 AM, Klise, Steve Steve.Klise@wwt.com wrote:
You can always get something from demo depot or a referb filer. I have used qnap fort home office and is a pretty sweet little box. As mentioned glacier is cheap but you need to look at the monthly cost and or recovery costs via amazon.
Sent from my iPhone
On Jun 19, 2015, at 11:34 AM, Tony Bar tbar@BERKCOM.com wrote:
SMtape or 3rd party backups over NDMP are valid options, but these
options preserve that filer at a point in time only so you'll have to keep multiple copies around if you needed to say restore system state to 7 years ago (plus have version compatible hardware to restore to as well).
You could also look at something like SnapVault where you set up custom
schedules and retentions to basically never expire snapshots and execute them over cron, you'll need to plan carefully that the destination volume has a large enough aggregate to support potentially years of growth. SnapVault would also let you restore a single file (or VM) through something like Windows explorer or UX shell (mount the SV volume in your namespace and then mount/map on your client). If you use OnCommand Protection Manager to manage the relationships it also allows you to browse the snaps in a friendly interface as well.
Good luck!
From: toasters-bounces@teaparty.net [toasters-bounces@teaparty.net] on
behalf of John Stoffel [john@stoffel.org]
Sent: Friday, June 19, 2015 11:15 AM To: Arnold de Leon Cc: toasters Subject: Re: Recommendations on preserving a filers data?
> "Arnold" == Arnold de Leon a-toasters@deleons.com writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Tapes will survive longer than disk that's offline, I think
On Fri, Jun 19, 2015 at 3:04 PM, Arnold de Leon a-toasters@deleons.com wrote:
Thank you for the replies so far. Some clarification and additional information.
The intent is that current archive filer will be turned off. That is going to be the primary backup and presumably the easiest to restore if some valid reason came about to try to retrieve that data. The second back up is the protection from the just in case the building housing it catches fire or enough disks fail to spin back up when the time comes.
There is probably a 3 to 6 month window where the potential value is high and it then start dropping as time goes by. So $400 to $600 a month on Glacier wouldn't be unreasonable for a year. Then a decision can be made to start dropping archives beyond that.
I don't have a rational case against tapes for this application although the person who may have to do this is not exactly excited about needing to jockey tapes.
arnold
On Fri, Jun 19, 2015 at 11:40 AM, Klise, Steve Steve.Klise@wwt.com wrote:
You can always get something from demo depot or a referb filer. I have used qnap fort home office and is a pretty sweet little box. As mentioned glacier is cheap but you need to look at the monthly cost and or recovery costs via amazon.
Sent from my iPhone
On Jun 19, 2015, at 11:34 AM, Tony Bar tbar@BERKCOM.com wrote:
SMtape or 3rd party backups over NDMP are valid options, but these
options preserve that filer at a point in time only so you'll have to keep multiple copies around if you needed to say restore system state to 7 years ago (plus have version compatible hardware to restore to as well).
You could also look at something like SnapVault where you set up custom
schedules and retentions to basically never expire snapshots and execute them over cron, you'll need to plan carefully that the destination volume has a large enough aggregate to support potentially years of growth. SnapVault would also let you restore a single file (or VM) through something like Windows explorer or UX shell (mount the SV volume in your namespace and then mount/map on your client). If you use OnCommand Protection Manager to manage the relationships it also allows you to browse the snaps in a friendly interface as well.
Good luck!
From: toasters-bounces@teaparty.net [toasters-bounces@teaparty.net] on
behalf of John Stoffel [john@stoffel.org]
Sent: Friday, June 19, 2015 11:15 AM To: Arnold de Leon Cc: toasters Subject: Re: Recommendations on preserving a filers data?
>> "Arnold" == Arnold de Leon a-toasters@deleons.com writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Yeah, agree with Basil. Snapmirror to tape (SMT) is probably going to be your friend in this case but in my opinion you will want to take a couple backups to be safe & if you have a way to test the backups even better.
If there is a way you can turn the shelves & head back on occasionally to make sure everything is good there it would probably buy you some peace of mind. I'd buy some extra spare disks too for the inevitable disk failures when you power it back on after a long period of being down.
On Jun 19, 2015, at 2:01 PM, Basil <basilberntsen@gmail.commailto:basilberntsen@gmail.com> wrote:
Tapes will survive longer than disk that's offline, I think
On Fri, Jun 19, 2015 at 3:04 PM, Arnold de Leon <a-toasters@deleons.commailto:a-toasters@deleons.com> wrote: Thank you for the replies so far. Some clarification and additional information.
The intent is that current archive filer will be turned off. That is going to be the primary backup and presumably the easiest to restore if some valid reason came about to try to retrieve that data. The second back up is the protection from the just in case the building housing it catches fire or enough disks fail to spin back up when the time comes.
There is probably a 3 to 6 month window where the potential value is high and it then start dropping as time goes by. So $400 to $600 a month on Glacier wouldn't be unreasonable for a year. Then a decision can be made to start dropping archives beyond that.
I don't have a rational case against tapes for this application although the person who may have to do this is not exactly excited about needing to jockey tapes.
arnold
On Fri, Jun 19, 2015 at 11:40 AM, Klise, Steve <Steve.Klise@wwt.commailto:Steve.Klise@wwt.com> wrote: You can always get something from demo depot or a referb filer. I have used qnap fort home office and is a pretty sweet little box. As mentioned glacier is cheap but you need to look at the monthly cost and or recovery costs via amazon.
Sent from my iPhone
On Jun 19, 2015, at 11:34 AM, Tony Bar <tbar@BERKCOM.commailto:tbar@BERKCOM.com> wrote:
SMtape or 3rd party backups over NDMP are valid options, but these options preserve that filer at a point in time only so you'll have to keep multiple copies around if you needed to say restore system state to 7 years ago (plus have version compatible hardware to restore to as well).
You could also look at something like SnapVault where you set up custom schedules and retentions to basically never expire snapshots and execute them over cron, you'll need to plan carefully that the destination volume has a large enough aggregate to support potentially years of growth. SnapVault would also let you restore a single file (or VM) through something like Windows explorer or UX shell (mount the SV volume in your namespace and then mount/map on your client). If you use OnCommand Protection Manager to manage the relationships it also allows you to browse the snaps in a friendly interface as well.
Good luck!
From: toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net [toasters-bounces@teaparty.netmailto:toasters-bounces@teaparty.net] on behalf of John Stoffel [john@stoffel.orgmailto:john@stoffel.org] Sent: Friday, June 19, 2015 11:15 AM To: Arnold de Leon Cc: toasters Subject: Re: Recommendations on preserving a filers data?
"Arnold" == Arnold de Leon <a-toasters@deleons.commailto:a-toasters@deleons.com> writes:
Arnold> What are the recommendations for the "best" way to preserve a Arnold> copy data on a filer for future use?
The problem with doing this, is that you haven't specified the time frame of how long they want to archive it.
Arnold> Background: A company has ceased operations and wants to Arnold> preserve copies of its data "just in case someone wants it in Arnold> the future". All the interesting data has been copied to the Arnold> a single "archive" filer. There are about eight volumes in Arnold> using about 50TB (with dedupe but no compression). This will Arnold> be backup copy #1. The other filers that previously held the Arnold> data are being sold and scheduled to be wiped.
I'd probably just dump the data to tape, ideally using an application aware tool, but in this case, just getting copies of the VMDKs is probably enough.
Arnold> The data is a mixed of virtual machines and file shares.
Blech...
Arnold> Now I'm trying to see if if a way to make another copy of the Arnold> data for safe keeping at a reasonable cost. Speed of restore Arnold> would not be high the feature list of this secondary archive.
Can't really beat tape for cost... esp if you want to be paranoid and make TWO copies to tape, so they you can shutoff the original filer as well to save on electricity and cooling costs.
Not sure why you hate tape, plain ole 'dump' from the filer console will do what you want. Esp if you dump each volume to it's own tape(s) to keep it easy to restore.
John
Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
_______________________________________________ Toasters mailing list Toasters@teaparty.netmailto:Toasters@teaparty.net http://www.teaparty.net/mailman/listinfo/toasters
Not sure of costs e.t.c but may want to consider using AWS S3 or Glacier depending on restore time needed e.t.c
You can now NDMP into S3:
http://www.jk-47.com/2015/04/a-look-at-netapps-ndmp-cloud-extension-software...
or investigate using Altavault (Steelstore VM) into AWS or Glacier?
One point to mention is S3 and Glacier have scanners to check for bit rot which tapes don't.
-- View this message in context: http://network-appliance-toasters.10978.n7.nabble.com/Recommendations-on-pre... Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.
-
Arnold de Leon -
Basil -
John Stoffel -
Jordan Slingerland -
Klise, Steve -
Martin -
Tony Bar