Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB of CIFS home dir data volumes backing up to tape via NDMP. And the tape backup is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point, the LTO-3 is shoe-shining as the minimum sustained throughput on that drive is 40mbps. And also, does anybody have any experience with NDMP to tell us why this is happening?
Thanks
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB of CIFS home dir data volumes backing up to tape via NDMP. And the tape backup is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point, the LTO-3 is shoe-shining as the minimum sustained throughput on that drive is 40mbps. And also, does anybody have any experience with NDMP to tell us why this is happening?
Thanks
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that drive
is
40mbps. And also, does anybody have any experience with NDMP to tell us
why
this is happening?
Thanks
-- --tmac
Askm Ammar,
You don't happen to have volumes with millions of files do you? This is what we have and it is slow backing these up... We're only doing 2Mb/s on those so you are doing a lot better... :-)
BTW, our other volumes are doing 40+ Mb/s so number of files seem to be a factor...
Best regards,
--saiful
________________________________
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB of CIFS home dir data volumes backing up to tape via NDMP. And the tape backup is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point, the LTO-3 is shoe-shining as the minimum sustained throughput on that drive is 40mbps. And also, does anybody have any experience with NDMP to tell us why this is happening?
Thanks
-- --tmac
You might want to look into snapmirror to tape. It's makes backing up lots and lots of little files very fast. The drawback is no incremental backups anymore, every backup is a full backup. And restores are always a full restore. If that's ok with you, Vertias and Bakbone both support snapmirror to tape.
-Blake
On 12/18/06, saiful-rizal_muhd-ramli@agilent.com saiful-rizal_muhd-ramli@agilent.com wrote:
Askm Ammar,
You don't happen to have volumes with millions of files do you? This is what we have and it is slow backing these up… We're only doing 2Mb/s on those so you are doing a lot better… J
BTW, our other volumes are doing 40+ Mb/s so number of files seem to be a factor…
Best regards,
--saiful
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that drive
is
40mbps. And also, does anybody have any experience with NDMP to tell us
why
this is happening?
Thanks
-- --tmac
Hi Blake,
Thanks a lot for the suggestion! I was not aware of such a solution.
I think this would work well as our plan is to keep snapshots stretching back to 1-2 months so we needn't go back to tape unless it's something drastic..
Best regards, --saiful
-----Original Message----- From: Blake Golliher [mailto:thelastman@gmail.com] Sent: Tuesday, December 19, 2006 4:21 PM To: saiful-rizal_muhd-ramli@agilent.com Cc: ammar.zolkipli.lists@gmail.com; toasters@mathworks.com Subject: Re: V-Series with NDMP backup
You might want to look into snapmirror to tape. It's makes backing up lots and lots of little files very fast. The drawback is no incremental backups anymore, every backup is a full backup. And restores are always a full restore. If that's ok with you, Vertias and Bakbone both support snapmirror to tape.
-Blake
On 12/18/06, saiful-rizal_muhd-ramli@agilent.com saiful-rizal_muhd-ramli@agilent.com wrote:
Askm Ammar,
You don't happen to have volumes with millions of files do you? This is what we have and it is slow backing these up... We're only doing 2Mb/s on those so you are doing a lot better... J
BTW, our other volumes are doing 40+ Mb/s so number of files seem to be a factor...
Best regards,
--saiful
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that drive
is
40mbps. And also, does anybody have any experience with NDMP to tell us
why
this is happening?
Thanks
-- --tmac
Hello Blake, You mentioned Veritas supports Snapmirror to Tape. Can you tell me which version? I have been trying to find documentation on Veritas's site on this but haven't found any. Are you or anyone else on this list using SM to tape with Veritas? Hope you can share some info on how this is done. TIA -G
On 12/19/06, Blake Golliher thelastman@gmail.com wrote:
You might want to look into snapmirror to tape. It's makes backing up lots and lots of little files very fast. The drawback is no incremental backups anymore, every backup is a full backup. And restores are always a full restore. If that's ok with you, Vertias and Bakbone both support snapmirror to tape.
-Blake
On 12/18/06, saiful-rizal_muhd-ramli@agilent.com saiful-rizal_muhd-ramli@agilent.com wrote:
Askm Ammar,
You don't happen to have volumes with millions of files do you? This is what we have and it is slow backing these up… We're only doing 2Mb/s on those so you are doing a lot better… J
BTW, our other volumes are doing 40+ Mb/s so number of files seem to be a factor…
Best regards,
--saiful
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that drive
is
40mbps. And also, does anybody have any experience with NDMP to tell us
why
this is happening?
Thanks
-- --tmac
I'll admit I was pretty sure it was supported back in 04, when Netapp announced that partnership with Veritas. But I think I was wrong, or at least I can't find any support for snapmirror to tape for Vertias.
-Blake
On 12/19/06, Sto Rage(c) netbacker@gmail.com wrote:
Hello Blake, You mentioned Veritas supports Snapmirror to Tape. Can you tell me which version? I have been trying to find documentation on Veritas's site on this but haven't found any. Are you or anyone else on this list using SM to tape with Veritas? Hope you can share some info on how this is done. TIA -G
On 12/19/06, Blake Golliher thelastman@gmail.com wrote:
You might want to look into snapmirror to tape. It's makes backing up lots and lots of little files very fast. The drawback is no incremental backups anymore, every backup is a full backup. And restores are always a full restore. If that's ok with you, Vertias and Bakbone both support snapmirror to tape.
-Blake
On 12/18/06, saiful-rizal_muhd-ramli@agilent.com saiful-rizal_muhd-ramli@agilent.com wrote:
Askm Ammar,
You don't happen to have volumes with millions of files do you? This is what we have and it is slow backing these up… We're only doing 2Mb/s on those so you are doing a lot better… J
BTW, our other volumes are doing 40+ Mb/s so number of files seem to be a factor…
Best regards,
--saiful
From: owner-toasters@mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes through a cisco MDS san switch. We have 6 tapes in the system, and normally run 4 ndmp backupin parallel. The path to tape should be a 2gbps FC link. The back end infrastructure obviously was built to sustain a much higher bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI or over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design. Both primary and secondary filers are v-series, and snapvaulted. We have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king slow, I know).
Does anybody have experience how to speed up NDMP backup? At this point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that drive
is
40mbps. And also, does anybody have any experience with NDMP to tell us
why
this is happening?
Thanks
-- --tmac
I know for a fact that Snapmirror to tape (SM2T) is definitely being used with a large oil and gas multinational right now. But their backup manager of choice is bakbone. From marketing info, AFAIK, Veritas supports SM2T officially in NBU 6. If it was earlier than that, I would have little clue.
My team is also looking at SM2T as a solution should NDMP not be able to provide better performance. We need at least 60-70MB/s throughput, with 4 parallel backups to meet backup window SLA. But if all information in this list is true, then NDMP itself is the culprit, and we will have to shift to a different strategy.
I haven't had time to do the test, will post the results when its done.
On 12/20/06, Blake Golliher thelastman@gmail.com wrote:
I'll admit I was pretty sure it was supported back in 04, when Netapp announced that partnership with Veritas. But I think I was wrong, or at least I can't find any support for snapmirror to tape for Vertias.
-Blake
On 12/19/06, Sto Rage(c) netbacker@gmail.com wrote:
Hello Blake, You mentioned Veritas supports Snapmirror to Tape. Can you tell me which version? I have been trying to find documentation on Veritas's site on this but haven't found any. Are you or anyone else on this list using SM to tape with Veritas? Hope you can share some info on how this is done. TIA -G
On 12/19/06, Blake Golliher <thelastman@gmail.com > wrote:
You might want to look into snapmirror to tape. It's makes backing up lots and lots of little files very fast. The drawback is no incremental backups anymore, every backup is a full backup. And restores are always a full restore. If that's ok with you, Vertias and Bakbone both support snapmirror to tape.
-Blake
On 12/18/06, saiful-rizal_muhd-ramli@agilent.com saiful-rizal_muhd-ramli@agilent.com wrote:
Askm Ammar,
You don't happen to have volumes with millions of files do
you? This is
what we have and it is slow backing these up… We're only doing
2Mb/s on
those so you are doing a lot better… J
BTW, our other volumes are doing 40+ Mb/s so number of files seem to
be a
factor…
Best regards,
--saiful
From: owner-toasters@mathworks.com [mailto:
owner-toasters@mathworks.com] On
Behalf Of Ammar Zolkipli Sent: Tuesday, December 19, 2006 10:20 AM To: toasters@mathworks.com Subject: Re: V-Series with NDMP backup
tmac,
The tape drives are fiber channel, and the connectivity all goes
through a
cisco MDS san switch. We have 6 tapes in the system, and normally
run 4 ndmp
backupin parallel. The path to tape should be a 2gbps FC link. The
back end
infrastructure obviously was built to sustain a much higher
bandwidth.
I'll try to run the null device dump and come back with the results
Thanks!
On 12/18/06, tmac tmacmd@gmail.com wrote:
try dumping to a null device instead of tape to test raw speed
without tape.
pick one of your dumps, and then do something like: dump 0f null /vol/vol0
You can see how long the dump takes and compare it to tape.
How many simultaneous dumps do you do? are the tapes on FC or SCSI
or
over the net? how fast is the path to tape?
if it is over the net, did you check for any errors, like using
netstat -a?
On 12/18/06, Ammar Zolkipli ammar.zolkipli.lists@gmail.com wrote:
Gents,
Let me first explain the overall layout. I've got a D2D2T design.
Both
primary and secondary filers are v-series, and snapvaulted. We
have 16TB
of
CIFS home dir data volumes backing up to tape via NDMP. And the
tape
backup
is on LTO-3 running at sustained throughput of 20mbps, (F**king
slow, I
know).
Does anybody have experience how to speed up NDMP backup? At this
point,
the
LTO-3 is shoe-shining as the minimum sustained throughput on that
drive
is
40mbps. And also, does anybody have any experience with NDMP to
tell us
why
this is happening?
Thanks
-- --tmac
-
Ammar Zolkipli -
Blake Golliher -
saiful-rizal_muhd-ramli＠agilent.com -
Sto Rage© -
tmac