Hello good toaster folks
We are planning to move a huge lot of data (> 3.0 T) from our 32 GB drives to 144 GB. I am carrying out some preliminary information about the quickest path to move the data across - the first option is to use ndmpcopy and the second is to copy it from shelf to shelf over fiber. Since this is a preliminary investigation, can someone please point me to relevant resources that may be helpful? I am after statistics of transfer rates between the two and perhaps other materials that others have found useful.
I have googled and looked through past toaster archives, and most of it is either outdated or lack of relevant information.
Thanks.
Clarence.
************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries. **************************************************
Clarence,
IAre the disks within the same box? If not...then snapmirror is a good way to go. You might be able to snapmirror within the same box. Performance really depends on how many disks you have in the volume(s) and what kind of head (700, 800, 900) that you have.
If you are able to snapmirror, then the down time can be on the order of minutes. It has become my favorite method for moving data. I am not positive about the source and destination being on the same box though.
Rough numbers though, you could get 60MByte/sec xfer rate on 820s. If you are on 960s expect to be on the order of 120MBytes/sec. But then it does depend on how many drives in the raid group and volumes.
Joe
Ngoh, Clarence wrote:
Hello good toaster folks
We are planning to move a huge lot of data (> 3.0 T) from our 32 GB drives to 144 GB. I am carrying out some preliminary information about the quickest path to move the data across - the first option is to use ndmpcopy and the second is to copy it from shelf to shelf over fiber. Since this is a preliminary investigation, can someone please point me to relevant resources that may be helpful? I am after statistics of transfer rates between the two and perhaps other materials that others have found useful.
I have googled and looked through past toaster archives, and most of it is either outdated or lack of relevant information.
Thanks.
Clarence.
************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
Clarence, Can you explain what you meant by "copy it from shelf to shelf over fiber"? How does one do this? Vol copy?? Doesn't ndmp copy also do it over fiber if one uses it to copy within the filer?
Last month we used ndmpcopy to merge 3 Qtrees into 1, all within the same volume. 555GB took about 3 1/2 hours. Using sysstat we didn't notice any network traffic, it was always disk I/Os (CPU was at 100% almost all the time though).
Here's the snip from our logs: ( unfortunately we didn't capture sysstat output during this copy )
dmp Sat Jul 19 10:43:01 PDT ndmpcopy:/vol/vol1/personal2/ /vol/vol1/personal/ filer-ip filer-ip(0) Start (Level 0, NDMP) rst Sat Jul 19 12:52:53 PDT /vol/vol1/personal/ End (272272 files, 276081 MB) dmp Sat Jul 19 12:52:53 PDT ndmpcopy:/vol/vol1/personal2/ /vol/vol1/personal/ filer-ip filer-ip(0) End (276081 MB)
dmp Sat Jul 19 11:35:36 PDT ndmpcopy:/vol/vol1/personal1/ /vol/vol1/personal/ filer-ip filer-ip(1) Start (Level 0, NDMP) rst Sat Jul 19 14:12:29 PDT /vol/vol1/personal/ End (411481 files, 292227 MB) dmp Sat Jul 19 14:12:29 PDT ndmpcopy:/vol/vol1/personal1/ /vol/vol1/personal/ filer-ip filer-ip(1) End (292227 MB)
-G
"Ngoh, Clarence" clarence.ngoh@cba.com.au wrote in message news:C37147635CECD611BA430002A5CDD38905333E76@C205960S12...
Hello good toaster folks
We are planning to move a huge lot of data (> 3.0 T) from our 32 GB drives to 144 GB. I am carrying out some preliminary information about the quickest path to move the data across - the first option is to use
ndmpcopy
and the second is to copy it from shelf to shelf over fiber. Since this
is
a preliminary investigation, can someone please point me to relevant resources that may be helpful? I am after statistics of transfer rates between the two and perhaps other materials that others have found useful.
I have googled and looked through past toaster archives, and most of it is either outdated or lack of relevant information.
Thanks.
Clarence.
************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains
information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.
Clarence, Can you explain what you meant by "copy it from shelf to shelf over fiber"? How does one do this? Vol copy?? Doesn't ndmp copy also do it over fiber if one uses it to copy within the filer?
Last month we used ndmpcopy to merge 3 Qtrees into 1, all within the same volume. 555GB took about 3 1/2 hours. Using sysstat we didn't notice any network traffic, it was always disk I/Os (CPU was at 100% almost all the time though).
Here's the snip from our logs: ( unfortunately we didn't capture sysstat output during this copy )
dmp Sat Jul 19 10:43:01 PDT ndmpcopy:/vol/vol1/personal2/ /vol/vol1/personal/ filer-ip filer-ip(0) Start (Level 0, NDMP) rst Sat Jul 19 12:52:53 PDT /vol/vol1/personal/ End (272272 files, 276081 MB) dmp Sat Jul 19 12:52:53 PDT ndmpcopy:/vol/vol1/personal2/ /vol/vol1/personal/ filer-ip filer-ip(0) End (276081 MB)
dmp Sat Jul 19 11:35:36 PDT ndmpcopy:/vol/vol1/personal1/ /vol/vol1/personal/ filer-ip filer-ip(1) Start (Level 0, NDMP) rst Sat Jul 19 14:12:29 PDT /vol/vol1/personal/ End (411481 files, 292227 MB) dmp Sat Jul 19 14:12:29 PDT ndmpcopy:/vol/vol1/personal1/ /vol/vol1/personal/ filer-ip filer-ip(1) End (292227 MB)
-G
----- Original Message ----- From: "Ngoh, Clarence" clarence.ngoh@cba.com.au To: toasters@mathworks.com Sent: Thursday, July 31, 2003 10:27 PM Subject: NDMPcopy and fibre disk shelve file copy.
Hello good toaster folks
We are planning to move a huge lot of data (> 3.0 T) from our 32 GB drives to 144 GB. I am carrying out some preliminary information about the quickest path to move the data across - the first option is to use
ndmpcopy
and the second is to copy it from shelf to shelf over fiber. Since this
is
a preliminary investigation, can someone please point me to relevant resources that may be helpful? I am after statistics of transfer rates between the two and perhaps other materials that others have found useful.
I have googled and looked through past toaster archives, and most of it is either outdated or lack of relevant information.
Thanks.
Clarence.
************** IMPORTANT MESSAGE ************** This e-mail message is intended only for the addressee(s) and contains
information which may be confidential. If you are not the intended recipient please advise the sender by return email, do not use or disclose the contents, and delete the message and any attachments from your system. Unless specifically indicated, this email does not constitute formal advice or commitment by the sender or the Commonwealth Bank of Australia (ABN 48 123 123 124) or its subsidiaries.