toasters

toasters@lists.teaparty.net

2 participants
13522 discussions

RE: Defragmentation
by Alan McLachlan 18 Dec '02

18 Dec '02

Hi Edward, It's a case of whether or not it makes sense... Firstly, since WAFL always writes to new locations (a basic reason for NetApp's outstanding write performance - avoidance of seeks), any defragmentation effort would quickly be wasted. Filers tend to stay in a state of "partially fragmented" as NVRAM write gathering keeps files in large contiguous "chunks". Secondly, since blocks in snapshots are shared with the active filesystem and several versions of a file are sharing blocks, defragmenting becomes an impossibly complex task. Even if you treated the active filesystem as the base for gathering file segments you would need to rearrange the pointers in the snapshot metadata inodes. Defragmentation would be VERY slow. Doing what you have done (copy off then back on again) may improve read transfer rate for some large files for a short time. But it would do little to nothing for transactional I/O, so for most applications there's no point. Also, this means that you have created new versions of the files that don't share blocks with any of your snapshots, so you lose huge amounts of disk space. Not knocking your intentions, it's just that this discussion has been coming up for years know, and the answer hasn't changed. If it had (or if it really mattered) NetApp would have done somthing about it long ago. regards, Alan McLachlan Senior Systems Engineer Storage Management Solutions ASI Solutions www.asi.com.au Ph +61 2 6230 1566 Fax +61 2 6230 5343 Mobile +61 428 655644 e-mail amclachlan(a)asi.com.au -----Original Message----- From: Edward Hibbert [mailto:EH@dataconnection.com] Sent: Wednesday, 18 December 2002 10:36 PM To: 'toasters(a)mathworks.com' Subject: Defragmentation I think it's general knowledge that once a NetApp gets above 90% full, performance starts to degrade quite noticeably. I've been testing an application recently and observed that this is the case on ONTAP 5.2: * When ~95% full, my performance test came up with a number of ~4 seconds. * Freeing up space to get to around 60% full, the number was ~3.5 seconds. * Copying off my data files and copying them back to defragment them, the number was 2.8s. The only thing I've been able to find about fragmentation is: * Some statements that WAFL tries hard to avoid fragmentation (which I'm sure it does) but can't always (which I'm sure of too). * Copying off data and copying it back again is the only way to resolve it. The latter solution doesn't seem practical if the data in question is supposed to be available 24*7. Some applications have a method of doing this at an application level, but does anyone know whether there are methods/future plans for background non-intrusive defragmentation at a filesystem level? Or is it just me who thinks this would be a good idea? Edward Hibbert Internet Applications Group Data Connection Ltd Tel: +44 131 662 1212 Fax: +44 131 662 1345 Email: eh(a)dataconnection.com Web: http://www.dataconnection.com **** ASI Solutions Disclaimer **** The material transmitted may contain confidential and/or privileged material and is intended only for the addressee. If you receive this in error, please notify the sender and destroy any copies of the material immediately. ASI will protect your Privacy according to the 10 Privacy Principles outlined under the new Privacy Act, Dec 2001. This email is also subject to copyright. Any use of or reliance upon this material by persons or entities other than the addressee is prohibited. E-mails may be interfered with, may contain computer viruses or other defects. Under no circumstances do we accept liability for any loss or damage which may result from your receipt of this message or any attachments. **** END OF MESSAGE ****

1 0

NDMPCOPY solved
by Haber, David J. 18 Dec '02

18 Dec '02

Let me thank everyone one more time. I was successful in copying file from the 840 filer (6.1.1R2) to the NearStore (6.2.1) without upgrading the 840. By running this command from the NearStore it worked: NEARSTORE> ndmpcopy -sa root:rootpassword -da root:rootpassword -st text -dt text 840:/vol/vol0/Test NearStore:/vol/vol0/test Thanks to everyone's input I realized I needed to add the -st option and, after awhile the obvious was revealed---the path needs is case sensitive. Once I had these two elements in place everything worked. So thanks again for everyone's help. I also now have fresh, new root passwords for all of my filers!!!

1 0

RE: Ndmpcopy
by Chitre, Amol 18 Dec '02

18 Dec '02

Hi David, Let me clear some of the confusion for you.... ndmpcopy command on the filer acts like an application. It was introduced in Ontap 6.2. Hence you need to have the filer running the ndmpcopy command at Ontap 6.2 or higher. You can run the ndmpcopy command on any one of the following filers: 1. the source filer for the copy [ source has to be atleast 6.2, destination atleast 5.3.7 ] 2. destination filer for the copy [ destination has to be atleast 6.2, source atleast 5.3.7 ] 3. filer which is neither the source nor destination. [ filer has to be atleast 6.2, source and destination atleast 5.3.7 ] Basically it makes 2 ndmp connections to the ndmp servers on the source filer and the destination filer. The source and destination must support ndmp version 3 or higher. Now, looking at your last email, I see you have waded through the initial issues. Currently, there seems to be trouble kicking off the dump. Check if the path supplied on the ndmpcopy command line is correct, and it exists. So from the command line, it seems like you are running the ndmpcopy command on a machine NEARSTORE. And you are copying (source) alifiler:/vol/vol0/test to (destination)pacsnas1:/vol/vol0/test. Let me know if this is what you intended. If so, please send me the ndmpd debug logs (level 50). Thanks, Amol. *-----Original Message----- *From: Haber, David J. [mailto:David.Haber@vtmednet.org] *Sent: Wednesday, December 18, 2002 10:31 AM *To: 'toasters(a)mathworks.com' *Subject: Ndmpcopy * * *Thanks to everyone for their responses. I am getting closer to *a solution. *The consensus from the NetApp engineers is that I need to *upgrade the source *filer (840) to at least version 6.2.1. Others have told me *that I should be *able to run ndmpcopy from a third machine and still be able to copy the *files from the 840 to the NearStore even though the 840 is not *at 6.2.1. *This is where I'm at before the upgrade: * *NEARSTORE> ndmpcopy -sa root:dedsam9ap -da root:dedsam9ap -st *text -dt text *alifiler:/vol/vol0/test p *acsnas1:/vol/vol0/test *Ndmpcopy: Starting copy [ 21 ] ... *Ndmpcopy: alifiler: Connect: Connection Established. *Ndmpcopy: pacsnas1: Connect: Connection Established. *Ndmpcopy: alifiler: Log: DUMP: *Ndmpcopy: alifiler: Log: cannot open /vol/vol0/test. *Ndmpcopy: alifiler: Log: DUMP: *Ndmpcopy: alifiler: Log: DUMP IS ABORTED *Ndmpcopy: alifiler: Notify: dump failed *Ndmpcopy: pacsnas1: Notify: restore failed *Ndmpcopy: Transfer failed *Ndmpcopy: Fatal error. Aborting ... * *I can make a connection but for some reason I can't open the *file on the *source. Any suggestions before I upgrade would be appreciated. *Also, I ran *the same command with ndmpd debug set to 50 and have those results. If *someone out there would like to look at this maybe they can understand *what's happening better than I. E-mail me off the list and I *will send a *copy of the debug output if you're up to it. *Thanks again for everyone's help. *

1 0

OOPS!!
by Haber, David J. 18 Dec '02

18 Dec '02

I have changed the root password on my filers! Thanks for pointing out my egregious oversight!

1 0

Ndmpcopy
by Haber, David J. 18 Dec '02

18 Dec '02

Thanks to everyone for their responses. I am getting closer to a solution. The consensus from the NetApp engineers is that I need to upgrade the source filer (840) to at least version 6.2.1. Others have told me that I should be able to run ndmpcopy from a third machine and still be able to copy the files from the 840 to the NearStore even though the 840 is not at 6.2.1. This is where I'm at before the upgrade: NEARSTORE> ndmpcopy -sa root:dedsam9ap -da root:dedsam9ap -st text -dt text alifiler:/vol/vol0/test p acsnas1:/vol/vol0/test Ndmpcopy: Starting copy [ 21 ] ... Ndmpcopy: alifiler: Connect: Connection Established. Ndmpcopy: pacsnas1: Connect: Connection Established. Ndmpcopy: alifiler: Log: DUMP: Ndmpcopy: alifiler: Log: cannot open /vol/vol0/test. Ndmpcopy: alifiler: Log: DUMP: Ndmpcopy: alifiler: Log: DUMP IS ABORTED Ndmpcopy: alifiler: Notify: dump failed Ndmpcopy: pacsnas1: Notify: restore failed Ndmpcopy: Transfer failed Ndmpcopy: Fatal error. Aborting ... I can make a connection but for some reason I can't open the file on the source. Any suggestions before I upgrade would be appreciated. Also, I ran the same command with ndmpd debug set to 50 and have those results. If someone out there would like to look at this maybe they can understand what's happening better than I. E-mail me off the list and I will send a copy of the debug output if you're up to it. Thanks again for everyone's help.

1 0

RE: Ndmpcopy again
by Kennedy, Jeffrey 18 Dec '02

18 Dec '02

I believe the problem is that ndmpcopy was only introduced as part of OT in 6.2; previously it required a platform specific binary. The original poster's 840 filer is running 6.1.1. ~JK > -----Original Message----- > From: Collin Park [mailto:collin@netapp.com] > Sent: Wednesday, December 18, 2002 7:47 AM > To: David J. Haber > Cc: 'toasters(a)mathworks.com' > Subject: Re: Ndmpcopy again > > >This is what I got: > >Ndmpcopy: starting copy [13] > >Ndmpcopy: Trouble connecting to destination NearStore > >Ndmpcopy: fatal error. Aborting.... > > Does your version of software support "ndmpd debug 50" ? If so, > issuing that command on both sides should tell you what IP address[es] > are being used/recommended, and at what point the connection isn't > connecting. > > === > Collin Park Not a statement of my employer. > 408-822-6878 3.1.039

2 1

RE: Ndmpcopy again
by Noll, Kevin 18 Dec '02

18 Dec '02

Dont know if this helps, but here is my how to on NDMP Task NDMP Copy Commands: options ndmpd Outputs the options for ndmp. Ndmpd status Status of NDMP Ndmpd on/off Switches ndmp on / off Options ndmpd.enable on Enables NDMP Options ndmpd.enable off Disables NDMP Options ndmpd.preferred_interface Specifies the interface through which ndmp would send data Steps to enable NDMP Options ndmpd.enable on Options ndmpd.preferred_interface <interface name> Commands to copy data through NDMP Volumes, Qtrees, Directories can be copied through NDMP using the below syntax ndmpcopy -sa username:password -da username:password <source> <destination> where sa and da are source/destination filer authentication <source> and <destination> are of the form [<filer>:]<path> Limitations: Individual file(s) cannot be copied through NDMP Not preferred for transfer for small data Kevin Noll tel +44 7801765042 Cap Gemini Ernst & Young -----Original Message----- From: Kennedy, Jeffrey [mailto:jkennedy@qualcomm.com] Sent: 18 December 2002 16:01 To: Collin Park; David J. Haber Cc: toasters(a)mathworks.com Subject: RE: Ndmpcopy again I believe the problem is that ndmpcopy was only introduced as part of OT in 6.2; previously it required a platform specific binary. The original poster's 840 filer is running 6.1.1. ~JK > -----Original Message----- > From: Collin Park [mailto:collin@netapp.com] > Sent: Wednesday, December 18, 2002 7:47 AM > To: David J. Haber > Cc: 'toasters(a)mathworks.com' > Subject: Re: Ndmpcopy again > > >This is what I got: > >Ndmpcopy: starting copy [13] > >Ndmpcopy: Trouble connecting to destination NearStore > >Ndmpcopy: fatal error. Aborting.... > > Does your version of software support "ndmpd debug 50" ? If so, > issuing that command on both sides should tell you what IP address[es] > are being used/recommended, and at what point the connection isn't > connecting. > > === > Collin Park Not a statement of my employer. > 408-822-6878 3.1.039 ******************************************************************************************** " This message contains information that may be privileged or confidential and is the property of the Cap Gemini Ernst & Young Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate, distribute, or use this message or any part thereof. If you receive this message in error, please notify the sender immediately and delete all copies of this message ". ********************************************************************************************

1 0

Ndmpcopy again
by Haber, David J. 18 Dec '02

18 Dec '02

I used this command from the Nearstore ndmpcopy -sa root:passwd -da root:passwd 840:/vol/vol0/test NearStore:/vol/vol0/ (I used the real names for the 840 and the Nearstore) This is what I got: Ndmpcopy: starting copy [13] Ndmpcopy: Trouble connecting to destination NearStore Ndmpcopy: fatal error. Aborting.... Ndmpcopy: Done

3 2

test1
by Roy Ruigrok 17 Dec '02

17 Dec '02

this is a test, please ignore __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com

1 0

Ndmpcopy
by Haber, David J. 17 Dec '02

17 Dec '02

I have to move 60G of data from a folder on an 840 filer to a folder on a NearStore. I want to use ndmpcopy however, I can't seem to get it to work. The 840 is running 6.1.1R1 and the NearStore is running 6.2.1. Does the 840 need to be upgraded to 6.2.1? Also, there is no ndmpcopy on the 840. Is this because of the downlevel version of software? NetApp tech support has not been helpful at all. I was on the phone with them about this issue for a few hours today and they couldn't answer these questions. Also, I think I got some bad advice on how to set up the /etc/hosts file and the /etc/hosts.equiv file. Any help would be appreciated.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters