toasters

toasters@lists.teaparty.net

2 participants
13529 discussions

Has anyone seen this ndmpcopy error message
by Michael Homa 01 Mar '10

01 Mar '10

Hi: I'm hoping someone has seen this message. I've been working with netapp tech support but they, so far, have not been able to duplicate the error in their lab. I'll start by apologizing for the length of the email. Brevity can result in key details being left out; details that might otherwise explain the situation and lead to the solution. On the other hand, too many details can annoy the crap out of people. We have two netapps, both model FAS270. Netapp1, the source for ndmpcopy, runs 7.0.4; netapp2, the destination, netapp2, runs 7.2.7. We use ndmpcopy to copy volumes from netapp1 to netapp2. Intermittently (though it seems to occur more frequently now), the ndmpcopy fails to run and gives the following message: Body error NDMP_ILLEGAL_STATE_ERR in reply message NdmpMessageDataStartRecover from destination Feb 25 06:00:05 CST [ndmpc:233]: Failed to start restore on destination There is no pattern regarding on which volume or the size of the volume the ndmpcopy operation will fail. It can occur on one of our smaller volumes (1g) or on our largest (765g). The error will occur when the ndmpcopy operation is initiated from the console or when it is started from a perl script on the administrative host for the destination filer. Though the two netapps are physically located in separate buildings that are miles apart, they are on the same subnet and no other network issues regarding them have occurred. There is only one instance of the error message in the knowledgebase and it concerns a SAN library with BakBone NetVault with NDMP plugin. A review of the ndmpcopy spec at ndmp.org suggests one of the following: 1) Illegal state. A data operation is not currently in progress. 2) Tape record size cannot be set in the current state. 3) A data operation is already in progress. Only one data operation is allowed to be executing at a time. 4) Illegal state. NDMP server not in the halted state 5) Message cannot be processed in current state I checked the state of the ndmpd server on the destination, netapp2, and there are no other sessions. All the others are things I have no control over. Some of the things I've tried: o Toggled the ndmpd daemon off and on for both source and destination o Set the ndmpd debug level to 70 o Set the debug option on the ndmpcopy command o Set the ndmpd level to three (had been four) Regarding the detailed ndmpdlog, there is no explanation as to what causes the error (only the error): Feb 25 06:00:05 CST [ndmpd:73]: Restore type: dump Feb 25 06:00:05 CST [ndmpd:73]: Error code: NDMP_ILLEGAL_STATE_ERR <--- Feb 25 06:00:05 CST [ndmpd:73]: Key: EXTRACT, Value: N Feb 25 06:00:05 CST [ndmpd:73]: Key: EXCLUDE_PATH, Value: /etc Feb 25 06:00:05 CST [ndmpd:73]: Key: FILESYSTEM, Value: /vol/vol0 Feb 25 06:00:05 CST [ndmpd:73]: Key: BASE_DATE, Value: 9856860996 Feb 25 06:00:05 CST [ndmpd:73]: Key: UPDATE, Value: Y Feb 25 06:00:05 CST [ndmpd:73]: Key: LEVEL, Value: 2 <--- Feb 25 06:00:05 CST [ndmpd:73]: Key: DMP_NAME, Value: ndmpcopy:/vol/vol0//vol/vol0/128.248.155.101128.248.155.102 As you can see, this failure occurred on a level two operation; it has occurred on the other two levels as well. I believe, though I can say for certain, that the failure rate increased after upgrading ONTAP on the destination from 7.2.2 to 7.2.7. Please do not suggest we license snapmirror. We are a university in Illinois, a state whose financial situation is only slightly better than that of California. If I can't solve the problem, then *** yes *** I'll try vol copy. Michael Homa Operating Systems Support and Database Group Academic Computing and Communication Center University of Illinois at Chicago email: mhoma(a)uic.edu

2 1

aggr dedupe in 7.3?
by Christopher Moll 27 Feb '10

27 Feb '10

I can't seem to find a definite answer to this one... Can ontap 7.3 actually dedupe across a whole aggregate or only within one volume? I know the dedupe _data_ moves to the aggr in 7.3 but does that change how it works? We have many cases where databases (each in their own flex vol in the same aggr) are near-duplicates of each other, would this get de-duped? (seemingly a good case for flex-clones but unfortunately when they're generated they're usually copied from one netapp to another). Chris Moll

2 1

Re: Has anyone seen this ndmpcopy error message
by Michael Homa 25 Feb '10

25 Feb '10

On Thu, 25 Feb 2010, steve rieger wrote: > are you running a snapvault snap create at the same time ? Nope. We have licenses for only two products on both the source and destination: nfs and iscsi (and we don't even use iscsi). The destination machine has only one volume that is not a backup of the corresponding volume on the source. That single production volume is used for the Oracle Calendar product. The backup volumes have all been configured with no snapshots. Another person asked if we use any other backup product on our administrative hosts. Both admin hosts are IBM machines running a flavor of AIX. Both use the IBM Tivoli ADSM product for backups. Michael

1 0

SM SQL DB per LUN limitation (?) Thi
by Raj Patel 25 Feb '10

25 Feb '10

Hi all, It appears SM SQL 5 reduces the maximum number of databases from 100 (SM SQL 2) to 35 due to a 'Microsoft Limitation' on a storage volume. https://now.netapp.com/Knowledgebase/solutionarea.asp?id=kb53310 This article references http://support.microsoft.com/kb/943471/en-us If the NetApp Snapshot occurs at the Volume level nearly instantly via SnapDrive api call, what part of SM SQL uses VDI/VSS ? I understand that SM SQL has to queisce the DB momentarily for a consistent snapshot - is this when VDI/VSS is called ? It would appear that upgrading our SD/SM SQL version has resulted in a chunk of extra work in splitting out databases out onto more volumes/luns to accommodate this limitation. Its tempting to go back to the previous version. Most of the DB's are small MS CRM / SharePoint ones with minimal loading (ie the SQL box isn't being hammered and the DB's themselves aren't big). Before we go down this track of adding more drives and splitting out the databases (our DBA's are less than pleased at the prospect) have other people struck SM SQL 5 backup issues since upgrading due to the number of databases on a lun ? Thanks in advance, Raj.

1 0

PAM cards or disks and some questions about the impact of running with PAM
by Page, Jeremy 23 Feb '10

23 Feb '10

I apologize that this is so long but it's kind of complicated and I don't want to blow my budget on the wrong stuff. I've got a mixed use 3070A at 7.3.1P2D11 hosting FC based AIX boxes (about 4k IOPS average, 90+% reads), 300 or so VMs (NFS, 3k IOPS average) and 1,000 users home/departmental shares via CIFS and NFS (2500 IOPS on a busy day. Oracle is starting to get a bit pokey and I've budgeted for some upgrades this coming quarter. The question is should I be buying spindles or PAM cards. Currently this load is spread across 231 spindles in wide stripes (minimum 12 disks per RG). So not great performance but it also takes a big spike to have a major negative impact as well. I have good performance metrics for everything except Oracle, which for some reason is seeing poor performance even though latency and throughput look good. Of course there is no DBA running this database so I suspect at least part of the problem is on the Oracle side but for now I have to assume that's not going to improve. Currently we are already observing a very good cache hit % so I am questioning if the PAM cards will give any benefit there. On the other hand statit shows that 2/3 of the IO from disk is still reads so I think maybe there would be a benefit and I know it would be great for file shares & VMs (which are also on deduped volumes so it's my understanding the cache from the PAM cards would be even more effective). * How do I determine what to purchase? * What's does the "seek time" for data read from PAM? * I'm getting to the point where the 3070 is going to work really hard in a fail over situation, if PAM cards can service a lot more requests in a short period if the data is cached on SSD will this put more load on my CPUs? * Why did NetApp name them Pam? It's not the 60's, they should be named Shelby or Logan cards or something? Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance. In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email.

3 5

How mixed are you?
by David N. Blank-Edelman 22 Feb '10

22 Feb '10

Hi- I realize this may be a simple question, but I don't think I've ever seen an answer to it. We've been mulling over the idea of trying to reduce or eliminate the use of mixed-mode qtrees at our site because of some of the hassles they cause. One big question is how much our users would notice a change like that which leads to my real question: is there any clever way to determine just how "mixed" a mixed qtree is? (i.e stats on which way parts of it have flipped)? -- dNb

1 0

OSSV and Protection Manager
by steve klise 17 Feb '10

17 Feb '10

Current setup: Windows 2003 clients with OSSV agent 2.6.1 and host agent 2.7: V31xx series Filer all licenses needed Doing some testing with OSSV. I have the policy backup from primary to backup and keep for 30 days via Snapvault... I then snapmirror the volume over to a FAS3170. Pretty basic stuff. Question 1: I backup the guest. It takes about 1 hour, and PEGS the CPU of all VM's tested so far. I am not backing up the vmdk option, just the file system walk. No biggie to be honest. Here is the kicker. If I add a hard drive after I have done my initial backup/basline, the protection manager does not create a new SV relationship and protect the new resouce. I then go into NMC, Host tab, OSSV, select my test host, click ok the paths tab at the bottom, and it see the new drive, and SHOWS IT AS PROTECTED. Wrong!! I take a backup to see if a relationship is created, but is not. Very bad design. I would expect to see the resouce as unprotected, or give me some sort of alert. Maybe I am wrong. Am I missing something here? I legacy backup client would automatically pick up the new drive. Maybe I need to check something that is not obvious. Question 2: I have about 80 windows servers to backup. The thought was to put all the clients in 1 volume. I am expecting around 1 to 1.5TB of data. Maybe 30 of them will be VM's. I am concerned that although the 1st backup is the most intensive, will this impact if I kick off 30 VM's and the CPU's hit the roof. Not good I would guess, but anyone have any thoughts, opinions, I would love them. So far I have ~ 10 windows boxes. Seeing 25% dedupe. Love it. Sorry for being so verbose. Any help is appreciated. -- View this message in context: http://old.nabble.com/OSSV-and-Protection-Manager-tp27557891p27557891.html Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.

1 1

Windows Groups and Quotas
by ifstat 10 Feb '10

10 Feb '10

Hi I know in the past Windows Groups where not supported for quotas only UNIX Groups are, this is true upto ONTAP 7.2.6.1. Does anyone know if this has changed in newer ONTAP versions or even if there is a plan to modify it? Also is there an alternate solution to enabling quotas for Windows Group, i.e. possibly using the usermap.cfg file Thanks -- View this message in context: http://old.nabble.com/Windows-Groups-and-Quotas-tp27527110p27527110.html Sent from the Network Appliance - Toasters mailing list archive at Nabble.com.

2 1

RE: Changing a LUN type
by Page, Jeremy 10 Feb '10

10 Feb '10

It can be pretty signifigant, it's not as well documented as VMDK misalignment but I'd expect similar numbers. It looks like the Solaris configuration shouldn't be an issue, it's got a larger offset but still aligned the same way as an AIX LUN. ________________________________ From: MRWILLIS(a)UP.COM [mailto:MRWILLIS@UP.COM] Sent: Wednesday, February 03, 2010 10:49 PM To: Roach, Steven; Page, Jeremy; toasters Subject: Re: Changing a LUN type Anyone ever get an IO performance delta on the misalignment created between the file system and the LUN? I suppose it would depend on the amount of IO, but just curious. Thanks. ________________________________ From: "Roach, Steven" [Steven.Roach(a)netapp.com] Sent: 02/03/2010 10:07 PM EST To: "Page, Jeremy" <jeremy.page(a)gilbarco.com>; <toasters(a)mathworks.com> Subject: Re: Changing a LUN type Jeremy, The LUN type setup up characteristics for that LUN like On-Disk Layout, while this may be working for you, you might not have optimal performance. Once it is created, I don't think you can change the LUN type, you would have to create a new one (same size) with the correct LUN type and copy over the data. On 2/3/10 9:22 PM, "Page, Jeremy" <jeremy.page(a)gilbarco.com> wrote: I am probably missing something simple but I have an AIX system running off a LUN that's configured for a Solaris box. I have two questions - *What's the impact? Are they essentially the same or is there a difference between the two that's going to cause me grief *How do I change it? Any suggestions would be much appreciated. Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance. In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email. ** This message and any attachments contain information from Union Pacific which may be confidential and/or privileged. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited by law. If you receive this message in error, please contact the sender immediately and delete the message and any attachments. ** Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance. In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email.

1 0

Changing a LUN type
by Page, Jeremy 04 Feb '10

04 Feb '10

I am probably missing something simple but I have an AIX system running off a LUN that's configured for a Solaris box. I have two questions - *What's the impact? Are they essentially the same or is there a difference between the two that's going to cause me grief *How do I change it? Any suggestions would be much appreciated. Please be advised that this email may contain confidential information. If you are not the intended recipient, please do not read, copy or re-transmit this email. If you have received this email in error, please notify us by email by replying to the sender and by telephone (call us collect at +1 202-828-0850) and delete this message and any attachments. Thank you in advance for your cooperation and assistance. In addition, Danaher and its subsidiaries disclaim that the content of this email constitutes an offer to enter into, or the acceptance of, any contract or agreement or any amendment thereto; provided that the foregoing disclaimer does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment to this email.

2 1

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters