May 2012 - toasters - lists.teaparty.net

NSF client listing
by Weasner, Matthew K 19 Oct '12

19 Oct '12

This is probably a silly question, but what is the most reliable way to get a list of NFS clients that have an export mounted? OnTap 7.3.3, and the share is exported R/W to the world. Thanks, Matt Sent from my iPad The information contained in this message may be privileged, confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify your representative immediately and delete this message from your computer. Thank you.

7 10

Expand SATA aggregate or buy SAS disks
by James Andrewartha 10 Aug '12

10 Aug '12

Hi, We have a small metrocluster with 2TB SATA disks in DS14mk2 trays, mostly used for VMware. We desperately need more IOPs, and are tossing up another 14x2TB SATA tray, which would have the benefit of just expanding the aggregate, vs 12 of 600GB 15krpm SAS (half a DS4243), which would give more IOPs but be more admin overhead since we'd have to move VMs amongst the shelves. Capacity isn't a problem, and when we built the metrocluster the ATTO 6500N fibre bridge wasn't available, so SATA was our only option at the time. We're getting quotes for both options, but does anyone have an opinion on expanding the SATA aggregate vs setting up a new SAS one? Thanks, -- James Andrewartha Network & Projects Engineer Christ Church Grammar School Claremont, Western Australia Ph. (08) 9442 1757 Mob. 0424 160 877

4 5

ndmpcopy from 6.x to 8.1
by Jon Bousselot 23 Jul '12

23 Jul '12

Has anyone used ndmpcopy to migrate volumes from ontap 6.5 to 8.1 without going through any intermediate steps? The 8.1 docs say qtree snapmirror doesn't support source volumes prior to 6.5, so I'm looking for success stories on possible alternatives in case it doesn't work as advertised. rsync takes a long long time, but it works fine. This is all linux data, nfs. -Jon

3 4

Getting NFS3ERR_ACCES for NFS Share
by Ray Van Dolson 30 May '12

30 May '12

IBM N6240 running ONTAP 8.0.2P3. Have a number of NFS shares set up with pretty straightforward permissions: /vol/napc2_p2_Data6 -sec=sys,rw,nosuid Filer is connected both to NIS and AD and most shares are shared out via both NFS and CIFS. When attempting to copy file content to the share above (all shares really, but using this one as an example), I get a Permission denied error (packet capture shows this to be an NFS3ERR_ACCES message). The file itself is successfully created, but is size zero. Once the file is created (with error message) I can then run the exact same copy command again and this time the data is populated. Packet capture seems to show the CREATE call succeeding, while the subsequent SETATTR call failing with the aforementioned error. Anyone run into something like this before? NFS client(s) in this case are Linux (RHEL, Fedora). NFSv3 is in use with NFSv4 explicitly disabled. Ray

8 16

Install OnCommand Core on Ubuntu?
by Fredrik Pettai 30 May '12

30 May '12

Hi, Earlier I got a tip on how to install and run DFM on an Ubuntu server instead of RedHat / SuSE (http://www.eigenmagic.com/2009/06/26/how-to-install-netapp-dfm-on-ubuntu/) However, I would like to do the same thing with OnCommand Core, but the former DFM routine doesn't seem to work well at all. So, I wonder if someone else has manage to do the same thing with OnCommand Core and succeeded? Thanks, /P

3 5

Snapvault stop on 8.1 7-mode
by Jeff Cleverley 30 May '12

30 May '12

Greetings, I posted a couple of weeks ago about snapvault relationships not going away. It turned out the snapvault stop had not cleaned up all the associated snapshots on the destination used for qtrees. Since that time I have had a similar problem on multiple filers. The problem seems to be that snapvault stop doesn't actually do anything in 8.1. I upgraded one of the filers from 8.0.1P1 to 8.1 GA. I was able to get the SV stop to work at 8.0.1, but it hasn't worked once since I've upgraded. One of my other filers is doing the same thing. Ultimately after multiple SV stop attempts (at least 5), rebooting the filer, then doing more multiple SV stops, I was able to get some of them to lose the SV lock. I've got a call open for this but haven't received any useful information from it. This is an example of the error message I get. isolation> snap delete former_mofab isolation(0118075592)_former_mofab_root-dst.0 Snapshot isolation(0118075592)_former_mofab_root-dst.0 is busy because of LUN clone, snapmirror, sync mirror, volume clone, snap restore, dump, CIFS share, volume copy, ndmp, WORM volume, SIS Clone The snapvault stop was the first thing run. Nothing else was done until after it completed. Everything else is cleaned up. The schedules, relationships, etc. A snapvault status doesn't show a source because that has already been cleaned up. The volumes are not part of a mirror and this snapshot is the only special snapshot in the volume. I've also taken the volume offline and brought it back online but that doesn't help either. Is there some magic recipe at 8.1 to make SV stop work? I can't keep rebooting filers and spending 30 minutes per volume doing multiple stops and snap delete attempts. Thanks, Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611

1 1

VSC Privilege
by Carl Howell 30 May '12

30 May '12

I'm trying to get the Virtual Storage Console working using a domain account. Every time I try to log in, it gives me "In order to determine which controller privileges are allowed for this username, the following controller privilege is required: useradmin-user-list" The domain user account I'm using is a member of a group on the filers that has a role with login-*, api-*, and cli-* capabilities. What am I missing? Thanks, Carl

3 7

Curiosity killed the NetApp admin: Empty lun not empty
by Peta Thames 30 May '12

30 May '12

Hi all, I have a question born mostly out of curiosity but also slight paranoia. I don't know VMware very well so forgive me if I'm missing something obvious. I have a VMware lun that is thin-provisioned and presented via FC. vSphere reports that the lun has no VMs in it, and yesterday I deleted the datastore from VMware. It was only mapped to one cluster. The lun is currently offline and no-one's complained (yet). On the filer, it reports that the lun occupies 962.9gb of its 1tb. Surely it should be empty? There are no snapshots. We don't use snapmirror. Fractional reserve was set to 100 but I've since set it to 0 and it made no difference. I haven't been here long so I don't know any more history I'm afraid. Any ideas? Thanks, Peta P.S. If anyone was curious about my last email sent to this list re reallocation and de-duped volumes, I can't get a detailed answer from NetApp support aside from "generally, don't reallocate de-duped volumes". So that's what I'm doing (or not, so to speak).

2 3

Poor NFS 10GbE performance on NetApp 6080s
by Dan Burkland 29 May '12

29 May '12

Hi all, My company just bought some Intel x520 10GbE cards which I recently installed into our Oracle EBS database servers (IBM 3850 X5s running RHEL 5.8). As the "linux guy" I have been tasked with getting these servers to communicate with our NetApp 6080s via NFS over the new 10GbE links. I have got everything working however ever after tuning the RHEL kernel I am only getting 160MB/s writes using the "dd if=/dev/zero of=/mnt/testfile bs=1024 count=5242880" command. For you folks that run 10GbE to your toasters, what write speeds are you seeing from your 10GbE connected servers? Did you have to do any tuning in order to get the best results possible? If so what did you change? Thanks! Dan

10 22

ONTAP & IPv6?
by Fredrik Pettai 29 May '12

29 May '12

Hi, IPv6 support in ONTAP has be there for a while, but is anyone really using it? and in production? I asked NetApp (the local Swedish office in Stockholm) and they tried to asked both their integrators and internally in NetApp worldwide to see if anyone knew about any customers running ONTAP and IPv6 enabled, but only got "no" as answer or no feedback at all. That was disappointing... So does anyone on the list run IPv6 on their NetApp(s)? or know of any other NetApp customer that enabled IPv6? We're starting to use it (on ONTAP 8.1 GA), but it would be comforting if we where not the only ones using it... Thx, /P

2 1

Diagnosing outage on FAS6040 running 8.0.1 7-mode
by Steve Losen 25 May '12

25 May '12

Hello Toasters, We had an incident where a FAS6040 running 8.0.1 gave very slow response to a variety of requests. This resulted in a bunch of Linux VMs experiencing failed disk I/O requests, which resulted in corrupted local Linux filesystems. One Linux VM logged that it waited 180 sec. for a disk I/O to complete. The filer did not reboot, and recovered on its own, but we are very interested in figuring out what happened and if we can avoid it (known bug?, should we upgrade ONTAP?) This 6040 does a variety of jobs -- NFS server for a Communigate Pro email system, NFS volumes for VMWARE, and it even has a few FC SAN LUNs used by sharepoint. In general it does not appear to be overloaded. It's been up for over 300 days with no problems. The CF partner filer experienced no problems at that time and it also holds VMWARE volumes and mail volumes. At the time of the outage the /etc/messages file indicated a slow NFS response (93 sec) to one of the Communigate servers. It also indicated that a FC LUN was reset by the sharepoint server. I'm guessing due to delayed response. The mail and sharepoint volumes are in the same aggregate. I see three resets for the LUN in /etc/messages. I looked in /etc/log/ems at about the time of the outage (Tue May 22 17:30 EDT) I see that a raid scrub of a raid group in the mail/sharepoint aggregate completed with no errors. I also see wafl_cp_toolong_warning_1 and wafl_cp_slovol_warning_1 for a different aggregate (which contains VMs). I see several of these, which I presume are caused by slow completion of CPs. I don't know if these caused the problem or were caused by the problem. Anyone have any suggestions for further investigation or diagnosis? Any other logs to look at? Everything is fine now and has been running fine since the outage. Thanks, Steve Losen scl(a)virginia.edu phone: 434-924-0640 University of Virginia ITC Unix Support

5 5

Two semi-related questions - Adding a shelf, and FC config
by Justin Popa 24 May '12

24 May '12

I put in a FAS2020HA last year, and we're looking to do a little expansion. We currently just have the base unit, with 600GB SAS drives. We'd like to add a shelf for our testing environment. We asked for a quote from our reseller but it was astronomical, so we've looked into the used market. Other than the added equipment not being covered by my service & support contract, is there any issue buying a ds14mk4 and installing it? Assuming I do add it, I need some help figuring out my fiber connections. We're currently NFS, but I've recently procured some HBA's, as well as switches. To make things more interesting, we currently do our backup over fiber to an HP Autoloader connected directly to the first filer. Here are my concerns: 1) I've been told that you can't put a tape backup on a fiber switch with a netapp. True? If so, can I just leave it connected to filer A, and then only connect the second FC port to the switch? I know I'm looking at a point of failure there, but that would be temporary until we change our backup infrastructure. 2) Can you hang a shelf off a fiber switch as well? Is there any concern of doing that given question 1?

3 2

encrypted SMB traffic?
by Randy Rue 23 May '12

23 May '12

We've had a request to consider providing encrypted access to a CIFS filer. From the quick search I've done so far it appears that SMB itself doesn't provide any encryption options so instead we'd need to use IPSec at the filer and workstation ends. I've found NetApp man entries for the commands to manage this. Before I dig too deep into this, I'm hoping to get a feeling of whether anyone is using this in the real world. If so, how hard is it to set up? Is there a significant performance overhead? Any other gotchas? Hope to hear from you, rrue

3 2

Im Hiring!
by Jeff Mohler 22 May '12

22 May '12

If you are a high end, super skilled, deductive reasoning data and storage guy overflowing with awesome, I may have an interesting opportunity for you in Field Performance. SF Bay Area Location. Please do NOT contact me thru the list, but Pvt Email at jmohler(a)netapp.com So lang und Dank für alle Fische --- I drank what?

1 0

Cluster Notification from filer-a (SCHEDULED SNAPSHOT CREATE FAILED) CRITICAL
by Randy Rue 18 May '12

18 May '12

Hello All, Got this autosupport from our FAS3020 running 7.2.4. We use the "vol options VOLXX schedsnapname create_time" so that snapshot names reflect when the snapshot was taken, and the autosupport email contains the line: VOLXX - hourly.2012-05-16_1200 : Snapshot already exists. Trying to take a snapshot twice? Failing to roll off an old one? Just noise? I've STFW'd a little but I'm not finding anything like this. Hope to hear from you, Randy

2 2

LUN migration and consolidation
by Milazzo Giacomo 18 May '12

18 May '12

Hi all, I have the following situation. Source file system: an AIX volume created on hundreds of very small LUNs (HDS storage array, something like 15 GB each!) Destination file system: a "mirrored" one to online migrate using LVM with no disruption on services but at the same time consolidating on a single LUN on NetApp (as you know the geometry is completely different!) In the past I did it but on a 1:1 relationship between source and destination LUNs. Not there's somebody stating that on AIX this is not possible... ?!? But this is possibile on any distribution of Linux too!!! My idea in other words is to map the new space to the same AIX host that will use to create a sort of R1 between file systems (Veritas Volume Manager also doed this i.e) regards

3 3

Rate Limiting an NFS client
by Steven Kreuzer 18 May '12

18 May '12

Offhand, does anyone know if 8.1 has the ability to do rate limiting for nfs traffic? I have been asked to configure a filer so that nfs traffic from a certain ip addresses will not be able to exceed a maximum transfer rate while other clients will not be subjected to a maximum transfer rate I am aware that this can be done with ACLs on a switch, but management's would prefer to do it on the filer side if possible. Many Thanks -- Steven Kreuzer

1 1

Reallocate redux: Reallocation and de-dupe
by Peta Thames 17 May '12

17 May '12

Hi all, So I've been able to reallocate about 20% of our luns so far, and it's already made a huge improvement. I still see some huge latency spikes (up to 500ms) on some luns, mostly ones I haven't reallocated yet, but the spikes I originally described, where I had spikes on every volume in the aggregate, sometimes the entire filer, at the same time, have disappeared. Our VMware guy and the Citrix team are also reporting performance improvements. I must admit, I'm surprised by how quickly reallocate has improved this environment, to the point where users are already noticing it, and I'm nowhere near finished. If you've never checked your reallocation measurement, I highly recommend it! Thanks especially to Jeff Mohler and Fletcher Cocquyt (I found your blog posts through google before you'd emailed and thank you for writing them, they are really good). I was thinking over the weekend, as you do, about why I'd sometimes see spikes on multiple aggregates at once, and a common denominator there would be the backend loops. CPU has never been an issue. Does it make sense that the loops become a bottleneck on systems with severe fragmentation? Everything on the filer queues up there behind the IO going to one disk? Anyway, the question I really would like answered this time is: How best to balance reallocation and de-dupe. On Friday I reallocated a lun that's been de-duplicated, and the measurement changed from 6, hotspot 0, to 4, hotspot 28. Obviously the hotspot is due to the de-duplicated blocks. So, is it better to leave a higher reallocate measure (in this case, 6), with no hotspot, or better to reallocate these luns to lower the overall measure, even though it creates a horrible hotspot? Thanks, Peta On 30 April 2012 03:29, Milazzo Giacomo <G.Milazzo(a)sinergy.it> wrote: > > At last am answer with sense J > > I explain it better. I’m referring to what Fletcher wrote: “It was maddening to me back in 2010 how netapp support could blockade support cases with a blanket "must align VMs first" without a real quantification of the impact of misalignment – see” > > We had some critical case of this “performances” issues and in both cases the first thing that (support) asked to do: realign! > > Well. After a long and tedious process of realignment things were better…meanly 5% better, not more!!! And don’t let customer see graphs coming from the CMPG portal! Something like this attached…this could be pure terrorism :-D and a lot of nightmare for you trying to explain (and understand) the meaning of buckets J > > > Alignment is important…but NOT SO important. I concentrate investigation first on other levels. > > > > Reagards, > > > > > Da: toasters-bounces(a)teaparty.net [mailto:toasters-bounces@teaparty.net] Per conto di Fletcher Cocquyt > Inviato: venerdì 27 aprile 2012 08:46 > A: Peta Thames > Cc: Toasters(a)teaparty.net > Oggetto: Re: Reallocation and VMware > > > > Peta - we were dealing with this very issue (unexplained latency spikes Netapp blamed on VM misalignment) > > back in 2010 - I wrote up how we deconstructed the IOPs after many wasted perfstat iterations > > to solve it pretty much on our own: > > > > http://www.vmadmin.info/2010/07/vmware-and-netapp-deconstructing.html > > > > It was maddening to me back in 2010 how netapp support could blockade support cases with a > > blanket "must align VMs first" without a real quantification of the impact of misalignment - see > > > > http://www.vmadmin.info/2010/07/quantifying-vmdk-misalignment.html > > > > We ended up taking the downtime back then to align all VMs > > But now, I would be one to encourage your making the leap to 8.x - we are on 8.1GA and we are not looking back. > > The data motion of vFilers is allowing us to upgrade clusters with no downtime > > > > http://www.vmadmin.info/2012/04/meta-storage-vmotion-netapp-datamotion.html > > > > They have me almost believing in cluster mode for scale out... > > > > > > > > > > > > On Apr 25, 2012, at 11:40 PM, Peta Thames wrote: > > > > Hi Jack, > > You're right, and I should have mentioned it before. Large numbers of > the VMDKs are misaligned. I'd estimate about 33%, but I don't know > exactly how many as the shiny new VSC scanner got stuck halfway > through the scan I ran, leaving several VMs in a "being scanned" > state. I have a case open with Netapp to find out how to get those > VMs out of that state so I can a) continue the scan b) schedule fixing > the misaligned luns. > > Not all the luns that have large latency spikes are misaligned > however. Mind you, by the same token, not all of them are fragmented, > although so far (I'm still getting through measuring them all) there's > definitely a strong correlation. > > I also have to admit that I read the scale wrong in perf advisor, and > the numbers I'm seeing are in microseconds, not milliseconds. Still > way more than the 10ms I would like, but an order of magnitude better! > > Peta > > On 26 April 2012 15:52, Jack Lyons <jack1729(a)gmail.com> wrote: > > Have you checked the alignment of the VMDK's? > > > > Jack > > Sent from my Verizon Wireless BlackBerry > > > > -----Original Message----- > > From: Peta Thames <petathames(a)gmail.com> > > Sender: toasters-bounces(a)teaparty.net > > Date: Thu, 26 Apr 2012 14:49:43 > > To: <Toasters(a)teaparty.net> > > Subject: Reallocation and VMware > > > > Hi all, > > > > I'd like to pick your collective brains about your experiences with > > reallocate, specifically when reallocating luns under VMware. > > > > For background, we're running ONTAP 8.0.1 on a 3170 that's over three > > years old. I've been going through measuring reallocation, and most > > of the volumes are over 3. We have no snapshots, and only a > > relatively small number of volumes are de-duplicated. All our volumes > > and luns are thin-provisioned, and no aggregate is more than 76% full > > (most are ~65%). We regularly have huge latency spikes (worst I've > > seen so far is 5000000ms, and there are far too many to even track > > over 50000ms daily), and on one filer head, but not its partner, I > > regularly see disk utilisation go to 100% or more. I'm hoping > > reallocate will help here. > > > > I have a brief note from a NetApp support person who says "It’s very > > important that you complete the reallocation in the following order: > > 1:OS 2:LUN 3: Volume". > > > > I have two questions about this: > > - is it absolutely necessary to defrag the OS before you reallocate > > the lun? I'm sure I've run reallocate without defraging the OS and > > still seen performance improvements. I'm also assuming that this is > > only relevant to Windows VMs, not Linux (in our case, Red Hat/CentOS) > > ones. > > - if you only have one lun per volume, do you still need to run > > reallocate on both the lun and the volume? If only one, which is > > preferable? > > > > All advice appreciated. > > > > Thanks, > > Peta > > > > _______________________________________________ > > Toasters mailing list > > Toasters(a)teaparty.net > > http://www.teaparty.net/mailman/listinfo/toasters > > > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > http://www.teaparty.net/mailman/listinfo/toasters > >

3 2

upgrade impact from 7.3.5 to 8
by Randy Rue 16 May '12

16 May '12

Hello All, We're starting the planning for an upgrade of a v3170 from 7.3.5.1P5 to 8.0x or possibly 8.1GA 7-mode (we've already upgraded another filer to 8.1 without trouble). The system is backed by a 3Par T800 serving up LUNs as "disks" for the v3170 with all NetApp aggregates in essentially RAID0 (redundancy is provided at the SAN level). The v3170 is heavily used with a long list of protocols and features enabled including autohomes, de-dupe, vfilers serving CIFS, NFS, and iSCSI. NFS is v3 and v4, sometimes Kerberized, sometimes auth_sys. iSCSI is CHAP authenticated, sometimes using native initiators and sometimes SnapDrive and/or SnapManager for SQL. Backups are via NDMP to a CommVault system. Our enterpise SQL clusters are served (iSCSI) as well as our main VMware Farm (datastores are via NFS). I welcome feedback from anyone with experience in this type of effort. My sense is that the biggest dependencies will be on connected SnapDrive and SnapManager clients. For example, is there a way to identify all connected SnapDrive and SnapManager clients and their versions? Hope to hear from you, Randy Rue

1 0

Opinions on SnapProtect
by Fletcher Cocquyt 16 May '12

16 May '12

This looks like exactly the level of snapshot + backup integration we've been looking for - especially for our 100's of VMs on Netapp NFS volumes. http://www.netapp.com/us/products/protection-software/snapprotect.html Can anyone offer any opinions on the product - especially integration with vmware and existing tape backups (eg Legato) thanks, Fletcher

2 3

snapshot cleanup and performance
by Peter D. Gray 16 May '12

16 May '12

OK, just wondering if anybody can shed light on this. We just had a massive performance problem on one of our netapps. One aggregate was amazingly busy, with disk drives 100% busy all the time. IO latency went through the roof for all the volumes on the aggregate. We spent a bit of time on this and we are not novice netapp users. In the end we could not identify the problem. There appeared to be no relationship between the amountof I/O coming from clients and the amount of I/O on the aggregate. So, we called netapp support. It took them a while (a few days) but eventually they suggested changing the snapshot schedules and removing snapshots off the aggregates and also removing hourly snapshots off the volumes on the aggregate. We complied. It took a while, but after 5 hours or so, relatively suddenly the problem went away and seems to have stayed away for a day now. I am assuming the snapshot cleanup was the problem and the problem stopped when it finally caught up. So, I guess my question is "why is snapshot cleanup so expensive". Do blocks freed from snapshots need to be written, if so why? It seems snapshot creation is cheap, but deletion expensive, which makes the entire snapshot management cycle rather more trouble then you might hope. Regards, pdg

4 3

NFS Lock and Oracle
by Craig A. Falls 15 May '12

15 May '12

Hi, I was wondering if anyone had a script for viewing/removing NFS locks on a filer? We have just started to use Oracle on NFS and have had an issue where we have had the host crash and leave locks which then stop the DB being started again (on that node) when that node is recovered and will probably (guessing) eventually stall the DB when other nodes try to access the data that is being locked. I was planning on something like: - check the locks for a filer/vfiler for the specific host - remove the locks using lock status/lock break Any thoughts/help or advice would be appreciated. thanks c

3 2

Writing to the list via Gmane
by Oliver Brakmann 15 May '12

15 May '12

Hello all, just a heads-up: if this post reaches the list, it means that posting to the list via gmane (http://gmane.org) works again. It broke after the move to the new domain, but I contacted the Gmane maintainer recently and asked him to please fix the address, which has now happened. Regards, Oliver

1 0

What is Flash Cache caching?
by Fletcher Cocquyt 15 May '12

15 May '12

We setup a custom view in NMC to show us the hit % and Disk Reads replaced per the Flash Cache Best Practices doc TR-3832. Is it possible to get more granular information about what is being cached? These are NFS data stores and one of the volumes is 500Gb of web content. So the web team would be very interested to know the net benefit to them - eg how much are the apache instances benefitting. We also have oracle instances, VMware volumes and would like per volume stats if possible I guess I am looking for the ability to apply the custom view to different vFilers and see per volume hits and hit %, Disk Reads replaced We have the 1Tb card installed and see the Disk Reads Replaced topping out around 5000 - with one spike up to ~8500 Does this look like what others are seeing in a mixed workload environment like I described? thanks, Fletcher

3 5

Cleaning up snapvault relationships.
by Jeff Cleverley 14 May '12

14 May '12

Greetings, We retired a SV primary volume and I need to keep the destination SV volume around until the snapshots age out. The backups are done via command line/cron/snapvault snap sched setups. They are not managed by DFM/PM or any other package. I did a snapvault stop on the destination which seems to have worked fine. I verified that the destination qtree is gone. I also did a snapvault release on the primary and a snapvault snap unsched on both primary and secondary. I've even rebooted the destination filer which would have the effect of stopping and starting snapvault. The primary shows no trace of the former relationship. The secondary got rid of everything, but some of the destinations still show up in the snapvault status output. I need to figure out how to clean these up. Here is the output of a snapvault -l of one of the destination qtrees: Snapvault secondary is ON. Source: source:/vol/carmel2/test Destination: destination:/vol/carmel2_i/test Status: Idle Progress: - State: Unknown Lag: - Mirror Timestamp: - Base Snapshot: - Current Transfer Type: - Current Transfer Error: - Contents: - Last Transfer Type: Update Last Transfer Size: 8 KB Last Transfer Duration: 00:00:06 Last Transfer From: source:/vol/carmel2/test Thanks, Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611

1 1

revision control for /etc/exports with post-commit exports -va API call?
by Fletcher Cocquyt 11 May '12

11 May '12

Is anyone doing revision control on /etc/exports with post-commit hooks to call exportfs -va? (including vfiler:/rootvol/etc/exports)? Or more generally what is the recommended best practice for these command line file updates in a team environment (multiple admins)? There used to be a Filerview UI for updating exports, but I can not see one in the new System Manager - is it there? In a team environment we'd like to avoid touching files directly with vi (and introducing errors) - but if we need to, we'd like revision control and automation with post-commit hooks which has worked well for our apache and puppet configs. thanks Fletcher

1 0

SMSQL question
by Klise, Steve 10 May '12

10 May '12

I just upgraded a SQL 2005 SQL cluster from snapdrive 5, to 6.3.1r1 and SMSQL 2.x to 5.0. I also upgraded the verification server (remote server). Now, after upgrading, on a set of drives (G:\data and L:\logs), the backup works, but DBCC fails. What is strange is there is another set of drives (H:\data M:\logs) DBCC works.. I checked for snapshots that would still be mounted, flex clones, etc. Nothing appears to have "busy snapshots". This is the error (names/ip's changed to protect the innocent). Even checked for LUNS that would be not assigned to an igroup.. nothing.. This is the event from the verification server: Event Type: Error Event Source: SnapDrive Event Category: Generic event Event ID: 331 Date: 5/4/2012 Time: 1:19:32 PM User: N/A Computer: VERIFICATIONSRV Description: Failed to connect FlexClone of Snapshot copy (sqlsnap__SQL__recent) as drive C:\Program Files\NetApp\SnapManager for SQL Server\SnapMgrMountPoint\MPDisk001\ on the computer (VERIFICATIONSRV). LUN Name = lun0.lun Storage Path = /vol/SQLSERVER/SQLSERVER/ Protocol Type = LUN Storage System Name = 10.10.10.10 Error code : ZAPI: An attempt to online LUN '/vol/sdw_cl_SQLSERVER_0/SQLSERVER/lun0.lun' failed on the storage system 10.10.10.10. Error description: Another LUN mapped at this number This is the snap list. Volume SQLSERVER working... %/used %/total date name ---------- ---------- ------------ -------- 0% ( 0%) 0% ( 0%) May 04 13:18 sqlsnap__SQLSERVER__recent 0% ( 0%) 0% ( 0%) May 04 11:33 sqlsnap__SQLSERVER_05-04-2012_11.33.12__weekly 0% ( 0%) 0% ( 0%) May 04 10:43 sqlsnap__SQLSERVER_05-04-2012_10.42.44__weekly 0% ( 0%) 0% ( 0%) May 04 10:03 panstor2(1573990916)_SQLSERVER_snap.65 (snapmirror) 0% ( 0%) 0% ( 0%) May 04 10:03 @snapmir@{EF1F6E8C-64BA-4C23-A8F1-1566E881A373} 0% ( 0%) 0% ( 0%) May 04 10:01 sqlsnap__SQLSERVER_05-04-2012_10.00.07__daily 0% ( 0%) 0% ( 0%) May 04 07:59 sqlsnap__SQLSERVER_05-04-2012_07.58.44__weekly 0% ( 0%) 0% ( 0%) May 04 07:20 sqlsnap__SQLSERVER_05-04-2012_07.19.32__weekly 0% ( 0%) 0% ( 0%) May 04 07:12 @snapmir@{5CF5E20A-6C44-4CE9-B212-0F4BAEB90636} 0% ( 0%) 0% ( 0%) May 04 07:11 sqlsnap__SQLSERVER_05-04-2012_07.10.46__daily 0% ( 0%) 0% ( 0%) May 03 20:40 sqlsnap__SQLSERVER_05-03-2012_20.39.16__daily 0% ( 0%) 0% ( 0%) May 03 19:00 sqlsnap__SQLSERVER_05-03-2012_19.00.00__weekly 0% ( 0%) 0% ( 0%) May 03 18:30 sqlsnap__SQLSERVER_05-03-2012_18.30.00__daily 0% ( 0%) 0% ( 0%) May 03 14:30 sqlsnap__SQLSERVER_05-03-2012_14.30.00__daily 0% ( 0%) 0% ( 0%) May 03 10:30 sqlsnap__SQLSERVER_05-03-2012_10.30.00__daily 0% ( 0%) 0% ( 0%) May 02 19:00 sqlsnap__SQLSERVER_05-02-2012_19.00.00__weekly 0% ( 0%) 0% ( 0%) May 01 19:00 sqlsnap__SQLSERVER_05-01-2012_19.00.00__weekly 0% ( 0%) 0% ( 0%) Apr 30 19:00 sqlsnap__SQLSERVER_04-30-2012_19.00.00__weekly Any help would be appreciated.

2 3

NDMP backups of named snapshots
by John Stoffel 09 May '12

09 May '12

Guys, Anyone know if it's possible do an NDMP backup (using CommVault) of a *named* snapshot? Basically I need to snapshot two or more volumes at the same time (I can do this in a pre-script) and then let the snapshots be backed up. I guess I could be silly and just create a flex clone volumes and then back those volumes up instead, which would get rid of the issue, but would complicate things a bit more. Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel(a)taec.toshiba.com - 508-486-1087

4 10

15/19 vFilers migrated - 64 bit aggr upgrade ontap next
by Fletcher Cocquyt 09 May '12

09 May '12

So we've reached the point where our 32->32 bit data motion (vFiler migrations) are almost complete - (pending one vFiler stuck in "Migrate Failed" state case, and a previously fun flex clone vFiler) we purposely destroyed the destination (> 19Tb 64bit) aggregates and recreated them < 16Tb to work around 32->64bit data motion incompatibility Next we will initiate non-disruptive 64 bit aggregate upgrades by adding disk (the only currently supported method) Its beaucoup hoops, but we are almost there - our goal is to data motion off all vFilers btw partner clusters to complete our flash cache hardware upgrade thanks Fletcher

1 1

SAS port swap.
by Jeff Cleverley 08 May '12

08 May '12

Greetings, I have a SAS stack (plus other FC ds14s) in a FAS60xx cluster running 7.3.5.1. It has 2 quad port SAS HBAs per filer. The stack currently uses ports 6a and 7d on each filer. I need to add a couple more stacks. To make the cabling consistent I would like to move the current 7d connection to port 7a. When I moved the cable from 7d to 7a on one of the heads, it never brought the port online and never saw the disks on that port. I tried the storage enable adapter 7a but it still showed it was offline. I ran various sysconfig, storage show, disk show, etc, type commands to get the system to go find the disks on the new path. Nothing seemed to help. I waited 5 minutes after doing this just in case there was some type of discovery lag, but they never showed up. When I plugged back into 7d, everything is fine. I thought moving to a new port should work. Is there something else I'm missing to make this work? Thanks, Jeff -- Jeff Cleverley Unix Systems Administrator 4380 Ziegler Road Fort Collins, Colorado 80525 970-288-4611

2 2

vsc 4.0 vm inaccessible
by John Sfetsoris 08 May '12

08 May '12

hello everybody there is a vsc 4.0 in a vsphere 4.0 vms are on 'others" folder and have operation status 'inaccessible", so cannot be align attached is a scrrenshot of the console vsc log reports the following 2012-04-20 16:27:31,991 (VMExaminer-t-vwks-lkattis-1334928446341) WARN [VmdkFile] - Unable to read sectors using VDDK java.lang.Exception: The handle is not a valid VIX object at com.netapp.alignment.virtualDisk.VirtualDiskInfo.readSectors(Native Method) at com.netapp.alignment.virtualDisk.VirtualDiskInfo.readSectors(VirtualDiskInfo.java:99) at com.netapp.kamino.server.duplicator.VmdkFile.getSectors(VmdkFile.java:665) at com.netapp.caster.server.scanmanager.VMExaminer.buildPartitions(VMExaminer.java:583) at com.netapp.caster.server.scanmanager.VMExaminer.run(VMExaminer.java:460) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2012-04-20 16:27:31,992 (VMExaminer-t-vwks-lkattis-1334928446341) ERROR [VMExaminer] - Unable to read first two sectors of [Development] t-vwks-lkattis/t-vwks-lkattis_1.vmdk java.lang.IllegalStateException: VDDK is not properly initialized at com.netapp.kamino.server.duplicator.VmdkFile.getSectors(VmdkFile.java:668) at com.netapp.caster.server.scanmanager.VMExaminer.buildPartitions(VMExaminer.java:583) at com.netapp.caster.server.scanmanager.VMExaminer.run(VMExaminer.java:460) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2012-04-20 16:27:31,992 (VMExaminer-t-vwks-lkattis-1334928446341) ERROR [VMExaminer] - Failed to examine VM java.lang.IllegalStateException: VDDK is not properly initialized at com.netapp.kamino.server.duplicator.VmdkFile.getSectors(VmdkFile.java:668) at com.netapp.caster.server.scanmanager.VMExaminer.buildPartitions(VMExaminer.java:583) at com.netapp.caster.server.scanmanager.VMExaminer.run(VMExaminer.java:460) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) 2012-04-20 16:27:32,050 (VMExaminer-t-vwks-lkattis-1334928446341) DEBUG [VcInventory] - Entering waitOnTask (which now calls waitForTask),KaminoCountDownLatch [look is 0 with a max of 3 (look starts as zero).] 2012-04-20 16:27:32,050 (VMExaminer-t-vwks-lkattis-1334928446341) DEBUG [KaminoCountDownLatch] - keepGoing: look (1) is less than maxLook (3). Returning true (keep going). 2012-04-20 16:27:32,067 (VMExaminer-t-vwks-lkattis-1334928446341) DEBUG [VcInventory] - waiting on vm.Snapshot.remove which is in the following state: running any sugestions? thnks in advance John

1 0

User 'root' denied access - missing required capability: 'cli-route'
by Fletcher Cocquyt 04 May '12

04 May '12

I'm seeing this error "User 'root' denied access - missing required capability: 'cli-route'" (on the console) at the cutover stage of vFiler migration under 8.1 GA The vFiler migrations are successful, but this error is troubling for production vFiler migrations which require zero downtime. I opened a netapp support case and posted more details here: https://communities.netapp.com/message/80796 I'm looking to confirm the proper useradmin user->group->role mappings for root in case 8.1GA introduced some bug. I have (note no groups for root user - should I create a root group and add the mapping to the root role?): useradmin user list Name: root Info: Default system administrator. Rid: 0 Groups: -- useradmin group list Name: Administrators Info: Members can fully administer the filer Rid: 544 Roles: admin Name: Backup Operators Info: Members can bypass file security to backup files Rid: 551 Roles: backup,none Name: Compliance Administrators Info: Members can perform compliance operations Rid: 131072 Roles: compliance Name: Guests Info: Users granted Guest Access Rid: 546 Roles: none Name: ndmp Info: Rid: 131077 Roles: ndmp_role Name: oragroup Info: Rid: 131075 Roles: oracle Name: Power Users Info: Members that can share directories Rid: 547 Roles: power Name: Replicators Info: not supported Rid: 552 Roles: none Name: Users Info: Ordinary Users Rid: 545 Roles: audit -- useradmin role list Name: admin Info: Allowed Capabilities: login-*,cli-*,api-*,security-* Name: audit Info: Allowed Capabilities: api-snmp-get,api-snmp-get-next Name: backup Info: Default role for NDMP privileges. Allowed Capabilities: login-ndmp Name: compliance Info: Default role for compliance privileges. Allowed Capabilities: cli-cifs*,cli-exportfs*,cli-nfs*,cli-useradmin*,api-cifs-*,api-nfs-*,login-telnet,login-http-admin,login-rsh,login-ssh,api-system-api-*,cli-snaplock*,api-snaplock-*,api-file-*,compliance-* Name: ndmp_role Info: Allowed Capabilities: login-ndmp Name: none Info: Allowed Capabilities: Name: oracle Info: Allowed Capabilities: login-ssh,cli-snap* Name: power Info: Allowed Capabilities: cli-cifs*,cli-exportfs*,cli-nfs*,cli-useradmin*,api-cifs-*,api-nfs-*,login-telnet,login-http-admin,login-rsh,login-ssh Name: root Info: Allowed Capabilities: * thanks

1 0

DataMotion for Volumes in 8.1?
by Randy Rue 03 May '12

03 May '12

Hello All, Running 8.1GA on a 3240 and trying to move two vfilers from one aggregate to another (both are 64 bit aggrs). Ultimately I just need to move the root and data volumes for these vfilers. Not finding a wealth of examples on the "vol move" command, not surprising as this is a pretty new feature. http://www.netapp.com/us/communities/tech-ontap/tot-data-motion-1102.html NetApp's blurb says this can be done without interruption, but also talks a lot about when to do the "cutover." They say the cutover can be triggered manually or automatically. But I can't make the "vol move" command do anything but return a usage blurb unless I include a "start" switch. Also, their website claims this can be done nondisruptively. But "vol move start" keeps failing because I have CIFS and/or NFS running. In fact, if I turn them both off it still complains that I have NFS exports configured and that I need to remove them first. Anyone have experience moving volumes from one aggr to another, hopefully live? Randy Rue

6 8

Getting a quick "connection refused"
by David Lee 02 May '12

02 May '12

A TCP connection towards an arbitrary high port number (>=1024) on the NetApp seems to return "connection refused" instantly. By contrast, a connection towards a low port number waits for several minutes, then something times out. I imagine this is related to assisting security etc. As part of our transition from our previous non-NetApp fileserver, it would be very useful if we could persuade the NetApp to return a quick "connection refused" on a particular low port number. But we cannot see a way to adjust this behaviour (either for a particular low port or for all low ports). Is this possible? If so, could you point us to the relevant documentation, please? (If it is not possible, we can probably put a workaround in the external application.) -- David Lee

4 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters May 2012