toasters October 2010

toasters@lists.teaparty.net

10 participants
6 discussions

Moving netapp hardware to different racks
by Steve Losen 20 Oct '10

20 Oct '10

Greetings Toasters, Please bear with me, I'm not an electrician. We are constructing a new data center and we are standardizing our racks and the power feeding them. We plan to move some netapp hardware to the new data center, which is currently installed in netapp cabinets where each cabinet is powered by 4 240V 30 amp circuits. The racks in the new data center will be powered by 6 120V 20 amp circuits each. That is less power than we have in our netapp cabinets, but we can simply put less equipment in the new racks if necessary. I am pretty sure that this will not cause us any problems. We already have several FAS2050 controller pairs and DS14MK2 disk shelves installed in non-netapp racks plugged into 120V outlets. Can FAS6040 and FAS3170 heads be plugged into 120V ? (I will double check the hardware specs.) We could move the netapp cabinets to the new data center, but then we need to reserve floor space for the cabinets and run 240V circuits to them. We would rather use the new standard racks/power. Steve Losen scl(a)virginia.edu phone: 434-924-0640 University of Virginia ITC Unix Support

6 6

FAS2050 adding 2 x DS14mk2 (ESH2) shelves
by John Stoffel 12 Oct '10

12 Oct '10

Hi all, We're planning on adding a pair of DS14mk2 ESH2 shelves to a FAS2050 running 7.2.6.1P8 and we're wondering if there are any gotchas we need to worry about? The disks are 300F ones. Neither of the two FC ports (0a & 0b) are currently in use. I think we're all set hardware wise, though we might need to do firmware upgrades. I know I'll need to assign the disks to the system once it boots up with them attached, and such like that. Thanks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel(a)taec.toshiba.com - 508-486-1087

1 0

/vol/vol0/etc permissions
by Kevin Sheen 06 Oct '10

06 Oct '10

Hi all, We have a new filer running ontap 7.3.3. We normally nfs export the /vol/vol0 to a select few hosts in order to do management on the filers (just the few things that filer view / system manager don't do well). I can't seem to do it on this new filer. filer3> exportfs /vol/vol0/home -sec=sys,rw,nosuid /vol/vol0 -sec=sys,rw=ewsapps,root=ewsapps I can do the following: ewsapps:root:/ $mount filer3:/vol/vol0 /mnt ewsapps:root:/ $cd /mnt ewsapps:root:/mnt $ls -al total 58 drwxrwxrwx 6 root root 4096 Oct 6 11:21 . drwxr-xr-x 39 root root 1024 Oct 2 17:30 .. dr-------- 2 root root 4096 Oct 6 09:46 .ha drwxrwxrwx 10 root root 4096 Oct 6 08:00 .snapshot drwx------ 28 root root 12288 Oct 6 11:20 etc drwxrwxrwt 3 root root 4096 Jun 7 11:19 home ewsapps:root:/mnt $cd etc ksh: etc: permission denied ewsapps:root:/mnt $mkdir test ewsapps:root:/mnt $cd test ewsapps:root:/mnt/test $cd .. ewsapps:root:/mnt $rmdir test ewsapps:root:/mnt $touch test ewsapps:root:/mnt $rm test ewsapps:root:/mnt $ls -al total 58 drwxrwxrwx 6 root root 4096 Oct 6 11:29 . drwxr-xr-x 39 root root 1024 Oct 2 17:30 .. dr-------- 2 root root 4096 Oct 6 09:46 .ha drwxrwxrwx 10 root root 4096 Oct 6 08:00 .snapshot drwx------ 28 root root 12288 Oct 6 11:20 etc drwxrwxrwt 3 root root 4096 Jun 7 11:19 home ewsapps:root:/mnt $cd etc ksh: etc: permission denied ewsapps:root:/mnt $chmod 755 etc chmod: WARNING: can't change etc I've looked at options nfs, options cifs on the filer. Can't see anything that stands out as would prevent me from changing etc. I've done searches on NOW and google but nothing jumps out. Anyone seen this? thanks, Kevin

1 1

Re: What do you monitor
by Page, Jeremy 03 Oct '10

03 Oct '10

Getting an hourly snapshot of per protocol latency of each volume and its IOPS for historical trending is priceless. Nick Bernstein <nick(a)nicholasbernstein.com> wrote: I was mainly looking for things that weren't just the obvious capacity planning / performance stuff, but this has been a good list - thanks everyone. I took people's responses and dumped them into a wiki, if people are interested in referring back to this later on. Feel free to add/edit. http://nicholasbernstein.com/netapp/wiki/doku.php?id=monitoring Cheers! Nick On Fri, Oct 1, 2010 at 5:59 AM, Collins, Brian <brianc(a)netapp.com> wrote: > Funny, nobody has mentioned volume capacity..... Just sayin' > > > On 9/30/10 11:38 AM, "Nicholas Bernstein" <nick(a)nicholasbernstein.com> > wrote: > >> I had an interesting question posed by a student yesterday - "aside >> from interfaces being up, and volumes being online, what do you >> typically monitor?" Some of my initial thoughts were: >> >> - inodes being available >> - ifstat on multimode vifs to make sure interfaces are being used >> - io counts on luns/qtrees/vols/ >> - client side nfsstat or the equivalent for that protocol from the >> client side >> >> Anyway, I was trying to think of some of the "non-standard" things to >> monitor, and thought I'd put it out to the list and see what other >> people are typically doing. >> >> Cheers, >> Nick > > -- > Brian M Collins > Virtualization Solutions Architect > NetApp, Inc. > +1 919.476.5316 Direct > +1 919.522.3348 Mobile > brianc(a)netapp.com > > > > > Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.

1 0

Re: What do you monitor
by Nick Bernstein 03 Oct '10

03 Oct '10

I was mainly looking for things that weren't just the obvious capacity planning / performance stuff, but this has been a good list - thanks everyone. I took people's responses and dumped them into a wiki, if people are interested in referring back to this later on. Feel free to add/edit. http://nicholasbernstein.com/netapp/wiki/doku.php?id=monitoring Cheers! Nick On Fri, Oct 1, 2010 at 5:59 AM, Collins, Brian <brianc(a)netapp.com> wrote: > Funny, nobody has mentioned volume capacity..... Just sayin' > > > On 9/30/10 11:38 AM, "Nicholas Bernstein" <nick(a)nicholasbernstein.com> > wrote: > >> I had an interesting question posed by a student yesterday - "aside >> from interfaces being up, and volumes being online, what do you >> typically monitor?" Some of my initial thoughts were: >> >> - inodes being available >> - ifstat on multimode vifs to make sure interfaces are being used >> - io counts on luns/qtrees/vols/ >> - client side nfsstat or the equivalent for that protocol from the >> client side >> >> Anyway, I was trying to think of some of the "non-standard" things to >> monitor, and thought I'd put it out to the list and see what other >> people are typically doing. >> >> Cheers, >> Nick > > -- > Brian M Collins > Virtualization Solutions Architect > NetApp, Inc. > +1 919.476.5316 Direct > +1 919.522.3348 Mobile > brianc(a)netapp.com > > > > >

1 0

diagnosing iSCSI performance on FAS2050
by John Stoffel 01 Oct '10

01 Oct '10

Hi all, I'm working to diagnose some problems with performance of a Lotus Notes server talking over iSCSI to a FAS2050 running 7.2.6.1P8 and figuring out what my options are. The system has 20 X289 420Gb SATA/SAS disks on it currently, and we are looking to add some FC disks in a new shelf. I'm not seeing any network collisions or anything like that, so I don't think it's a hardware problem, but more of a resource problem. There is some CIFS on the system in use as well, but it's mostly iSCSI traffic. Looking at the info, I don't see any terrible numbers, and the performance looks ok. Not great, but ok. Here's some output from 'statit' which I left running for about 2 minutes or so. I can go longer if need be obviously. Hostname: irvfs1 ID: 0135054398 Memory: 1792 MB NetApp Release 7.2.6.1P8: Fri Jul 17 09:03:14 PDT 2009 <8O> Start time: Fri Oct 1 11:30:24 PDT 2010 CPU Statistics 316.694026 time (seconds) 100 % 74.098481 system time 23 % 4.159199 rupt time 1 % (905915 rupts x 5 usec/rupt) 69.939282 non-rupt system time 22 % 242.595544 idle time 77 % 73.954047 time in CP 23 % 100 % 1.130218 rupt time in CP 2 % (226012 rupts x 5 usec/rupt) Miscellaneous Statistics (per second) 10966.86 hard context switches 0.00 NFS operations 125.38 CIFS operations 0.00 HTTP operations 0.00 NetCache URLs 0.00 streaming packets 2870.35 network KB received 8529.71 network KB transmitted 11154.87 disk KB read 4770.64 disk KB written 2698.64 NVRAM KB written 0.00 nolog KB written 2034.43 WAFL bufs given to clients 0.00 checksum cache hits ( 0%) 0.00 no checksum - partial buffer 0.00 FCP operations 434.87 iSCSI operations WAFL Statistics (per second) 218.61 name cache hits ( 80%) 55.35 name cache misses ( 20%) 20887.17 buf hash hits ( 82%) 4548.72 buf hash misses ( 18%) 1813.94 inode cache hits ( 100%) 6.17 inode cache misses ( 0%) 5871.88 buf cache hits ( 75%) 1963.55 buf cache misses ( 25%) 171.92 blocks read 2101.86 blocks read-ahead 384.55 chains read-ahead 88.86 dummy reads 344.25 blocks speculative read-ahead 831.96 blocks written 6.58 stripes written 0.00 blocks over-written 0.08 wafl_timer generated CP 0.00 snapshot generated CP 0.00 wafl_avail_bufs generated CP 0.00 dirty_blk_cnt generated CP 0.00 full NV-log generated CP 0.00 back-to-back CP 0.00 flush generated CP 0.00 sync generated CP 0.00 wafl_avail_vbufs generated CP 0.00 deferred back-to-back CP 0.00 low mbufs generated CP 0.00 low datavecs generated CP 1183.82 non-restart messages 121.62 IOWAIT suspends 314438 buffers RAID Statistics (per second) 312.93 xors 0.00 long dispatches [0] 0.00 long consumed [0] 0.00 long consumed hipri [0] 0.00 long low priority [0] 0.00 long high priority [0] 0.00 long monitor tics [0] 0.00 long monitor clears [0] 0.00 long dispatches [1] 0.00 long consumed [1] 0.00 long consumed hipri [1] 0.00 long low priority [1] 0.00 long high priority [1] 0.00 long monitor tics [1] 0.00 long monitor clears [1] 18 max batch 0.56 blocked mode xor 112.52 timed mode xor 0.05 fast adjustments 0.05 slow adjustments 0 avg batch start 0 avg stripe/msec 6.73 tetrises written 0.00 master tetrises 0.00 slave tetrises 179.93 stripes written 133.00 partial stripes 46.93 full stripes 832.11 blocks written 361.72 blocks read 21.89 1 blocks per stripe size 7 17.47 2 blocks per stripe size 7 16.67 3 blocks per stripe size 7 20.33 4 blocks per stripe size 7 24.39 5 blocks per stripe size 7 32.24 6 blocks per stripe size 7 46.93 7 blocks per stripe size 7 Network Interface Statistics (per second) iface side bytes packets multicasts errors collisions pkt drops e0a recv 55529.33 122.23 0.00 0.00 0.00 xmit 227177.99 228.14 0.00 0.00 0.00 e0b recv 39175.77 117.07 0.00 0.00 0.00 xmit 79516.28 126.61 0.27 0.00 0.00 e1a recv 2844533.72 2570.46 0.00 0.00 0.00 xmit 8427700.87 6604.89 0.00 0.00 0.00 e1b recv 0.40 0.01 0.00 0.00 0.00 xmit 29.44 0.27 0.27 0.00 0.00 vh recv 0.00 0.00 0.00 0.00 0.00 xmit 0.00 0.00 0.00 0.00 0.00 vif1 recv 94724.52 239.47 0.08 0.00 0.00 xmit 306742.10 354.89 0.27 0.00 0.00 iscsi recv 2863035.80 2585.27 0.02 0.00 0.00 xmit 8459234.73 6632.46 0.27 0.00 0.00 Disk Statistics (per second) ut% is the percent of time the disk was busy. xfers is the number of data-transfer commands issued per second. xfers = ureads + writes + cpreads + greads + gwrites chain is the average number of 4K blocks per command. usecs is the average disk round-trip time per 4K block. disk ut% xfers ureads--chain-usecs writes--chain-usecs cpreads-chain-usecs greads--chain-usecs gwrites-chain-usecs /aggr0/plex0/rg0: 0c.00.0 3 8.11 0.16 1.36 6206 5.20 15.58 423 2.75 6.96 377 0.00 .... . 0.00 .... . 0c.00.1 3 8.27 0.16 1.36 21309 5.38 15.13 423 2.74 7.08 402 0.00 .... . 0.00 .... . 0c.00.2 23 48.11 40.63 4.28 1733 4.37 11.31 875 3.11 8.07 616 0.00 .... . 0.00 .... . 0c.00.15 23 47.38 39.91 4.23 1708 4.27 11.16 936 3.19 8.09 544 0.00 .... . 0.00 .... . 0c.00.4 23 46.19 39.16 4.31 1739 3.89 12.76 813 3.14 7.67 631 0.00 .... . 0.00 .... . 0c.00.5 23 46.99 39.86 4.28 1761 3.98 12.16 814 3.15 8.14 529 0.00 .... . 0.00 .... . 0c.00.6 23 47.39 40.20 4.33 1711 3.95 11.92 832 3.25 8.61 542 0.00 .... . 0.00 .... . 0c.00.7 23 47.00 39.86 4.33 1738 3.92 11.96 806 3.21 8.51 567 0.00 .... . 0.00 .... . 0c.00.8 22 45.78 38.71 4.37 1688 3.87 12.57 839 3.20 7.94 563 0.00 .... . 0.00 .... . /aggr0/plex0/rg1: 0c.00.9 2 5.78 0.00 .... . 4.16 23.81 222 1.62 6.71 399 0.00 .... . 0.00 .... . 0c.00.10 2 5.78 0.00 .... . 4.18 23.68 231 1.60 6.80 384 0.00 .... . 0.00 .... . 0c.00.11 23 47.92 39.78 4.24 1740 4.77 14.60 656 3.37 8.05 550 0.00 .... . 0.00 .... . 0c.00.12 23 47.55 39.45 4.14 1860 4.79 14.57 649 3.31 8.08 514 0.00 .... . 0.00 .... . 0c.00.13 23 47.62 39.49 4.26 1725 4.94 14.52 649 3.19 7.79 532 0.00 .... . 0.00 .... . 0c.00.19 24 48.22 39.78 4.18 1849 5.07 13.97 689 3.37 7.91 512 0.00 .... . 0.00 .... . 0c.00.17 23 48.19 40.11 4.16 1811 4.82 14.66 641 3.25 8.15 553 0.00 .... . 0.00 .... . 0c.00.14 23 47.77 39.81 4.16 1834 4.74 14.92 609 3.21 8.00 530 0.00 .... . 0.00 .... . 0c.00.16 24 48.31 40.05 4.13 1829 4.98 14.28 688 3.27 7.88 584 0.00 .... . 0.00 .... . Aggregate statistics: Minimum 2 5.78 0.00 3.87 1.60 0.00 0.00 Mean 18 38.46 30.95 4.52 2.99 0.00 0.00 Maximum 24 48.31 40.63 5.38 3.37 0.00 0.00 Spares and other disks: 0c.00.3 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . Spares and other disks: 0c.00.18 0 0.00 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . 0.00 .... . FCP Statistics (per second) 0.00 FCP Bytes recv 0.00 FCP Bytes sent 0.00 FCP ops iSCSI Statistics (per second) 2672116.49 iSCSI Bytes recv 8064194.15 iSCSI Bytes xmit 434.87 iSCSI ops Interrupt Statistics (per second) 999.38 Clock (IRQ 0) 0.05 Uart (IRQ 4) 1766.58 PCI direct (IRQ 16) 94.52 PCI direct (IRQ 17) 0.00 RTC 2860.54 total NVRAM Statistics (per second) 0.00 total dma transfer KB 0.00 wafl write req data KB 0.00 dma transactions 0.00 dma destriptors 28.36 transactions not queued 67.01 transactions queued 12.38 total waittime (MS) 29.34 completion proc wakeups 18.39 completion proc waitdone 186.57 waitdone preempts 0.00 waitdone delays 2698.66 total nvlog KB 0.00 nvlog shadow header array full 0.00 channel1 dma transfer KB 0.00 channel1 dma transactions 0.00 channel1 dma descriptors E7520 Data Mover Statistics (per second) 3810.35 total dma transfer KB 1.18 total bcopy transfer KB 4.22 total waittime (MS) I've also got some 'sysstat -i 5' numbers: > sysstat -i 5 CPU NFS CIFS iSCSI Net kB/s Disk kB/s iSCSI kB/s Cache in out read write in out age 15% 0 195 247 1232 4944 7678 4267 1104 4629 1 16% 0 238 352 1959 6185 7414 3201 1770 4791 1 11% 0 135 264 1908 4533 4946 0 1770 4257 1 18% 0 171 246 1155 4372 6915 7230 1046 4084 1 14% 0 230 298 1868 4478 4892 0 1717 4184 2 17% 0 164 284 1113 4955 7014 5319 1001 4683 2 9% 0 18 238 1395 3836 4217 0 1303 3654 2 15% 0 12 245 1759 4110 7539 3378 1653 3912 3 13% 0 21 244 900 5029 6302 3205 824 4814 2 11% 0 22 282 1569 5152 5866 0 1461 4910 2 19% 0 21 389 2280 5927 9750 6982 2132 5644 2 14% 0 49 401 1746 6072 6506 0 1614 5775 3 27% 0 83 491 5246 8447 12100 7693 4961 7985 2 11% 0 126 263 1772 4800 5258 0 1642 4553 2 16% 0 20 487 802 7104 10182 718 694 6800 6 25% 0 15 518 1763 6440 10771 9322 1621 6144 2 15% 0 81 389 1745 6271 6844 0 1607 5979 3 20% 0 140 287 906 6280 10088 7681 782 5965 4 8% 0 70 201 1163 4381 4638 0 937 4148 2 Any thoughts? Thnaks, John John Stoffel - Senior Staff Systems Administrator - System LSI Group Toshiba America Electronic Components, Inc. - http://www.toshiba.com/taec john.stoffel(a)taec.toshiba.com - 508-486-1087

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters October 2010