toasters March 2013

toasters@lists.teaparty.net

26 participants
15 discussions

File Access / Size Usage Reports
by Ray Van Dolson 13 Mar '13

13 Mar '13

We have several large CIFS shares managed by internal departments. I'd like to be able to get reports based on which directories contain the most data / files (inodes) and also which files aren't being accessed very frequently and as such are candidates for archival / deletion. We've historically used CommVault's SRM for the latter, but the Backup team is telling me it won't work on CIFS shares as it needs an agent. Essentially looking for something like JdiskReport plus file access reporting. Could probably run tools direct from OS's accessing the shares (and that may be the right thing to do). Am wondering what all of you use out there? Ray

4 3

FCP Fan In question
by Milazzo Giacomo 11 Mar '13

11 Mar '13

Hi all, we're setting up a very large FCP environment on a 6240HA with 4 target HBA each dedicated to that SAN. Hosts are IBM AIX LPAR each with 4 NPIV HBAs coming from two different VIOS. We will have more or less 300 LPARs. Both NetApp controllers and hosts are simmetrically zoned and divided on two FC switches, as usual. Well, the FC & iSCSI Configuration Guide report the limits for each family of controller and for 6200 one the FC port fan-in is 64. [cid:image001.png@01CE1E58.FE336720] Is it really the maximum number of hosts they mean? Not the inititators number? I ask it because if you look at the current fcp stat you can see we already have roughly 40 initiators connected to each target and we'll quickly reach that limit. If that is the host number we can easily add more target ports to balance workloads...but if that number is "initiators" we're in trouble. Thanks in advance, Regards V6240A*> fcp stats Statistics for adapter 0c Read Ops: 142646 Write Ops: 5387410 Other Ops: 5501 KBytes In: 606619208 KBytes Out: 18650223 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 2 LIP Resets: 7 SCSI Requests Dropped: 0 Interrupts: 20804631 Spurious Interrupts: 0 Total Logins: 415 Total Logouts: 415 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 1 OLS Received: 8 Queue Depth: 0 Statistics for SFP on 0c Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53SSX Date Code: 12:42:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 442.5 (uWatts) SFP TX Optical Power: 460.1 (uWatts) Statistics for adapter 0d Read Ops: 0 Write Ops: 0 Other Ops: 0 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 0 LIP Resets: 0 SCSI Requests Dropped: 0 Interrupts: 4 Spurious Interrupts: 0 Total Logins: 0 Total Logouts: 0 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 0 Queue Depth: 0 Statistics for SFP on 0d Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM537SY Date Code: 12:24:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 0.0 (uWatts) (out of range) SFP TX Optical Power: 0.0 (uWatts) (out of range) Statistics for adapter 1a Read Ops: 56009868 Write Ops: 91968480 Other Ops: 7178759 KBytes In: 9625731996 KBytes Out: 1622330068 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 41 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 411105094 Spurious Interrupts: 0 Total Logins: 2143 Total Logouts: 2102 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 1a Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM5379D Date Code: 12:04:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 486.4 (uWatts) SFP TX Optical Power: 463.5 (uWatts) Statistics for adapter 1b Read Ops: 53939443 Write Ops: 48576966 Other Ops: 6672592 KBytes In: 1819149800 KBytes Out: 1220340034 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 47 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 242434524 Spurious Interrupts: 0 Total Logins: 2572 Total Logouts: 2525 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 1b Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53PGM Date Code: 12:17:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 619.2 (uWatts) SFP TX Optical Power: 463.3 (uWatts) Statistics for adapter 1c Read Ops: 0 Write Ops: 0 Other Ops: 5 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 587980 Spurious Interrupts: 0 Total Logins: 5 Total Logouts: 5 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 1c Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53V1V Date Code: 12:39:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 521.8 (uWatts) SFP TX Optical Power: 472.0 (uWatts) Statistics for adapter 1d Read Ops: 0 Write Ops: 0 Other Ops: 5 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 586249 Spurious Interrupts: 0 Total Logins: 5 Total Logouts: 5 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 1d Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM536P9 Date Code: 12:36:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 661.6 (uWatts) SFP TX Optical Power: 461.9 (uWatts) Statistics for adapter 0a Read Ops: 329944 Write Ops: 2818801 Other Ops: 949 KBytes In: 124038934 KBytes Out: 14749111 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 4 LIP Resets: 9 SCSI Requests Dropped: 0 Interrupts: 9006114 Spurious Interrupts: 0 Total Logins: 57 Total Logouts: 57 CRC Errors: 1 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 10 Queue Depth: 0 Statistics for SFP on 0a Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53NRB Date Code: 12:42:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 464.7 (uWatts) SFP TX Optical Power: 461.8 (uWatts) Statistics for adapter 0b Read Ops: 0 Write Ops: 0 Other Ops: 0 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 0 LIP Resets: 0 SCSI Requests Dropped: 0 Interrupts: 4 Spurious Interrupts: 0 Total Logins: 0 Total Logouts: 0 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 0 Queue Depth: 0 Statistics for SFP on 0b Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM537T0 Date Code: 12:28:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 0.0 (uWatts) (out of range) SFP TX Optical Power: 0.0 (uWatts) (out of range) Statistics for adapter 11a Read Ops: 56021935 Write Ops: 94502028 Other Ops: 7138201 KBytes In: 9659958208 KBytes Out: 1621021005 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 41 Link Breaks: 2 LIP Resets: 7 SCSI Requests Dropped: 0 Interrupts: 417616681 Spurious Interrupts: 0 Total Logins: 1925 Total Logouts: 1884 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 8 Queue Depth: 0 Statistics for SFP on 11a Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53SA4 Date Code: 12:18:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 440.0 (uWatts) SFP TX Optical Power: 463.0 (uWatts) Statistics for adapter 11b Read Ops: 53734018 Write Ops: 46687962 Other Ops: 6700517 KBytes In: 1799406809 KBytes Out: 1216393589 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 47 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 237637872 Spurious Interrupts: 0 Total Logins: 2443 Total Logouts: 2396 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 11b Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53V1Y Date Code: 12:16:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 639.0 (uWatts) SFP TX Optical Power: 470.1 (uWatts) Statistics for adapter 11c Read Ops: 0 Write Ops: 0 Other Ops: 6 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 2 LIP Resets: 7 SCSI Requests Dropped: 0 Interrupts: 587951 Spurious Interrupts: 0 Total Logins: 6 Total Logouts: 6 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 8 Queue Depth: 0 Statistics for SFP on 11c Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM538VV Date Code: 12:27:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 491.8 (uWatts) SFP TX Optical Power: 460.7 (uWatts) Statistics for adapter 11d Read Ops: 0 Write Ops: 0 Other Ops: 5 KBytes In: 0 KBytes Out: 0 Adapter Resets: 0 Frame Overruns: 0 Frame Underruns: 0 Initiators Connected: 0 Link Breaks: 1 LIP Resets: 6 SCSI Requests Dropped: 0 Interrupts: 586210 Spurious Interrupts: 0 Total Logins: 5 Total Logouts: 5 CRC Errors: 0 Adapter Qfulls: 0 Protocol Errors: 0 Invalid Transmit Words: 0 LR Sent: 0 LR Received: 0 Discarded Frames: 0 NOS Received: 0 OLS Received: 7 Queue Depth: 0 Statistics for SFP on 11d Vendor Name: FINISAR CORP. Vendor OUI: 0:144:101 Vendor PN: FTLF8528P2BNV Vendor Rev: A Serial No: PM53SS4 Date Code: 12:31:0 Media Form: SFP Connector: LC Wavelength: 850 (nm) Encoding: 8B10B FC Speed Capabilities: 2, 4, 8 Gbit/Sec SFP RX Optical Power: 607.2 (uWatts) SFP TX Optical Power: 465.4 (uWatts) Statistics for adapter vtic Read Ops: 246105 Write Ops: 2907529 Other Ops: 168582 KBytes In: 539680064 KBytes Out: 4695478 out_of_vtic_cmdblks: 0 out_of_vtic_msgs: 0 out_of_vtic_resp_msgs: 0 out_of_bulk_msgs: 0 out_of_bulk_buffers: 0 out_of_r2t_buffers: 0 Message: sent completed received VTIC_MESSAGE_L2P_NEW: 0 0 3322216 VTIC_MESSAGE_L2P_DATA_IN: 0 0 61561 VTIC_MESSAGE_L2P_DATA_IN_RESP: 0 0 271383 VTIC_MESSAGE_L2P_RESP: 0 0 3050832 VTIC_MESSAGE_L2P_DATA_OUT: 0 0 9196613 VTIC_MESSAGE_L2P_ABORT: 0 0 1 VTIC_MESSAGE_L2P_TASK_MGT: 112 112 1763 VTIC_MESSAGE_P2L_DATA_IN: 61561 61561 0 VTIC_MESSAGE_P2L_DATA_IN_RESP: 271383 271383 0 VTIC_MESSAGE_P2L_RESP: 3050833 3050833 0 VTIC_MESSAGE_P2L_R2T: 9196613 9196613 0 VTIC_MESSAGE_P2L_ABORT: 0 0 0 VTIC_MESSAGE_FCP_CONFIG_INITIALIZE: 3 3 0 VTIC_MESSAGE_FCP_CONFIG_UPDATE: 0 0 4 VTIC_MESSAGE_L2P_ADD_INITIATOR: 9576 9576 9056 VTIC_MESSAGE_L2P_REMOVE_INITIATOR: 9400 9400 8880 VTIC_MESSAGE_L2P_GET_LUN_LIST: 2139 2139 25 VTIC_MESSAGE_P2L_RETURN_LUN_LIST: 25 25 2139 VTIC_MESSAGE_DUMMY_UPDATE: 2213 2213 106 Dott. Giacomo Milazzo [cid:image001.jpg@01C75422.0D316270] Technical Account Manager Sinergy SpA Filiale di Roma * 00198. viale Regina Margherita, 269 ' (+39) 3406001045 0644245272 7 (+39) 0226922048 * Giacomo.Milazzo(a)Sinergy.it

1 0

wafl_cp_slovol_warning_1 with big latency spikes
by Fletcher Cocquyt 08 Mar '13

08 Mar '13

Yesterday morning one of the heads on our 3270 experienced large NFS latency spikes causing our VMware hosts and their VMs to log storage timeouts. This latency does not correlate to any external metrics like CPU, network, OPS etc. But in the logs do show CP events on the aggregate hosting the VMs: Jan 14 05:27:56 [n04:wafl.cp.slovol:warning]: aggregate aggr2 is holding up the CP. And the EMS log has CP events logged for the duration of the episode - what can we do to prevent these issues? <wafl_cp_toolong_warning_1 total_ms="117825" total_dbufs="32276" clean="4312" v_ino="3" v_bm="29" a_ino="0" a_bm="3428" flush="1209"/> </LR> <LR d="14Jan2013 05:19:38" n="irt-na04" t="1358169578" id="1335304168/148007" p="4" s="Ok" o="wafl_CP_proc" vf="" type="0" seq="633232" > <wafl_cp_slovol_warning_1 voltype="aggregate" volowner="" volname="aggr2" volident="" nt="35" nb="22045" clean="1346852" v_ino="0" v_bm="113" a_ino="0" a_bm="4" flush="0" rgid="2"/> Netapp support wants me to run perfstats, but the issue is not ongoing - things are idle thanks Fletcher Cocquyt Principal Engineer Information Resources and Technology (IRT) Stanford University School of Medicine Email: fcocquyt(a)stanford.edu Phone: (650) 724-7485

5 4

Anti Virus Scanning
by John Constable 05 Mar '13

05 Mar '13

Hi Folks, Does anyone have any recommendations (or horrible warnings, off list is fine!) for antivirus scanning integration with NetApp? We're starting to look into all this (again), and I'd be interested in people's experiences and recommendations, especially if anyone has used products like Avere to get round the performance hit that may result? We have a low double digit number of 7-mode HA systems, and are about to bring in a cluster, so we'd be looking at something that could do both, in an ideal world (I'd also like a pony). cheers John -- John Constable, Storage Team x6924 (+44 1223 496924) Wellcome Trust Sanger Institute, Wellcome Trust Genome Campus, Hinxton, Cambridge, UK -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

3 2

Re: wafl_cp_slovol_warning_1 with big latency spikes
by Brian Beaulieu 04 Mar '13

04 Mar '13

I had opened a case when this happened and let the TSE know that I'm not the only one having this issue.. asked for it to be escalated and treated as a possible bug. We'll see where it goes. Brian On Thu, Jan 17, 2013 at 4:16 PM, Scott Eno <s.eno(a)icloud.com> wrote: > We're on 8.1.1P1 across the environment. > > 7-mode, of course. > > > On Jan 17, 2013, at 3:35 PM, Brian Beaulieu <brian.beaulieu(a)gmail.com> > wrote: > > Scott, what version of OnTAP are you on? > We're on 8.1.1P1 > > I saw a BURT that sounded related to this but it was apparently fixed by > 8.1.1P1. > > Brian > > > On Thu, Jan 17, 2013 at 2:21 PM, Jeff Mohler <speedtoys.racing(a)gmail.com>wrote: > >> :s is wafl updating special files in the CP process. >> >> >> Going on _that_ long....??? A few seconds of special file updates in a >> CP sure, but that much? >> >> I'd be pretty pushy on getting an answer, id put that in the "its a big >> bug" bucket. That's not normal IO activity in a healthy system. >> >> >> On Thu, Jan 17, 2013 at 8:07 PM, Scott Eno <s.eno(a)me.com> wrote: >> >>> >>> Yes, this is what I see. ":s" and all the other protocols go to "0". >>> >>> There's been some correlation, when this happens, to cleanup of VMware >>> snapshots (not NetApp snaps on the volumes, but VMware snapshots of vm's >>> via vcenter). But it happens other times too. >>> >>> >>> >>> On Jan 17, 2013, at 1:47 PM, Brian Beaulieu <brian.beaulieu(a)gmail.com> >>> wrote: >>> >>> 3rd time is the charm. >>> >>> I've attached my sysstat from the other night when NFS/CIFS hung up... >>> is this what you've seen as well? >>> >>> During that issue, FCP was also slow.. had some MPIO failovers happening >>> on our AIX LPARs. But, AIX handles that just fine and at least has an >>> alternate path through the other filer. NFS isn't so lucky. >>> >>> I have a 3250+1TB PAM sitting on deck.. you'd think that the 3240+512GB >>> PAM would be sufficient for what we do. >>> While I do have SATA in use for VMWare, it's not heavy hitting VMs.. >>> it's the dormant stuff, mostly. >>> I'm moving a lot of it, though, to 6xDS4243x600GB-15k shelves ASAP. >>> >>> I'm drinking the PAM kool-aid too but do have some measurable results >>> primarily on our PeopleSoft DB2 databases. >>> I definitely wouldn't bet on SATA+PAM == FC/SAS performance. >>> >>> Brian >>> <sysstat - Copy.txt>_______________________________________________ >>> >>> Toasters mailing list >>> Toasters(a)teaparty.net >>> http://www.teaparty.net/mailman/listinfo/toasters >>> >>> >>> >>> _______________________________________________ >>> Toasters mailing list >>> Toasters(a)teaparty.net >>> http://www.teaparty.net/mailman/listinfo/toasters >>> >>> >> >> >> -- >> --- >> Gustatus Similis Pullus > > > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > http://www.teaparty.net/mailman/listinfo/toasters > > >

4 3

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters March 2013