toasters

toasters@lists.teaparty.net

13513 discussions

ssh timing out (filer high CPU load)
by Edward Rolison 11 Aug '16

11 Aug '16

On the off chance - I'm having trouble with a filer. I can't ssh to it reliably (at all, mostly). I'm pretty sure that's correlated with some high CPU load - my system console has it 'spiked' at >95% for the last 24h, and that's much higher than 'normal'. What i'm not sure of is quite what's causing it - the filer is busy, but not abnormally so. The only thing I can think of that _might_ have changed it, is api calls (qtree-list, get-file-info) - I've recently started doing quota snmp trap enrichment. (but thats 'every few minutes' at most). But otherwise - I'm not sure what might be causing sshd to stall, and if there's a way to 'kick' it? This is a 7 mode filer, on 8.2.1 I've got a case open, but would appreciate any further insight on how to track a high CPU-causing ssh to not respond type issue. I'm pretty sure a failover/failback will do the trick, but that'll have to wait until the weekend - I'd like not to if I can manage it. My current ps list looks like: Process statistics over 67.328 seconds... ID State Domain %CPU StackUsed %StackUsed Name 195 RR N 47% 6928 10% NwkThd_00 196 RR N 47% 7880 12% NwkThd_01 197 RR 0 47% 6928 10% NwkThd_02 223 BR s 7% 7648 46% pmcsas_intrd_1 259 BR e 5% 2440 19% fal_io_thread2 502 BR R 7% 7448 45% raidio_thread 503 BR R 7% 7448 45% raidio_thread 635 BG k 6% 15184 11% snmpd 1614 BR 0 5% 3464 10% ntm_main 1711 RR w 35% 14256 21% wafl_exempt00 1712 BR w 35% 14136 21% wafl_exempt01 1713 BR w 35% 14136 21% wafl_exempt02 2599 BR k 5% 2752 8% gr_scheduler That seems pretty busy for a 4cpu system... Thanks and regards, Ed.

2 5

cdot initial setup help
by Mr. John W. Miller 10 Aug '16

10 Aug '16

Greetings, I have been using Netapp and ONTAP for years. I am new to CDOT. I am setting up a FAS2552 system running NetApp Release v8.3P1. I have a single HA Pair and have created a two node cluster. This should be very simple. I am looking for simple examples of how to initially configure my cluster. Where do I find such examples or templates? thanks

2 1

Snapshot deletion performance impact
by Basil 05 Aug '16

05 Aug '16

Hi folks, I need to delete a bunch of snapshots that were orphaned by CDOT when someone changed the schedule. It's a lot of data- 500gb for a 1tb LUN. How does ontap prioritize releasing these blocks after the delete? What's the best way in 8.2.3 to measure the impact, if any?

2 1

OnTAP + Brocade's Target Driven Zoning (TDZ)
by Momonth 03 Aug '16

03 Aug '16

Hi, I'm looking at simplifying FC zone management. And since I am 100% Brocade shop, then "Target Driven Zoning (TDZ)" sounds like a possible solution - http://www.brocade.com/content/html/en/administration-guide/fos-741-admingu… I couldn't google anything NetApp's OnTAP wise in this regard, so I guess the TDZ is not supported, but maybe I'm missing something here. Does anyone have any experience with $subj? Cheers, Vladimir

1 0

Kerberized NFSv4 in 7-Mode (AD)
by Ray Van Dolson 30 Jul '16

30 Jul '16

Need some guidance in getting this working. Have an 8.2.2P1 7-mode filer, joined to AD (working fine w/ CIFS) and have used 'nfs setup' to enable Kerberized NFS. However, I noticed that no nfs/* entries are present when I run setspn -L <FILER> from a Windows box. I was under the impression that nfs setup should create these? Or, since we're joined to AD already, perhaps the established machine account is sufficient to obtain one on the fly... I have the following export on a volume with ntfs security style: /vol/raytest -sec=krb5:krb5i:krb5p,rw,anon=0,nosuid On my client (RHEL7+SSSD), I'm fairly confident my setup for Kerberos is good. I can SSH in with a Kerberos login, do kinit, etc and connect to other Kerberized services (like doing an ldapearch do AD without being prompted for a password using -Y GSSAPI as long as I have a valid tgt). I've tried my krb5.conf with and without the following: default_tgs_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC default_tkt_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC preferred_enctypes = RC4-HMAC DES-CBC-MD5 DES-CBC-CRC I also ran 'net ads keytab add nfs' on my client and see that I have a valid nfs/ entry in my machine's keytab file. If I run the following (either as root or as a user -- below is as a user with a valid TGT): $ sudo mount.nfs4 -o vers=4,sec=krb5 red-str-napc2-p12.esri.com:/vol/raytest /mnt/nfs I get a successful mount and also see a valid nfs ticket in klist. However, when I try to cd to /mnt/nfs, I get a "Permission Denied" error. I have rpc.gssd running in the foreground in verbose mode but don't really see anything odd. Wireshark tells me I've asked for read, lookup, modify, extend and delete permissions but that access was denied and only delete was allowed. I can see that the RPC portion of the packets are speaking Kerberos, but I can't tell much more than that. DNS, NTP, etc. is all set up and working on both sides. I've used sectrace on the /vol/raytest path above to try and catch issues, but get no hits making me think it's NFS on the NetApp rejecting things, not the file system layer... fsecurity shows that everything is wide open and from a CIFS client using the same user as on the Linux box, I can create files on the same path with no issues. Kinda stumped at this point. Any suggestions? Ray

3 8

displayed unix permissions on ntfs qtree
by Fred Grieco 22 Jul '16

22 Jul '16

I'm having an issue on the displayed permissions in linux, on an ntfs qtree. This is in cDOT 8.2.3. I have a vserver that's joined to an AD domain and NIS-enabled. Basically, most of the permissions display rwxrwxrwx on the linux, and it's not clear where it's getting these. The NIS/nfs permission themselves are obeyed -- I can only get to where I have access, on the linux side. This is a snapmirrored volume/qtree from a 7-mode filer. It's user directories The linux permissions from the 7-mode filer are almost exclusively rwx------. The ntfs permissions on the source and destinations match, and the NIS/AD/namemapping configs are the same. I"m not sure why it's not displaying the same permissions from linux on the source and destination. Fred

3 7

oracle linux 5.6 iscsi client multipath question
by Natxo Asenjo 18 Jul '16

18 Jul '16

hi, I am migrating 2 7mode iscsi luns to an iscsi svm on a cdot cluster (8.3.2). The migration is fine, I am now testing the luns from the client os (oracle linux 5.6, I do not have any say on the client os,). So I have followed the instructions in here: https://library.netapp.com/ecmdocs/ECMP1654943/html/index.html I see the luns, but now I see 2x 4 luns, because the cdot cluster has 4 iscsi lifs. So I need to configure multipathd. I have this /etc/multipath.conf defaults { user_friendly_names yes } blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^sd[a-c]" } multipaths { multipath { wwid 3600a098038303370562b4946426b6742 alias netapp_oracle_recovery } multipath { wwid 3600a098038303370562b4946426b6743 alias netapp_oracle_datafile } devices { device { vendor "NETAPP" product "LUN" path_grouping_policy group_by_prio features "1 queue_if_no_path" prio_callout "/sbin/mpath_prio_alua /dev/%n" path_checker directio path_selector "round-robin 0" failback immediate hardware_handler "1 alua" rr_weight uniform rr_min_io 128 getuid_callout "/sbin/scsi_id -g -u -s /block/%n" } } I blacklist sda, sdb and sdc (local vmware disks). This is the output of multipath -ll: # multipath -ll netapp_oracle_datafile (3600a098038303370562b4946426b6743) dm-3 NETAPP,LUN C-Mode size=350G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=4 status=active | |- 6:0:0:1 sdg 8:96 active ready running | `- 3:0:0:1 sdf 8:80 active ready running `-+- policy='round-robin 0' prio=1 status=enabled |- 4:0:0:1 sdj 8:144 active ready running `- 5:0:0:1 sdk 8:160 active ready running netapp_oracle_recovery (3600a098038303370562b4946426b6742) dm-2 NETAPP,LUN C-Mode size=300G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=4 status=active | |- 3:0:0:0 sdd 8:48 active ready running | `- 6:0:0:0 sde 8:64 active ready running `-+- policy='round-robin 0' prio=1 status=enabled |- 5:0:0:0 sdh 8:112 active ready running `- 4:0:0:0 sdi 8:128 active ready running And after installing the netapp host tools, this is the output of sanlun lun show -p # sanlun lun show -p ONTAP Path: ALR-SVM-iSCSI:/vol/vol_oracle_datafile/q_oracle_datafile/lun_oracle_datafile LUN: 1 LUN Size: 350.1g Product: cDOT Host Device: netapp_oracle_datafile(3600a098038303370562b4946426b6743) Multipath Policy: round-robin 0 Multipath Provider: Native --------- ---------- ------- ------------ ---------------------------------------------- host vserver path path /dev/ host vserver state type node adapter LIF --------- ---------- ------- ------------ ---------------------------------------------- up primary sdg host6 iscsi_lif02b up primary sdf host3 iscsi_lif02a up secondary sdj host4 iscsi_lif01a up secondary sdk host5 iscsi_lif01b ONTAP Path: ALR-SVM-iSCSI:/vol/vol_oracle_recovery/q_oracle_recovery/lun_oracle_recovery LUN: 0 LUN Size: 300g Product: cDOT Host Device: netapp_oracle_recovery(3600a098038303370562b4946426b6742) Multipath Policy: round-robin 0 Multipath Provider: Native --------- ---------- ------- ------------ ---------------------------------------------- host vserver path path /dev/ host vserver state type node adapter LIF --------- ---------- ------- ------------ ---------------------------------------------- up primary sdd host3 iscsi_lif02a up primary sde host6 iscsi_lif02b up secondary sdh host5 iscsi_lif01b up secondary sdi host4 iscsi_lif01a and this is sanlun lun show: sanlun lun show controller(7mode/E-Series)/ device host lun vserver(cDOT/FlashRay) lun-pathname filename adapter protocol size product ----------------------------------------------------------------------------------------------------------------------------------------------- ALR-SVM-iSCSI /vol/vol_oracle_datafile/q_oracle_datafile/lun_oracle_datafile /dev/sdk host5 iSCSI 350.1g cDOT ALR-SVM-iSCSI /vol/vol_oracle_recovery/q_oracle_recovery/lun_oracle_recovery /dev/sdi host4 iSCSI 300g cDOT ALR-SVM-iSCSI /vol/vol_oracle_datafile/q_oracle_datafile/lun_oracle_datafile /dev/sdj host4 iSCSI 350.1g cDOT ALR-SVM-iSCSI /vol/vol_oracle_recovery/q_oracle_recovery/lun_oracle_recovery /dev/sdh host5 iSCSI 300g cDOT ALR-SVM-iSCSI /vol/vol_oracle_datafile/q_oracle_datafile/lun_oracle_datafile /dev/sdf host3 iSCSI 350.1g cDOT ALR-SVM-iSCSI /vol/vol_oracle_datafile/q_oracle_datafile/lun_oracle_datafile /dev/sdg host6 iSCSI 350.1g cDOT ALR-SVM-iSCSI /vol/vol_oracle_recovery/q_oracle_recovery/lun_oracle_recovery /dev/sde host6 iSCSI 300g cDOT ALR-SVM-iSCSI /vol/vol_oracle_recovery/q_oracle_recovery/lun_oracle_recovery /dev/sdd host3 iSCSI 300g cDOT In /dev/mapper I see this: ls -l /dev/mapper/ total 0 crw------- 1 root root 10, 62 Jul 13 14:00 control brw-rw---- 1 root disk 253, 3 Jul 13 14:01 netapp_oracle_datafile brw-rw---- 1 root disk 253, 4 Jul 13 14:01 netapp_oracle_datafilep1 brw-rw---- 1 root disk 253, 2 Jul 13 14:01 netapp_oracle_recovery brw-rw---- 1 root disk 253, 5 Jul 13 14:01 netapp_oracle_recoveryp1 brw-rw---- 1 root disk 253, 0 Jul 13 14:00 VolGroup00-LogVol00 brw-rw---- 1 root disk 253, 1 Jul 13 14:00 VolGroup00-LogVol01 Is this ok? What do I have to use in /etc/fstab? /dev/mapper/netapp_oracle_datafile en /dev/mapper/netapp_oracle_recovery? Or am I doing something terribly wrong. I called with support but as this is no break or fix stuff, they recommend calling professional services ... Apologies for the wall of text. Thanks in advance. -- Groeten, natxo

2 3

cDOT cluster replication aka peering via specific LIFs
by Momonth 13 Jul '16

13 Jul '16

Hi All, I have two production cDOT clusters out of 8 nodes in two DCs, running OnTAP 8.3.2. I ended up having a bit of a weird network topology where 6 nodes (3 pairs) are only 1Gb enabled and 2 nodes (1 pair) are 10Gb enabled. I wonder how can I enforce cluster peering traffic via those 10Gb enabled nodes? I know that I have to have ICL LIFs on every node in the clusters, and that's fine, I can do that, but would like to set something like a preference for ICL traffic to flow via 10Gb enabled LIFs. Cheers, Vladimir

5 20

is this a bug ?
by Peter D. Gray 11 Jul '16

11 Jul '16

Hi guys I am running 8.2.3 cluster mode. We usually run 30 days of daily snaps on our volumes. We also use snapmirror dr extensively. What I am noticing is that we are missing daily snapshots with regularity. I think the problem is that than the time for dailys rolls around, if the volumes are involved in a snapmirror operation at the time, the daily snap is missed. Anybody seen this and if so, is it a known bug or is this documented behaviour or am I barking up the wrong tree entirely? Regards, pdg

3 2

cDOT .snapshot access over NFS?
by andrei.borzenkov＠ts.fujitsu.com 10 Jul '16

10 Jul '16

I always configured 7-Mode systems with "nosnapdir on" (because quite a lot of programs were confused by .snapshot directory and its content) but still was able to explicitly mount any snapshot for purposes of restore. This seems to be no more possible in cDOT - snapdir-access is apparently all or nothing for NFS clients; either I can mount snapshots but also get .snapshot visible, or .snapshot is hidden but then I cannot mount them. This is different in case of CIFS where access and visibility are controlled separately. I just verified that this is still the case under ONTAP 9.0RC1. Anyone knows some (hidden) knob that provides 7-Mode semantic on NFS? Thank you! --- With best regards Andrei Borzenkov Senior system engineer FTS WEMEAI RUC RU SC TMS FOS FUJITSU Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation Tel.: +7 495 730 62 20 ( reception) Mob.: +7 916 678 7208 Fax: +7 495 730 62 14 E-mail: Andrei.Borzenkov(a)ts.fujitsu.com Web: ru.fujitsu.com Company details: ts.fujitsu.com/imprint This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation. Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters