toasters August 2021

toasters@lists.teaparty.net

20 participants
24 discussions

Upgrade from 9.3 to 9.7
by Philbert Rupkins 21 Sep '21

21 Sep '21

Hello Toasters, Anybody have experience, positive or negative, upgrading from 9.3 to 9.7? It is a supported automated upgrade path but requires packages for both 9.5 and 9.7. My understanding is each node will be upgraded twice (once to 9.5 then to 9.7) but otherwise should follow the typical automated upgrade routine. Thank You Phil

11 17

options for adding a shelf/disks to an AFF-A220
by Rue, Randy 01 Sep '21

01 Sep '21

Hi All, I've cabled up the second external shelf for our AFF-A220 and it sees the disks. Now I'm at a fork in the road. We currently have two local data tiers made up of 24 disks each, one on each node. Best practices at https://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.dot-cm-ps… say a disk group of SSDs should be from 20-28 SSDs each. If I go to create a new local tier the default is to create two new tiers, one on each node. That would give me two disks groups of 12, below the best practice. How badly would performance suffer? Any other caveats? Reliability? I could put them all on one disk group, and on one node? Bleah. I could add half of them to each existing tier/disk group and exceed the best practice by eight SSDs. I believe I've seen discussions (get it?) claiming that top limit is flexible with SSDs, was it here? But when I add another shelf I'll have the same problem. Seems like the clearest path forward for consistency and future expansions is to add two tiers with small disk groups. So, can anyone guess if I'll have a significant performance impact? Any other caveats? Grateful for any guidance, Randy in Seattle

4 5

Bad Labels. . .
by Carl Howell 20 Aug '21

20 Aug '21

I have a couple of shelves of 900GB SAS that were just sanitized from a now retired 7-Mode cluster. I'd like to use these drives as spares in a C-Mode cluster, but they come up as "bad label." I've looked on the NOW/MySupport site, but wanted to check here before I attempt to do anything. . . Thanks, --Carl

2 1

wedged updates to changes to users' group memberships
by Randy Rue 19 Aug '21

19 Aug '21

Hello All, We recently cut over from a FAS to an AFF running 9.8P1. Serving CIFS and NFSv3 with authentication tied to openLDAP. When we change group memberships in LDAP the filer fails to see the change in group memberships for some varying amount of time. We had been able to use the flush command to force the filer to update it's user/group membership lists. This was changed in our current version, 9.8P1, and we have switched to using the replacement command. This used to work: set diag; diag nblade credentials flush -node $NODE -vserver $VSERVER -unix-user-name $USER We think this should be the new command but does not seem to perform as expected: set -privilege advanced -confirmations off; credentials flush -node $NODE -vserver $VSERVER -unix-user-name $USER The new flush command does not seem to perform with any reliability. It's worth noting that we have many groups so have set the group limit to 512 as described here: https://library.netapp.com/ecmdocs/ECMP1610208/html/GUID-1D3D018C-DF37-4C87… As of this writing, we have 2 users whose group memberships were updated over 24 hours ago and they are still unable to access the relevant files over CIFS. We were able to demonstrate that the user can in fact access those files over NFS using the `newgrp` command to select the new group and access the files however after dropping the `newgrp` identity, we were once again denied access to the location. Are we missing something? Grateful for any help... Randy Rue in Seattle

3 4

Re:
by Basil 18 Aug '21

18 Aug '21

Not from a position of experience, but if I had to hazard a guess, I think that deduplication would eventually cause all the blocks to be recognized and squished. On Wed, Aug 18, 2021 at 12:49 PM Scott Eno via Toasters < toasters(a)teaparty.net> wrote: > > > > ---------- Forwarded message ---------- > From: Scott Eno <cse(a)hey.com> > To: Toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Wed, 18 Aug 2021 12:46:10 -0400 > Subject: move files without snapshot growth? > Hi, > > Anyone on this list have a favorite way of moving files from one folder to > another folder on the same CIFS volume without causing snapshot growth? > > I gotta move 10TB+ of data from one folder structure to another folder > structure on the same CIFS vol. We keep snapshots for a year so I really > don't want to hang on to 10TB+ of snapshot data for a year. > > Can XCP do this somehow? Powershell? > > Sorry if this ends up wrapped in a no subject email. I don't know what > causing that. > > > > ---------- Forwarded message ---------- > From: Scott Eno via Toasters <toasters(a)teaparty.net> > To: Toasters <toasters(a)teaparty.net> > Cc: > Bcc: > Date: Wed, 18 Aug 2021 17:46:13 +0100 > Subject: > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters

2 1

move files without snapshot growth?
by Scott Eno 18 Aug '21

18 Aug '21

Hi, Anyone on this list have a favorite way of moving files from one folder to another folder on the same CIFS volume without causing snapshot growth? I gotta move 10TB+ of data from one folder structure to another folder structure on the same CIFS vol. We keep snapshots for a year so I really don't want to hang on to 10TB+ of snapshot data for a year. Can XCP do this somehow? Powershell? Sorry if this ends up wrapped in a no subject email. I don't know what causing that.

1 0

Re: pNFS
by Michael Bergman 17 Aug '21

17 Aug '21

On 2021-08-17 14:26, Justin Parisi wrote: > NFSv4.1 performance is nearly identical to v3 in ONTAP for sequential > reads/writes. The performance issues mostly come from high metadata > workloads. If you do want to use v4.1, move to ONTAP 9.9.1 and the latest > Linux kernels you can. Yes. And therein lies the problem. Anything which has many files (high inode density), many open() and close() operations and lots of deletes of [small] files will suffer significantly (higher processing util in WAFL and extra latency) from using v4.1 The current performance penalties in v4.1 w.r.t. ONTAP compared to v3, are not easy to fix. Maybe something can be done inside ONTAP to mitigate but it's never going to "go away" easily to become on par overall with v3 performance any day soon. Metadata ops (esp GETATTRv3) is very very optimised in ONTAP in v3, for example I agree 100%: use 9.1.1 (latest stable Px whatever it is) and make sure that you have the latest Linux [kernel] version(s) you possibly can. It will help. (One of the very latest is highly preferable.) If you want to get proper enterprise level support, then use the latest RHEL 8.x, keep it well patched all the time, and you'll at least lower the risk if having issues. If you have a specific application system(s), e.g. one which really depends on v4.1 ACLs (I have yet to hear about such an animal that isn't esoteric) or some other feature v3 doesn't have (there aren't many IMO) then you can probably "contain", and hence control that environment much better including the involved NFS clients. If not... caveat emptor. /M -- Michael Bergman Sr Systems Analyst / Storage Architect michael.bergman(a)ericsson.com BNEW Service Area Networks, Lab Ops Phone +46 10 7152945 Global Network & Tech, Architecture SMS/MMS +46 70 5480835 Ericsson Torshamnsg 36, 16480 Sthlm, Sweden -- This communication is confidential. We only send and receive email on the basis of the terms set out at www.ericsson.com/email_disclaimer

1 0

Re: pNFS
by Michael Bergman 17 Aug '21

17 Aug '21

On 2021-08-17 14:23, Justin Parisi wrote: > Referrals probably have the same issue. Will need to test when I get back > into the office. Unless you seek to eliminate/avoid, all CI (Cluster Interconnect) traffic at all cost, I don't really see that that fact matters too much [at mount time]. (Good point about FlexGroups though... I don't have a good solid solution for those scenarios) We have since long (years) strived to avoid CI traffic as much as we can, but we're not as diligent anymore. Note: we do *not* use that DNS function ONTAP has in it which is intended to help spreading all the traffic (mount from clients) to various LIFs (based on the "load" on the nblades). Again, this takes place at mount time, once a mounts sits on a LIF it stays there... That DNS function (it is based on an algorithm using various "weights") suffers from serious drawbacks in some situations and we can't take that risk in our environment. Anyone who has seen and experienced a "mount storm" will know what I mean. How important locality is (minimal CI traffic) for you I guess can depend a bit on what model/power of CI switch you have. If there's enough bandwidth for the workload streams going across it, it only induces a little extra latency -- that may matter for some workloads, but I'd argue that for most it won't. Once you're on CI switch with 100 GbE, my take is that most people don't have to bother that much about it. With 100 GbE the latency is lower as well /M Justin Parisi wrote: > But referrals only help with initial mount; after the mount is established, > the connections stay where they were established. If a volume moves to a new > node, that doesn't keep locality. They're also not much help with FlexGroup > volumes. >

1 0

Re:
by Parisi, Justin 17 Aug '21

17 Aug '21

Nconnect can help with some performance issues, provided the performance issues are related to things that nconnect can help with - such as slot table exhaustion or a single TCP connection that has maxed out what it can do bandwidth wise. NFSv4.1 performance is nearly identical to v3 in ONTAP for sequential reads/writes. The performance issues mostly come from high metadata workloads. If you do want to use v4.1, move to ONTAP 9.9.1 and the latest Linux kernels you can. ________________________________ From: Toasters <toasters-bounces(a)teaparty.net> on behalf of Michael Bergman via Toasters <toasters(a)teaparty.net> Sent: Tuesday, August 17, 2021 7:57 AM To: Toasters Subject: NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. _______________________________________________ Toasters mailing list Toasters(a)teaparty.net https://www.teaparty.net/mailman/listinfo/toasters

1 0

Re: pNFS
by Sebastian Goetze 17 Aug '21

17 Aug '21

Hi Justin, I'm wondering: Does that also go for *referrals*? Would the IP returned to the client respect the incoming LIF's Broadcast Domain and return the IP of a LIF in the same BD? Or should you refrain from referrals in that situation, too? I checked TR-4067, but didn't find a mention about that. @Florian: Could be, that NFS4.0/4.1 referrals are the solution to keeping your paths local, especially, if your volumes do not move around... Sebastian On 16.08.2021 18:41, Parisi, Justin via Toasters wrote: FYI TR-4067 calls out this exact issue as well. There’s currently no way to “blacklist” inelegible interfaces for pnfs. If it’s in the SVM, it will be used with pNFS, regardless of if it’s reachable. We have an open RFE to allow you to specify LIFs for pNFS to avoid this problem. In the meantime you can blacklist devices from the client as per page 54 of the tr. > _______________________________________________ > Toasters mailing list > Toasters(a)teaparty.net > https://www.teaparty.net/mailman/listinfo/toasters

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters August 2021