toasters November 2005

toasters@lists.teaparty.net

59 participants
55 discussions

RE: howto document for initial volume architecting?
by Palmer, Jason 10 Nov '05

10 Nov '05

If you use Qtrees you can snapmirror on a Qtree basis (QSM) rather than a whole volume - does not bring across the snapshot history like a volume snapmirror (VSM) You can also set Qtree Quotas based upon the Qtree directory, and also you can have different security styles set on a Qtree by Qtree basis. Some backup vendors (Legatto / Symantec nee Veritas) also recognise Qtrees and allows administrators to make the backups more granular, breaking down a potentially multi Tb volume backup into a number of more manageable Gb Qtrees. Qtrees make manipulating and managing data a little easier - sometimes if you don't think you will need to use them, you find that you wish you had at a later date :o) Been there ! Jason -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: 10 November 2005 18:49 To: Jerry Cc: John Stoffel; toasters(a)mathworks.com Subject: Re: howto document for initial volume architecting? On 11/10/05, Jerry <juanino(a)yahoo.com> wrote: > Whats the primary reason to use qtrees under flexvols? from my reading, it looks like the only way to define access rights in a non-unix way is to have qtrees. i'd think this could be critical to admins in heterogeneous environments. i found qtrees to be useful yesterday for their nfs ops stats. if you are stuck with one big volume of data, and all your other stats are aggregate, that's a blessing. ...lori -------------------------------------------------------------------------------- The information contained herein is confidential and is intended solely for the addressee. Access by any other party is unauthorised without the express written permission of the sender. If you are not the intended recipient, please contact the sender either via the company switchboard on +44 (0)20 7623 8000, or via e-mail return. If you have received this e-mail in error or wish to read our e-mail disclaimer statement and monitoring policy, please refer to http://www.drkw.com/disc/email/ or contact the sender. 3167 --------------------------------------------------------------------------------

1 0

howto document for initial volume architecting?
by Lori Barfield 10 Nov '05

10 Nov '05

we have the opportunity to build several filers from scratch for our LAN, which will serve all departments in the company, fileshare through Samba, host htdocs for the internal web site, etc. can someone please point me to a good document that offers guidelines for doing the rearchitecting? our architecture is currently flat- one volume per filer with the data only split up into separate directories. all hosts can mount all directories on all volumes if needed. we are overpowered so i'm not too concerned about performance. it does makes sense to have multiple volumes for varying snap schedules, for instance; however i'd like to learn more about what the trade-offs are because i do understand there is a storage penalty. the security paradigm is to depend on firewall protection on the perimeter and to have a relatively open internal network. backups will be snap mirrors to a sibling filer at our offsite colo, with an occasional archival to tape for some of the data. thanks in advance. ...lori

3 3

RE: snapmirroring over WAN
by Lai, Derek 10 Nov '05

10 Nov '05

Lori, Local initial snapmirror is a good idea whenever it's an option. I've setup DR to St. Louis first and now Plano, Texas and it's been very reliable. Snapmirroring is also pretty good at setting up a checkpoint re-start if it has problem getting through. I am also pretty confident that it knows how to handle overlapping snapmirror. I've seen it happen to us when there is networking problem. If it detects that a snapmirroring is still in progress it will not kick off the same snapmirror. Snapmirror has been really reliable. I've had lots more problem with the WAN. The only problem I've seen with Snapmirror is when you run out of disk space (We are using mostly qtree snapmirror). Derek -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 9:30 AM To: toasters(a)mathworks.com Subject: Re: snapmirroring over WAN On 11/10/05, Brian Parent <bparent(a)calvin.ucsd.edu> wrote: > If I recall correctly, snapmirror uses very little bandwidth, and > shouldn't be a problem on any modern network. It only copies the > blocks which were changed, rather than entire files. ah yes, sorry that i didn't mention this point...we're doing an initial ndmpcopy over a little gig switch here at the office before moving the filer to the colo. some of our data is going to be large image files, so we aren't sure yet how much bandwidth the syncs are going to want. i was more concerned about the jerks and jolts and irregular paths you get when you fling data out into the wild...not sure how well netapps handle hiccups in the middle of snapping. ...lori

1 0

RE: snapmirroring over WAN
by Johnson, John T. 10 Nov '05

10 Nov '05

SnapMirror will consume as much bandwidth as it can get. I have a DS3 between Minneapolis and Denver and SnapMirror will flood it 100% no matter if it's a full or incremental transfer. The only difference is how long it floods the pipe. There's two areas where you can change this. If you can set QOS on your VPN link between Filers IP addresses and set a reasonable bandwidth parameter, I would do that. Otherwise you can set in in Filerview when setting up the Snapmirror volume relationship under Maximum Transfer Rate. This can be tedious as you have to set it on each replicated volume. We set it on our Cisco router to run at 10 Mbit during the day and automatically jump to 40 Mbit after 10:00 PM. I replicate Exchange and SQL data daily with logs getting replicated every 2 hours. My delta changes are roughly 300 gigs per night and at 40 Mbits (the other 5 Mbits we reserve for AD replication and normal network traffic), it takes about 6 hours. SnapMirror also doesn't thread its transfers. It just blasts the pipe so if you can stagger your transfers you should get better throughput. Good luck, John -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 10:50 AM To: toasters(a)mathworks.com Subject: snapmirroring over WAN for offsite backups, we hope to snapmirror our new fas 3020 on a regular schedule via vpn over the internet to an older filer at our colo. obviously, intermittent latency is a concern since we don't have a dedicated link. and so is the potential for overpowering the modest firewall/network hardware currently in place. does anyone have a suggestion for how we might throttle the snapmirrors to improve reliability in the transmission? and how are you folks handling encryption for remote mirroring? we have a small pix, sonicwall, or cisco to choose from...i'm guessing we may have to try all three but i hope to pick the one most likely to succeed the first time. ...lori

1 0

RE: snapmirroring over WAN
by Leeds, Daniel 10 Nov '05

10 Nov '05

yeah but that initial transfer can be fun if you are mirroring a couple terabyte volumes or if the application is generating alot of changes. a DS3 wan link would be no problem but they may need to rate limit snapmirror on much smaller wan links. -----Original Message----- From: owner-toasters(a)mathworks.com on behalf of Brian Parent Sent: Thu 11/10/2005 9:08 AM To: toasters(a)mathworks.com Subject: Re: snapmirroring over WAN If I recall correctly, snapmirror uses very little bandwidth, and shouldn't be a problem on any modern network. It only copies the blocks which were changed, rather than entire files. Re: > Date: Thu, 10 Nov 2005 08:49:38 -0800 > From: Lori Barfield <itdirector(a)gmail.com> > To: toasters(a)mathworks.com > Subject: snapmirroring over WAN > > for offsite backups, we hope to snapmirror our new fas 3020 on a > regular schedule via vpn over the internet to an older filer at our colo. > obviously, intermittent latency is a concern since we don't have a > dedicated link. and so is the potential for overpowering the modest > firewall/network hardware currently in place. does anyone have a > suggestion for how we might throttle the snapmirrors to improve > reliability in the transmission? > > and how are you folks handling encryption for remote mirroring? > we have a small pix, sonicwall, or cisco to choose from...i'm > guessing we may have to try all three but i hope to pick the one > most likely to succeed the first time. > > > ...lori

1 0

RE: snapmirroring over WAN
by Frank, Jason 10 Nov '05

10 Nov '05

Well, that all depends on how many changed blocks you expect in a day, as well as what kind of link you have, how busy the link is currently, what exact model of PIX or Sonicwall you have, and what the operating hours of the business are. Snapmirror does include a throttle, but if you throttle it too much, you'll be perpetually behind. And if you don't throttle it enough, it's about the same as not having one. It might be possible to do your snapmirror overnight, if it's acceptable to your client, and not throttle it at all. There are really a lot of options here. -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com] On Behalf Of Lori Barfield Sent: Thursday, November 10, 2005 10:50 AM To: toasters(a)mathworks.com Subject: snapmirroring over WAN for offsite backups, we hope to snapmirror our new fas 3020 on a regular schedule via vpn over the internet to an older filer at our colo. obviously, intermittent latency is a concern since we don't have a dedicated link. and so is the potential for overpowering the modest firewall/network hardware currently in place. does anyone have a suggestion for how we might throttle the snapmirrors to improve reliability in the transmission? and how are you folks handling encryption for remote mirroring? we have a small pix, sonicwall, or cisco to choose from...i'm guessing we may have to try all three but i hope to pick the one most likely to succeed the first time. ...lori

3 2

High network traffic due to RPC Authoriztion (nfs mounts)
by Tavis Gustafson 10 Nov '05

10 Nov '05

We are seeing extremely high network traffic from single clients to nfs mounted filers. In most cases running nfswatch on the client shows almost all of the traffic due to "RPC Authorization". We are talking 50-80Mbits of traffic with nothing sticking out on the clients process list. Has anyone seen weird high network traffic via nfs over udp? thank -tavis -- ___ /\__\ /:/ / /:/__/ /::\ \ /:/\:\ \ \/__\:\ \ \:\__\ \/__/ avis tavis(a)hq.newdream.net New Dream Network

2 3

samba acls and Re: howto document for initial volume architecting?
by Lori Barfield 09 Nov '05

09 Nov '05

thank you, greg and john, for the recommendations and the pdf links. i was not aware of FlexVols and it looks like this is suddenly going to be a much more challenging exercise. :) i do have two followup questions right now, before i start untangling the reading in detail. On 11/9/05, John Stoffel <john.stoffel(a)taec.toshiba.com> wrote: > Lori> our architecture is currently flat- one volume per filer with > Lori> the data only split up into separate directories. > > Are you using qtrees at all? I find them to be very useful. i am still learning about the existing architecture but don't believe qtrees are in use. originally i was concerned about setting any up because i am not a permanent fixture here, and i should not leave an overly complex design in my wake. so it is important to create an architecture that makes sense but does not require a lot of close watching to avoid failures and admins learning a whole new command set to stay afloat. please correct me if my vague impression about qtrees is ignorant, but don't they make it that much more difficult to do the day-to-day work of maintaining filers? > Lori> all hosts can mount all directories on all volumes if needed. > > Sure, makes sense. I assume you use permissions to keep people from > touching what they shouldn't? er, some work does need to be done here. with root and non-privileged access paradigms as well. good thing we get to work from scratch with the new filer. :) i do have a question about samba; hope it's okay to present that here. right now our permissions are group-based...meaning that we seem to create a new custom group for every single shared dir. (want to add another user? new group on every unix workstation! :) and there is no NIS here in this Windows house; nsswitch.conf is not even understood.) so it's time to switch to acls, but here again is unfamiliar territory for me and concern about making it too hard for the next guy to figure out. is there a good how-to acls document for samba that might include the pitfalls of this approach? thanks again! ...lori (busy printing pdfs :)

2 1

RE: samba acls and Re: howto document for initial volume architecting?
by Hirschorn, Steven 09 Nov '05

09 Nov '05

Hi Lori, Just thought I'd follow up about qtrees - we use just Windows and CIFS here generally, so I might be missing some subtlety about qtrees in a multiprotocol environment, but as I understand it, the one disadvantage of qtrees is that they can't be set up within Explorer. You need to use the console or Filerview. The advantages of Qtrees on the other hand: - You can apply tree quotas to them - You can do NDMP backups of a single Qtree, rather than an entire volume (I believe you can do this on a non-qtree subtree, but it's slower) - You can do NDMPCopies of a single qtree - You can use QTree Snapmirror on a Qtree - If you use DFM, it will monitor the QTrees for you and produce pretty graphs of the qtrees over time, so you can tell approximately where the data growth in your organisation is. The fact that most of our data is in QTrees is also aiding our migration to 7G as we can QSM all the data to new flexvols. We have one chunk of data outside of a qtree due to an oversight, and if it didn't consume such a large proportion of the volume, I'd have moved it to a qtree long ago. I've deliberately restricted access to the volume root shares to just my team and advised them never to create a root level folder without making it a qtree. I don't suppose anyone on the list can answer these questions? - Is there an easy way of changing a non qtree root level directory to a qtree? I'm guessing the only way is to NDMPCopy the data to a new qtree? - Is there any way of getting DFM to monitor subtree growth, as well as qtrees? I'm guessing again that DFM uses the filer's in-built quota monitoring to do the qtree size measurement, and so there isn't any flexibility here? Thanks! -----Original Message----- From: owner-toasters(a)mathworks.com [mailto:owner-toasters@mathworks.com]On Behalf Of Lori Barfield Sent: 09 November 2005 17:43 To: John Stoffel; Greg.Libert(a)plexus.com Cc: toasters(a)mathworks.com Subject: samba acls and Re: howto document for initial volume architecting? thank you, greg and john, for the recommendations and the pdf links. i was not aware of FlexVols and it looks like this is suddenly going to be a much more challenging exercise. :) i do have two followup questions right now, before i start untangling the reading in detail. On 11/9/05, John Stoffel <john.stoffel(a)taec.toshiba.com> wrote: > Lori> our architecture is currently flat- one volume per filer with > Lori> the data only split up into separate directories. > > Are you using qtrees at all? I find them to be very useful. i am still learning about the existing architecture but don't believe qtrees are in use. originally i was concerned about setting any up because i am not a permanent fixture here, and i should not leave an overly complex design in my wake. so it is important to create an architecture that makes sense but does not require a lot of close watching to avoid failures and admins learning a whole new command set to stay afloat. please correct me if my vague impression about qtrees is ignorant, but don't they make it that much more difficult to do the day-to-day work of maintaining filers? > Lori> all hosts can mount all directories on all volumes if needed. > > Sure, makes sense. I assume you use permissions to keep people from > touching what they shouldn't? er, some work does need to be done here. with root and non-privileged access paradigms as well. good thing we get to work from scratch with the new filer. :) i do have a question about samba; hope it's okay to present that here. right now our permissions are group-based...meaning that we seem to create a new custom group for every single shared dir. (want to add another user? new group on every unix workstation! :) and there is no NIS here in this Windows house; nsswitch.conf is not even understood.) so it's time to switch to acls, but here again is unfamiliar territory for me and concern about making it too hard for the next guy to figure out. is there a good how-to acls document for samba that might include the pitfalls of this approach? thanks again! ...lori (busy printing pdfs :) ------------------------------------------------------------------------------------ Free to all: Catalogue Awareness Day, 9th December at The National Archives in Kew www.nationalarchives.gov.uk/events/calendar/catagenda.htm ------------------------------------------------------------------------------------ National Archives Disclaimer This e-mail message (and attachments) may contain information that is confidential to The National Archives. If you are not the intended recipient you cannot use, distribute or copy the message or attachments. In such a case, please notify the sender by return e-mail immediately and erase all copies of the message and attachments. Opinions, conclusions and other information in this message and attachments that do not relate to the official business of The National Archives are neither given nor endorsed by it. ------------------------------------------------------------------------------------

1 0

Find a sexoholic tonight
by Leta Noel 09 Nov '05

09 Nov '05

"Where love reigns the impossible may be attained" a insightful man once said;) And soon you can discover passion for yourself aswell, without the pointless dates;) If its either a One-Nite stand or maybe something more serious, you'll find it here;-) http://www.look-for-singles-online.us

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters November 2005