toasters

toasters@lists.teaparty.net

2 participants
13529 discussions

RE: File too large Linux fix
by Hubbard, Dan 30 Mar '01

30 Mar '01

Thanks to everyone for the help on this. Here is what we have found to date with large file support and Linux. * you must be running RHat Kernel 2.4.2 (this includes large FS support in ext2) * you must have an application that supports large FS's (as per Aaron's idea we modified the gzip.c header and recompiled gunzip) * you need to be mounting with NFS v3 for > 2GB Thanks to all for the help again.. -----Original Message----- From: Chris Thompson [mailto:cet1@cus.cam.ac.uk] Sent: Thursday, March 29, 2001 3:37 AM To: toasters(a)mathworks.com Cc: dhubbard(a)websense.com; Aaron.Sims(a)netapp.com; crandall(a)matchlogic.com Subject: Re: File too large ? Just concentrating on the Solaris parts of this thread... Dan Hubbard <dhubbard(a)websense.com> wrote: > > We are using a ONTAP v 6.0.1 and are NFS mounting Solaris and Redhat Linux > boxes. We are in the process of creating some large text files and cannot > either gunzip and/or create files greater than 2GB's. We have bumped up our > maxfiles and set the options nfs.v2.df_2gb_lim ON without any luck. You don't say which version of Solaris you are running. I don't believe the version of gunzip included in Solaris 8 is large-file-aware. Charles Randall <crandall(a)matchlogic.com> wrote: > > 1. NFS v2 doesn't support files larger than 2 GB. Switch to NFS v3. Only > very recently has the Linux NFS implementation supported NFS v3 (see below). > > 2. Can you work with files larger than 2 GB using the same tools on a local > file system? If so, NFS v3 will help. If not, you'll have to find out how to > "large file enable" your applications. On Solaris, look at the "largefile" > and "lfcompile" manpages. It's perhaps worth adding that large file support (i.e. > 2 GB) started with Solaris 2.6. Some people run amazingly ancient versions of Solaris... Aaron Sims <Aaron.Sims(a)netapp.com> wrote: [ ... NFS v2 comments omitted ... ] > > I've seen this before (at least on Solaris) using NFSv3. Some utilities in > Solaris (like 'ls','tar' etc.) will utilize 32bit versions system calls like > open(),stat(),read(), etc. In order to do operations on a file larger than > 2gb, your program must use open64(),stat64(),etc. I've seen Solaris' 'ls' > command cough up hair balls because the file is larger than 2gb. I ran > 'truss ls -l largefile' and noticed that 'ls' was using the wrong system > calls. Although, on that same system, if I used '/usr/ucb/ls' it worked > just fine. Using 'truss' on '/usr/ucb/ls' showed that it was using the > correct system calls. This rather amazes me. Solaris /usr/bin/ls is supposed to have been large-file-aware right from the start of Solaris 2.6, and I've never seen it behave as described above since then. I *have* observed problems with Solaris ls(1) applied to files larger than 2^40 bytes (the UFS limit) but smaller than 2^43 bytes (the WAFL limit). Of course, even on a filer such files are necessarily sparse! That was some time ago, I'll try and repeat the experiment with up-to-date versions of both Solaris and ONTAP. > I finally found an answer on how to force your application (I'll assume > you have source code) to use the 64bit version of the system calls. You > need to define _FILE_OFFSET_BITS 64 in your source code. That'll force > your app (under Solaris) to use open64(),stat64(), etc. You might want > to peruse through your gunzip code to see if it is defined. This is all described in the lfcompile(5) man page, of course. Note also that if you are compiling in 64-bit architecture mode (e.g. -xarch=v9 with the SUNWspro compilers) in Solaris 7 or later, then the largefile support comes along with it automatically. > I'm not sure that upgrading to 6.0.1R3 will help if you're using NFSv2. I don't think it will help if you are using NFSv3 either! Chris Thompson University of Cambridge Computing Service, Email: cet1(a)ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.

1 0

RE: W2k profile and Netapp
by Mike Sphar 29 Mar '01

29 Mar '01

Yes, that's my point. You couldn't do that with NT/9X. The original poster's NT environment was one of the only ways to handle automated creation and mapping of user home directories. It looks like they're being forced to "correct" this model to upgrade to Win2K. -- Mike Sphar - Sr Systems Administrator - Engineering Support Services - Remedy Corporation BOFH, GWP, MCP, MCP+I, MCSE, BFD -----Original Message----- From: Nail, Larry [mailto:lnail@ti.com] Sent: Thursday, March 29, 2001 6:14 AM To: 'Mike Sphar'; toasters(a)mathworks.com Subject: RE: W2k profile and Netapp W2K will map down an entire path to a drive letter now, ie: net use X: \\filer01\share\users\joed\data\docs\word\ I went down 16 levels & it still worked... lost interest after that. -----Original Message----- From: Mike Sphar [mailto:mikey@Remedy.COM] Sent: Wednesday, March 28, 2001 5:38 PM To: toasters(a)mathworks.com Subject: RE: W2k profile and Netapp Also, correct me if I'm wrong, but I believe part of the "problem" MS tried to fix in Win2K is that prior to Win2K you couldn't map a directory below the share point. I.e, if you shared \\server\users you couldn't specifically map a drive letter to \\server\users\username, which was sometimes a problem. It's been a while since I dealt with any of this and I normally don't deal with the Windows side of things much, so I could be completely off base here. -- Mike Sphar - Sr Systems Administrator - Engineering Support Services - Remedy Corporation BOFH, GWP, MCP, MCP+I, MCSE, BFD -----Original Message----- From: Nail, Larry [mailto:lnail@ti.com] Sent: Wednesday, March 28, 2001 1:39 PM To: 'Tim Longo'; toasters(a)mathworks.com Subject: RE: W2k profile and Netapp You can map the share anyway you'd like. The preferred & taught method is to create 1 single share and then let the OS create subdirectories on that share. All our PC home directories are individual shares. 1 reason is that we control all access at the share level if at all possible... don't have to mess with file level ACL's. This also relieves most of the issues of Unix file permisssions vs. NTFS permissions. -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 2:25 PM To: toasters(a)mathworks.com Subject: W2k profile and Netapp Greetings: Currently on my NT 4 domain, we map a user share on the netapp,(also the unix home), to a drive letter in the NT account profile. This is in the form of: Z: \\toaster\share We have noticed that with Windows 2000, they want this share to be mapped in the form of: Z: \\toaster\share\folder This makes migrating people from the old convention to the new convention quite a bit of work. Has anyone else encountered this, and how do you work around this problem? Needless to say, we are extremely angry with Microsoft. Thanks

1 0

RE: W2k profile and Netapp
by Nail, Larry 29 Mar '01

29 Mar '01

W2K will map down an entire path to a drive letter now, ie: net use X: \\filer01\share\users\joed\data\docs\word\ I went down 16 levels & it still worked... lost interest after that. -----Original Message----- From: Mike Sphar [mailto:mikey@Remedy.COM] Sent: Wednesday, March 28, 2001 5:38 PM To: toasters(a)mathworks.com Subject: RE: W2k profile and Netapp Also, correct me if I'm wrong, but I believe part of the "problem" MS tried to fix in Win2K is that prior to Win2K you couldn't map a directory below the share point. I.e, if you shared \\server\users you couldn't specifically map a drive letter to \\server\users\username, which was sometimes a problem. It's been a while since I dealt with any of this and I normally don't deal with the Windows side of things much, so I could be completely off base here. -- Mike Sphar - Sr Systems Administrator - Engineering Support Services - Remedy Corporation BOFH, GWP, MCP, MCP+I, MCSE, BFD -----Original Message----- From: Nail, Larry [mailto:lnail@ti.com] Sent: Wednesday, March 28, 2001 1:39 PM To: 'Tim Longo'; toasters(a)mathworks.com Subject: RE: W2k profile and Netapp You can map the share anyway you'd like. The preferred & taught method is to create 1 single share and then let the OS create subdirectories on that share. All our PC home directories are individual shares. 1 reason is that we control all access at the share level if at all possible... don't have to mess with file level ACL's. This also relieves most of the issues of Unix file permisssions vs. NTFS permissions. -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 2:25 PM To: toasters(a)mathworks.com Subject: W2k profile and Netapp Greetings: Currently on my NT 4 domain, we map a user share on the netapp,(also the unix home), to a drive letter in the NT account profile. This is in the form of: Z: \\toaster\share We have noticed that with Windows 2000, they want this share to be mapped in the form of: Z: \\toaster\share\folder This makes migrating people from the old convention to the new convention quite a bit of work. Has anyone else encountered this, and how do you work around this problem? Needless to say, we are extremely angry with Microsoft. Thanks

1 0

RE: File too large ?
by Sims, Aaron 29 Mar '01

29 Mar '01

The nfs.v2.2gb.lim option only relates to how some NFS clients determine if there is free space available or not on the volume. It has nothing to do with how file sizes are returned. Now, the problem with NFSv2 is that the size attribute is a 32bit unsigned integer, thus a 2gb file max. NFSv3 has a unsigned 64bit integer value for file size. If you're using NFSv2, you're going to run into problems. I've seen this before (at least on Solaris) using NFSv3. Some utilities in Solaris (like 'ls','tar' etc.) will utilize 32bit versions system calls like open(),stat(),read(), etc. In order to do operations on a file larger than 2gb, your program must use open64(),stat64(),etc. I've seen Solaris' 'ls' command cough up hair balls because the file is larger than 2gb. I ran 'truss ls -l largefile' and noticed that 'ls' was using the wrong system calls. Although, on that same system, if I used '/usr/ucb/ls' it worked just fine. Using 'truss' on '/usr/ucb/ls' showed that it was using the correct system calls. I finally found an answer on how to force your application (I'll assume you have source code) to use the 64bit version of the system calls. You need to define _FILE_OFFSET_BITS 64 in your source code. That'll force your app (under Solaris) to use open64(),stat64(), etc. You might want to peruse through your gunzip code to see if it is defined. I'm not sure that upgrading to 6.0.1R3 will help if you're using NFSv2. Do you have a case number to reference? I hope that helps some. Aaron Professional Services Consultant Network Appliance -----Original Message----- From: Hubbard, Dan [mailto:dhubbard@websense.com] Sent: Wednesday, March 28, 2001 2:10 PM To: toasters(a)mathworks.com Subject: File too large ? All; We are using a ONTAP v 6.0.1 and are NFS mounting Solaris and Redhat Linux boxes. We are in the process of creating some large text files and cannot either gunzip and/or create files greater than 2GB's. We have bumped up our maxfiles and set the options nfs.v2.df_2gb_lim ON without any luck. Our friends in tech support at Netapp told me that I must upgrade to 6.0.1r3 to fix this. Any input from the field ? Also, any input on issues with NFS and Redhat Linux kernel 2.2.14 (RH7.0) ? Thanks

2 1

RE: W2k profile and Netapp
by Nail, Larry 29 Mar '01

29 Mar '01

Purely administrative relief... MS preferred method---------- Create a share, give every one FULL Control access at the share level ACL the files in the root to EVERYONE:READ, Admins:Full, System:Full Load usrmgr, in the profile area, enter \\filer01\share\%username% Usrmgr will create a new directory in the share for the username and set file acls to username:Full You're done Issue 1 - It sets the \username directory to user:Full only, if you as an admin need access, you have to take ownership & push permissions down the tree... more work down the line when you have to troubleshoot. Issue 2 - 1 single share with EVERYONE FULL control as access... first off, this opens the root to EVERYONE on the network with a valid domain account & when you make/have a mistake with the file level ACL's, you've opened up the customer's data to EVERYONE. In this event, it also opens it up to the dummy who launched EXPLORE.WORM.KAK and starts to zero out files. Issue 3 - This setup is required to store W2K profiles/home directories on filers... the user must have FULL control as the OS impersonates the user when it downloads the profile. Other Method-------------- Use a script to create shares... you eliminate having to deal with ACL's as the root of .../users is set for EVERYONE:Full which is inherited when a new directory is created. We never give our user's FULL control b/c that enables them to mess with permissions (which overwrites UNIX permissions if you're running mixed). I can't say this creates any overhead for me. I have seen no performance issues between either method & we're running over 700 user shares at this point. Sample Script: : Create user share on \\filer01\d$\users : Required : rmtshare from NT (or W2K) Resource Kit in path : rsh access to filer : Create directory mkdir \\filer01\d$\users\%1 : Create Share rsh filer01 cifs shares -add %1 /vol/vol1/users/%1 : Comment Share rsh filer01 cifs shares -change %1 -comment \"%1 Home Directory\" : rmtshare used to set permissions : This is a hold over from early filer maintenance where cifs shares comment : didn't work correctly... never check it afterward to see if they fixed it rmtshare \\filer01\%1 /grant DOMAIN\%1:c /grant "deskside":c /grant filer01\administrators:f /remove everyone : Send email to server team that share was created echo User Share \\filer01\%1 Created |postmail -Hsomewhere.usa.com serverteam(a)somewhere.usa.com To unshare-------- rmtshare \\filer01\sharename\ /d rd \\filer01\d$\users\username /q/s Hope this helps... Larry -----Original Message----- From: neil lehrer [mailto:nlehrer@ibb.gov] Sent: Wednesday, March 28, 2001 4:19 PM To: Nail, Larry Cc: 'Tim Longo'; toasters(a)mathworks.com Subject: Re: W2k profile and Netapp what's the overhead for having 1 share per person? "Nail, Larry" wrote: > > You can map the share anyway you'd like. The preferred & taught method is > to create 1 single share and then let the OS create subdirectories on that > share. All our PC home directories are individual shares. 1 reason is that > we control all access at the share level if at all possible... don't have to > mess with file level ACL's. This also relieves most of the issues of Unix > file permisssions vs. NTFS permissions. > > -----Original Message----- > From: Tim Longo [mailto:tlongo@avaya.com] > Sent: Wednesday, March 28, 2001 2:25 PM > To: toasters(a)mathworks.com > Subject: W2k profile and Netapp > > Greetings: > > Currently on my NT 4 domain, we map a user share on the netapp,(also the > unix home), to a drive letter in the NT account profile. This is in the > form of: Z: \\toaster\share > > We have noticed that with Windows 2000, they want this share to be mapped in > the form of: Z: \\toaster\share\folder > > This makes migrating people from the old convention to the new convention > quite a bit of work. > > Has anyone else encountered this, and how do you work around this problem? > Needless to say, we are extremely angry with Microsoft. > > Thanks -- regards

2 1

RE: W2k profile and Netapp
by Mike Sphar 29 Mar '01

29 Mar '01

Also, correct me if I'm wrong, but I believe part of the "problem" MS tried to fix in Win2K is that prior to Win2K you couldn't map a directory below the share point. I.e, if you shared \\server\users you couldn't specifically map a drive letter to \\server\users\username, which was sometimes a problem. It's been a while since I dealt with any of this and I normally don't deal with the Windows side of things much, so I could be completely off base here. -- Mike Sphar - Sr Systems Administrator - Engineering Support Services - Remedy Corporation BOFH, GWP, MCP, MCP+I, MCSE, BFD -----Original Message----- From: Nail, Larry [mailto:lnail@ti.com] Sent: Wednesday, March 28, 2001 1:39 PM To: 'Tim Longo'; toasters(a)mathworks.com Subject: RE: W2k profile and Netapp You can map the share anyway you'd like. The preferred & taught method is to create 1 single share and then let the OS create subdirectories on that share. All our PC home directories are individual shares. 1 reason is that we control all access at the share level if at all possible... don't have to mess with file level ACL's. This also relieves most of the issues of Unix file permisssions vs. NTFS permissions. -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 2:25 PM To: toasters(a)mathworks.com Subject: W2k profile and Netapp Greetings: Currently on my NT 4 domain, we map a user share on the netapp,(also the unix home), to a drive letter in the NT account profile. This is in the form of: Z: \\toaster\share We have noticed that with Windows 2000, they want this share to be mapped in the form of: Z: \\toaster\share\folder This makes migrating people from the old convention to the new convention quite a bit of work. Has anyone else encountered this, and how do you work around this problem? Needless to say, we are extremely angry with Microsoft. Thanks

1 0

RE: W2k profile and Netapp
by Loux, Linda 29 Mar '01

29 Mar '01

We encountered the same thing. What I discovered is that we are mounting Z: to the users home directory in their user account profile. That user home directory is \\toaster\share\user. You can change the mount to be \\toaster\share in the logon script. Linda Loux Systems Administrator Rosetta Inpharmatics -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 12:25 PM To: toasters(a)mathworks.com Subject: W2k profile and Netapp Greetings: Currently on my NT 4 domain, we map a user share on the netapp,(also the unix home), to a drive letter in the NT account profile. This is in the form of: Z: \\toaster\share We have noticed that with Windows 2000, they want this share to be mapped in the form of: Z: \\toaster\share\folder This makes migrating people from the old convention to the new convention quite a bit of work. Has anyone else encountered this, and how do you work around this problem? Needless to say, we are extremely angry with Microsoft. Thanks **************************************************************************** This e-mail message is the property of Rosetta Inpharmatics and may contain information that is confidential and/or privileged. If you are not the intended recipient, any use, disclosure, copying, or communication of the contents of this message is prohibited. If this message was received in error, please forward a copy to mail_admin(a)rii.com and delete the message and any attachments from your computer. Thank you. ****************************************************************************

1 0

RE: File too large ?
by Kenley, Martin 29 Mar '01

29 Mar '01

More specifically, I believe that NFS v2 wont display the correct size; but if the file is on a multiprotocal box, like a filer, and if your client is running NFS v2, the file only looks like 2 GB, but looked at under CIFS, you will see the correct file size. -----Original Message----- From: Charles Randall [mailto:crandall@matchlogic.com] Sent: Wednesday, March 28, 2001 1:04 PM To: 'Hubbard, Dan'; toasters(a)mathworks.com Subject: RE: File too large ? Two things, 1. NFS v2 doesn't support files larger than 2 GB. Switch to NFS v3. Only very recently has the Linux NFS implementation supported NFS v3 (see below). 2. Can you work with files larger than 2 GB using the same tools on a local file system? If so, NFS v3 will help. If not, you'll have to find out how to "large file enable" your applications. On Solaris, look at the "largefile" and "lfcompile" manpages. For more info on the Linux NFS implementation, see this page http://nfs.sourceforge.net/ No offense intended to our resident Linux aficionados, but I cringed when I read most of it. Charles -----Original Message----- From: Hubbard, Dan [mailto:dhubbard@websense.com] Sent: Wednesday, March 28, 2001 12:10 PM To: toasters(a)mathworks.com Subject: File too large ? All; We are using a ONTAP v 6.0.1 and are NFS mounting Solaris and Redhat Linux boxes. We are in the process of creating some large text files and cannot either gunzip and/or create files greater than 2GB's. We have bumped up our maxfiles and set the options nfs.v2.df_2gb_lim ON without any luck. Our friends in tech support at Netapp told me that I must upgrade to 6.0.1r3 to fix this. Any input from the field ? Also, any input on issues with NFS and Redhat Linux kernel 2.2.14 (RH7.0) ? Thanks

1 0

RE: W2k profile and Netapp
by Nail, Larry 29 Mar '01

29 Mar '01

You can map the share anyway you'd like. The preferred & taught method is to create 1 single share and then let the OS create subdirectories on that share. All our PC home directories are individual shares. 1 reason is that we control all access at the share level if at all possible... don't have to mess with file level ACL's. This also relieves most of the issues of Unix file permisssions vs. NTFS permissions. -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 2:25 PM To: toasters(a)mathworks.com Subject: W2k profile and Netapp Greetings: Currently on my NT 4 domain, we map a user share on the netapp,(also the unix home), to a drive letter in the NT account profile. This is in the form of: Z: \\toaster\share We have noticed that with Windows 2000, they want this share to be mapped in the form of: Z: \\toaster\share\folder This makes migrating people from the old convention to the new convention quite a bit of work. Has anyone else encountered this, and how do you work around this problem? Needless to say, we are extremely angry with Microsoft. Thanks

2 1

RE: Toasters and WINS
by Nail, Larry 28 Mar '01

28 Mar '01

If you have downlevel clients (NT, 9x), you'll need still need WINS, however the W2K implementation is quite a bit more stable & less inclinded to corrupt than the NT version. -----Original Message----- From: Tim Longo [mailto:tlongo@avaya.com] Sent: Wednesday, March 28, 2001 2:27 PM To: toasters(a)mathworks.com Subject: Toasters and WINS We are migrating users to a Windows 2000 domain. Wondering if anyone has run a Netapp in a Win2k environment without WINS? We will probably try to add the domain, and domain crontroller entries to the toaster hosts file?

1 0

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

1997

toasters