Brian Parent wrote:
I haven't seen any responses yet. Have you opened a ticket with NetApp?
No; since the mount itself seems to succeed I think the Netapp side is doing the right thing. Why the credentials aren't being passed and why rpc.gssd isn't seeing anything is still a mystery.
Posting to the Kerberos list produced no responses; posting to the Linux NFS list got a reply from one of the developers, but no fixes yet.
Please post a summary back to the list if you ever do make any progress.
Big "if"... but I'll be glad to do so.
Hi Jim,
I am very new to this myself - I have been setting this up in the lab, for my own learning. Since you seem to be stuck hard, I thought I'd hazard this a shot from my side.
From what you have reported so far, I am not sure if I ran into that same
issue, but I was seeing similar symptoms at one point - from a lab linux client (RHEL AS / 2.6.9-42.ELsmp x86_64), mount to a filer was succeeding, but then doing a cd to or ls on the mount point, was giving permission denied. More on this at the end.
Looking at the rpc.gssd strace o/p (and the rpc.gssd messages) that you had sent - I -think- that was till the point you did the mount?
If so, can you do a cd to the dir in question (which results in the permission denied), and send strace o/p + rpc.gssd messages corresponding to that? (or confirm that the strace o/p and rpc.gssd messages that you had sent, covered that step already.)
That might help me in suggesting something further to try. (but I might well not come up with any suggestion either; so please use your discretion in putting in more time for the above data.)
Back to the problem that I had hit - it seemed a lot like this issue -
http://linux-nfs.org/pipermail/nfsv4/2006-April/004101.html http://linux-nfs.org/pipermail/nfsv4/2006-April/004088.html
- but the solution mentioned there (upgrading librpcsecgss to 0.11) didn't do it for me. I finally upgraded nfs-utils itself (w/o an understanding of what exactly I had hit) - to 1.0.10 I believe (from what version, am not sure now) - and that resolved the problem.
(you mentioned already upgrading the nfs-* stuff, so probably the above doesn't apply to your case.)
thanks,
-dhawal.
{ jdavis@cs.arizona.edu, Mon, 19 Feb 2007, 3:08pm (-0700 ) }<
} Brian Parent wrote: } > I haven't seen any responses yet. Have you opened a ticket with NetApp? } } No; since the mount itself seems to succeed I think the Netapp side is } doing the right thing. Why the credentials aren't being passed and why } rpc.gssd isn't seeing anything is still a mystery. } } Posting to the Kerberos list produced no responses; posting to the Linux } NFS list got a reply from one of the developers, but no fixes yet. } } > Please post a summary back to the list if you ever do make any progress. } } Big "if"... but I'll be glad to do so. } } }
I have a 3020 filers running 7.1.1, I am confuse by the sysstat:
Disk KB/s is 20MB/s, while FCP(no NFS) is doing 9MB/s, how big is the disk strip size 64KB, I am guessing disk is always IOs at strip size.
Also the FCP IO number is 581 IO/s , and I used statsit command to collect disk usage, it showed there are 23 disks are doing 55 IO/s, so total disk is doing 1265 IO/s, does anybody know why backend disk are doing more IOs than FCP?
Thanks
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s in out read write read write age hit time ty util in out 9% 0 0 0 581 1 2 20120 16 0 0 2 98% 0% - 13% 581 0 67 9261 3% 0 0 0 135 0 0 2584 12 0 0 2 95% 0% - 9% 134 0 290 1844 2% 0 0 0 121 0 0 2064 4 0 0 2 95% 0% - 20% 121 0 509 1566 5% 0 0 0 146 0 0 5161 2503 0 0 2 96% 23% T 12% 146 0 255 2053 3% 0 0 0 185 0 0 3404 12 0 0 2 95% 0% - 12% 185 0 136 2499 8% 0 0 0 694 0 0 5694 4 0 0 2 98% 0% - 15% 694 0 2982 3724 9% 0 0 0 400 1 1 18294 12 0 0 2 98% 0% - 18% 400 0 792 11466 6% 0 0 0 306 0 0 12856 16 0 0 2 97% 0% - 25% 306 0 550 7419 7% 0 0 0 242 1 1 6905 4482 0 0 3 96% 28% T 21% 242 0 366 3709 2% 0 0 0 127 0 0 2476 12 0 0 3 95% 0% - 9% 127 0 38 1889 2% 0 0 0 120 1 1 2348 16 0 0 3 96% 0% - 11% 120 0 128 1697 3% 0 0 0 156 0 0 4476 0 0 0 3 96% 0% - 14% 156 0 272 2486 4% 0 0 0 214 1 2 4084 16 0 0 3 95% 0% - 16% 214 0 773 2608 5% 0 0 0 183 1 1 5228 2890 0 0 3 95% 21% Tf 29% 183 0 264 2407 4% 0 0 0 209 7 18 3952 16 0 0 3 95% 2% : 16% 209 0 9 3026 7% 0 0 0 462 0 0 12176 16 0 0 3 97% 0% - 20% 462 0 484 6376
Jackie(Yanhui) Li Global Infrastructure Services Electronic Arts 650-6285746
On 2/20/07, Li, Jackie (Yanhui) YLi@ea.com wrote:
I have a 3020 filers running 7.1.1, I am confuse by the sysstat:
Disk KB/s is 20MB/s, while FCP(no NFS) is doing 9MB/s
It could be that there are other processes running that are causing some IO, such as reallocates, disk scrubs, or other background maintenance tasks.
Kind regards,
Nils
There could be several reasons why the #s don't always match
1) All of this is sampling over time. So if you take a "snapshot" of the stats over a given second or few seconds, there could be I/Os left over from the previous interval or coming into play. 2) WAFL could be doing read-aheads to try and get more info in cache. So you ask for a certain size read, and the filer intentionally pulls more to try and pull more into the read cache. 3) In general, I throw out the first sample from sysstat, which is the example you are using. It's usually off, probably again due to getting the sampling up and running. Most of the rest of your output doesn't show quite the same disparity.
There could be other reasons as well, but these are off the top of my head.
-- Adam Fox adamfox@netapp.com
________________________________
From: owner-dl-toasters@jhereg.corp.netapp.com [mailto:owner-dl-toasters@jhereg.corp.netapp.com] On Behalf Of Li, Jackie (Yanhui) Sent: Tuesday, February 20, 2007 1:55 PM To: toasters@mathworks.com Subject: sysstat question:
I have a 3020 filers running 7.1.1, I am confuse by the sysstat:
Disk KB/s is 20MB/s, while FCP(no NFS) is doing 9MB/s, how big is the disk strip size 64KB, I am guessing disk is always IOs at strip size.
Also the FCP IO number is 581 IO/s , and I used statsit command to collect disk usage, it showed there are 23 disks are doing 55 IO/s, so total disk is doing 1265 IO/s, does anybody know why backend disk are doing more IOs than FCP?
Thanks
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s
in out read write read write age hit time ty util in out
9% 0 0 0 581 1 2 20120 16 0 0 2 98% 0% - 13% 581 0 67 9261
3% 0 0 0 135 0 0 2584 12 0 0 2 95% 0% - 9% 134 0 290 1844
2% 0 0 0 121 0 0 2064 4 0 0 2 95% 0% - 20% 121 0 509 1566
5% 0 0 0 146 0 0 5161 2503 0 0 2 96% 23% T 12% 146 0 255 2053
3% 0 0 0 185 0 0 3404 12 0 0 2 95% 0% - 12% 185 0 136 2499
8% 0 0 0 694 0 0 5694 4 0 0 2 98% 0% - 15% 694 0 2982 3724
9% 0 0 0 400 1 1 18294 12 0 0 2 98% 0% - 18% 400 0 792 11466
6% 0 0 0 306 0 0 12856 16 0 0 2 97% 0% - 25% 306 0 550 7419
7% 0 0 0 242 1 1 6905 4482 0 0 3 96% 28% T 21% 242 0 366 3709
2% 0 0 0 127 0 0 2476 12 0 0 3 95% 0% - 9% 127 0 38 1889
2% 0 0 0 120 1 1 2348 16 0 0 3 96% 0% - 11% 120 0 128 1697
3% 0 0 0 156 0 0 4476 0 0 0 3 96% 0% - 14% 156 0 272 2486
4% 0 0 0 214 1 2 4084 16 0 0 3 95% 0% - 16% 214 0 773 2608
5% 0 0 0 183 1 1 5228 2890 0 0 3 95% 21% Tf 29% 183 0 264 2407
4% 0 0 0 209 7 18 3952 16 0 0 3 95% 2% : 16% 209 0 9 3026
7% 0 0 0 462 0 0 12176 16 0 0 3 97% 0% - 20% 462 0 484 6376
Jackie(Yanhui) Li
Global Infrastructure Services
Electronic Arts
650-6285746
I think we are going to need more data to solve this one. Can you start by collecting a set of 30 second long statit samples for about 5 minutes? And maybe some longer sysstats? A perfstat would be awesome.
-Blake
On 2/20/07, Fox, Adam Adam.Fox@netapp.com wrote:
There could be several reasons why the #s don't always match
- All of this is sampling over time. So if you take a "snapshot" of the
stats over a given second or few seconds, there could be I/Os left over from the previous interval or coming into play. 2) WAFL could be doing read-aheads to try and get more info in cache. So you ask for a certain size read, and the filer intentionally pulls more to try and pull more into the read cache. 3) In general, I throw out the first sample from sysstat, which is the example you are using. It's usually off, probably again due to getting the sampling up and running. Most of the rest of your output doesn't show quite the same disparity.
There could be other reasons as well, but these are off the top of my head.
-- Adam Fox adamfox@netapp.com
From: owner-dl-toasters@jhereg.corp.netapp.com [mailto:owner-dl-toasters@jhereg.corp.netapp.com] On Behalf Of Li, Jackie (Yanhui) Sent: Tuesday, February 20, 2007 1:55 PM To: toasters@mathworks.com Subject: sysstat question:
I have a 3020 filers running 7.1.1, I am confuse by the sysstat:
Disk KB/s is 20MB/s, while FCP(no NFS) is doing 9MB/s, how big is the disk strip size 64KB, I am guessing disk is always IOs at strip size.
Also the FCP IO number is 581 IO/s , and I used statsit command to collect disk usage, it showed there are 23 disks are doing 55 IO/s, so total disk is doing 1265 IO/s, does anybody know why backend disk are doing more IOs than FCP?
Thanks
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk FCP iSCSI FCP kB/s
in out read writeread write age hit time ty util in out
9% 0 0 0 581 1 2 20120 16 0 0 2 98% 0% - 13% 581 0 67 9261
3% 0 0 0 135 0 0 2584 12 0 0 2 95% 0% - 9% 134 0 290 1844
2% 0 0 0 121 0 0 2064 4 0 0 2 95% 0% - 20% 121 0 509 1566
5% 0 0 0 146 0 0 5161 2503 0 0 2 96% 23% T 12% 146 0 255 2053
3% 0 0 0 185 0 0 3404 12 0 0 2 95% 0% - 12% 185 0 136 2499
8% 0 0 0 694 0 0 5694 4 0 0 2 98% 0% - 15% 694 0 2982 3724
9% 0 0 0 400 1 1 18294 12 0 0 2 98% 0% - 18% 400 0 792 11466
6% 0 0 0 306 0 0 12856 16 0 0 2 97% 0% - 25% 306 0 550 7419
7% 0 0 0 242 1 1 6905 4482 0 0 3 96% 28% T 21% 242 0 366 3709
2% 0 0 0 127 0 0 2476 12 0 0 3 95% 0% - 9% 127 0 38 1889
2% 0 0 0 120 1 1 2348 16 0 0 3 96% 0% - 11% 120 0 128 1697
3% 0 0 0 156 0 0 4476 0 0 0 3 96% 0% - 14% 156 0 272 2486
4% 0 0 0 214 1 2 4084 16 0 0 3 95% 0% - 16% 214 0 773 2608
5% 0 0 0 183 1 1 5228 2890 0 0 3 95% 21% Tf 29% 183 0 264 2407
4% 0 0 0 209 7 18 3952 16 0 0 3 95% 2% : 16% 209 0 9 3026
7% 0 0 0 462 0 0 12176 16 0 0 3 97% 0% - 20% 462 0 484 6376
Jackie(Yanhui) Li
Global Infrastructure Services
Electronic Arts
650-6285746
I've been a lurker for a while, and I'm learning lots, thanks! The technical content is great.
But, are "I need to hire a NetApp admin" job postings permitted?
--tep
Hi Tom,
This is *not* the appropriate forum for job postings. Let's keep it technical.
Thanks.
Kingman
-----Original Message----- From: Tom Perrine [mailto:tperrine@scea.com] Sent: Tuesday, February 20, 2007 11:17 AM To: toasters@mathworks.com Subject: job postings permitted?
I've been a lurker for a while, and I'm learning lots, thanks! The technical content is great.
But, are "I need to hire a NetApp admin" job postings permitted?
--tep
-
Blake Golliher -
dhawal bhagwat -
Fox, Adam -
Jim Davis -
Li, Jackie (Yanhui) -
Nils Vogels -
Tang, Kingman -
Tom Perrine