Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real space on the filer.
Tom
-----Original Message----- From: Chris Thompson [mailto:cet1@cus.cam.ac.uk] Sent: Tuesday, April 03, 2001 4:25 PM To: MadDog@fool.com Cc: toasters@mathworks.com Subject: Re: Inodes a go go
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Okay, so the maximum number of inodes is 95% of the true (not usable) size of a volume divided by four.
And if you're daring enough to type this into your filer, it will tell you that anyway:
usvasvfile-001> maxfiles inquisition 50000000000000000 Max inode count cannot exceed 201249946
Now here's the problem. I just passed this information on to the
developers
who wrote the program that uses up this exceedingly ridiculous number of inodes, and they tell me that they are going to start using them up even faster than in the past, and we may run out within six months, and is
there
anything we can do to get more inodes?
I suggested they re-write their program to not generate tens of thousands
of
directories every day, but I don't know how well that will fly.
Can anyone think of a way to get more inodes? NetApp folks, any hope a
RFE
for a higher inode limit would go through?
Ummm... can you tell us more about this application that eats thousands of inodes for breakfast every day? What are the contents of these inodes?
One would normally expect every inode to have at least one 4K block of data associated with it unless
1. It is a regular file with length <= 64 bytes (fits in the inode). 2. It is a symbolic link with length <= 64 bytes (ditto). 3. It is a device, or socket, or named pipe, or some other implictly zero-length object.
That is presumably the rationale behind NetApp's "you can't possibly want more than 1 inode per 4 KB of data, can you?" upper limit.
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
This sounds like a dreadful rehash of INN circa 1995, an updated version of a self-inflicted denial of service attack. :-)
Actually, it reminds me of our "doit shell" idea, designed in honor of a friend who (shall remain anonymous :-) named all his shell scripts "doit". Sometimes Very Bad Things could happen if you were in the wrong directory and typed "doit" -- cured us of being lazy and having '.' in root's path real quick. :-)
In "doitsh" you'd have only a few commands to learn: "cd", and "doit".
# cd /usr/bin/mkdir # ./doit /home/blah
# cd /usr/sbin/shutdown # ./doit -h now
Yeah, okay, so we were drunk or something. :-)
So, all these directories take up lots and lots of inodes, but no real space on the filer.
"A great deal of sound and fury, signifying nothing..."
:-)
-- Chris
-- Chris Lamb, Unix Guy MeasureCast, Inc. 503-241-1469 x247 skeezics@measurecast.com
Tom "Mad Dog" Yergeau wrote:
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache. So, all these directories take up lots and lots of inodes, but no real space on the filer.
Crikey, thats a hell of a lot of reports. I thought what we did was mad: We need to store status information for each and every web server on the planet. Rather than store it in a huge database with tiny records we do as a zero-length file. Basically all the information we need (status flags and various time information) can be stored as the files permissions and the files ctime and mtime. The benefit for us of this route is that to check the status of a server we just have to do:
[create hash from host to give path] stat(/whatever/hash/host)
I guess this is one of the things that we can only consider doing because we have a filer. On anything else it would be sheer lunacy and we would have to use a database and live with the higher overhead.
Chris
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real space on the filer.
But each directory will still occupy at least one 4 KB block, so what's all this about "no real space"?
The space allocated to directories isn't counted against quotas in ONTAP, for some historical reason I have never really understood [it's not like that with BSD-derived quotas in Unix]. But it certainly does eat up real disc space, and counts against, say, used space as displayed by "df".
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
Many dirents fit into a single 4KB block. While dirents do take up space, it's a very small amount of space (no real space is pretty accurate).
*********** REPLY SEPARATOR ***********
On 4/4/01 at 2:21 PM Chris Thompson wrote:
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real space on the filer.
But each directory will still occupy at least one 4 KB block, so what's all this about "no real space"?
The space allocated to directories isn't counted against quotas in ONTAP, for some historical reason I have never really understood [it's not like that with BSD-derived quotas in Unix]. But it certainly does eat up real disc space, and counts against, say, used space as displayed by "df".
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
Marty Johnson
marty.johnson@erols.com writes [please excuse de-jeopardisation!]:
On 4/4/01 at 2:21 PM Chris Thompson wrote:
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each report can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real space on the filer.
But each directory will still occupy at least one 4 KB block, so what's all this about "no real space"?
The space allocated to directories isn't counted against quotas in ONTAP, for some historical reason I have never really understood [it's not like that with BSD-derived quotas in Unix]. But it certainly does eat up real disc space, and counts against, say, used space as displayed by "df".
Many dirents fit into a single 4KB block. While dirents do take up space, it's a very small amount of space (no real space is pretty accurate).
Yes, in WAFL usually 128 dirents can fit in a 4 KB block. But this doesn't stop the minimum amount of disc space allocated to a directory being one block. When one sees
$ mkdir empty $ ls -dls empty 8 drwxr-xr-x 2 cet1 cet1 4096 Apr 4 16:46 empty ^ ^^^^
there really is 4096 bytes of disc space (alternatively, 8*512) involved!
So even if most of Tom's inodes are going on empty or almost-empty directories, I don't see how he is going to run out of inodes much before he runs out of disc space, if he has 1 inode per 4 KB (or thereabouts).
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
I'll betcha that the block allocation pattern goes like this Dirent Block New 1 1 1 2 1 0 3 1 0 ... 128 1 0 129 2 1 130 2 0 ... Which means that only every 128th dirent causes a 4KB block to be allocated.
I don't know for certain how WAFL does this ... but I do know how others do it ... it would be pretty darn wasteful to allocate a 4KB block for every dirent.
I think that 'ls -l' is "being less than truthful".
*********** REPLY SEPARATOR ***********
On 4/4/01 at 5:12 PM Chris Thompson wrote:
marty.johnson@erols.com writes [please excuse de-jeopardisation!]:
On 4/4/01 at 2:21 PM Chris Thompson wrote:
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each
report
can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real
space
on the filer.
But each directory will still occupy at least one 4 KB block, so what's all this about "no real space"?
The space allocated to directories isn't counted against quotas in ONTAP, for some historical reason I have never really understood [it's not like that with BSD-derived quotas in Unix]. But it certainly does eat up real disc space, and counts against, say, used space as displayed by "df".
Many dirents fit into a single 4KB block. While dirents do take up space,
it's
a very small amount of space (no real space is pretty accurate).
Yes, in WAFL usually 128 dirents can fit in a 4 KB block. But this doesn't stop the minimum amount of disc space allocated to a directory being one block. When one sees
$ mkdir empty $ ls -dls empty 8 drwxr-xr-x 2 cet1 cet1 4096 Apr 4 16:46 empty ^ ^^^^
there really is 4096 bytes of disc space (alternatively, 8*512) involved!
So even if most of Tom's inodes are going on empty or almost-empty directories, I don't see how he is going to run out of inodes much before he runs out of disc space, if he has 1 inode per 4 KB (or thereabouts).
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
Marty Johnson
Even if I am wrong and a 4KB block gets allocated for every dirent, you're going to eat up all of your storage with these 4KB dirents instead of with files. You might not run out of inodes, but you still won't be able to write because you'll have no storage left.
*********** REPLY SEPARATOR ***********
On 4/4/01 at 12:21 PM Marty Johnson wrote:
I'll betcha that the block allocation pattern goes like this Dirent Block New 1 1 1 2 1 0 3 1 0 ... 128 1 0 129 2 1 130 2 0 ... Which means that only every 128th dirent causes a 4KB block to be allocated.
I don't know for certain how WAFL does this ... but I do know how others do it ... it would be pretty darn wasteful to allocate a 4KB block for every dirent.
I think that 'ls -l' is "being less than truthful".
*********** REPLY SEPARATOR ***********
On 4/4/01 at 5:12 PM Chris Thompson wrote:
marty.johnson@erols.com writes [please excuse de-jeopardisation!]:
On 4/4/01 at 2:21 PM Chris Thompson wrote:
MadDog@fool.com (Tom "Mad Dog" Yergeau) writes:
Agreed. You can't want more than 1 inode per 4K WAFL block, right?
This application generates reports by generating directories. Each
report
can generate hundreds of directories on the filer. I don't know why it works this way but I'm told changing it will be quite a headache.
So, all these directories take up lots and lots of inodes, but no real
space
on the filer.
But each directory will still occupy at least one 4 KB block, so what's all this about "no real space"?
The space allocated to directories isn't counted against quotas in ONTAP, for some historical reason I have never really understood [it's not like that with BSD-derived quotas in Unix]. But it certainly does eat up real disc space, and counts against, say, used space as displayed by "df".
Many dirents fit into a single 4KB block. While dirents do take up space,
it's
a very small amount of space (no real space is pretty accurate).
Yes, in WAFL usually 128 dirents can fit in a 4 KB block. But this doesn't stop the minimum amount of disc space allocated to a directory being one block. When one sees
$ mkdir empty $ ls -dls empty 8 drwxr-xr-x 2 cet1 cet1 4096 Apr 4 16:46 empty ^ ^^^^
there really is 4096 bytes of disc space (alternatively, 8*512) involved!
So even if most of Tom's inodes are going on empty or almost-empty directories, I don't see how he is going to run out of inodes much before he runs out of disc space, if he has 1 inode per 4 KB (or thereabouts).
Chris Thompson University of Cambridge Computing Service, Email: cet1@ucs.cam.ac.uk New Museums Site, Cambridge CB2 3QG, Phone: +44 1223 334715 United Kingdom.
Marty Johnson
Marty Johnson
-
Chris Lamb -
Chris Thompson -
chris＠webtop.com -
Marty Johnson -
Tom "Mad Dog" Yergeau