reducing imaxpct on linux

reducing imaxpct on linux

[CZEH, Istvan]

Hi,

I have a 34TB xfs partition. First time we were run out of the space I
thought that the reason was that the fs was not mounted with inode64
option. So I unmounted the fs and mounted again with inode64 mount
option. (Before I did that I deleted some files because I needed a quick
solution, even if it was temporary.) I also moved the oldest files away
and back as the XFS FAQ suggests. A few days ago the "No space left on
device" message appeared again, but the free space shown by df is still
about 9TB and df -i shows that only 1% of inodes is uses. 

When the file system was created it was much smaller and it was created
with the default maxpct value which is 25%. Now that the size is 34TBs,
the 25% seems to be too big, we run out of the space. The actual inode
usage is about 1% so I decided to reduce maxpct to 5%.

I made a test on a 5GB fs, and it was successful with 'xfs_growfs -m
5 /dev/sdb1' but I'm still worrying about the result in the production
environment. Also, the production system is using LVM, the test was
using native disk.

What could happen if I reduce imaxpct? Is it safe or painful? 
What is really a chance that the 25% value is causing the error?

thanks very much,

Istvan




_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: reducing imaxpct on linux

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 924 bytes --]

On Freitag, 12. November 2010 CZEH, Istvan wrote:
> What could happen if I reduce imaxpct? Is it safe or painful? 
> What is really a chance that the 25% value is causing the error?

I don't believe reducing imaxpct results in more free space, because 
that's the *maximum* Inodes can have.

I'm not a dev, but AFAIK the "no free space" problem comes when the 
space in the first TeraByte is completely filled. I don't remember 
exactly, either the FAQ tells you or look in the mailinglist archive. 
The question about "no free space despite enough free" comes rather 
often.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// ****** Radiointerview zum Thema Spam ******
// http://www.it-podcast.at/archiv.html#podcast-100716
// 
// Haus zu verkaufen: http://zmi.at/langegg/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: reducing imaxpct on linux

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 1902 bytes --]

On Freitag, 12. November 2010 CZEH, Istvan wrote:
> I already made the things written in the FAQ (question 32.) but still
> having the problem. Maybe I should find the oldest data again? It
> won't be easy...

I guess the FAQ is a bit wrong here:

http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_receive_No_space_left_on_device_after_xfs_growfs.3F

Because it is not the "oldest" data, but the data that is within the 
space in the first TeraByte of the partition.

Dave Chinner explained to me, that XFS will split data equally amongst 
all AGs. So if you have a partition with 4TB and 4 AGs, and you put 1TB 
data on that partition, each AG will have around 250GB data, meaining 
you still have 750GB free within the first TB.
When you do NOT use the inode64 option, all inodes will be placed in the 
first TB *only*. Even when later you use the inode64 option, it might 
not succeed in finding space for the inodes.

I can imaginge this would help
1) Run xfs_fsr, if it finds files below 1TB and defrags them to a space 
>1TB, you're lucky
2) use the "noikeep" option before doing what XFS Q32 describes. If you 
remove a large portion of inodes, and later move them back, they should 
be newly allocated, and when you use inode64 then, chances are good the 
go at a place >1TB.
3) Find files that are placed at <1TB. I think some xfs_db magic can do 
that, but I never used it so someone else should say.

Anyway: Dave, I'd like to "repair" the FAQ question 32 to be more 
correct and descriptive, would my description be correct?

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// ****** Radiointerview zum Thema Spam ******
// http://www.it-podcast.at/archiv.html#podcast-100716
// 
// Haus zu verkaufen: http://zmi.at/langegg/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: reducing imaxpct on linux

[Eric Sandeen]

On 11/12/2010 09:18 AM, Michael Monnerie wrote:
> On Freitag, 12. November 2010 CZEH, Istvan wrote:
>> I already made the things written in the FAQ (question 32.) but still
>> having the problem. Maybe I should find the oldest data again? It
>> won't be easy...
> 
> I guess the FAQ is a bit wrong here:
> 
> http://xfs.org/index.php/XFS_FAQ#Q:_Why_do_I_receive_No_space_left_on_device_after_xfs_growfs.3F
> 
> Because it is not the "oldest" data, but the data that is within the 
> space in the first TeraByte of the partition.
> 
> Dave Chinner explained to me, that XFS will split data equally amongst 
> all AGs. So if you have a partition with 4TB and 4 AGs, and you put 1TB 
> data on that partition, each AG will have around 250GB data, meaining 
> you still have 750GB free within the first TB.
> When you do NOT use the inode64 option, all inodes will be placed in the 
> first TB *only*. Even when later you use the inode64 option, it might 
> not succeed in finding space for the inodes.

I don't think that follows; when you mount with inode64, inodes can
be allocated anywhere.  Even if you previously mounted without it.

What is required, though, is some contiguous space to allocate an
inode cluster into.  That could be the problem.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: reducing imaxpct on linux

[Michael Monnerie]

[-- Attachment #1.1: Type: Text/Plain, Size: 878 bytes --]

On Freitag, 12. November 2010 Eric Sandeen wrote:
> I don't think that follows; when you mount with inode64, inodes can
> be allocated anywhere.  Even if you previously mounted without it.
> 
> What is required, though, is some contiguous space to allocate an
> inode cluster into.  That could be the problem.

In his first mail, the OP said there are 9TB free space. I can't believe 
that space would be so scattered that you can't find a free space for 
new inodes.

BTW: What size requirement is there for contig. inode space?

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// ****** Radiointerview zum Thema Spam ******
// http://www.it-podcast.at/archiv.html#podcast-100716
// 
// Haus zu verkaufen: http://zmi.at/langegg/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs