XFS memory recomendation?

XFS memory recomendation?

[Assarsson, Emil]

Hi,

Are there any recommendations about how much memory I need based on the size of the file system and/or amount of files?
For example: how much memory would be optimal for a 20TB file system with 3000000 files? 

I guess it depends on the usage pattern?
Is there any way to make sure that the file allocation table stays in the memory?

Best regards
Emil Assarsson

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Dave Chinner]

On Thu, Feb 16, 2012 at 12:50:57PM +0100, Assarsson, Emil wrote:
> Hi,
> 
> Are there any recommendations about how much memory I need based
> on the size of the file system and/or amount of files?  For
> example: how much memory would be optimal for a 20TB file system
> with 3000000 files? 

/me shrugs

> I guess it depends on the usage pattern?

Totally.

> Is there any way to make sure that the file allocation table stays in the memory?

What's a "file allocation table"? XFS doesn't have one. :)

XFS does have a sophisticated metadata cache and reclaim scheme that
keeps frequently referenced metadata hot in cache, but you as a
user have very little control over that because the cache size is
effectively controlled by memory pressure. i.e. it's size is
completely workload dependent.....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Michael Monnerie]

[-- Attachment #1.1: Type: text/plain, Size: 486 bytes --]

Am Freitag, 17. Februar 2012, 13:49:41 schrieb Dave Chinner:
> What's a "file allocation table"? XFS doesn't have one.

I guess he means the inode/dentry cache and such.

BTW, what's the difference of inode and dentry caches? I'd like to 
understand the setting of
/proc/sys/vm/vfs_cache_pressure

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Assarsson, Emil]

Thanks for the answer Dave and Michael.

As you might realize, I have no knowledge if the internal works of
XFS :-P 

My problem is that the file system sometimes seems to be a bit laggy
when listing directories and so on. I really don't know how to deal with
it. I guess more memory always will help but where should I put the
limit?

Can it be a solution (ugly) to warm up the cache by making a complete
"find" on the file system sending output to /dev/null? but then again it
would probably be a stupid way to use precious memory for something that
isn't used that often.

--
Emil


fre 2012-02-17 klockan 14:05 +0100 skrev Michael Monnerie:
> Am Freitag, 17. Februar 2012, 13:49:41 schrieb Dave Chinner:
> > What's a "file allocation table"? XFS doesn't have one.
> 
> I guess he means the inode/dentry cache and such.
> 
> BTW, what's the difference of inode and dentry caches? I'd like to 
> understand the setting of
> /proc/sys/vm/vfs_cache_pressure
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Eric Sandeen]

On 2/17/12 7:27 AM, Assarsson, Emil wrote:
> Thanks for the answer Dave and Michael.
> 
> As you might realize, I have no knowledge if the internal works of
> XFS :-P 
> 
> My problem is that the file system sometimes seems to be a bit laggy
> when listing directories and so on. 

This would have been a better starting point for your question. :)

> I really don't know how to deal with
> it. I guess more memory always will help but where should I put the
> limit?

How slow is slow?  What kernel are you using, and what does your storage
look like?  How many files are in the directory?  How much memory do you
have, and what else is going on in the system at the time?

I think you'll need to quantify it a bit more.

-Eric

> Can it be a solution (ugly) to warm up the cache by making a complete
> "find" on the file system sending output to /dev/null? but then again it
> would probably be a stupid way to use precious memory for something that
> isn't used that often.
> 
> --
> Emil
> 
> 
> fre 2012-02-17 klockan 14:05 +0100 skrev Michael Monnerie:
>> Am Freitag, 17. Februar 2012, 13:49:41 schrieb Dave Chinner:
>>> What's a "file allocation table"? XFS doesn't have one.
>>
>> I guess he means the inode/dentry cache and such.
>>
>> BTW, what's the difference of inode and dentry caches? I'd like to 
>> understand the setting of
>> /proc/sys/vm/vfs_cache_pressure
>>
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Stan Hoeppner]

On 2/16/2012 8:49 PM, Dave Chinner wrote:
> On Thu, Feb 16, 2012 at 12:50:57PM +0100, Assarsson, Emil wrote:
>> Hi,
>>
>> Are there any recommendations about how much memory I need based
>> on the size of the file system and/or amount of files?  For
>> example: how much memory would be optimal for a 20TB file system
>> with 3000000 files? 
> 
> /me shrugs
> 
>> I guess it depends on the usage pattern?
> 
> Totally.

Allow me to drag the OP's question under a different light...

I have a 20TB XFS filesystem with 3000000 files.  What is the minimum
amount of system RAM I will need to run an xfs_check or xfs_repair or
xfs_[tool] on this filesystem, assuming Linux has been put into a low
overhead state, and said tool[s] has access to the bulk of the system
memory?

Is there a formula available so any XFS user can calculate this
xfs_[tools] RAM requirement, given FS size X and file count Y?

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Eric Sandeen]

On 2/17/12 5:58 PM, Stan Hoeppner wrote:
> On 2/16/2012 8:49 PM, Dave Chinner wrote:
>> On Thu, Feb 16, 2012 at 12:50:57PM +0100, Assarsson, Emil wrote:
>>> Hi,
>>>
>>> Are there any recommendations about how much memory I need based
>>> on the size of the file system and/or amount of files?  For
>>> example: how much memory would be optimal for a 20TB file system
>>> with 3000000 files? 
>>
>> /me shrugs
>>
>>> I guess it depends on the usage pattern?
>>
>> Totally.
> 
> Allow me to drag the OP's question under a different light...
> 
> I have a 20TB XFS filesystem with 3000000 files.  What is the minimum
> amount of system RAM I will need to run an xfs_check or xfs_repair or
> xfs_[tool] on this filesystem, assuming Linux has been put into a low
> overhead state, and said tool[s] has access to the bulk of the system
> memory?
> 
> Is there a formula available so any XFS user can calculate this
> xfs_[tools] RAM requirement, given FS size X and file count Y?
> 

http://xfs.org/index.php/XFS_FAQ#Q:_Which_factors_influence_the_memory_usage_of_xfs_repair.3F

-Eric



_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Michael Monnerie]

[-- Attachment #1.1: Type: text/plain, Size: 1243 bytes --]

Am Freitag, 17. Februar 2012, 18:17:46 schrieb Eric Sandeen:
> http://xfs.org/index.php/XFS_FAQ#Q:_Which_factors_influence_the_memory
> _usage_of_xfs_repair.3F

I tried that, and it said "use 434":

xfs_repair -n -vv -m 1 /dev/mapper/vg_orion-lv_orion_data 
Phase 1 - find and verify superblock...
        - max_mem = 1024, icount = 339648, imem = 1326, dblock = 
805304256, dmem = 393214
Required memory for repair is greater that the maximum specified with 
the -m option. Please increase it to at least 434.

But when I tried with 
# xfs_repair -n -vv -m 434 /dev/mapper/vg_orion-lv_orion_data 
it said the same again. It only worked with 435:
# xfs_repair -n -vv -m 435 /dev/mapper/vg_orion-lv_orion_data 
(is that what you call an off-by-1 error?)

Maybe that has been fixed already? This is
# xfs_repair -V
xfs_repair Version 3.1.6

BTW, this XFS is 3219644160 KB (3,2TB), used 2,9TB, has (df -i) 325364 
inodes used, 293884 files in 31643 dirs. It seems mem usage primarily 
comes from inodes, not from the size of the filesystem.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: XFS memory recomendation?

[Eric Sandeen]

On 2/19/12 7:16 AM, Michael Monnerie wrote:
> Am Freitag, 17. Februar 2012, 18:17:46 schrieb Eric Sandeen:
>> http://xfs.org/index.php/XFS_FAQ#Q:_Which_factors_influence_the_memory
>> _usage_of_xfs_repair.3F
> 
> I tried that, and it said "use 434":

That's megabytes, FWIW.

> xfs_repair -n -vv -m 1 /dev/mapper/vg_orion-lv_orion_data 
> Phase 1 - find and verify superblock...
>         - max_mem = 1024, icount = 339648, imem = 1326, dblock = 
> 805304256, dmem = 393214
> Required memory for repair is greater that the maximum specified with 
> the -m option. Please increase it to at least 434.
> 
> But when I tried with 
> # xfs_repair -n -vv -m 434 /dev/mapper/vg_orion-lv_orion_data 
> it said the same again. It only worked with 435:
> # xfs_repair -n -vv -m 435 /dev/mapper/vg_orion-lv_orion_data 
> (is that what you call an off-by-1 error?)

Yep, but really not too serious, I guess, still worth fixing though.

It's only used to try to enforce the bare minimum - in reality you'd
want more than that.

> Maybe that has been fixed already? This is
> # xfs_repair -V
> xfs_repair Version 3.1.6
> 
> BTW, this XFS is 3219644160 KB (3,2TB), used 2,9TB, has (df -i) 325364 
> inodes used, 293884 files in 31643 dirs. It seems mem usage primarily 
> comes from inodes, not from the size of the filesystem.

        _("        - max_mem = %lu, icount = %" PRIu64 ", imem = %" PRIu64 ", db
lock = %" PRIu64 ", dmem = %" PRIu64 "\n"),
                                max_mem, mp->m_sb.sb_icount,
                                mp->m_sb.sb_icount >> (10 - 2),
                                mp->m_sb.sb_dblocks,
                                mp->m_sb.sb_dblocks >> (10 + 1));

so yes, inodes in use count for more in the approximation.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs