Re: XFS and DPX files

From: "AndrewL733@aol.com" <AndrewL733@aol.com>
To: Emmanuel Florac <eflorac@intellique.com>
Cc: Michael Monnerie <michael.monnerie@is.it-management.at>, xfs@oss.sgi.com
Subject: Re: XFS and DPX files
Date: Mon, 02 Nov 2009 16:50:48 -0500	[thread overview]
Message-ID: <4AEF5438.5050801@aol.com> (raw)
In-Reply-To: <20091102185249.0da8e388@harpe.intellique.com>

>> I believe for a 15 drive RAID-6, where 2 disks are used forredundancy, the correct mkfs would be:
>> mkfs -t xfs -d su=65536,sw=13 /dev/sdXX
>>     
>
> Yes you're right, I replied a bit too quickly :)
>
>
>   
>> Another thing to try is if it would help to turn disk cache writes
>> *on*, despite all warnings if the FAQ. 
>>     

Thank you for your suggestions.  Yes I have write caching enabled. And I 
have StorSave set to "Performance". And I have a UPS on the system at 
all times!

The information about barriers was useful.  In years past I was running 
much older firmware for the 3ware 9650 cards and that did not support 
barriers.  But it is true the current firmware does support barriers.  I 
also believe the 3ware StorSave "Performance" setting will disable 
barriers as well -- at least it makes the card ignore FUA commands.

Anyway, I have mounted the XFS filesystem with the "nobarrier" flag and 
I'm still seeing the same behavior.  If you want to take a closer look 
at what I mean, please go to this link:

    http://sites.google.com/site/andrewl733info/xfs_and_dpx

At this point, I have tried the following -- and none of these 
approaches seems to fix the problem:

    -- preallocation of DPX files
    -- reservation of DXP files (Make 10,000 zero-byte files named
    0000001.dpx through 0010000.dpx)
    -- creating xfs filesystem with external log device (also a 16-drive
    RAID array, because that's what I have available)
    -- mounting with large logbsize
    -- mounting with more logbufs
    -- mounting with larger allocsize

Again, I want to point out that I don't have any problem with the 
underlying RAID device.  On Linux itself, I get Bonnie++ scores of 
around 740 MB/sec reading and 650 MB/sec writing, minimum.  Over 10 
Gigabit Ethernet, I can write uncompressed HD streams (160 MB/sec) and I 
can read 2K DPX files (300+ MB/sec).  DD shows similar results.

My gut feeling is that XFS is falling over after creating a certain 
number of new files.  Because the DPX format creates one file for every 
frame (30 files/sec), it's not really a video stream. It's really like 
making 30 photoshop files per second. It seems as if some resource that 
XFS needs is being used up after a certain number of files are created, 
and that it is very disruptive and costly to get more of that resource. 
Why ext3 and ext4 can keep going past 60,000 files and xfs falls over 
after 4000 or 5000 files, I do not understand.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs