EXTENT BOUNDARIES

EXTENT BOUNDARIES

[Les Oxley]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]


Hello,

We are looking into running XFS on a 3TB FLASH MEMORY MODULE.  We have a 
question  regarding the extent boundaries.
See the attached PowerPoint drawing, xfs.ppt We are running Linux.
Our  media is 3 million contiguous 4KB blocks. We would like to define 
an extent size of 1MB and this tracks the erasure block size
of the flash memory, and that greatly improves perfomance. We are trying 
to understand where XFS places the extent boundaries with reference to 
the contiguous block sequence.
Is this deterministic as indicated in the drawing ? That is, are the 
extent boundaries on 256 block boundaries.

Any help would be greatly appreciated.

Les Oxley
Ampex Corporation
Redwood City
California.

[-- Attachment #2: xfs.ppt --]
[-- Type: application/vnd.ms-powerpoint, Size: 23040 bytes --]

Re: EXTENT BOUNDARIES

[Chris Wedgwood]

On Fri, Jan 19, 2007 at 01:27:38PM -0800, Les Oxley wrote:

> We are looking into running XFS on a 3TB FLASH MEMORY MODULE.

neat, you need to send me some of those

does it wear level and do other clever stuff?  most conventional flash
storage devices don't last long with journalling filesystems...

> We have a question regarding the extent boundaries.  See the
> attached PowerPoint drawing, xfs.ppt We are running Linux.

you're asking linux people to read a powerpoint file?  :-)

you might have better luck with something else

> Our media is 3 million contiguous 4KB blocks.

That's not 3TB, surely it's < 12GB?

> We would like to define an extent size of 1MB and this tracks the
> erasure block size of the flash memory, and that greatly improves
> perfomance. We are trying to understand where XFS places the extent
> boundaries with reference to the contiguous block sequence.  Is this
> deterministic as indicated in the drawing ? That is, are the extent
> boundaries on 256 block boundaries.

i didn't open the powerpoint file, so i migth not be answering this
very well...

extent boundaries do not have to be on any boundary

you can tweak mkfs.xfs to affect where AGs start, is that of value to
you?

now, there are also rt volumes which can have larger extents so those
might be of value to you, but you would still need somewhere for the
metadata

Re: EXTENT BOUNDARIES

[Eric Sandeen]

Les Oxley wrote:
> 
> Hello,
> 
> We are looking into running XFS on a 3TB FLASH MEMORY MODULE.  We have a 
> question  regarding the extent boundaries.
> See the attached PowerPoint drawing, xfs.ppt We are running Linux.
> Our  media is 3 million contiguous 4KB blocks. We would like to define 
> an extent size of 1MB and this tracks the erasure block size
> of the flash memory, and that greatly improves perfomance. We are trying 
> to understand where XFS places the extent boundaries with reference to 
> the contiguous block sequence.
> Is this deterministic as indicated in the drawing ? That is, are the 
> extent boundaries on 256 block boundaries.
> 
> Any help would be greatly appreciated.
> 
> Les Oxley
> Ampex Corporation
> Redwood City
> California.

extents by definition land on filesystem block boundaries, and can in 
general be any number of filesystem blocks, starting & ending most 
anywhere on the block device.

If you wish to always allocate in 1m chunks, you might consider using 
the xfs realtime subvolume, see the extsize description in the mkfs.xfs 
man page.  I'm not sure how much buffered IO to the realtime subvol has 
been tested; pretty sure it works at this point, though the sgi guys 
will correct me if I'm wrong... it's not exactly the normal mode of 
operation.

Using the realtime subvol, however, all your file -metadata- will still 
be allocated on the main data volume, in much smaller pieces.

-Eric

RE: EXTENT BOUNDARIES

[Barry Naujok]

 

> -----Original Message-----
> From: xfs-bounce@oss.sgi.com [mailto:xfs-bounce@oss.sgi.com] 
> On Behalf Of Eric Sandeen
> Sent: Saturday, 20 January 2007 3:35 PM
> To: Les Oxley
> Cc: xfs@oss.sgi.com
> Subject: Re: EXTENT BOUNDARIES
> 
> Les Oxley wrote:
> > 
> > Hello,
> > 
> > We are looking into running XFS on a 3TB FLASH MEMORY 
> MODULE.  We have a 
> > question  regarding the extent boundaries.
> > See the attached PowerPoint drawing, xfs.ppt We are running Linux.
> > Our  media is 3 million contiguous 4KB blocks. We would 
> like to define 
> > an extent size of 1MB and this tracks the erasure block size
> > of the flash memory, and that greatly improves perfomance. 
> We are trying 
> > to understand where XFS places the extent boundaries with 
> reference to 
> > the contiguous block sequence.
> > Is this deterministic as indicated in the drawing ? That 
> is, are the 
> > extent boundaries on 256 block boundaries.
> > 
> > Any help would be greatly appreciated.
> > 
> > Les Oxley
> > Ampex Corporation
> > Redwood City
> > California.
> 
> extents by definition land on filesystem block boundaries, and can in 
> general be any number of filesystem blocks, starting & ending most 
> anywhere on the block device.
> 
> If you wish to always allocate in 1m chunks, you might consider using 
> the xfs realtime subvolume, see the extsize description in 
> the mkfs.xfs 
> man page.  I'm not sure how much buffered IO to the realtime 
> subvol has 
> been tested; pretty sure it works at this point, though the sgi guys 
> will correct me if I'm wrong... it's not exactly the normal mode of 
> operation.
> 
> Using the realtime subvol, however, all your file -metadata- 
> will still 
> be allocated on the main data volume, in much smaller pieces.
> 
> -Eric

If you don't need to use 100% of the space for your data, you can give
XFS a hint to align on a stripe unit if it's applicable.

If you allocate 1MB chunks at a time (either via a write or prealloc)
with an filesystem with sunit=1MB (2048 sectors if using 512 bytes
sectors for mkfs.xfs command), it will align to the stripe unit where
there is space available.

Once the aligned space is full, it will allocate in the remaining space.
Metadata such as inodes, directories, etc will not be nicely aligned.

If your total write or prealloc is smaller than 512KB it will not nicely
align, but find suitable space.

I would recommending some experimentation to see if either of the above
ideas are suitable for your purpose.

For the sunit idea and 512 byte sector size, the following mkfs command
should work:

# mkfs.xfs -b 4096 -d sunit=2048,swidth=4096 <device>

To see it in action using dd:

# dd if=/dev/zero of=<file> bs=1048576
# xfs_bmap -v <file>

You should see the block range aligned to 2048 sectors.