How to reserve disk space in XFS to make the blocks over many files continuous?

public inbox for linux-xfs@vger.kernel.org
 help / color / mirror / Atom feed

* How to reserve disk space in XFS to make the blocks over many files continuous?
@ 2012-11-07  1:02 huubby zhou
  2012-11-07  3:19 ` Dave Chinner
  0 siblings, 1 reply; 7+ messages in thread
From: huubby zhou @ 2012-11-07  1:02 UTC (permalink / raw)
  To: xfs

[-- Attachment #1.1: Type: text/plain, Size: 1363 bytes --]

Hi, folks,

I'm using *CentOS5.8*, with *XFS* filesystem extend storage disks. What I
want to do is, pre-allocating many files, with continuous blocks in
filesystem, both single file and crossing files. That is the start block ID
of the next file is following the end block ID of current file.

I could do space pre-allocation by *posix_allocate()*, problem is the API
zeros all disk space, I can't afford time consuming(I'm not sure if this
API could make the blocks continuous). then I tried
*xfsctl()*<http://linux.die.net/man/3/xfsctl>,
with *XFS_IOC_RESVSP* flag, I can reserve space faster.

The problem with *xfsctl()* is, it could make the blocks continuous for
individual file, but the blocks over files are *not* continuous. For
example, 10 files, a/b/c/d/e/f... reserved. After I do the real writing to
these files, it turns out the file 'b' isn't next to file 'a', and some
file could be far from both previous one and next one, though other files
may be neighboring with each other, rarely.

I use the following code to do the pre-allocation:

ftruncate(fd, FILE_SIZE);

xfs_flock_t flag = {0};
flag.l_whence = SEEK_SET;
flag.l_start  = 0;
flag.l_len    = 512*1024*1024;
xfsctl(fileName, fd, XFS_IOC_RESVSP64, &flag);

My question is, how can I guarantee the file system blocks over files
continuous? Thanks for your time and appreciate your answer.

[-- Attachment #1.2: Type: text/html, Size: 4250 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-07  1:02 How to reserve disk space in XFS to make the blocks over many files continuous? huubby zhou
@ 2012-11-07  3:19 ` Dave Chinner
  2012-11-09  2:04   ` huubby zhou
  2012-11-09 15:03   ` Roger Willcocks
  0 siblings, 2 replies; 7+ messages in thread
From: Dave Chinner @ 2012-11-07  3:19 UTC (permalink / raw)
  To: huubby zhou; +Cc: xfs

On Wed, Nov 07, 2012 at 09:02:11AM +0800, huubby zhou wrote:
> Hi, folks,
> 
> I'm using *CentOS5.8*, with *XFS* filesystem extend storage disks. What I
> want to do is, pre-allocating many files, with continuous blocks in
> filesystem, both single file and crossing files. That is the start block ID
> of the next file is following the end block ID of current file.
> 
> I could do space pre-allocation by *posix_allocate()*, problem is the API
> zeros all disk space, I can't afford time consuming(I'm not sure if this
> API could make the blocks continuous).

it doesn't on more recent releases - it uses fallocate() in the back
end, which does the same as:

> then I tried
> *xfsctl()*<http://linux.die.net/man/3/xfsctl>,
> with *XFS_IOC_RESVSP* flag, I can reserve space faster.

This command.

> The problem with *xfsctl()* is, it could make the blocks continuous for
> individual file, but the blocks over files are *not* continuous. For
> example, 10 files, a/b/c/d/e/f... reserved. After I do the real writing to
> these files, it turns out the file 'b' isn't next to file 'a', and some
> file could be far from both previous one and next one, though other files
> may be neighboring with each other, rarely.
> 
> I use the following code to do the pre-allocation:
> 
> ftruncate(fd, FILE_SIZE);
> 
> xfs_flock_t flag = {0};
> flag.l_whence = SEEK_SET;
> flag.l_start  = 0;
> flag.l_len    = 512*1024*1024;
> xfsctl(fileName, fd, XFS_IOC_RESVSP64, &flag);
> 
> My question is, how can I guarantee the file system blocks over files
> continuous? Thanks for your time and appreciate your answer.

You can't, directly. If you have enough contiguous free space in the
AG that you are allocating in, then you will get contiguous files if
the allocation size lines up with the filesystem geometry:

$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0 512m" foo.$i ; done
$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
 EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
 sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
   0: [0..1048575]:    8096..1056671     0 (8096..1056671)  1048576 10000
   0: [0..1048575]:    1056672..2105247  0 (1056672..2105247) 1048576 10000
   0: [0..1048575]:    2105248..3153823  0 (2105248..3153823) 1048576 10000
   0: [0..1048575]:    3153824..4202399  0 (3153824..4202399) 1048576 10000
   0: [0..1048575]:    4202400..5250975  0 (4202400..5250975) 1048576 10000
   0: [0..1048575]:    5250976..6299551  0 (5250976..6299551) 1048576 10000
   0: [0..1048575]:    6299552..7348127  0 (6299552..7348127) 1048576 10000
   0: [0..1048575]:    7348128..8396703  0 (7348128..8396703) 1048576 10000
   0: [0..1048575]:    8396704..9445279  0 (8396704..9445279) 1048576 10000
   0: [0..1048575]:    9445280..10493855  0 (9445280..10493855) 1048576 10000

So all those files are contiguous both internally and externally. If
there isn't sufficient contiguous freespace, or there is allocator
contention, this won't happen - it's best effort behaviour....

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-07  3:19 ` Dave Chinner
@ 2012-11-09  2:04   ` huubby zhou
  2012-11-09  3:08     ` Dave Chinner
  2012-11-09 15:03   ` Roger Willcocks
  1 sibling, 1 reply; 7+ messages in thread
From: huubby zhou @ 2012-11-09  2:04 UTC (permalink / raw)
  To: Dave Chinner; +Cc: xfs


[-- Attachment #1.1: Type: text/plain, Size: 7632 bytes --]

Hi, Dave,

Thanks for the answer, it's great, and I apologize for the terrible format.

>You can't, directly. If you have enough contiguous free space in the
>AG that you are allocating in, then you will get contiguous files if
>the allocation size lines up with the filesystem geometry:
>
>$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
512m" foo.$i ; done
>$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
> EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
> sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
>   0: [0..1048575]:    8096..1056671     0 (8096..1056671)  1048576 10000
>   0: [0..1048575]:    1056672..2105247  0 (1056672..2105247) 1048576 10000
>   0: [0..1048575]:    2105248..3153823  0 (2105248..3153823) 1048576 10000
>   0: [0..1048575]:    3153824..4202399  0 (3153824..4202399) 1048576 10000
>   0: [0..1048575]:    4202400..5250975  0 (4202400..5250975) 1048576 10000
>   0: [0..1048575]:    5250976..6299551  0 (5250976..6299551) 1048576 10000
>   0: [0..1048575]:    6299552..7348127  0 (6299552..7348127) 1048576 10000
>   0: [0..1048575]:    7348128..8396703  0 (7348128..8396703) 1048576 10000
>   0: [0..1048575]:    8396704..9445279  0 (8396704..9445279) 1048576 10000
>   0: [0..1048575]:    9445280..10493855  0 (9445280..10493855) 1048576
10000
>
>So all those files are contiguous both internally and externally. If
>there isn't sufficient contiguous freespace, or there is allocator
>contention, this won't happen - it's best effort behaviour....

I believe you got these in a single AG, but I do the allocation in
filesystem
with multi-AGs, specifically, it is a 6T storage space, and I run the
mkfs.xfs
without setting the AG number/size, it ends up with 32 AGs.
My files layout:
    - 0                         - dir
    | - 0                       - dir
    | | - 1                     - file
    | | - 2                     - file
    | | - 3                     - file
    | | - 4                     - file
    | | - 5                     - file
    | | - ...                   - file
    | | - 128                   - file
    | - 1                       - dir
    | | - 1                     - file
    | | - 2                     - file
    | | - 3                     - file
    | | - 4                     - file
    | | - 5                     - file
    | | - ...                   - file
    | | - 128                   - file
    | - ...                     - dir
Every file is 512MB, every directory holds 512MB*128=64GB.

According to your advice and XFS document, I tried to set the AG size to
64GB,
for avoiding the allocator contention and keeping all files in single
directory
fall in the same AG, but it didn't work. The files are still in different
AGs.

My xfs_info:
meta-data=/dev/sdc2              isize=256    agcount=96, agsize=16777216
blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=1610116329, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal log           bsize=4096   blocks=32768, version=1
         =                       sectsz=512   sunit=0 blks, lazy-count=0
realtime =none                   extsz=4096   blocks=0, rtextents=0

The files:
$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
512m" foo.$i ; done
$ sudo xfs_bmap -vp *| grep " 0:"
   0: [0..1048575]:    2147483712..2148532287 16 (64..1048639)    1048576
10000
   0: [0..1048575]:    3355443264..3356491839 25 (64..1048639)    1048576
10000
   0: [0..1048575]:    2281701440..2282750015 17 (64..1048639)    1048576
10000
   0: [0..1048575]:    2415919168..2416967743 18 (64..1048639)    1048576
10000
   0: [0..1048575]:    2550136896..2551185471 19 (64..1048639)    1048576
10000
   0: [0..1048575]:    2684354624..2685403199 20 (64..1048639)    1048576
10000
   0: [0..1048575]:    2818572352..2819620927 21 (64..1048639)    1048576
10000
   0: [0..1048575]:    2952790080..2953838655 22 (64..1048639)    1048576
10000
   0: [0..1048575]:    3087007808..3088056383 23 (64..1048639)    1048576
10000
   0: [0..1048575]:    3221225536..3222274111 24 (64..1048639)    1048576
10000

Any way I can specify which AG a file should be allocated?

Again, Thanks for your time.


On Wed, Nov 7, 2012 at 11:19 AM, Dave Chinner <david@fromorbit.com> wrote:

> On Wed, Nov 07, 2012 at 09:02:11AM +0800, huubby zhou wrote:
> > Hi, folks,
> >
> > I'm using *CentOS5.8*, with *XFS* filesystem extend storage disks. What I
> > want to do is, pre-allocating many files, with continuous blocks in
> > filesystem, both single file and crossing files. That is the start block
> ID
> > of the next file is following the end block ID of current file.
> >
> > I could do space pre-allocation by *posix_allocate()*, problem is the API
> > zeros all disk space, I can't afford time consuming(I'm not sure if this
> > API could make the blocks continuous).
>
> it doesn't on more recent releases - it uses fallocate() in the back
> end, which does the same as:
>
> > then I tried
> > *xfsctl()*<http://linux.die.net/man/3/xfsctl>,
> > with *XFS_IOC_RESVSP* flag, I can reserve space faster.
>
> This command.
>
> > The problem with *xfsctl()* is, it could make the blocks continuous for
> > individual file, but the blocks over files are *not* continuous. For
> > example, 10 files, a/b/c/d/e/f... reserved. After I do the real writing
> to
> > these files, it turns out the file 'b' isn't next to file 'a', and some
> > file could be far from both previous one and next one, though other files
> > may be neighboring with each other, rarely.
> >
> > I use the following code to do the pre-allocation:
> >
> > ftruncate(fd, FILE_SIZE);
> >
> > xfs_flock_t flag = {0};
> > flag.l_whence = SEEK_SET;
> > flag.l_start  = 0;
> > flag.l_len    = 512*1024*1024;
> > xfsctl(fileName, fd, XFS_IOC_RESVSP64, &flag);
> >
> > My question is, how can I guarantee the file system blocks over files
> > continuous? Thanks for your time and appreciate your answer.
>
> You can't, directly. If you have enough contiguous free space in the
> AG that you are allocating in, then you will get contiguous files if
> the allocation size lines up with the filesystem geometry:
>
> $ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
> 512m" foo.$i ; done
> $ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
>  EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
>  sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
>    0: [0..1048575]:    8096..1056671     0 (8096..1056671)  1048576 10000
>    0: [0..1048575]:    1056672..2105247  0 (1056672..2105247) 1048576 10000
>    0: [0..1048575]:    2105248..3153823  0 (2105248..3153823) 1048576 10000
>    0: [0..1048575]:    3153824..4202399  0 (3153824..4202399) 1048576 10000
>    0: [0..1048575]:    4202400..5250975  0 (4202400..5250975) 1048576 10000
>    0: [0..1048575]:    5250976..6299551  0 (5250976..6299551) 1048576 10000
>    0: [0..1048575]:    6299552..7348127  0 (6299552..7348127) 1048576 10000
>    0: [0..1048575]:    7348128..8396703  0 (7348128..8396703) 1048576 10000
>    0: [0..1048575]:    8396704..9445279  0 (8396704..9445279) 1048576 10000
>    0: [0..1048575]:    9445280..10493855  0 (9445280..10493855) 1048576
> 10000
>
> So all those files are contiguous both internally and externally. If
> there isn't sufficient contiguous freespace, or there is allocator
> contention, this won't happen - it's best effort behaviour....
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david@fromorbit.com
>

[-- Attachment #1.2: Type: text/html, Size: 10184 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-09  2:04   ` huubby zhou
@ 2012-11-09  3:08     ` Dave Chinner
  2012-11-09  6:50       ` huubby zhou
  0 siblings, 1 reply; 7+ messages in thread
From: Dave Chinner @ 2012-11-09  3:08 UTC (permalink / raw)
  To: huubby zhou; +Cc: xfs

On Fri, Nov 09, 2012 at 10:04:57AM +0800, huubby zhou wrote:
> Hi, Dave,
> 
> Thanks for the answer, it's great, and I apologize for the terrible format.
> 
> >You can't, directly. If you have enough contiguous free space in the
> >AG that you are allocating in, then you will get contiguous files if
> >the allocation size lines up with the filesystem geometry:
> >
> >$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
> 512m" foo.$i ; done
> >$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
> > EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
> > sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
> >   0: [0..1048575]:    8096..1056671     0 (8096..1056671)  1048576 10000
> >   0: [0..1048575]:    1056672..2105247  0 (1056672..2105247) 1048576 10000
> >   0: [0..1048575]:    2105248..3153823  0 (2105248..3153823) 1048576 10000
> >   0: [0..1048575]:    3153824..4202399  0 (3153824..4202399) 1048576 10000
> >   0: [0..1048575]:    4202400..5250975  0 (4202400..5250975) 1048576 10000
> >   0: [0..1048575]:    5250976..6299551  0 (5250976..6299551) 1048576 10000
> >   0: [0..1048575]:    6299552..7348127  0 (6299552..7348127) 1048576 10000
> >   0: [0..1048575]:    7348128..8396703  0 (7348128..8396703) 1048576 10000
> >   0: [0..1048575]:    8396704..9445279  0 (8396704..9445279) 1048576 10000
> >   0: [0..1048575]:    9445280..10493855  0 (9445280..10493855) 1048576
> 10000
> >
> >So all those files are contiguous both internally and externally. If
> >there isn't sufficient contiguous freespace, or there is allocator
> >contention, this won't happen - it's best effort behaviour....
> 
> I believe you got these in a single AG, but I do the allocation in
> filesystem
> with multi-AGs, specifically, it is a 6T storage space, and I run the
> mkfs.xfs
> without setting the AG number/size, it ends up with 32 AGs.
> My files layout:
>     - 0                         - dir
>     | - 0                       - dir
>     | | - 1                     - file
>     | | - 2                     - file
>     | | - 3                     - file
>     | | - 4                     - file
>     | | - 5                     - file
>     | | - ...                   - file
>     | | - 128                   - file
>     | - 1                       - dir
>     | | - 1                     - file
>     | | - 2                     - file
>     | | - 3                     - file
>     | | - 4                     - file
>     | | - 5                     - file
>     | | - ...                   - file
>     | | - 128                   - file
>     | - ...                     - dir
> Every file is 512MB, every directory holds 512MB*128=64GB.

Yup, that's exactly by design. That's how the inode64 allocation
policy is supposed to work.

> According to your advice and XFS document, I tried to set the AG size to
> 64GB,

What advice might that be? I don't thikn I've ever recommended
anyone use 96*64GB AGs. Unless you have 96 allocations all occurring
at the same time (very rare, in my experience), there is no need for
some many AGs.


> for avoiding the allocator contention and keeping all files in single
> directory
> fall in the same AG, but it didn't work. The files are still in different
> AGs.
> My xfs_info:
> meta-data=/dev/sdc2              isize=256    agcount=96, agsize=16777216
> blks
>          =                       sectsz=512   attr=0
> data     =                       bsize=4096   blocks=1610116329, imaxpct=25
>          =                       sunit=0      swidth=0 blks, unwritten=1
> naming   =version 2              bsize=4096
> log      =internal log           bsize=4096   blocks=32768, version=1
>          =                       sectsz=512   sunit=0 blks, lazy-count=0
> realtime =none                   extsz=4096   blocks=0, rtextents=0
> 
> The files:
> $ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
> 512m" foo.$i ; done
> $ sudo xfs_bmap -vp *| grep " 0:"
>    0: [0..1048575]:    2147483712..2148532287 16 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    3355443264..3356491839 25 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2281701440..2282750015 17 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2415919168..2416967743 18 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2550136896..2551185471 19 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2684354624..2685403199 20 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2818572352..2819620927 21 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    2952790080..2953838655 22 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    3087007808..3088056383 23 (64..1048639)    1048576
> 10000
>    0: [0..1048575]:    3221225536..3222274111 24 (64..1048639)    1048576
> 10000

That's inode32 allocator behaviour (rotoring each new allocation
across a different AG). Mount with inode64 - it's the default in the
latest kernels - and it will behave as I demonstrated.

Cheers,

Dave.

-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-09  3:08     ` Dave Chinner
@ 2012-11-09  6:50       ` huubby zhou
  0 siblings, 0 replies; 7+ messages in thread
From: huubby zhou @ 2012-11-09  6:50 UTC (permalink / raw)
  To: Dave Chinner; +Cc: xfs


[-- Attachment #1.1: Type: text/plain, Size: 5544 bytes --]

By "According to your advice...", I mean what your demonstrated.

I mount with inode64, and everything is working perfectly.
Many thanks, really appreciate!


On Fri, Nov 9, 2012 at 11:08 AM, Dave Chinner <david@fromorbit.com> wrote:

> On Fri, Nov 09, 2012 at 10:04:57AM +0800, huubby zhou wrote:
> > Hi, Dave,
> >
> > Thanks for the answer, it's great, and I apologize for the terrible
> format.
> >
> > >You can't, directly. If you have enough contiguous free space in the
> > >AG that you are allocating in, then you will get contiguous files if
> > >the allocation size lines up with the filesystem geometry:
> > >
> > >$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp
> 0
> > 512m" foo.$i ; done
> > >$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
> > > EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
> > > sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"
> > >   0: [0..1048575]:    8096..1056671     0 (8096..1056671)  1048576
> 10000
> > >   0: [0..1048575]:    1056672..2105247  0 (1056672..2105247) 1048576
> 10000
> > >   0: [0..1048575]:    2105248..3153823  0 (2105248..3153823) 1048576
> 10000
> > >   0: [0..1048575]:    3153824..4202399  0 (3153824..4202399) 1048576
> 10000
> > >   0: [0..1048575]:    4202400..5250975  0 (4202400..5250975) 1048576
> 10000
> > >   0: [0..1048575]:    5250976..6299551  0 (5250976..6299551) 1048576
> 10000
> > >   0: [0..1048575]:    6299552..7348127  0 (6299552..7348127) 1048576
> 10000
> > >   0: [0..1048575]:    7348128..8396703  0 (7348128..8396703) 1048576
> 10000
> > >   0: [0..1048575]:    8396704..9445279  0 (8396704..9445279) 1048576
> 10000
> > >   0: [0..1048575]:    9445280..10493855  0 (9445280..10493855) 1048576
> > 10000
> > >
> > >So all those files are contiguous both internally and externally. If
> > >there isn't sufficient contiguous freespace, or there is allocator
> > >contention, this won't happen - it's best effort behaviour....
> >
> > I believe you got these in a single AG, but I do the allocation in
> > filesystem
> > with multi-AGs, specifically, it is a 6T storage space, and I run the
> > mkfs.xfs
> > without setting the AG number/size, it ends up with 32 AGs.
> > My files layout:
> >     - 0                         - dir
> >     | - 0                       - dir
> >     | | - 1                     - file
> >     | | - 2                     - file
> >     | | - 3                     - file
> >     | | - 4                     - file
> >     | | - 5                     - file
> >     | | - ...                   - file
> >     | | - 128                   - file
> >     | - 1                       - dir
> >     | | - 1                     - file
> >     | | - 2                     - file
> >     | | - 3                     - file
> >     | | - 4                     - file
> >     | | - 5                     - file
> >     | | - ...                   - file
> >     | | - 128                   - file
> >     | - ...                     - dir
> > Every file is 512MB, every directory holds 512MB*128=64GB.
>
> Yup, that's exactly by design. That's how the inode64 allocation
> policy is supposed to work.
>
> > According to your advice and XFS document, I tried to set the AG size to
> > 64GB,
>
> What advice might that be? I don't thikn I've ever recommended
> anyone use 96*64GB AGs. Unless you have 96 allocations all occurring
> at the same time (very rare, in my experience), there is no need for
> some many AGs.
>
>
> > for avoiding the allocator contention and keeping all files in single
> > directory
> > fall in the same AG, but it didn't work. The files are still in different
> > AGs.
> > My xfs_info:
> > meta-data=/dev/sdc2              isize=256    agcount=96, agsize=16777216
> > blks
> >          =                       sectsz=512   attr=0
> > data     =                       bsize=4096   blocks=1610116329,
> imaxpct=25
> >          =                       sunit=0      swidth=0 blks, unwritten=1
> > naming   =version 2              bsize=4096
> > log      =internal log           bsize=4096   blocks=32768, version=1
> >          =                       sectsz=512   sunit=0 blks, lazy-count=0
> > realtime =none                   extsz=4096   blocks=0, rtextents=0
> >
> > The files:
> > $ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0
> > 512m" foo.$i ; done
> > $ sudo xfs_bmap -vp *| grep " 0:"
> >    0: [0..1048575]:    2147483712..2148532287 16 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    3355443264..3356491839 25 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2281701440..2282750015 17 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2415919168..2416967743 18 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2550136896..2551185471 19 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2684354624..2685403199 20 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2818572352..2819620927 21 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    2952790080..2953838655 22 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    3087007808..3088056383 23 (64..1048639)    1048576
> > 10000
> >    0: [0..1048575]:    3221225536..3222274111 24 (64..1048639)    1048576
> > 10000
>
> That's inode32 allocator behaviour (rotoring each new allocation
> across a different AG). Mount with inode64 - it's the default in the
> latest kernels - and it will behave as I demonstrated.
>
> Cheers,
>
> Dave.
>
> --
> Dave Chinner
> david@fromorbit.com
>

[-- Attachment #1.2: Type: text/html, Size: 7445 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-07  3:19 ` Dave Chinner
  2012-11-09  2:04   ` huubby zhou
@ 2012-11-09 15:03   ` Roger Willcocks
  2012-11-09 23:15     ` Dave Chinner
  1 sibling, 1 reply; 7+ messages in thread
From: Roger Willcocks @ 2012-11-09 15:03 UTC (permalink / raw)
  To: Dave Chinner; +Cc: huubby zhou, xfs


> > My question is, how can I guarantee the file system blocks over files
> > continuous? Thanks for your time and appreciate your answer.
> 
> You can't, directly. 

We needed to do this so I added code to swapext to support transferring
leading blocks from one (preallocated single extent) file to another
empty file. It's pretty straightforward but perhaps too special case for
general consumption.

-- 
Roger Willcocks <roger@filmlight.ltd.uk>

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: How to reserve disk space in XFS to make the blocks over many files continuous?
  2012-11-09 15:03   ` Roger Willcocks
@ 2012-11-09 23:15     ` Dave Chinner
  0 siblings, 0 replies; 7+ messages in thread
From: Dave Chinner @ 2012-11-09 23:15 UTC (permalink / raw)
  To: Roger Willcocks; +Cc: huubby zhou, xfs

On Fri, Nov 09, 2012 at 03:03:21PM +0000, Roger Willcocks wrote:
> 
> > > My question is, how can I guarantee the file system blocks over files
> > > continuous? Thanks for your time and appreciate your answer.
> > 
> > You can't, directly. 
> 
> We needed to do this so I added code to swapext to support transferring
> leading blocks from one (preallocated single extent) file to another
> empty file. It's pretty straightforward but perhaps too special case for
> general consumption.

Please post the patch - swapping arbitrary ranges between two inodes
is something that is definitely useful. It is needed, for example,
to do directory defragmentation, and I know that non-linear video
editting apps would love ioctls to do similar things within the same
file (e.g. punching ads out of a video stream without having to copy
data around at all).

That's a note for anyone that has implemented stuff like this -
regardless of whether you think it is useful or not, having the
patches out in the open (not matter what the state of the code)
makes it 100x more valuable than keeping it to yourself, and it
allows people to build functionality off them rather than having to
re-invent the wheel.

And, of course, there is the possibility we add the functionality to
the main tree, and you no longer have to maintain and test it
yourself.... :)

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2012-11-09 23:14 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-11-07  1:02 How to reserve disk space in XFS to make the blocks over many files continuous? huubby zhou
2012-11-07  3:19 ` Dave Chinner
2012-11-09  2:04   ` huubby zhou
2012-11-09  3:08     ` Dave Chinner
2012-11-09  6:50       ` huubby zhou
2012-11-09 15:03   ` Roger Willcocks
2012-11-09 23:15     ` Dave Chinner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox