Bug#661580: mkfs.xfs fails to detect correct sector size

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Package: xfsprogs
Version: 3.1.7
Severity: important
File: /sbin/mkfs.xfs

Hi,

since I recently bought a harddisk with 4K sector size I've been
checking for correct alignment and blocksizes in various things. One
of them being xfs.

When I partition the disk in parted it shows correctly to have a
logical blocksize of 512 byte and physical blocksize of 4096 byte.
But mkfs.xfs does not detect that the physical blocksize is 4096 byte
and creates a filesystem with:

meta-data=/dev/sde1              isize=256    agcount=32, agsize=22892696 blks
         =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=732566272, imaxpct=5
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal               bsize=4096   blocks=357698, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


This often results in access to the disk that is not aligned to the
blocksize with the resulting degradation in performance.

Further, manually selecting the right sector size of 4096 still causes
lots of unaligned access while formating and while mounting the
filesystem. It does seem to fix the problem for actual use though.

MfG
	Goswin

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.1.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=de_DE (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/dash

Versions of packages xfsprogs depends on:
ii  libblkid1     2.20.1-1
ii  libc6         2.13-21 
ii  libreadline5  5.2-11  
ii  libuuid1      2.19.1-5

xfsprogs recommends no packages.

Versions of packages xfsprogs suggests:
pn  acl      2.2.51-4  
pn  attr     1:2.4.46-3
pn  quota    <none>    
pn  xfsdump  <none>    

-- no debconf information


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Christoph Hellwig]

Carlos, didn't you plan to look into this issue?

Goswin, how do you determin that mkfs is still doing unaligned I/O
when forcing the large sevtor size?  Once we set the sector size XFS
can't do I/O smaller than it.

On Tue, Feb 28, 2012 at 08:49:14AM +0100, Goswin von Brederlow wrote:
> Package: xfsprogs
> Version: 3.1.7
> Severity: important
> File: /sbin/mkfs.xfs
> 
> Hi,
> 
> since I recently bought a harddisk with 4K sector size I've been
> checking for correct alignment and blocksizes in various things. One
> of them being xfs.
> 
> When I partition the disk in parted it shows correctly to have a
> logical blocksize of 512 byte and physical blocksize of 4096 byte.
> But mkfs.xfs does not detect that the physical blocksize is 4096 byte
> and creates a filesystem with:
> 
> meta-data=/dev/sde1              isize=256    agcount=32, agsize=22892696 blks
>          =                       sectsz=512   attr=2
> data     =                       bsize=4096   blocks=732566272, imaxpct=5
>          =                       sunit=0      swidth=0 blks
> naming   =version 2              bsize=4096   ascii-ci=0
> log      =internal               bsize=4096   blocks=357698, version=2
>          =                       sectsz=512   sunit=0 blks, lazy-count=1
> realtime =none                   extsz=4096   blocks=0, rtextents=0
> 
> 
> This often results in access to the disk that is not aligned to the
> blocksize with the resulting degradation in performance.
> 
> Further, manually selecting the right sector size of 4096 still causes
> lots of unaligned access while formating and while mounting the
> filesystem. It does seem to fix the problem for actual use though.
> 
> MfG
> 	Goswin
> 
> -- System Information:
> Debian Release: wheezy/sid
>   APT prefers unstable
>   APT policy: (500, 'unstable')
> Architecture: amd64 (x86_64)
> 
> Kernel: Linux 3.1.0-1-amd64 (SMP w/4 CPU cores)
> Locale: LANG=C, LC_CTYPE=de_DE (charmap=ISO-8859-1)
> Shell: /bin/sh linked to /bin/dash
> 
> Versions of packages xfsprogs depends on:
> ii  libblkid1     2.20.1-1
> ii  libc6         2.13-21 
> ii  libreadline5  5.2-11  
> ii  libuuid1      2.19.1-5
> 
> xfsprogs recommends no packages.
> 
> Versions of packages xfsprogs suggests:
> pn  acl      2.2.51-4  
> pn  attr     1:2.4.46-3
> pn  quota    <none>    
> pn  xfsdump  <none>    
> 
> -- no debconf information
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
---end quoted text---


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Carlos Maiolino]

On Tue, Feb 28, 2012 at 04:11:51AM -0500, Christoph Hellwig wrote:
> Carlos, didn't you plan to look into this issue?
> 
> Goswin, how do you determin that mkfs is still doing unaligned I/O
> when forcing the large sevtor size?  Once we set the sector size XFS
> can't do I/O smaller than it.
> 
> On Tue, Feb 28, 2012 at 08:49:14AM +0100, Goswin von Brederlow wrote:

Yes, I was looking to it some time ago, but iirc it was already implemented and
I was just fixins some issues on the code already implemented. I just got a new
box and I'm recovering my backup (including code trees). I'll take a look on
this later today and let you know on which steps this is.

-- 
--Carlos


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Christoph Hellwig <hch@infradead.org> writes:

> Carlos, didn't you plan to look into this issue?
>
> Goswin, how do you determin that mkfs is still doing unaligned I/O
> when forcing the large sevtor size?  Once we set the sector size XFS
> can't do I/O smaller than it.

I'm writing a nbd-server that uses async IO via libaio (the linux kernel
AIO interface). This has some alignment requirements so I set nbd to use
a 4k blocksize and added asserts that all requests are page aligned
(both offset and size of the request). That worked fine for dd and
mkfs.ext4 but fails on mount in general or mkfs.xfs.

So next I added a printf("WARNING: unaligned read/write") and handled
unaligned access in the server to see how extensive the problem is. With
ext4 I get ONE single read request of 512 Byte on mount and everything
else properly 4k aligned. With xfs I get a ton of them.

If you want I can add a log of the read/write requests a mkfs.xfs and
mount generates.

MfG
        Goswin


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Eric Sandeen]

On 2/28/12 3:11 AM, Christoph Hellwig wrote:
> Carlos, didn't you plan to look into this issue?
> 
> Goswin, how do you determin that mkfs is still doing unaligned I/O
> when forcing the large sevtor size?  Once we set the sector size XFS
> can't do I/O smaller than it.

I did think this was supposed to be working already:

        get_topology(&xi, &ft);

        if (ft.sectoralign) {
                /*
                 * Older Linux software RAID versions want the sector size
                 * to match the block size to avoid switching I/O sizes.
                 * For the legacy libdisk case we thus set the sector size to
                 * match the block size.  For systems using libblkid we assume
                 * that the kernel is recent enough to not require this and
                 * ft.sectoralign will never be set.
                 */
                sectorsize = blocksize;
        } else if (!ssflag) {
                /*
                 * Unless specified manually on the command line use the
                 * advertised sector size of the device.
                 */
                sectorsize = ft.sectorsize ? ft.sectorsize : XFS_MIN_SECTORSIZE;
        }

but it may depend on whether or not it is being built with libblkid support or not...?

I'll try to test it when I have some time, unless Carlos beats me to it.

-Eric




_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Eric Sandeen <sandeen@sandeen.net> writes:

> On 2/28/12 3:11 AM, Christoph Hellwig wrote:
>> Carlos, didn't you plan to look into this issue?
>> 
>> Goswin, how do you determin that mkfs is still doing unaligned I/O
>> when forcing the large sevtor size?  Once we set the sector size XFS
>> can't do I/O smaller than it.
>
> I did think this was supposed to be working already:
>
>         get_topology(&xi, &ft);
>
>         if (ft.sectoralign) {
>                 /*
>                  * Older Linux software RAID versions want the sector size
>                  * to match the block size to avoid switching I/O sizes.
>                  * For the legacy libdisk case we thus set the sector size to
>                  * match the block size.  For systems using libblkid we assume
>                  * that the kernel is recent enough to not require this and
>                  * ft.sectoralign will never be set.
>                  */
>                 sectorsize = blocksize;
>         } else if (!ssflag) {
>                 /*
>                  * Unless specified manually on the command line use the
>                  * advertised sector size of the device.
>                  */
>                 sectorsize = ft.sectorsize ? ft.sectorsize : XFS_MIN_SECTORSIZE;
>         }
>
> but it may depend on whether or not it is being built with libblkid support or not...?
>
> I'll try to test it when I have some time, unless Carlos beats me to it.
>
> -Eric

I'm asuming libblkid is used, otherwise a sectorsize of 512 would make
no sense.

It might be that NBD advertises the geometry wrong, as in ft.sectorsize
isn't set correctly. That would make that part at least a kernel bug.

MfG
        Goswin


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Christoph Hellwig]

On Thu, Mar 01, 2012 at 04:53:32AM +0100, Goswin von Brederlow wrote:
> I'm asuming libblkid is used, otherwise a sectorsize of 512 would make
> no sense.
> 
> It might be that NBD advertises the geometry wrong, as in ft.sectorsize
> isn't set correctly. That would make that part at least a kernel bug.

Now that I've looked at nbd: I can't find any way to transfer the
physical or logical block size information to the nbd client.  How
do you try set it on the server?



_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Christoph Hellwig <hch@infradead.org> writes:

> On Thu, Mar 01, 2012 at 04:53:32AM +0100, Goswin von Brederlow wrote:
>> I'm asuming libblkid is used, otherwise a sectorsize of 512 would make
>> no sense.
>> 
>> It might be that NBD advertises the geometry wrong, as in ft.sectorsize
>> isn't set correctly. That would make that part at least a kernel bug.
>
> Now that I've looked at nbd: I can't find any way to transfer the
> physical or logical block size information to the nbd client.  How
> do you try set it on the server?

I don't. The protocol doesn't (yet) have an option for this. There are a
number of new options planed for the new handshake protocol but the nbd
maintainer hasn't had time to implement them yet.

What I do is set the blocksize = 4096 on the nbd-client commandline
knowing that that is what the server expects.

MfG
        Goswin


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Package: xfsprogs
Version: 3.1.7
Followup-For: Bug #661580

Hi again,

I've patched the NBD driver to set the physical sector size. Was way
easier than I imagined:

----------------------------------------------------------------------
--- linux/drivers/block/nbd.c~      2012-03-01 09:10:35.000000000 +0100
+++ linux/drivers/block/nbd.c       2012-03-01 08:57:49.000000000 +0100
@@ -413,6 +413,9 @@
                return ret;
        }
 
+       /* Set physical block size to blocksize */
+       blk_queue_physical_block_size(lo->disk->queue, lo->blksize);
+
        while ((req = nbd_read_stat(lo)) != NULL)
                nbd_end_request(req);
 
----------------------------------------------------------------------

And I can confirm that the sector size detection is still broken in
3.1.7.

root@frosties:~$ parted /dev/nbd0
GNU Parted 2.3
Using /dev/nbd0
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p                                                                
Model: Unknown (unknown)
Disk /dev/nbd0: 16.8MB
Sector size (logical/physical): 512B/4096B
Partition Table: loop

Number  Start  End     Size    File system  Flags
 1      0.00B  16.8MB  16.8MB  xfs

(parted) q                                                                
root@frosties:~$ mkfs.xfs -f /dev/nbd0 
meta-data=/dev/nbd0              isize=256    agcount=1, agsize=4096 blks
         =                       sectsz=512   attr=2, projid32bit=0
data     =                       bsize=4096   blocks=4096, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0
log      =internal log           bsize=4096   blocks=1200, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

mrvn@frosties:~% tail /sys/block/nbd0/queue/*
==> /sys/block/nbd0/queue/add_random <==
1

==> /sys/block/nbd0/queue/discard_granularity <==
0

==> /sys/block/nbd0/queue/discard_max_bytes <==
0

==> /sys/block/nbd0/queue/discard_zeroes_data <==
0

==> /sys/block/nbd0/queue/hw_sector_size <==
512

==> /sys/block/nbd0/queue/iosched <==
tail: error reading `/sys/block/nbd0/queue/iosched': Is a directory

==> /sys/block/nbd0/queue/iostats <==
1

==> /sys/block/nbd0/queue/logical_block_size <==
512

==> /sys/block/nbd0/queue/max_hw_sectors_kb <==
127

==> /sys/block/nbd0/queue/max_integrity_segments <==
0

==> /sys/block/nbd0/queue/max_sectors_kb <==
127

==> /sys/block/nbd0/queue/max_segment_size <==
65536

==> /sys/block/nbd0/queue/max_segments <==
128

==> /sys/block/nbd0/queue/minimum_io_size <==
4096

==> /sys/block/nbd0/queue/nomerges <==
0

==> /sys/block/nbd0/queue/nr_requests <==
128

==> /sys/block/nbd0/queue/optimal_io_size <==
0

==> /sys/block/nbd0/queue/physical_block_size <==
4096

==> /sys/block/nbd0/queue/read_ahead_kb <==
128

==> /sys/block/nbd0/queue/rotational <==
0

==> /sys/block/nbd0/queue/rq_affinity <==
1

==> /sys/block/nbd0/queue/scheduler <==
noop deadline [cfq] 


MfG
	Goswin

-- System Information:
Debian Release: wheezy/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.1.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=de_DE (charmap=ISO-8859-1)
Shell: /bin/sh linked to /bin/dash

Versions of packages xfsprogs depends on:
ii  libblkid1     2.20.1-1
ii  libc6         2.13-21 
ii  libreadline5  5.2-11  
ii  libuuid1      2.19.1-5

xfsprogs recommends no packages.

Versions of packages xfsprogs suggests:
pn  acl      2.2.51-4  
pn  attr     1:2.4.46-3
pn  quota    <none>    
pn  xfsdump  <none>    

-- no debconf information


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Christoph Hellwig]

On Thu, Mar 01, 2012 at 09:14:09AM +0100, Goswin von Brederlow wrote:
> And I can confirm that the sector size detection is still broken in
> 3.1.7.

That's a known bug that Carlos wanted to look into.  If you really can't
accept smaller I/O without major pain make sure to use a larger logical
blocksize, which should be handled fine by everything.



_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Goswin von Brederlow]

Christoph Hellwig <hch@infradead.org> writes:

> On Thu, Mar 01, 2012 at 09:14:09AM +0100, Goswin von Brederlow wrote:
>> And I can confirm that the sector size detection is still broken in
>> 3.1.7.
>
> That's a known bug that Carlos wanted to look into.  If you really can't
> accept smaller I/O without major pain make sure to use a larger logical
> blocksize, which should be handled fine by everything.

Tried that too. Setting a locigal block size of 4096 makes all access
aligned to 4096 byte and makes mkfs.xfs detect a sector size of 4096
too.

But that breaks partitioning support for example. Linux simply won't
read the partition table on /dev/nbd0 anymore. Not sure why. Plus that
doesn't solve the problem for real harddisks.

MfG
        Goswin


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Eric Sandeen]

On 3/1/12 2:33 AM, Christoph Hellwig wrote:
> On Thu, Mar 01, 2012 at 09:14:09AM +0100, Goswin von Brederlow wrote:
>> And I can confirm that the sector size detection is still broken in
>> 3.1.7.
> 
> That's a known bug that Carlos wanted to look into.  If you really can't
> accept smaller I/O without major pain make sure to use a larger logical
> blocksize, which should be handled fine by everything.

Carlos is a bit busy right now, I'll send a patch shortly; want to whip
up an xfstest for this too.

-Eric


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Bug#661580: mkfs.xfs fails to detect correct sector size

[Eric Sandeen]

I've pushed a commit which should fix this, assuming xfsprogs is built with libblkid support enabled:

http://git.kernel.org/?p=fs/xfs/xfsprogs-dev.git;a=commitdiff;h=287d168b550857ce40e04b5f618d7eb91b87022f

-Eric


_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs