[PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[Zorro Lang]

When I run xfstests xfs/032 in ppc64le, I hit a failure:

     xfs_copy: read failed: Invalid argument
     xfs_copy: size check failed
     xfs_copy: /dev/sda5 filesystem failed to initialize
     xfs_copy: Aborting.
     Copy failed for Sector size 4096 Block size 4096

I try to use gdb trace xfs_copy. I find it try to open the
source device with DIRECT flag, then read the device with
((1<<BBSHIFT))=512 bytes length. If the source device is
4k sector, directly read 512 bytes will be failed.

xfs_copy '-b' option only work for target file/device, to
sure it will open the target without DIRECT flag. But useless
for source device open.

So I make DIRECT flag only be enabled when source device
sector size equal BBSIZE.

Signed-off-by: Zorro Lang <zlang@redhat.com>
---

Hi,

I'm not so familiar for xfsprogs, so I'm not sure this's the best
idea to fix this problem. Please help to check, and maybe give me
better suggestions.

Thanks very much,
Zorro Lang

 copy/xfs_copy.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/copy/xfs_copy.c b/copy/xfs_copy.c
index e13f468..1cd6253 100644
--- a/copy/xfs_copy.c
+++ b/copy/xfs_copy.c
@@ -666,7 +666,6 @@ main(int argc, char **argv)
 	/* prepare the libxfs_init structure */
 
 	memset(&xargs, 0, sizeof(xargs));
-	xargs.isdirect = LIBXFS_DIRECT;
 	xargs.isreadonly = LIBXFS_ISREADONLY;
 
 	if (source_is_file)  {
@@ -689,6 +688,8 @@ main(int argc, char **argv)
 			     1 << (XFS_MAX_SECTORSIZE_LOG - BBSHIFT), 0, NULL);
 	sb = &mbuf.m_sb;
 	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
+	if (sb->sb_sectsize == BBSIZE)
+		xargs.isdirect = LIBXFS_DIRECT;
 
 	/* Do it again, now with proper length and verifier */
 	libxfs_putbuf(sbp);
-- 
1.9.3

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[Eric Sandeen]

On 8/13/15 10:07 AM, Zorro Lang wrote:
> When I run xfstests xfs/032 in ppc64le, I hit a failure:
> 
>      xfs_copy: read failed: Invalid argument
>      xfs_copy: size check failed
>      xfs_copy: /dev/sda5 filesystem failed to initialize
>      xfs_copy: Aborting.
>      Copy failed for Sector size 4096 Block size 4096
> 
> I try to use gdb trace xfs_copy. I find it try to open the
> source device with DIRECT flag, then read the device with
> ((1<<BBSHIFT))=512 bytes length. If the source device is
> 4k sector, directly read 512 bytes will be failed.
> 
> xfs_copy '-b' option only work for target file/device, to
> sure it will open the target without DIRECT flag. But useless
> for source device open.
> 
> So I make DIRECT flag only be enabled when source device
> sector size equal BBSIZE.

Which version of xfsprogs did you test?  This is recently upstream:

commit c63ce10a9450020382bbbe2c48788961b19830f4
Author: Eric Sandeen <sandeen@redhat.com>
Date:   Fri Jul 31 09:03:11 2015 +1000

    xfs_copy: fix copy of hard 4k devices
    
    If we have a pure 4k device with no 512 emulation, xfs_copy
    fails straightaway because it tries to do a 512-byte direct
    IO read of the superblock.
    
    Do like we do in xfs_db, and read in the max possible sector size,
    because we don't yet know what the filesystem's sector size is.
    
    This fixes a failure in xfs/032 on a hard 4k device.

-Eric

> Signed-off-by: Zorro Lang <zlang@redhat.com>
> ---
> 
> Hi,
> 
> I'm not so familiar for xfsprogs, so I'm not sure this's the best
> idea to fix this problem. Please help to check, and maybe give me
> better suggestions.
> 
> Thanks very much,
> Zorro Lang
> 
>  copy/xfs_copy.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/copy/xfs_copy.c b/copy/xfs_copy.c
> index e13f468..1cd6253 100644
> --- a/copy/xfs_copy.c
> +++ b/copy/xfs_copy.c
> @@ -666,7 +666,6 @@ main(int argc, char **argv)
>  	/* prepare the libxfs_init structure */
>  
>  	memset(&xargs, 0, sizeof(xargs));
> -	xargs.isdirect = LIBXFS_DIRECT;
>  	xargs.isreadonly = LIBXFS_ISREADONLY;
>  
>  	if (source_is_file)  {
> @@ -689,6 +688,8 @@ main(int argc, char **argv)
>  			     1 << (XFS_MAX_SECTORSIZE_LOG - BBSHIFT), 0, NULL);
>  	sb = &mbuf.m_sb;
>  	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
> +	if (sb->sb_sectsize == BBSIZE)
> +		xargs.isdirect = LIBXFS_DIRECT;
>  
>  	/* Do it again, now with proper length and verifier */
>  	libxfs_putbuf(sbp);
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[Zirong Lang]

----- 原始邮件 -----
> 发件人: "Eric Sandeen" <sandeen@sandeen.net>
> 收件人: "Zorro Lang" <zlang@redhat.com>, xfs@oss.sgi.com
> 发送时间: 星期五, 2015年 8 月 14日 上午 12:23:22
> 主题: Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device
> 
> On 8/13/15 10:07 AM, Zorro Lang wrote:
> > When I run xfstests xfs/032 in ppc64le, I hit a failure:
> > 
> >      xfs_copy: read failed: Invalid argument
> >      xfs_copy: size check failed
> >      xfs_copy: /dev/sda5 filesystem failed to initialize
> >      xfs_copy: Aborting.
> >      Copy failed for Sector size 4096 Block size 4096
> > 
> > I try to use gdb trace xfs_copy. I find it try to open the
> > source device with DIRECT flag, then read the device with
> > ((1<<BBSHIFT))=512 bytes length. If the source device is
> > 4k sector, directly read 512 bytes will be failed.
> > 
> > xfs_copy '-b' option only work for target file/device, to
> > sure it will open the target without DIRECT flag. But useless
> > for source device open.
> > 
> > So I make DIRECT flag only be enabled when source device
> > sector size equal BBSIZE.
> 
> Which version of xfsprogs did you test?  This is recently upstream:

Sorry I didn't notice that you have fix this problem. I test on the
newest version from git://oss.sgi.com/xfs/cmds/xfsprogs, it haven't
fix this problem. I test this patch on it at first. Then I find
there is another version in git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git

I don't know they are too different, so I didn't check if this xfsprogs-dev has
 fixed it, and just did this patch on it and send out.

Sorry for this mistake, and your patch is really better:)

Thanks,
Zorro Lang

> 
> commit c63ce10a9450020382bbbe2c48788961b19830f4
> Author: Eric Sandeen <sandeen@redhat.com>
> Date:   Fri Jul 31 09:03:11 2015 +1000
> 
>     xfs_copy: fix copy of hard 4k devices
>     
>     If we have a pure 4k device with no 512 emulation, xfs_copy
>     fails straightaway because it tries to do a 512-byte direct
>     IO read of the superblock.
>     
>     Do like we do in xfs_db, and read in the max possible sector size,
>     because we don't yet know what the filesystem's sector size is.
>     
>     This fixes a failure in xfs/032 on a hard 4k device.
> 
> -Eric
> 
> > Signed-off-by: Zorro Lang <zlang@redhat.com>
> > ---
> > 
> > Hi,
> > 
> > I'm not so familiar for xfsprogs, so I'm not sure this's the best
> > idea to fix this problem. Please help to check, and maybe give me
> > better suggestions.
> > 
> > Thanks very much,
> > Zorro Lang
> > 
> >  copy/xfs_copy.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/copy/xfs_copy.c b/copy/xfs_copy.c
> > index e13f468..1cd6253 100644
> > --- a/copy/xfs_copy.c
> > +++ b/copy/xfs_copy.c
> > @@ -666,7 +666,6 @@ main(int argc, char **argv)
> >  	/* prepare the libxfs_init structure */
> >  
> >  	memset(&xargs, 0, sizeof(xargs));
> > -	xargs.isdirect = LIBXFS_DIRECT;
> >  	xargs.isreadonly = LIBXFS_ISREADONLY;
> >  
> >  	if (source_is_file)  {
> > @@ -689,6 +688,8 @@ main(int argc, char **argv)
> >  			     1 << (XFS_MAX_SECTORSIZE_LOG - BBSHIFT), 0, NULL);
> >  	sb = &mbuf.m_sb;
> >  	libxfs_sb_from_disk(sb, XFS_BUF_TO_SBP(sbp));
> > +	if (sb->sb_sectsize == BBSIZE)
> > +		xargs.isdirect = LIBXFS_DIRECT;
> >  
> >  	/* Do it again, now with proper length and verifier */
> >  	libxfs_putbuf(sbp);
> > 
> 
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[Eric Sandeen]

On 8/13/15 11:40 AM, Zirong Lang wrote:
> 
> 
> ----- 原始邮件 -----
>> 发件人: "Eric Sandeen" <sandeen@sandeen.net>
>> 收件人: "Zorro Lang" <zlang@redhat.com>, xfs@oss.sgi.com
>> 发送时间: 星期五, 2015年 8 月 14日 上午 12:23:22
>> 主题: Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device
>>
>> On 8/13/15 10:07 AM, Zorro Lang wrote:
>>> When I run xfstests xfs/032 in ppc64le, I hit a failure:
>>>
>>>      xfs_copy: read failed: Invalid argument
>>>      xfs_copy: size check failed
>>>      xfs_copy: /dev/sda5 filesystem failed to initialize
>>>      xfs_copy: Aborting.
>>>      Copy failed for Sector size 4096 Block size 4096
>>>
>>> I try to use gdb trace xfs_copy. I find it try to open the
>>> source device with DIRECT flag, then read the device with
>>> ((1<<BBSHIFT))=512 bytes length. If the source device is
>>> 4k sector, directly read 512 bytes will be failed.
>>>
>>> xfs_copy '-b' option only work for target file/device, to
>>> sure it will open the target without DIRECT flag. But useless
>>> for source device open.
>>>
>>> So I make DIRECT flag only be enabled when source device
>>> sector size equal BBSIZE.
>>
>> Which version of xfsprogs did you test?  This is recently upstream:
> 
> Sorry I didn't notice that you have fix this problem. I test on the
> newest version from git://oss.sgi.com/xfs/cmds/xfsprogs, it haven't
> fix this problem. I test this patch on it at first. Then I find
> there is another version in git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
> 
> I don't know they are too different, so I didn't check if this xfsprogs-dev has
>  fixed it, and just did this patch on it and send out.
> 
> Sorry for this mistake, and your patch is really better:)

No problem, thanks for looking into it!  I'm not sure why the sgi git tree
is a bit behind right now.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device

[Dave Chinner]

On Thu, Aug 13, 2015 at 11:54:27AM -0500, Eric Sandeen wrote:
> On 8/13/15 11:40 AM, Zirong Lang wrote:
> > 
> > 
> > ----- 原始邮件 -----
> >> 发件人: "Eric Sandeen" <sandeen@sandeen.net>
> >> 收件人: "Zorro Lang" <zlang@redhat.com>, xfs@oss.sgi.com
> >> 发送时间: 星期五, 2015年 8 月 14日 上午 12:23:22
> >> 主题: Re: [PATCH][RFC] xfs_copy: don't use DIRECT IO to copy 4k sector device
> >>
> >> On 8/13/15 10:07 AM, Zorro Lang wrote:
> >>> When I run xfstests xfs/032 in ppc64le, I hit a failure:
> >>>
> >>>      xfs_copy: read failed: Invalid argument
> >>>      xfs_copy: size check failed
> >>>      xfs_copy: /dev/sda5 filesystem failed to initialize
> >>>      xfs_copy: Aborting.
> >>>      Copy failed for Sector size 4096 Block size 4096
> >>>
> >>> I try to use gdb trace xfs_copy. I find it try to open the
> >>> source device with DIRECT flag, then read the device with
> >>> ((1<<BBSHIFT))=512 bytes length. If the source device is
> >>> 4k sector, directly read 512 bytes will be failed.
> >>>
> >>> xfs_copy '-b' option only work for target file/device, to
> >>> sure it will open the target without DIRECT flag. But useless
> >>> for source device open.
> >>>
> >>> So I make DIRECT flag only be enabled when source device
> >>> sector size equal BBSIZE.
> >>
> >> Which version of xfsprogs did you test?  This is recently upstream:
> > 
> > Sorry I didn't notice that you have fix this problem. I test on the
> > newest version from git://oss.sgi.com/xfs/cmds/xfsprogs, it haven't
> > fix this problem. I test this patch on it at first. Then I find
> > there is another version in git://git.kernel.org/pub/scm/fs/xfs/xfsprogs-dev.git
> > 
> > I don't know they are too different, so I didn't check if this xfsprogs-dev has
> >  fixed it, and just did this patch on it and send out.
> > 
> > Sorry for this mistake, and your patch is really better:)
> 
> No problem, thanks for looking into it!  I'm not sure why the sgi git tree
> is a bit behind right now.

I didn't push 4.2.0-rc1 in case anyone objected to the change of
version numbering I was proposing. Nobody has commented, so when I
push 4.2.0-rc2 out, I'll also update the oss.sgi.com repo.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs