The first character disappears

The first character disappears

[Markus Suvanto]

After bcp the first character disappears.
Kernel:=A0 2.6.31-rc7
Btrfs v0.19-1-g4f89b6e


Way to reproduce:

htpc ~ # lvm lvcreate -L 5G -n btrfs_test /dev/vg_md1
=A0 Logical volume "btrfs_test" created

htpc ~ # mkbtrfs /dev/mapper/vg_md1-btrfs_test

WARNING! - Btrfs v0.19-1-g4f89b6e IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

fs created label (null) on /dev/mapper/vg_md1-btrfs_test
=A0=A0=A0=A0=A0=A0=A0 nodesize 4096 leafsize 4096 sectorsize 4096 size =
5.00GB
Btrfs v0.19-1-g4f89b6e

mount -t btrfs -o subvol=3D. /dev/mapper/vg_md1-btrfs_test /tmp/btrfs/

htpc ~ # btrfsctl -S test /tmp/btrfs/
operation complete
Btrfs v0.19-1-g4f89b6e

htpc ~ # cd /tmp/btrfs/test/

htpc test # git clone /usr/src/linux-2.6/
Initialized empty Git repository in /tmp/btrfs/test/linux-2.6/.git/
Checking out files: 100% (29110/29110), done.

htpc test # btrfs-bcp linux-2.6/ linux-2.6_copy
htpc test # ls -la linux-2.6_copy/
total 350
drwxr-xr-x 1 root root=A0=A0=A0 324 Aug 24 07:53 .
drwx------ 1 root root=A0=A0=A0=A0 46 Aug 24 07:53 ..
-rw-r--r-- 1 root root 152148 Aug 24 07:53 AINTAINERS
-rw-r--r-- 1 root root=A0 17459 Aug 24 07:53 EADME
-rw-r--r-- 1 root root=A0=A0 3371 Aug 24 07:53 EPORTING-BUGS
-rw-r--r-- 1 root root=A0 18693 Aug 24 07:53 OPYING
-rw-r--r-- 1 root root=A0 94027 Aug 24 07:53 REDITS
-rw-r--r-- 1 root root=A0 55145 Aug 24 07:53 akefile
drwxr-xr-x 1 root root=A0=A0=A0 118 Aug 24 07:54 amples
=2E...
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: The first character disappears

[Chris Mason]

On Mon, Aug 24, 2009 at 08:17:21AM +0300, Markus Suvanto wrote:
> After bcp the first character disappears.
> Kernel:=A0 2.6.31-rc7
> Btrfs v0.19-1-g4f89b6e
>=20
>=20
> Way to reproduce:
>=20
> htpc ~ # lvm lvcreate -L 5G -n btrfs_test /dev/vg_md1
> =A0 Logical volume "btrfs_test" created
>=20
> htpc ~ # mkbtrfs /dev/mapper/vg_md1-btrfs_test
>=20
> WARNING! - Btrfs v0.19-1-g4f89b6e IS EXPERIMENTAL
> WARNING! - see http://btrfs.wiki.kernel.org before using
>=20
> fs created label (null) on /dev/mapper/vg_md1-btrfs_test
> =A0=A0=A0=A0=A0=A0=A0 nodesize 4096 leafsize 4096 sectorsize 4096 siz=
e 5.00GB
> Btrfs v0.19-1-g4f89b6e
>=20
> mount -t btrfs -o subvol=3D. /dev/mapper/vg_md1-btrfs_test /tmp/btrfs=
/
>=20
> htpc ~ # btrfsctl -S test /tmp/btrfs/
> operation complete
> Btrfs v0.19-1-g4f89b6e
>=20
> htpc ~ # cd /tmp/btrfs/test/
>=20
> htpc test # git clone /usr/src/linux-2.6/
> Initialized empty Git repository in /tmp/btrfs/test/linux-2.6/.git/
> Checking out files: 100% (29110/29110), done.
>=20
> htpc test # btrfs-bcp linux-2.6/ linux-2.6_copy
> htpc test # ls -la linux-2.6_copy/

Well, that's not good.  Thanks for this report, I'll take a look as I g=
o
through my mountain of vacation backlog.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH]Re: The first character disappears

[Atila]

*
Signed-off-by: Atila <atila.alr@dpf.gov.br>
---
 bcp |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/bcp b/bcp
index 5729e91..7c6b351 100755
--- a/bcp
+++ b/bcp
@@ -107,7 +107,7 @@ if src_args > 1:
         exit(1)
 
 for srci in xrange(0, src_args):
-    src = args[srci]
+    src = args[srci].strip('/')
     if os.path.isfile(src):
         statinfo = os.lstat(src)
         force_name = None
@@ -136,7 +136,7 @@ for srci in xrange(0, src_args):
             srcname = os.path.join(dirpath, x)
             statinfo = os.lstat(srcname)
 
-            if srcname.startswith(src):
+            if srcname.startswith(src + '/'):
                 part = srcname[len(src) + 1:]
 
             if stat.S_ISLNK(statinfo.st_mode):
@@ -152,7 +152,7 @@ for srci in xrange(0, src_args):
 
         for f in filenames:
             srcname = os.path.join(dirpath, f)
-            if srcname.startswith(src):
+            if srcname.startswith(src + '/'):
                 part = srcname[len(src) + 1:]
 
             statinfo = os.lstat(srcname)
-- 
1.6.4

*

Re: [PATCH]Re: The first character disappears

[Markus Suvanto]

After this patch against the latest
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstabl=
e.git
bcp don't eat the first character anymore, thanks :)


2009/8/24 Atila <atila.alr@dpf.gov.br>:
> *
> Signed-off-by: Atila <atila.alr@dpf.gov.br>
> ---
> =A0bcp | =A0 =A06 +++---
> =A01 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/bcp b/bcp
> index 5729e91..7c6b351 100755
> --- a/bcp
> +++ b/bcp
> @@ -107,7 +107,7 @@ if src_args > 1:
> =A0 =A0 =A0 =A0 exit(1)
>
> =A0for srci in xrange(0, src_args):
> - =A0 =A0src =3D args[srci]
> + =A0 =A0src =3D args[srci].strip('/')
> =A0 =A0 if os.path.isfile(src):
> =A0 =A0 =A0 =A0 statinfo =3D os.lstat(src)
> =A0 =A0 =A0 =A0 force_name =3D None
> @@ -136,7 +136,7 @@ for srci in xrange(0, src_args):
> =A0 =A0 =A0 =A0 =A0 =A0 srcname =3D os.path.join(dirpath, x)
> =A0 =A0 =A0 =A0 =A0 =A0 statinfo =3D os.lstat(srcname)
>
> - =A0 =A0 =A0 =A0 =A0 =A0if srcname.startswith(src):
> + =A0 =A0 =A0 =A0 =A0 =A0if srcname.startswith(src + '/'):
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 part =3D srcname[len(src) + 1:]
>
> =A0 =A0 =A0 =A0 =A0 =A0 if stat.S_ISLNK(statinfo.st_mode):
> @@ -152,7 +152,7 @@ for srci in xrange(0, src_args):
>
> =A0 =A0 =A0 =A0 for f in filenames:
> =A0 =A0 =A0 =A0 =A0 =A0 srcname =3D os.path.join(dirpath, f)
> - =A0 =A0 =A0 =A0 =A0 =A0if srcname.startswith(src):
> + =A0 =A0 =A0 =A0 =A0 =A0if srcname.startswith(src + '/'):
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 part =3D srcname[len(src) + 1:]
>
> =A0 =A0 =A0 =A0 =A0 =A0 statinfo =3D os.lstat(srcname)
> --
> 1.6.4
>
> *
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH]Re: The first character disappears

[Markus Suvanto]

Ok, the first problem is fixed but after bcp
git status
error: bad index file sha1 signature
fatal: index file corrupt

Way to reproduce:

htpc 17:07 masu >git clone /usr/src/linux-2.6/
Initialized empty Git repository in /mnt/btrfs/masu/linux-2.6/.git/
Checking out files: 100% (29110/29110), done.
htpc 17:08 masu >cd linux-2.6/
htpc 17:08 linux-2.6 >git status
# On branch master
nothing to commit (working directory clean)

htpc 17:08 linux-2.6 >cd ..
htpc 17:08 masu >/tmp/btrfs-progs-unstable/bcp linux-2.6/ linux-2.6_copy
htpc 17:09 masu >cd linux-2.6_copy
htpc 17:09 linux-2.6_copy >git status
error: bad index file sha1 signature
fatal: index file corrupt


2009/8/24 Markus Suvanto <markus.suvanto@gmail.com>:
> After this patch against the latest
> git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs-unstable.git
> bcp don't eat the first character anymore, thanks :)
>
>
> 2009/8/24 Atila <atila.alr@dpf.gov.br>:
>> *
>> Signed-off-by: Atila <atila.alr@dpf.gov.br>
>> ---

Re: [PATCH]Re: The first character disappears

[Markus Suvanto]

Now I notice that I can reproduce this index file corrupt only if
compress is enabled.

lvm lvcreate -L 5G -n btrfs_test /dev/vg_md1
mkbtrfs /dev/mapper/vg_md1-btrfs_test
mount -t btrfs -o subvol=.,compress  /dev/mapper/vg_md1-btrfs_test /tmp/btrfs/
btrfsctl -S test /tmp/btrfs/
cd /tmp/btrfs/test/
git clone /usr/src/linux-2.6/
/tmp/btrfs-progs-unstable/bcp linux-2.6/ linux-2.6_copy
cd linux-2.6_copy/
git status

error: bad index file sha1 signature
fatal: index file corrupt


But without compress:

mkbtrfs /dev/mapper/vg_md1-btrfs_test
mount -t btrfs -o subvol=.  /dev/mapper/vg_md1-btrfs_test /tmp/btrfs/
btrfsctl -S test /tmp/btrfs/
cd /tmp/btrfs/test/
git clone /usr/src/linux-2.6/
/tmp/btrfs-progs-unstable/bcp linux-2.6/ linux-2.6_copy
cd linux-2.6_copy/
git status

# On branch master
nothing to commit (working directory clean)


2009/8/24 Markus Suvanto <markus.suvanto@gmail.com>:
> Ok, the first problem is fixed but after bcp
> git status
> error: bad index file sha1 signature
> fatal: index file corrupt

Re: [PATCH]Re: The first character disappears

[Sage Weil]

On Mon, 24 Aug 2009, Markus Suvanto wrote:
> Now I notice that I can reproduce this index file corrupt only if
> compress is enabled.

The file clone ioctl doesn't fully support cloning compressed inline 
extents.  IIRC the problem is that the clone ioctl can operate on an 
arbitrary (block-aligned) file range, which may require splitting a 
compressed inline extent.  That means decompressing into a temporary 
buffer and then recompressing, and I didn't get around to doing that.

But as long as the full compressed inline extent is getting cloned it is 
supposed to get copied.  I think the problem is that the check for the 
uncompressed extent size is using ram_bytes, which is only an upper bound, 
so it's incorrectly deciding the whole extent isn't wanted and bailing 
out.

Can you tell me if the change below fixes the problem?

Thanks-
sage


diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index 7cdddd1..e8e4ebe 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -1100,7 +1100,8 @@ static long btrfs_ioctl_clone(struct file *file, unsigned long srcfd,
                                        new_key.offset += skip;
                                }
 
-                               if (key.offset + datal > off+len)
+                               if (key.offset + datal > off+len &&
+                                   key.offset + datal < src->i_size)
                                        trim = key.offset + datal - (off+len);
 
                                if (comp && (skip || trim)) {





> 
> lvm lvcreate -L 5G -n btrfs_test /dev/vg_md1
> mkbtrfs /dev/mapper/vg_md1-btrfs_test
> mount -t btrfs -o subvol=.,compress  /dev/mapper/vg_md1-btrfs_test /tmp/btrfs/
> btrfsctl -S test /tmp/btrfs/
> cd /tmp/btrfs/test/
> git clone /usr/src/linux-2.6/
> /tmp/btrfs-progs-unstable/bcp linux-2.6/ linux-2.6_copy
> cd linux-2.6_copy/
> git status
> 
> error: bad index file sha1 signature
> fatal: index file corrupt
> 
> 
> But without compress:
> 
> mkbtrfs /dev/mapper/vg_md1-btrfs_test
> mount -t btrfs -o subvol=.  /dev/mapper/vg_md1-btrfs_test /tmp/btrfs/
> btrfsctl -S test /tmp/btrfs/
> cd /tmp/btrfs/test/
> git clone /usr/src/linux-2.6/
> /tmp/btrfs-progs-unstable/bcp linux-2.6/ linux-2.6_copy
> cd linux-2.6_copy/
> git status
> 
> # On branch master
> nothing to commit (working directory clean)
> 
> 
> 2009/8/24 Markus Suvanto <markus.suvanto@gmail.com>:
> > Ok, the first problem is fixed but after bcp
> > git status
> > error: bad index file sha1 signature
> > fatal: index file corrupt
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: [PATCH]Re: The first character disappears

[Markus Suvanto]

2009/8/24 Sage Weil <sage@newdream.net>:
>
> Can you tell me if the change below fixes the problem?
>
No. Freshly created filesystem with compress:

htpc linux-2.6_copy # git status
error: bad index file sha1 signature
fatal: index file corrupt

[PATCH] Btrfs: fix arithmetic error in clone ioctl

[Sage Weil]

Markus,

This should fix your problem!

sage

---
Subject: [PATCH] Btrfs: fix arithmetic error in clone ioctl

Fix an arithmetic error that was breaking extents cloned via the clone
ioctl starting in the second half of a file.

Signed-off-by: Sage Weil <sage@newdream.net>
---
 fs/btrfs/ioctl.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index e8e4ebe..63ca12a 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -1072,8 +1072,7 @@ static long btrfs_ioctl_clone(struct file *file, unsigned long srcfd,
 					datao += off - key.offset;
 					datal -= off - key.offset;
 				}
-				if (key.offset + datao + datal + key.offset >
-				    off + len)
+				if (key.offset + datao + datal > off + len)
 					datal = off + len - key.offset - datao;
 				/* disko == 0 means it's a hole */
 				if (!disko)
-- 
1.5.6.5

Re: [PATCH] Btrfs: fix arithmetic error in clone ioctl

[Markus Suvanto]

2009/8/25 Sage Weil <sage@newdream.net>:
> Markus,
>
> This should fix your problem!
>

It works now , thanks!

-Markus


> ---
> Subject: [PATCH] Btrfs: fix arithmetic error in clone ioctl
>
> Fix an arithmetic error that was breaking extents cloned via the clon=
e
> ioctl starting in the second half of a file.
>
> Signed-off-by: Sage Weil <sage@newdream.net>
> ---
> =A0fs/btrfs/ioctl.c | =A0 =A03 +--
> =A01 files changed, 1 insertions(+), 2 deletions(-)
>
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index e8e4ebe..63ca12a 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -1072,8 +1072,7 @@ static long btrfs_ioctl_clone(struct file *file=
, unsigned long srcfd,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0datao +=3D off - key.offset;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0datal -=3D off - key.offset;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0}
> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (key=
=2Eoffset + datao + datal + key.offset >
> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 off + len)
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (key=
=2Eoffset + datao + datal > off + len)
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0datal =3D off + len - key.offset - datao;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* dis=
ko =3D=3D 0 means it's a hole */
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (!d=
isko)
> --
> 1.5.6.5
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH]Re: The first character disappears

[Atila]

Should be rstrip instead of strip.

Atila


Signed-off-by: Atila <atila.alr@dpf.gov.br>
---
  bcp |    6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/bcp b/bcp
index 5729e91..7c6b351 100755
--- a/bcp
+++ b/bcp
@@ -107,7 +107,7 @@ if src_args > 1:
          exit(1)

  for srci in xrange(0, src_args):
-    src = args[srci]
+    src = args[srci].rstrip('/')
      if os.path.isfile(src):
          statinfo = os.lstat(src)
          force_name = None
@@ -136,7 +136,7 @@ for srci in xrange(0, src_args):
              srcname = os.path.join(dirpath, x)
              statinfo = os.lstat(srcname)

-            if srcname.startswith(src):
+            if srcname.startswith(src + '/'):
                  part = srcname[len(src) + 1:]

              if stat.S_ISLNK(statinfo.st_mode):
@@ -152,7 +152,7 @@ for srci in xrange(0, src_args):

          for f in filenames:
              srcname = os.path.join(dirpath, f)
-            if srcname.startswith(src):
+            if srcname.startswith(src + '/'):
                  part = srcname[len(src) + 1:]

              statinfo = os.lstat(srcname)
-- 
1.6.4

*