[BUG] btrfs send/receive, page allocation failure

[BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

# btrfs send /mnt/320G/2012april_HDD_fullbackup.ro.0/ | btrfs receive /mnt/500G/
At subvol /mnt/320G/2012april_HDD_fullbackup.ro.0/
At subvol 2012april_HDD_fullbackup.ro.0
ERROR: send ioctl failed with -12: Cannot allocate memory
ERROR: unexpected EOF in stream.


kernel-3.16.0-1.fc21.x86_64
btrfs-progs-3.14.2-3.fc21.x86_64


Both the source and destination were recently (< 24 hours) created with the above versions.

Bug report here:
https://bugzilla.kernel.org/show_bug.cgi?id=81781

dmesg excerpt with debug kernel here:
https://bugzilla.kernel.org/attachment.cgi?id=145331


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

On Aug 7, 2014, at 9:31 AM, Chris Murphy <lists@colorremedies.com> wrote:

> # btrfs send /mnt/320G/2012april_HDD_fullbackup.ro.0/ | btrfs receive /mnt/500G/
> At subvol /mnt/320G/2012april_HDD_fullbackup.ro.0/
> At subvol 2012april_HDD_fullbackup.ro.0
> ERROR: send ioctl failed with -12: Cannot allocate memory
> ERROR: unexpected EOF in stream.
> 
> 
> kernel-3.16.0-1.fc21.x86_64
> btrfs-progs-3.14.2-3.fc21.x86_64
> 
> 
> Both the source and destination were recently (< 24 hours) created with the above versions.
> 
> Bug report here:
> https://bugzilla.kernel.org/show_bug.cgi?id=81781
> 
> dmesg excerpt with debug kernel here:
> https://bugzilla.kernel.org/attachment.cgi?id=145331

This isn't reproducible by only changing kernels to 3.14.15-100.fc19.x86_64, so it's a regression in 3.16. What's also interesting is the performance is much worse with 3.14 compared to 3.16. I get ~8.3MB/s with 3.14, and ~21.8MB/s with 3.16; all other factors are identical.


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Filipe David Manana]

On Sun, Aug 10, 2014 at 10:01 PM, Chris Murphy <lists@colorremedies.com> wrote:
>
> On Aug 7, 2014, at 9:31 AM, Chris Murphy <lists@colorremedies.com> wrote:
>
>> # btrfs send /mnt/320G/2012april_HDD_fullbackup.ro.0/ | btrfs receive /mnt/500G/
>> At subvol /mnt/320G/2012april_HDD_fullbackup.ro.0/
>> At subvol 2012april_HDD_fullbackup.ro.0
>> ERROR: send ioctl failed with -12: Cannot allocate memory
>> ERROR: unexpected EOF in stream.
>>
>>
>> kernel-3.16.0-1.fc21.x86_64
>> btrfs-progs-3.14.2-3.fc21.x86_64
>>
>>
>> Both the source and destination were recently (< 24 hours) created with the above versions.
>>
>> Bug report here:
>> https://bugzilla.kernel.org/show_bug.cgi?id=81781
>>
>> dmesg excerpt with debug kernel here:
>> https://bugzilla.kernel.org/attachment.cgi?id=145331
>
> This isn't reproducible by only changing kernels to 3.14.15-100.fc19.x86_64, so it's a regression in 3.16.

Hi Chris,

Can you try the following patch and confirm if it helps?
https://patchwork.kernel.org/patch/4705171/

> What's also interesting is the performance is much worse with 3.14 compared to 3.16. I get ~8.3MB/s with 3.14, and ~21.8MB/s with 3.16; all other factors are identical.

Yes, not surprising however because 3.15/16 brought in some
optimizations such as using readahead when reading file data for
example.

Thanks

>
>
> Chris Murphy--
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: [BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

> 
> 
> Can you try the following patch and confirm if it helps?
> https://patchwork.kernel.org/patch/4705171/

This one applies without problems, I didn't build it because I saw v4. The v4 patch I get:

+ patch -p1 -F1 -s
4 out of 4 hunks FAILED -- saving rejects to file fs/btrfs/send.c.rej

Should v4 alone be applied over 3.16.0? Or each version in succession?


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Filipe David Manana]

On Mon, Aug 11, 2014 at 4:07 AM, Chris Murphy <lists@colorremedies.com> wrote:
>>
>>
>> Can you try the following patch and confirm if it helps?
>> https://patchwork.kernel.org/patch/4705171/
>
> This one applies without problems, I didn't build it because I saw v4. The v4 patch I get:
>
> + patch -p1 -F1 -s
> 4 out of 4 hunks FAILED -- saving rejects to file fs/btrfs/send.c.rej
>
> Should v4 alone be applied over 3.16.0? Or each version in succession?

Alone. How did you try to apply it to 3.16? Try "cd source_dir && git
am patchfile" if you didn't (e.g. you used patch command directly).

thanks

>
>
> Chris Murphy



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: [BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

On Aug 11, 2014, at 2:46 AM, Filipe David Manana <fdmanana@gmail.com> wrote:

> On Mon, Aug 11, 2014 at 4:07 AM, Chris Murphy <lists@colorremedies.com> wrote:
>>> 
>>> 
>>> Can you try the following patch and confirm if it helps?
>>> https://patchwork.kernel.org/patch/4705171/
>> 
>> This one applies without problems, I didn't build it because I saw v4. The v4 patch I get:
>> 
>> + patch -p1 -F1 -s
>> 4 out of 4 hunks FAILED -- saving rejects to file fs/btrfs/send.c.rej
>> 
>> Should v4 alone be applied over 3.16.0? Or each version in succession?
> 
> Alone. How did you try to apply it to 3.16?

rpmbuild
https://fedoraproject.org/wiki/Building_a_custom_kernel

Gist is, save the patch, create patch filename entry into kernel.spec, then run rpmbuild. rpmbuild uses 'patch -p1 -F1 -s' to apply the patch. The v1 patch applies, as does Liu Bo's patch from July 29 "Btrfs: fix regression of btrfs device replace" which I was also going to test. But the v4 patch isn't applying.

> Try "cd source_dir && git
> am patchfile" if you didn't (e.g. you used patch command directly).

I'd kinda prefer to build an rpm since I need to test it on baremetal for this bug, and a VM for the device replace bug.


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Filipe David Manana]

On Mon, Aug 11, 2014 at 4:55 PM, Chris Murphy <lists@colorremedies.com> wrote:
>
> On Aug 11, 2014, at 2:46 AM, Filipe David Manana <fdmanana@gmail.com> wrote:
>
>> On Mon, Aug 11, 2014 at 4:07 AM, Chris Murphy <lists@colorremedies.com> wrote:
>>>>
>>>>
>>>> Can you try the following patch and confirm if it helps?
>>>> https://patchwork.kernel.org/patch/4705171/
>>>
>>> This one applies without problems, I didn't build it because I saw v4. The v4 patch I get:
>>>
>>> + patch -p1 -F1 -s
>>> 4 out of 4 hunks FAILED -- saving rejects to file fs/btrfs/send.c.rej
>>>
>>> Should v4 alone be applied over 3.16.0? Or each version in succession?
>>
>> Alone. How did you try to apply it to 3.16?
>
> rpmbuild
> https://fedoraproject.org/wiki/Building_a_custom_kernel
>
> Gist is, save the patch, create patch filename entry into kernel.spec, then run rpmbuild. rpmbuild uses 'patch -p1 -F1 -s' to apply the patch. The v1 patch applies, as does Liu Bo's patch from July 29 "Btrfs: fix regression of btrfs device replace" which I was also going to test. But the v4 patch isn't applying.
>
>> Try "cd source_dir && git
>> am patchfile" if you didn't (e.g. you used patch command directly).
>
> I'd kinda prefer to build an rpm since I need to test it on baremetal for this bug, and a VM for the device replace bug.

Sorry, I don't know anything about fedora's way of kernel patching.
Either way, it seems the problem is simple to solve:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
git checkout v3.16
git am /path/to/my_patch_file
git diff HEAD^..HEAD > /tmp/diff

The resulting patch file [1] /tmp/diff then applies cleanly with
"patch -p1 -F1 -s"

https://friendpaste.com/Bgwdjk31P3pZHtArr341G

>
>
> Chris Murphy



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: [BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

On Aug 11, 2014, at 10:04 AM, Filipe David Manana <fdmanana@gmail.com> wrote:

> On Mon, Aug 11, 2014 at 4:55 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> 
>> On Aug 11, 2014, at 2:46 AM, Filipe David Manana <fdmanana@gmail.com> wrote:
>> 
>>> On Mon, Aug 11, 2014 at 4:07 AM, Chris Murphy <lists@colorremedies.com> wrote:
>>>>> 
>>>>> 
>>>>> Can you try the following patch and confirm if it helps?
>>>>> https://patchwork.kernel.org/patch/4705171/
>>>> 
>>>> This one applies without problems, I didn't build it because I saw v4. The v4 patch I get:
>>>> 
>>>> + patch -p1 -F1 -s
>>>> 4 out of 4 hunks FAILED -- saving rejects to file fs/btrfs/send.c.rej
>>>> 
>>>> Should v4 alone be applied over 3.16.0? Or each version in succession?
>>> 
>>> Alone. How did you try to apply it to 3.16?
>> 
>> rpmbuild
>> https://fedoraproject.org/wiki/Building_a_custom_kernel
>> 
>> Gist is, save the patch, create patch filename entry into kernel.spec, then run rpmbuild. rpmbuild uses 'patch -p1 -F1 -s' to apply the patch. The v1 patch applies, as does Liu Bo's patch from July 29 "Btrfs: fix regression of btrfs device replace" which I was also going to test. But the v4 patch isn't applying.
>> 
>>> Try "cd source_dir && git
>>> am patchfile" if you didn't (e.g. you used patch command directly).
>> 
>> I'd kinda prefer to build an rpm since I need to test it on baremetal for this bug, and a VM for the device replace bug.
> 
> Sorry, I don't know anything about fedora's way of kernel patching.
> Either way, it seems the problem is simple to solve:
> 
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> git checkout v3.16
> git am /path/to/my_patch_file
> git diff HEAD^..HEAD > /tmp/diff
> 
> The resulting patch file [1] /tmp/diff then applies cleanly with
> "patch -p1 -F1 -s"
> 
> https://friendpaste.com/Bgwdjk31P3pZHtArr341G

OK that friendpaste is completely different than the [PATCH v4] email.

# from above URL
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 6528aa6..95891c0 100644

# from [PATCH v4] email
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 3c63b29..b29fc5c 100644

The lines numbers are all completely different also. I'll try the patch from the above URL.


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[lists]

On 2014-08-11 10:25, Chris Murphy wrote:
> On Aug 11, 2014, at 10:04 AM, Filipe David Manana <fdmanana@gmail.com> 
> wrote:
> 
>> On Mon, Aug 11, 2014 at 4:55 PM, Chris Murphy 
>> <lists@colorremedies.com> wrote:
>>> 
>>> On Aug 11, 2014, at 2:46 AM, Filipe David Manana <fdmanana@gmail.com> 
>>> wrote:
>>> 
>>>> On Mon, Aug 11, 2014 at 4:07 AM, Chris Murphy 
>>>> <lists@colorremedies.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> Can you try the following patch and confirm if it helps?
>>>>>> https://patchwork.kernel.org/patch/4705171/
>>>>> 
>>>>> This one applies without problems, I didn't build it because I saw 
>>>>> v4. The v4 patch I get:
>>>>> 
>>>>> + patch -p1 -F1 -s
>>>>> 4 out of 4 hunks FAILED -- saving rejects to file 
>>>>> fs/btrfs/send.c.rej
>>>>> 
>>>>> Should v4 alone be applied over 3.16.0? Or each version in 
>>>>> succession?
>>>> 
>>>> Alone. How did you try to apply it to 3.16?
>>> 
>>> rpmbuild
>>> https://fedoraproject.org/wiki/Building_a_custom_kernel
>>> 
>>> Gist is, save the patch, create patch filename entry into 
>>> kernel.spec, then run rpmbuild. rpmbuild uses 'patch -p1 -F1 -s' to 
>>> apply the patch. The v1 patch applies, as does Liu Bo's patch from 
>>> July 29 "Btrfs: fix regression of btrfs device replace" which I was 
>>> also going to test. But the v4 patch isn't applying.
>>> 
>>>> Try "cd source_dir && git
>>>> am patchfile" if you didn't (e.g. you used patch command directly).
>>> 
>>> I'd kinda prefer to build an rpm since I need to test it on baremetal 
>>> for this bug, and a VM for the device replace bug.
>> 
>> Sorry, I don't know anything about fedora's way of kernel patching.
>> Either way, it seems the problem is simple to solve:
>> 
>> git clone 
>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>> git checkout v3.16
>> git am /path/to/my_patch_file
>> git diff HEAD^..HEAD > /tmp/diff
>> 
>> The resulting patch file [1] /tmp/diff then applies cleanly with
>> "patch -p1 -F1 -s"
>> 
>> https://friendpaste.com/Bgwdjk31P3pZHtArr341G
> 
> OK that friendpaste is completely different than the [PATCH v4] email.
> 
> # from above URL
> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
> index 6528aa6..95891c0 100644
> 
> # from [PATCH v4] email
> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
> index 3c63b29..b29fc5c 100644
> 
> The lines numbers are all completely different also. I'll try the
> patch from the above URL.

The above friendpaste URL patch has applied, and I'm now building.


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Chris Murphy]

On Aug 11, 2014, at 10:41 AM, lists@colorremedies.com wrote:
>>> 
>>> https://friendpaste.com/Bgwdjk31P3pZHtArr341G
>> OK that friendpaste is completely different than the [PATCH v4] email.
>> # from above URL
>> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
>> index 6528aa6..95891c0 100644


I can confirm that the above patch fixes the reported bug. 

I couldn't get this one to apply so it's not tested:
http://www.spinics.net/lists/linux-btrfs/msg36556.html


Chris Murphy

Re: [BUG] btrfs send/receive, page allocation failure

[Filipe David Manana]

On Mon, Aug 11, 2014 at 9:30 PM, Chris Murphy <lists@colorremedies.com> wrote:
>
> On Aug 11, 2014, at 10:41 AM, lists@colorremedies.com wrote:
>>>>
>>>> https://friendpaste.com/Bgwdjk31P3pZHtArr341G
>>> OK that friendpaste is completely different than the [PATCH v4] email.
>>> # from above URL
>>> diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
>>> index 6528aa6..95891c0 100644
>
>
> I can confirm that the above patch fixes the reported bug.
>
> I couldn't get this one to apply so it's not tested:
> http://www.spinics.net/lists/linux-btrfs/msg36556.html

It's exactly the same as the diff you tried. The "git am" command is
able to deal with any fuzz while the patch command can't (or not
always at least). That patch is based on the integration branch, while
the diff I pasted (and showed you to generate it) is for the v3.16 tag
from linus' repository.

Thanks for testing and reporting back.

>
>
> Chris Murphy



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."