Btrfs stable updates for v3.16

Btrfs stable updates for v3.16

[Chris Mason]

Hi everyone,

For 3.16, please pull these into stable, I've cherry picked and tested
them here.  For 3.15 and earlier there are a few conflicts, so I'll make
a git tree with things to pull.

8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+
38c1c2e44bacb37efd68b90b3f70386a8ee370ee v3.11+
f6dc45c7a93a011dff6eb9b2ffda59c390c7705a v3.15+
9e0af23764344f7f1b68e4eefbe7dc865018b63d v3.15+

-chris

Re: Btrfs stable updates for v3.16

[Greg KH]

On Wed, Sep 03, 2014 at 04:50:47PM -0400, Chris Mason wrote:
> Hi everyone,
> 
> For 3.16, please pull these into stable, I've cherry picked and tested
> them here.  For 3.15 and earlier there are a few conflicts, so I'll make
> a git tree with things to pull.
> 
> 8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+
> 38c1c2e44bacb37efd68b90b3f70386a8ee370ee v3.11+
> f6dc45c7a93a011dff6eb9b2ffda59c390c7705a v3.15+
> 9e0af23764344f7f1b68e4eefbe7dc865018b63d v3.15+

Now applied to the trees I manage.

thanks,

greg k-h

Re: Btrfs stable updates for v3.16

[Holger Hoffstätte]

On Wed, 03 Sep 2014 16:50:47 -0400, Chris Mason wrote:

> Hi everyone,
> 
> For 3.16, please pull these into stable, I've cherry picked and tested
> them here.  For 3.15 and earlier there are a few conflicts, so I'll make
> a git tree with things to pull.
> 
> 8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+

This ("fix filemap_flush call in btrfs_file_release") is the only one 
that requires some work for 3.14.

There is one conflict in ordered.data.c - just a sligh work queue 
submission change - and the second in transaction.c where the patch does 
not delete enough from btrfs_flush_all_pending_stuffs(), since 3.14 still 
has the old qgroup calls in place. I removed it wholesale and that makes 
everything fit.

The followup ("fix filemap_flush call in btrfs_file_release") then also 
applies.

Should they also go into the next 3.14.x stable cycle? This rename 
deadlock sounds like a possible problem with rsync, which seems like a 
popular use case, and I guess nobody will complain about slightly better 
performance either.

Holger

Re: Btrfs stable updates for v3.16

[Chris Mason]

On 09/03/2014 07:36 PM, Holger Hoffstätte wrote:
> On Wed, 03 Sep 2014 16:50:47 -0400, Chris Mason wrote:
> 
>> Hi everyone,
>>
>> For 3.16, please pull these into stable, I've cherry picked and tested
>> them here.  For 3.15 and earlier there are a few conflicts, so I'll make
>> a git tree with things to pull.
>>
>> 8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+
> 
> This ("fix filemap_flush call in btrfs_file_release") is the only one 
> that requires some work for 3.14.
> 
> There is one conflict in ordered.data.c - just a sligh work queue 
> submission change - and the second in transaction.c where the patch does 
> not delete enough from btrfs_flush_all_pending_stuffs(), since 3.14 still 
> has the old qgroup calls in place. I removed it wholesale and that makes 
> everything fit.
> 
> The followup ("fix filemap_flush call in btrfs_file_release") then also 
> applies.
> 
> Should they also go into the next 3.14.x stable cycle? This rename 
> deadlock sounds like a possible problem with rsync, which seems like a 
> popular use case, and I guess nobody will complain about slightly better 
> performance either.

Right, the btrfs_flush_all_pending_stuffs function can just be deleted.
 But, Liu Bo's patch isn't required on 3.14 (since the regression he
fixed came with 3.15).

And these changes are big enough that I like to test them a little here
before sending out.  I did mark that patch as 3.15+, but really that
deadlock has been there forever.  We only started seeing it with 3.15+
because other waitqueue problems made it stand out.

-chris

Re: Btrfs stable updates for v3.16

[Holger Hoffstätte]

(removed stable@ for now)

Just to follow up on this -

On Wed, 03 Sep 2014 23:36:00 +0000, Holger Hoffstätte wrote:

> [on backporting both:
>  8d875f95da43c6a8f18f77869f2ef26e9594fecc and
>  f6dc45c7a93a011dff6eb9b2ffda59c390c7705a to 3.14]
>
> ...and I guess nobody will complain about slightly better performance
> either.

So "slightly" is an understatement: rename/replace/truncate without full
immediate flushing - as done by rsync's copy+rename - is now indeed
*much* faster, to the point that backups with many small files finish
in a fraction of the time. Wonderful!

Please do get this into the next 3.14 cycle, I'll gladly help and test.

Thanks!

Holger

Re: Btrfs stable updates for v3.16

[Marc MERLIN]

On Wed, Sep 03, 2014 at 04:50:47PM -0400, Chris Mason wrote:
> Hi everyone,
> 
> For 3.16, please pull these into stable, I've cherry picked and tested
> them here.  For 3.15 and earlier there are a few conflicts, so I'll make
> a git tree with things to pull.
> 
> 8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+

Hi Chris,

Would this patch you submitted be likely to fix the chrome profile
corruption needing restore from earlier snapshot problem we discussed a
few weeks back?

If so, yeah and thanks! :)
(compiling 3.16.2 now)


    btrfs: disable strict file flushes for renames and truncates
    
    commit 8d875f95da43c6a8f18f77869f2ef26e9594fecc upstream.
    
    Truncates and renames are often used to replace old versions of a file
    with new versions.  Applications often expect this to be an atomic
    replacement, even if they haven't done anything to make sure the new
    version is fully on disk.
    
    Btrfs has strict flushing in place to make sure that renaming over an
    old file with a new file will fully flush out the new file before
    allowing the transaction commit with the rename to complete.
    
    This ordering means the commit code needs to be able to lock file pages,
    and there are a few paths in the filesystem where we will try to end a
    transaction with the page lock held.  It's rare, but these things can
    deadlock.
    
    This patch removes the ordered flushes and switches to a best effort
    filemap_flush like ext4 uses. It's not perfect, but it should fix the
    deadlocks.
    
    Signed-off-by: Chris Mason <clm@fb.com>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>


-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/                         | PGP 1024R/763BE901