linux-next: manual merge of the mm-stable tree with the ext4 tree

linux-next: manual merge of the mm-stable tree with the ext4 tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

Hi all,

Today's linux-next merge of the mm-stable tree got a conflict in:

  fs/ext4/inode.c

between commits:

  726432969963 ("ext4: use nr_to_write directly in mpage_prepare_extent_to_map()")
  9ff6a9153c8f ("ext4: move page unlocking out of mpage_submit_page()")
  b4d26e70a755 ("ext4: move mpage_page_done() calls after error handling")
  9b18c23c131a ("ext4: convert data=journal writeback to use ext4_writepages()")

from the ext4 tree and commits:

  50ead2537441 ("ext4: convert mpage_prepare_extent_to_map() to use filemap_get_folios_tag()")
  d585bdbeb79a ("fs: convert writepage_t callback to pass a folio")

from the mm-stable tree.

I have no idea how to fix this up, so I have used the ext4 tree from
next-20230217 for today.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Matthew Wilcox]

On Mon, Feb 20, 2023 at 03:29:38PM +1100, Stephen Rothwell wrote:
> Hi all,
> 
> Today's linux-next merge of the mm-stable tree got a conflict in:
> 
>   fs/ext4/inode.c
> 
> between commits:
> 
>   726432969963 ("ext4: use nr_to_write directly in mpage_prepare_extent_to_map()")
>   9ff6a9153c8f ("ext4: move page unlocking out of mpage_submit_page()")
>   b4d26e70a755 ("ext4: move mpage_page_done() calls after error handling")
>   9b18c23c131a ("ext4: convert data=journal writeback to use ext4_writepages()")
> 
> from the ext4 tree and commits:
> 
>   50ead2537441 ("ext4: convert mpage_prepare_extent_to_map() to use filemap_get_folios_tag()")
>   d585bdbeb79a ("fs: convert writepage_t callback to pass a folio")
> 
> from the mm-stable tree.
> 
> I have no idea how to fix this up, so I have used the ext4 tree from
> next-20230217 for today.

Yeah, this one's a mess.  Probably the easiest solution is for Ted to
pull in 247f9e1feef4 and 50ead2537441 and then redo Jan's patches on
top of them.

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

Hi all,

On Mon, 20 Feb 2023 14:27:00 +0000 Matthew Wilcox <willy@infradead.org> wrote:
>
> On Mon, Feb 20, 2023 at 03:29:38PM +1100, Stephen Rothwell wrote:
> > Hi all,
> > 
> > Today's linux-next merge of the mm-stable tree got a conflict in:
> > 
> >   fs/ext4/inode.c
> > 
> > between commits:
> > 
> >   726432969963 ("ext4: use nr_to_write directly in mpage_prepare_extent_to_map()")
> >   9ff6a9153c8f ("ext4: move page unlocking out of mpage_submit_page()")
> >   b4d26e70a755 ("ext4: move mpage_page_done() calls after error handling")
> >   9b18c23c131a ("ext4: convert data=journal writeback to use ext4_writepages()")
> > 
> > from the ext4 tree and commits:
> > 
> >   50ead2537441 ("ext4: convert mpage_prepare_extent_to_map() to use filemap_get_folios_tag()")
> >   d585bdbeb79a ("fs: convert writepage_t callback to pass a folio")
> > 
> > from the mm-stable tree.
> > 
> > I have no idea how to fix this up, so I have used the ext4 tree from
> > next-20230217 for today.  
> 
> Yeah, this one's a mess.  Probably the easiest solution is for Ted to
> pull in 247f9e1feef4 and 50ead2537441 and then redo Jan's patches on
> top of them.

I am still using the ext4 tree from next-20230217 (which was empty
relative to Linus' tree).

So, I have to ask why all these commits were only published to
linux-next over the weekend?

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 1761 bytes --]

Hi all,

On Tue, 21 Feb 2023 17:54:36 +1100 Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> On Mon, 20 Feb 2023 14:27:00 +0000 Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Mon, Feb 20, 2023 at 03:29:38PM +1100, Stephen Rothwell wrote:  
> > > 
> > > Today's linux-next merge of the mm-stable tree got a conflict in:
> > > 
> > >   fs/ext4/inode.c
> > > 
> > > between commits:
> > > 
> > >   726432969963 ("ext4: use nr_to_write directly in mpage_prepare_extent_to_map()")
> > >   9ff6a9153c8f ("ext4: move page unlocking out of mpage_submit_page()")
> > >   b4d26e70a755 ("ext4: move mpage_page_done() calls after error handling")
> > >   9b18c23c131a ("ext4: convert data=journal writeback to use ext4_writepages()")
> > > 
> > > from the ext4 tree and commits:
> > > 
> > >   50ead2537441 ("ext4: convert mpage_prepare_extent_to_map() to use filemap_get_folios_tag()")
> > >   d585bdbeb79a ("fs: convert writepage_t callback to pass a folio")
> > > 
> > > from the mm-stable tree.
> > > 
> > > I have no idea how to fix this up, so I have used the ext4 tree from
> > > next-20230217 for today.    
> > 
> > Yeah, this one's a mess.  Probably the easiest solution is for Ted to
> > pull in 247f9e1feef4 and 50ead2537441 and then redo Jan's patches on
> > top of them.  
> 
> I am still using the ext4 tree from next-20230217 (which was empty
> relative to Linus' tree).
> 
> So, I have to ask why all these commits were only published to
> linux-next over the weekend?

I still have not been able to figure out a merge resolution for this,
so the ext4 tree still has had no testing in linux-next and the merge
window is progressing.  Does anyone want to help with this?

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Bagas Sanjaya]

On 2/23/23 10:47, Stephen Rothwell wrote:
> I still have not been able to figure out a merge resolution for this,
> so the ext4 tree still has had no testing in linux-next and the merge
> window is progressing.  Does anyone want to help with this?
> 

I think at worst, Linus will have to resolve this mess himself when PR
for ext4 is submitted and merged in this merge window.

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 724 bytes --]

Hi Bagas,

On Fri, 24 Feb 2023 09:42:49 +0700 Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>
> On 2/23/23 10:47, Stephen Rothwell wrote:
> > I still have not been able to figure out a merge resolution for this,
> > so the ext4 tree still has had no testing in linux-next and the merge
> > window is progressing.  Does anyone want to help with this?
> 
> I think at worst, Linus will have to resolve this mess himself when PR
> for ext4 is submitted and merged in this merge window.

In which case we will have an ext4 filesystem in Linus' kernel that has
no testing ... and if someone has done the testing, then they can at
least supply the merge resolution that they tested.

-- 
Cheers,
Stephen Rothwell

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Linus Torvalds]

On Thu, Feb 23, 2023 at 9:11 PM Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> In which case we will have an ext4 filesystem in Linus' kernel that has
> no testing ... and if someone has done the testing, then they can at
> least supply the merge resolution that they tested.

Well, the good news is that it will have more testing than the cifs
resolution I did - if only because I actually have ext4 filesystems
that I'd test with.

I'm not claiming that's great either, of course. I think this whole
"convert from find_get_pages_range_tag() to filemap_get_folios_tag()
with very different semantics at the same time as all the writepages
work is done" was an absolutely *horrible* idea.

I actually considered doing the cifs merge by just keeping the old
format, and making a wrapper around "filemap_get_folios_tag()" that
would work one page at a time, just to avoid the whole "do two changes
at once" issue.

But then I - probably from some excessive hubris - decided I could
just rewrite the whole thing without testing, and just get it right on
the first try.

I did have DavidH's suggestion to look at, but decided that was buggy,
so mine is actually fairly different.

Yes. I do think "excessive hubris" is probably the right word choice.
But "insanity" probably also comes close ;)

           Linus

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Theodore Ts'o]

Sorry for the delay in taking a look at things; this week has been
horribly crazy busy for me.

The conflict is with the folio changes in mm-stable and with Jan's
"ext4: Cleanup data=journal writeback path" patch series.  Both of
these touch mpage_prepare_extent_to_map() in fs/ext4/inode.c quite
extensively.

[1] https://lore.kernel.org/r/20230111152736.9608-1-jack@suse.cz

I've just tried taking Jan's patch series and applying it linux-next
(which contains the mm-stable branch).  Dealing with each chage one by
one, there were conflits with patches #1, #4, #5, and #7.  One at the
time, the conflict resolutions weren't _that_ bad, but looking at the
combined conflict after doing a "git merge" it was quite scary indeed.

Maybe Linus's git merging-fu is much stronger than mine, but I
certainly wouldn't wnat to try to resolve it just using "git merge"!

So here's my proposed path forawrd.

1)  So far the only testing Jan's patch series (modified so it sits on
top of linux-next) is "it builds, ship it".  So I'll kick off a full
xfstests test on that series, and make sure I don't see any
regressions.  After that, I'll post it on linux-ext4 for Jan to
examine.

(Since it's well after work hours in Europe on a Friday, Jan probably
won't get to it until Monday, which is fine.)

2) I'll drop Jan's patch set from the ext4 dev branch, and run the
following full xfstests runs on this dev "lite" branch.  (a) the dev
"lite" branch by itself.  (b) the dev "lite" branch merged with tip of
Linux's tree.  (c) the dev "lite" branch merged with linux-next.

99.9% of the time, when there are problems, they are detected by my
full set of xfstests regression testing, since random users running
linux-next tend not stress file system all that much, and the 0-day
bot doesn't do nearly as much testing as I do.  (I test a dozen
different ext4 fs configs[2], while the 0-day tests only a single
one.)

[2] https://github.com/tytso/xfstests-bld/blob/master/test-appliance/files/root/fs/ext4/cfg/all.list

3)  I'll then send the ext4 dev branch (minus the data=writepage
cleanups) as a pull request to Linus.  Next week, after Jan has a
chance to review my patch conflict resolutions, I'll send a second
pull request with the data=writepage cleanups.  (As usual, I'll do my
full set of test runs before sending the pull request.)

Linus, are you OK with this plan?

						- Ted

Re: linux-next: manual merge of the mm-stable tree with the ext4 tree

[Linus Torvalds]

On Fri, Feb 24, 2023 at 2:20 PM Theodore Ts'o <tytso@mit.edu> wrote:
>
> 3)  I'll then send the ext4 dev branch (minus the data=writepage
> cleanups) as a pull request to Linus.  Next week, after Jan has a
> chance to review my patch conflict resolutions, I'll send a second
> pull request with the data=writepage cleanups.  (As usual, I'll do my
> full set of test runs before sending the pull request.)
>
> Linus, are you OK with this plan?

Yeah. I think the whole "do two completely different things to the
same area in parallel" was horrible, and we should strive to avoid
that kind of thing in filesystems in general.

So let's just serialize them instead of doing them at the same time
and try to merge them together.

Even if the merge is then successful, it just seems unnecessarily
risky from a "what if somebody reports a problem, and it bisects to
the merge, and then we have something subtle going on".

              Linus