[U-Boot] [PATCH v4 0/3] mtd, ubi, ubifs: resync with Linux-3.14

[Heiko Schocher <hs@denx.de>]

Hello Tom, Scott,

Am 18.06.2014 23:42, schrieb Tom Rini:
> On Wed, Jun 18, 2014 at 04:24:39PM -0500, Scott Wood wrote:
>> On Wed, 2014-06-18 at 09:55 +0200, Heiko Schocher wrote:
>>> Hello Scott,
> [snip]
>>>> history to see what Linux version corresponded to the last mtd sync, and
>>>> generate a diff relative to that.
>>>
>>> I think, we should have the linux code as the reference, and on top
>>> of this, we should add our U-Boot changes ... Thats the reason
>>> for the git tree [1].
>>
>> One of the strengths of version control is the ability to do three-way
>> merges, rather than throw away the downstream code and reassemble it
>> from scratch.
>
> Indeed.
>
> [snip]
>>
>> What does #ifndef __UBOOT__ tell you that a diff between the U-Boot and
>> Linux files, as of the SHA1 that was last merged into U-Boot, doesn't
>> tell you?
>>
>>> There we have the "simple copy from linux patch" [2]. It overwrites
>>> the U-Boot specific changes, yes, but only in an intermediate step
>>> in this git tree ... (Maybe I should add in the commit message, that
>>> this is a simple copy from linux sources)
>>>
>>> Then a U-Boot specifc patch [3], which introduces again hopefully
>>> all U-Boot specific changes... so *all* U-Boot specific changs are
>>> documented. At last a seperate licencse file patch [4] added which
>>> changes the license text.
>>>
>>> My hope was, that future linux mtd syncs could look like:
>>>
>>> a) copy new linux source files to u-boot tree [1] (make a new branch
>>>      go back to commit [2] ... and copy new files from linux)
>>>      (commit this, to have all linux changes since last sync
>>>       with linux documented)
>>> b) apply [3] and fix problems
>>>      All U-Boot changes documented
>>> c) apply [4] and fix problems
>>> d) add newly added U-Boot specific patches
>>>      (mark them with _UBOOT_ define if they are not yet marked with it)
>>>      (maybe squash them to step b ?)
>>>
>>>      ->  new git tree ready
>>>
>>> e) squash all patches into one patch and send it to mainline ...
>>>      so the mainline patch has only new changes with all U-Boot
>>>      specific changes.
>>
>> If you take over as NAND custodian, you can of course do the updates as
>> you please (unless Tom or Wolfgang disagree), but until then I insist on
>> doing a proper merge, and not littering the code with #ifndef __UBOOT__.
>>
>>> Thats sounds easy ... and you have all linux and all U-Boot
>>> changes visible and documented ... I must admit, I did not
>>> tried your proposal to look into the linux changes since
>>> the last mtd sync ... but that sounds the more tricky way
>>> to get this sync, as the history can be long ... and shouldn;t
>>> be the result the same?
>>
>> Why does the length of the history matter?  I'm not asking you to go
>> patch by patch.  Just take a diff between the SHA1 of last merge and the
>> SHA1 you want to merge now, and apply that patch.
>
> But it's not even that hard to do so!  git format-patch -o mtd-resync
> v3.7..v3.14 drivers/mtd/nand will give you only the relevant patches,

"git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand"
results in 440 patches ... :-(

"git am" first patch immediately fails ...

Ok, one step back ... I try only "drivers/mtd/mtdcore.c":

$ git format-patch -o mtdcore v3.7..v3.14 drivers/mtd/mtdcore.c
mtdcore/0001-mtd-convert-to-idr_alloc.patch
mtdcore/0002-mtd-add-const-qualifier-to-a-couple-of-register-func.patch
mtdcore/0003-mtd-mtdcore-remove-few-useless-ifdef-s.patch
mtdcore/0004-mtd-merge-mtdchar-module-with-mtdcore.patch
mtdcore/0005-Include-missing-linux-slab.h-inclusions.patch
mtdcore/0006-drivers-avoid-format-string-in-dev_set_name.patch
mtdcore/0007-mtd-add-a-new-sys-node-to-show-the-ecc-step-size.patch
mtdcore/0008-mtd-add-MTD_MLCNANDFLASH-case-for-mtd_type_show.patch
mtdcore/0009-mtd-Move-major-number-definitions-to-major.h.patch
mtdcore/0010-mtd-remove-duplicated-include-from-mtdcore.c.patch
mtdcore/0011-mtd-convert-to-use-ATTRIBUTE_GROUPS.patch
$

Only 11 patches, great ... but:

$ git am -3 mtdcore/0001-mtd-convert-to-idr_alloc.patch
Wende an: mtd: convert to idr_alloc()
fatal: sha1 information is lacking or useless (drivers/mtd/mtdcore.c).
Dem Projektarchiv fehlen notwendige Blobs um auf eine 3-Wege-Zusammenf?hrung zur?ckzufallen.
Kann nicht zu 3-Wege-Zusammenf?hrung zur?ckfallen.
Anwendung des Patches fehlgeschlagen bei 0001 mtd: convert to idr_alloc()
Die Kopie des fehlgeschlagenen Patches befindet sich in:
    /home/hs/i2c/u-boot/.git/rebase-apply/patch
Wenn Sie das Problem gel?st haben, f?hren Sie "git am --resolved" aus.
Falls Sie diesen Patch auslassen m?chten, f?hren Sie stattdessen
"git am --skip" aus.
Um den urspr?nglichen Zweig wiederherzustellen und die Anwendung der
Patches abzubrechen, f?hren Sie "git am --abort" aus.
pollux:u-boot hs [mtdubiubifs_try] $

$ patch -p1 < mtdcore/0001-mtd-convert-to-idr_alloc.patch
patching file drivers/mtd/mtdcore.c
Hunk #1 FAILED@349.
1 out of 1 hunk FAILED -- saving rejects to file drivers/mtd/mtdcore.c.rej
$

:-(

> and they should mostly apply, and then you can squash them after the
> fact.

I try to try this ASAP and report (first steps see above ...). But as a
lot of linux code is missing in U-Boot code, I feel, that a lot of chunks
will not apply ... but maybe (hopefully) I am wrong ...

So therefore I copied and introduced the complete linux code, because
it is the easier way for generating this patch...

What I new introduced is:
mark in U-Boot unused/different Code with __UBOOT__ ... to get faster,
U-Boot relevant changes, by simple looking into the diff ...

Thats the question we must solve, do we want the complete Linux MTD code
in U-Boot and mark not used code?

If no, I delete this pieces of code...

Hmm... looking in the history of drivers/mtd/ubi it seems the code
is basically from 2008!
http://git.denx.de/?p=u-boot.git;a=history;f=drivers/mtd/ubi;hb=7d2357c1999ff1f93f795282526230a8bd176106

Hmm.. what was the Linux base of this ?

and ubifs from 2009?
http://git.denx.de/?p=u-boot.git;a=history;f=fs/ubifs;hb=7d2357c1999ff1f93f795282526230a8bd176106
Base: "2.6.29-rc6" ... uh, very, very old!

I see there no real syncs with newer linux code ... is this true?
Just some fixups ...

Hmm reading the commit mesage from 9eefe2a2b37a838558e3d213a9f5519503d0c180:
"UBIFS: Implement read-only UBIFS support in U-Boot"

"The U-Boot UBIFS implementation is largely a direct copy from the current
Linux version (2.6.29-rc6). As already done in the UBI version we have an
"abstraction layer" to redefine or remove some OS calls (e.g. mutex_lock()
...). This makes it possible to use the original Linux code with very
little changes. And by this we can better update to later Linux versions.

I removed some of the Linux features that are not used in the U-Boot
version (e.g. garbage-collection, write support)."

Sounds like my current approach ... Ok, again, not used code is deleted.

I really fear a patch "git format-patch -o ubifs-resync 2.6.29-rc6..v3.14 fs/ubifs"
(The result from this command are 355 patches ...)

So, in sum, this are 795 patches + missing ubi patches which must be
applied/reworked/reviewed... Sounds painful...

So another question is:
How to generate an update-sync patch:

version a):
- copy the new linux files from Linux v3.14 to U-Boot
- work in the U-Boot specific changes
- make a patch from this

Is this really different to:

version b):
- git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand
- apply them
- make a patch from this

Currently it is differrent, as I introduced the complete linux code
and marked unused/different code in U-Boot with __UBOOT__. If I delete
this pieces ... version a) and b) should be the same.

I try it ... if I not give up due to count of patches ...

Maybe it was not a so good idea to remove the "RFC" state from this
patches ;-)

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany