[U-Boot] [PATCH v4 0/3] mtd, ubi, ubifs: resync with Linux-3.14

[Heiko Schocher <hs@denx.de>]

Hello Scott,

Am 20.06.2014 01:45, schrieb Scott Wood:
> On Thu, 2014-06-19 at 12:34 +0200, Heiko Schocher wrote:
>> Hello Tom, Scott,
>>
>> Am 18.06.2014 23:42, schrieb Tom Rini:
>>> On Wed, Jun 18, 2014 at 04:24:39PM -0500, Scott Wood wrote:
>>>> On Wed, 2014-06-18 at 09:55 +0200, Heiko Schocher wrote:
>>>>> Hello Scott,
>>> [snip]
>>>>>> history to see what Linux version corresponded to the last mtd sync, and
>>>>>> generate a diff relative to that.
>>>>>
>>>>> I think, we should have the linux code as the reference, and on top
>>>>> of this, we should add our U-Boot changes ... Thats the reason
>>>>> for the git tree [1].
>>>>
>>>> One of the strengths of version control is the ability to do three-way
>>>> merges, rather than throw away the downstream code and reassemble it
>>>> from scratch.
>>>
>>> Indeed.
>>>
>>> [snip]
>>>>
>>>> What does #ifndef __UBOOT__ tell you that a diff between the U-Boot and
>>>> Linux files, as of the SHA1 that was last merged into U-Boot, doesn't
>>>> tell you?
>>>>
>>>>> There we have the "simple copy from linux patch" [2]. It overwrites
>>>>> the U-Boot specific changes, yes, but only in an intermediate step
>>>>> in this git tree ... (Maybe I should add in the commit message, that
>>>>> this is a simple copy from linux sources)
>>>>>
>>>>> Then a U-Boot specifc patch [3], which introduces again hopefully
>>>>> all U-Boot specific changes... so *all* U-Boot specific changs are
>>>>> documented. At last a seperate licencse file patch [4] added which
>>>>> changes the license text.
>>>>>
>>>>> My hope was, that future linux mtd syncs could look like:
>>>>>
>>>>> a) copy new linux source files to u-boot tree [1] (make a new branch
>>>>>       go back to commit [2] ... and copy new files from linux)
>>>>>       (commit this, to have all linux changes since last sync
>>>>>        with linux documented)
>>>>> b) apply [3] and fix problems
>>>>>       All U-Boot changes documented
>>>>> c) apply [4] and fix problems
>>>>> d) add newly added U-Boot specific patches
>>>>>       (mark them with _UBOOT_ define if they are not yet marked with it)
>>>>>       (maybe squash them to step b ?)
>>>>>
>>>>>       ->   new git tree ready
>>>>>
>>>>> e) squash all patches into one patch and send it to mainline ...
>>>>>       so the mainline patch has only new changes with all U-Boot
>>>>>       specific changes.
>>>>
>>>> If you take over as NAND custodian, you can of course do the updates as
>>>> you please (unless Tom or Wolfgang disagree), but until then I insist on
>>>> doing a proper merge, and not littering the code with #ifndef __UBOOT__.
>>>>
>>>>> Thats sounds easy ... and you have all linux and all U-Boot
>>>>> changes visible and documented ... I must admit, I did not
>>>>> tried your proposal to look into the linux changes since
>>>>> the last mtd sync ... but that sounds the more tricky way
>>>>> to get this sync, as the history can be long ... and shouldn;t
>>>>> be the result the same?
>>>>
>>>> Why does the length of the history matter?  I'm not asking you to go
>>>> patch by patch.  Just take a diff between the SHA1 of last merge and the
>>>> SHA1 you want to merge now, and apply that patch.
>>>
>>> But it's not even that hard to do so!  git format-patch -o mtd-resync
>>> v3.7..v3.14 drivers/mtd/nand will give you only the relevant patches,
>>
>> "git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand"
>> results in 440 patches ... :-(
>
> This is why I suggested taking one big diff rather than going
> patch-to-patch.  Then you don't have to resolve the same conflicts over
> and over.
>
> You should also exclude files that U-Boot doesn't have, and include
> header files that are relevant.

Yep.

>> "git am" first patch immediately fails ...
>>
>> Ok, one step back ... I try only "drivers/mtd/mtdcore.c":
>>
>> $ git format-patch -o mtdcore v3.7..v3.14 drivers/mtd/mtdcore.c
>> mtdcore/0001-mtd-convert-to-idr_alloc.patch
>> mtdcore/0002-mtd-add-const-qualifier-to-a-couple-of-register-func.patch
>> mtdcore/0003-mtd-mtdcore-remove-few-useless-ifdef-s.patch
>> mtdcore/0004-mtd-merge-mtdchar-module-with-mtdcore.patch
>> mtdcore/0005-Include-missing-linux-slab.h-inclusions.patch
>> mtdcore/0006-drivers-avoid-format-string-in-dev_set_name.patch
>> mtdcore/0007-mtd-add-a-new-sys-node-to-show-the-ecc-step-size.patch
>> mtdcore/0008-mtd-add-MTD_MLCNANDFLASH-case-for-mtd_type_show.patch
>> mtdcore/0009-mtd-Move-major-number-definitions-to-major.h.patch
>> mtdcore/0010-mtd-remove-duplicated-include-from-mtdcore.c.patch
>> mtdcore/0011-mtd-convert-to-use-ATTRIBUTE_GROUPS.patch
>> $
>>
>> Only 11 patches, great ... but:
>>
>> $ git am -3 mtdcore/0001-mtd-convert-to-idr_alloc.patch
>> Wende an: mtd: convert to idr_alloc()
>> fatal: sha1 information is lacking or useless (drivers/mtd/mtdcore.c).
>
> If you fetch the Linux tree into your U-boot repository, then -3 will
> work.

Ah, try this.

>>> and they should mostly apply, and then you can squash them after the
>>> fact.
>>
>> I try to try this ASAP and report (first steps see above ...). But as a
>> lot of linux code is missing in U-Boot code, I feel, that a lot of chunks
>> will not apply ... but maybe (hopefully) I am wrong ...
>
> If the conflict is due to a change that applies to code that is missing
> because it doesn't make sense in U-Boot, then just delete that conflict
> and move on.  Unlike having the change silently apply to an ifndef
> region, you can first evaluate the change to see if it should apply to
> some alternative U-Boot version of the code.

Ok, I step into this direction.

>> Thats the question we must solve, do we want the complete Linux MTD code
>> in U-Boot and mark not used code?
>
> No.

Ok, prevent this.

>> If no, I delete this pieces of code...
>>
>> Hmm... looking in the history of drivers/mtd/ubi it seems the code
>> is basically from 2008!
>> http://git.denx.de/?p=u-boot.git;a=history;f=drivers/mtd/ubi;hb=7d2357c1999ff1f93f795282526230a8bd176106
>>
>> Hmm.. what was the Linux base of this ?
>
> See commit dfe64e2c89731a3f9950d7acd8681b68df2bae03 "mtd: resync with
> Linux-3.7.1"

Ah, yes, there are also UBI files.

>> and ubifs from 2009?
>> http://git.denx.de/?p=u-boot.git;a=history;f=fs/ubifs;hb=7d2357c1999ff1f93f795282526230a8bd176106
>> Base: "2.6.29-rc6" ... uh, very, very old!
>>
>> I see there no real syncs with newer linux code ... is this true?
>> Just some fixups ...
>
> CCing Kyungmin who is the UBI maintainer.

Thanks.

>> I really fear a patch "git format-patch -o ubifs-resync 2.6.29-rc6..v3.14 fs/ubifs"
>> (The result from this command are 355 patches ...)
>>
>> So, in sum, this are 795 patches + missing ubi patches which must be
>> applied/reworked/reviewed... Sounds painful...
>
> I don't know about ubifs, but the core NAND code (and apparently the
> core UBI code) is currently synced with Linux 3.7.1.
>
>> So another question is:
>> How to generate an update-sync patch:
>>
>> version a):
>> - copy the new linux files from Linux v3.14 to U-Boot
>> - work in the U-Boot specific changes
>> - make a patch from this
>>
>> Is this really different to:
>>
>> version b):
>> - git format-patch -o mtd-resync v3.7..v3.14 drivers/mtd/nand
>> - apply them
>> - make a patch from this
>
> It is different because "work in the U-Boot specific changes" is a
> manual, error-prone step compared to letting git do most of the work.
>
>> Currently it is differrent, as I introduced the complete linux code
>> and marked unused/different code in U-Boot with __UBOOT__. If I delete
>> this pieces ... version a) and b) should be the same.
>>
>> I try it ... if I not give up due to count of patches ...
>
> Again, there's no need to go patch-by-patch.  We haven't done so on
> prior updates.

I try it this way.

bye,
Heiko
-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany