Generating patches/Cherry Picking for a large number of commits

Generating patches/Cherry Picking for a large number of commits

[Alydis]

Hi,

Newly converted git user here.  I've stopped working with Subversion
ever since working with the Linux kernel has introduced me to git, and
never looked back!

I am currently working on a project in which I'm trying to backport
the powerpc boot-wrapper and device tree handling from Linux 2.6.30
into Linux 2.6.21.  We have some third party vendor code that is not
ready for the (new) completely fair scheduler just yet.  Yet, our BSP
relies on cuImage and the DTS files.

The approach I'm taking is to try to cherry-pick any commits between
v2.6.21 and v2.6.30 on the Linux kernel git repository that touch any
file in the arch/powerpc/boot directory into a v2.6.21 branch (and
pray that the conflicts are manageable).

I've tried something along these lines:

git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
cd linux-2.6
git checkout -b mybranch v2.6.21
git format-patch -o patches v2.6.21..v2.6.30 arch/powerpc/boot
git am -3 patches/*

But, to my dismay, format-patch here tears apart the commits and
applies ONLY the hunks that apply to the arch/powerpc/boot directory.
What I'd much rather do is obtain a list of commits that apply to
arch/powerpc/boot; but, then apply the entire patch.

I'm a bit lost at how to proceed at this point?  Do I need to write my
own script that uses git-log to grab the commit hashes that I want,
then call format-patch (or diff/apply/etc) individually on each of
them?  I realize I'll have to handle patch numbering myself, if I were
to do that (luckily, I believe format-patch allows you to specify the
starting number).  I also know at some point, I'll have to drill down
into individual commits and do some more detailed cherry picking to
form the desired patch series.  This script seems like a good deal of
work for me... so before I go off and start it, I was wondering if the
git gurus have any clever solutions for me?

Thanks,
Tommy Wang

Re: Generating patches/Cherry Picking for a large number of commits

[Jeff King]

On Fri, Aug 28, 2009 at 02:26:43PM -0500, Alydis wrote:

> I've tried something along these lines:
> 
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
> cd linux-2.6
> git checkout -b mybranch v2.6.21
> git format-patch -o patches v2.6.21..v2.6.30 arch/powerpc/boot
> git am -3 patches/*
> 
> But, to my dismay, format-patch here tears apart the commits and
> applies ONLY the hunks that apply to the arch/powerpc/boot directory.
> What I'd much rather do is obtain a list of commits that apply to
> arch/powerpc/boot; but, then apply the entire patch.

By default, format-patch (and log, gitk, etc) when given a path limiter
will also limit the diff shown. You can override it with --full-diff.

-Peff

Re: Generating patches/Cherry Picking for a large number of commits

[Alydis]

Ack!  Embarrassing RTFM.

While I have your attention, however, I noticed that git am <path>
will apply the list patches generated by format-patch.  The
documentation said something about mbox/maildir directories, which I
actually am not that familiar with.  Is it safe to say that git am
<path> will read the path and apply patches in numerical order?  Does
it allow skipping?

Thanks again,
Tommy Wang


On Fri, Aug 28, 2009 at 2:45 PM, Jeff King<peff@peff.net> wrote:
> On Fri, Aug 28, 2009 at 02:26:43PM -0500, Alydis wrote:
>
>> I've tried something along these lines:
>>
>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
>> cd linux-2.6
>> git checkout -b mybranch v2.6.21
>> git format-patch -o patches v2.6.21..v2.6.30 arch/powerpc/boot
>> git am -3 patches/*
>>
>> But, to my dismay, format-patch here tears apart the commits and
>> applies ONLY the hunks that apply to the arch/powerpc/boot directory.
>> What I'd much rather do is obtain a list of commits that apply to
>> arch/powerpc/boot; but, then apply the entire patch.
>
> By default, format-patch (and log, gitk, etc) when given a path limiter
> will also limit the diff shown. You can override it with --full-diff.
>
> -Peff
>

Re: Generating patches/Cherry Picking for a large number of commits

[Jakub Narebski]

[Alydis, please do not toppost]

Alydis <alydis@august8.net> writes:
> On Fri, Aug 28, 2009 at 2:45 PM, Jeff King<peff@peff.net> wrote:
>> On Fri, Aug 28, 2009 at 02:26:43PM -0500, Alydis wrote:
>>
>>> I've tried something along these lines:
>>>
[...]
>>> git format-patch -o patches v2.6.21..v2.6.30 arch/powerpc/boot
>>> git am -3 patches/*
>>>
>>> But, to my dismay, format-patch here tears apart the commits and
>>> applies ONLY the hunks that apply to the arch/powerpc/boot directory.
>>> What I'd much rather do is obtain a list of commits that apply to
>>> arch/powerpc/boot; but, then apply the entire patch.
>>
>> By default, format-patch (and log, gitk, etc) when given a path limiter
>> will also limit the diff shown. You can override it with --full-diff.
>
> Ack!  Embarrassing RTFM.
> 
> While I have your attention, however, I noticed that git am <path>
> will apply the list patches generated by format-patch.  The
> documentation said something about mbox/maildir directories, which I
> actually am not that familiar with.  Is it safe to say that git am
> <path> will read the path and apply patches in numerical order? 

git-format-patch | git-am pipeline has to work correctly, as it
originally was the way (modulo extra options) git-rebase was
implemented.  So yes, "git am <dir>" should understand and apply in
correct order result of "git format-patch -o <dir> <revspec>".

> Does it allow skipping?

There is "git am -i".

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: Generating patches/Cherry Picking for a large number of commits

[Junio C Hamano]

Jakub Narebski <jnareb@gmail.com> writes:

> git-format-patch | git-am pipeline has to work correctly, as it
> originally was the way (modulo extra options) git-rebase was
> implemented.  So yes, "git am <dir>" should understand and apply in
> correct order result of "git format-patch -o <dir> <revspec>".

You are not making any sense.  rebase is done using --stdout and does not
rely on the intermediate files nor ordering of their names.

Re: Generating patches/Cherry Picking for a large number of commits

[Jeff King]

On Fri, Aug 28, 2009 at 02:50:53PM -0500, Alydis wrote:

> While I have your attention, however, I noticed that git am <path>
> will apply the list patches generated by format-patch.  The
> documentation said something about mbox/maildir directories, which I
> actually am not that familiar with.  Is it safe to say that git am
> <path> will read the path and apply patches in numerical order?  Does
> it allow skipping?

It will apply the patches in the order given on the command line. When
your shell expands the "patches/*" glob, it will do so in lexically
sorted order. Meaning "0001" comes before "0002", etc, which is the
reason that format-patch zero-pads the filenames.

You can edit or delete the patches in your patch directory before
applying, and they should apply the same (unless you create a patch that
cannot actually be applied).

-Peff

Re: Generating patches/Cherry Picking for a large number of commits

[Junio C Hamano]

Alydis <alydis@august8.net> writes:

> Ack!  Embarrassing RTFM.
>
> While I have your attention, however, I noticed that git am <path>
> will apply the list patches generated by format-patch.  The
> documentation said something about mbox/maildir directories, which I
> actually am not that familiar with.  Is it safe to say that git am
> <path> will read the path and apply patches in numerical order?  Does
> it allow skipping?

If you say "git am <directory>/*", like you have in your original message
shown *below* (sheesh, why am I responding to somebody who top-posts?),
you let your shell perform a numerical sort, so you would be Ok.

>>> git format-patch -o patches v2.6.21..v2.6.30 arch/powerpc/boot
>>> git am -3 patches/*

I do not think the above format-patch, even with --full-diff, is
necessarily a good idea nor would work in general.

Often, when a series is concocted, you would see this pattern:

 * Early parts of the patch series to lay groundwork by introducing non
   platform specific infrastructure; and then

 * Later parts of the series utilizes the infrastructure to implement the
   feature for particular platforms.

Obviously you are interested only for powerpc parts in the latter category
and would want to omit anything irrelevant to powerpc.  But you do want to
include all of the former class, even if they do not touch anything inside
the powerpc area.