Creating diff from 2.6.16 from cryptodev-2.6 git tree

Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Michal Ludvig]

Hi Herbert and others,

just a quick question: how can I create a patch with all changes in
cryptodev-2.6 tree against tag v2.6.16 in Linus tree? I've got
git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
cloned here and want to extract all your commits in this tree since
2.6.16. Is there a way to do it in Git/Cogito?

I was playing with cg-mkpatch but that gave me only the last commit
(Twofish updates). I'm sure I could search for your commits in the log
and create those patches one by one but that doesn't seem to be too
efficient. Also tried "cg-mkpatch -r 6246b61[...].." which is your merge
of v2.6.17-rc1 into cryptodev-2.6 but that extracted _all_ commits since
that tag, not only those in your tree.

Any help appreciated :-)

Thanks

Michal

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Herbert Xu]

On Fri, Jun 30, 2006 at 01:18:24PM +1200, Michal Ludvig wrote:
> 
> just a quick question: how can I create a patch with all changes in
> cryptodev-2.6 tree against tag v2.6.16 in Linus tree? I've got
> git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
> cloned here and want to extract all your commits in this tree since
> 2.6.16. Is there a way to do it in Git/Cogito?

OK, it's easier if you break this into three problems.

You start by getting all the changes merged right after 2.6.16.  This
can be done by locating the merge changeset in Linus's tree.  It looks
like this:

commit ec1248e70edc5cf7b485efcc7b41e44e10f422e5
Merge: 3d1f337b3e7378923c89f37afb573a918ef40be5 55e9dce37ddf3ab358ba1d1e9eef4ee4bd8174a6
Author: Linus Torvalds <torvalds@g5.osdl.org>
Date:   Tue Mar 21 09:33:19 2006 -0800

    Merge master.kernel.org:/pub/scm/linux/kernel/git/herbert/crypto-2.6

    * master.kernel.org:/pub/scm/linux/kernel/git/herbert/crypto-2.6:
      [CRYPTO] aes: Fixed array boundary violation
      [CRYPTO] tcrypt: Fix key alignment
      [CRYPTO] all: Add missing cra_alignmask
      [CRYPTO] all: Use kzalloc where possible
      [CRYPTO] api: Align tfm context as wide as possible
      [CRYPTO] twofish: Use rol32/ror32 where appropriate

You can get the diff for this merge by doing

$ git-diff-tree -p 3d1f337b3e7378923c89f37afb573a918ef40be5 ec1248e70edc5cf7b485efcc7b41e44e10f422e5

You then repeat this for 2.6.17 where the merge changeset is:

commit 972d19e837833b93466c6f6a8ef2a7d653000aa3
Merge: cdf4f383a4b0ffbf458f65380ecffbeee1f79841 b9d0a25a484a90c1d60b974d115eff2fe580ce16
Author: Linus Torvalds <torvalds@g5.osdl.org>
Date:   Mon Jun 26 11:03:29 2006 -0700

Finally you get the changes since that merge by find the common changeset
between cryptodev-2.6 and Linus's tree and getting all the changes after
that.

This method works because all crypto changes since 2.6.16 were merged
from my tree.  If this wasn't the case, then you'd have to search
through the history since 2.6.16 to identify the crypto patches that
were merged from other sources.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Linus Torvalds]

On Fri, 30 Jun 2006, Herbert Xu wrote:
>
> On Fri, Jun 30, 2006 at 01:18:24PM +1200, Michal Ludvig wrote:
> > 
> > just a quick question: how can I create a patch with all changes in
> > cryptodev-2.6 tree against tag v2.6.16 in Linus tree? I've got
> > git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
> > cloned here and want to extract all your commits in this tree since
> > 2.6.16. Is there a way to do it in Git/Cogito?
> 
> OK, it's easier if you break this into three problems.
> 
> You start by getting all the changes merged right after 2.6.16.  This
> can be done by locating the merge changeset in Linus's tree.  It looks
> like this:

Actually, there are certainly other, potentially easier, ways to do this.

It depends a bit on what Michal wants, though. Since the current trees 
(both mine and the cryptodev tree) have been merging things back and 
forth, it's _not_ as easy as just saying "pick all commits that exist in 
one branch but not the other", but depending on what Michal wants to do, 
git gives other ways to prune out just the info he wants.

The easiest by far is if you only care about a certain sub-directory. 
Then, assuming the branch "crypto" is the top-most commit of the cryptodev 
repo, just do

	git diff v2.6.16..crypto -- crypto/

and that will give you a diff of all the changes since v2.6.16 inside that 
subdirectory. That may or may not be sufficient and what Michal wants.

Now, the cryptodev-2.6.git tree doesn't even contain the v2.6.16 tags, but 
you can fix that by just doing

	git fetch --tags git://git.kernel.org//pub/scm/linux/kernel/git/torvalds/linux-2.6

even if your clone is actually from just the cryptodev-2.6 archive.

Alternatively, if you want to see the individual changes, you can just do

	git log -p --full-diff v2.6.16..crypto -- crypto/

which shows you all the commits that changed the crypto/ subdirectory, AND 
it shows the other changes those same commits did to other subdirectories 
too (which is usually something you want in a case like this).

Finally, what you can also do is that instead of matching for stuff that 
changed the crypto/ subdirectory, you could try to match commits where the 
committer is somebody special, eg Herbert Xu. We don't have that kind of 
thing automated, but here's one way to do it:

	git-rev-list --header v2.6.16..crypto |
		grep -z 'committer Herbert Xu' |
		tr '\0' '\n' |
		sed -n '/^[a-f0-9][a-f0-9]*$/p' |
		git diff-tree --pretty -p --stdin |
		less -S

where the "git-rev-list --header | grep -z" part picks out any commits 
committed by Herbert, the "tr '\0' '\n' | sed -n" part then picks up just 
the commit ID's from those lines, and the "git-diff-tree" part then shows 
those commits as diffs.

(The above should really be quite possible to shorten as

	git log -p --committer="Herbert Xu"

but we don't actually support git-rev-list doing matching on 
committer/author names - although it should be easy to do in case somebody 
wants to have a small git project to get their toes wet, hint hint)

			Linus

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Herbert Xu]

On Thu, Jun 29, 2006 at 07:25:01PM -0700, Linus Torvalds wrote:
> 
> The easiest by far is if you only care about a certain sub-directory. 
> Then, assuming the branch "crypto" is the top-most commit of the cryptodev 
> repo, just do
> 
> 	git diff v2.6.16..crypto -- crypto/

Yes, you should always do something like this when backporting to make
sure that you haven't missed patches merged in ways that you didn't
anticipate.

Although I'm not aware of any such patches in the time frame that Michal
has in mind.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Linus Torvalds]

On Fri, 30 Jun 2006, Herbert Xu wrote:

> On Thu, Jun 29, 2006 at 07:25:01PM -0700, Linus Torvalds wrote:
> > 
> > The easiest by far is if you only care about a certain sub-directory. 
> > Then, assuming the branch "crypto" is the top-most commit of the cryptodev 
> > repo, just do
> > 
> > 	git diff v2.6.16..crypto -- crypto/
> 
> Yes, you should always do something like this when backporting to make
> sure that you haven't missed patches merged in ways that you didn't
> anticipate.
> 
> Although I'm not aware of any such patches in the time frame that Michal
> has in mind.

Note that this is why

	git log -p --full-diff v2.6.16.. crypto/

is so great. It will show everything that touched that subdirectory, along 
with the incidentals. I think that in this case, that's exactly what 
Michal wants.

		Linus

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Michal Ludvig]

Linus Torvalds wrote:

> Note that this is why
> 
> 	git log -p --full-diff v2.6.16.. crypto/
> 
> is so great. It will show everything that touched that subdirectory, along 
> with the incidentals. I think that in this case, that's exactly what 
> Michal wants.

Exactly!

I just want to have the latest greatest CryptoAPI on 2.6.16, it's that
simple ;-)

Thanks both for a quick response

Michal

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Michal Ludvig]

Linus Torvalds wrote:

> 	git log -p --full-diff v2.6.16.. crypto/

Can I somehow get the result in a reverse order, i.e. oldest commits first?

As it is the patch conflicts on files that were changed multiple times
since 2.6.16 :-(

Will see if patchutils can help me somehow...

Michal

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Linus Torvalds]

On Fri, 30 Jun 2006, Michal Ludvig wrote:
>
> Linus Torvalds wrote:
> 
> > 	git log -p --full-diff v2.6.16.. crypto/
> 
> Can I somehow get the result in a reverse order, i.e. oldest commits first?

Not that way, no. "git log" generates the data on-the-fly, so a simple 
"git log" will always give most recent first.

However, you can do this

	git log --pretty=oneline v2.6.16.. crypto/

to generate a list of commits, one per line. Then, reverse that list 
("tac" is your friend), and feed it back to "git-diff-tree --stdin 
--pretty -p" to get the diffs.

So

	git log --pretty=oneline v2.6.16.. crypto/ |
		tac |
		git-diff-tree --stdin --pretty -p

should do what you want.

Of course, the patches (to other files than the crypt/ subdirectory) may 
still clash due to changes that are unrelated to the crypto changes. That 
is unavoidable, and you'll just have to fix that up by hand.

			Linus

Re: Creating diff from 2.6.16 from cryptodev-2.6 git tree

[Junio C Hamano]

Michal Ludvig <michal@logix.cz> writes:

> Linus Torvalds wrote:
>
>> 	git log -p --full-diff v2.6.16.. crypto/
>
> Can I somehow get the result in a reverse order, i.e. oldest commits first?

Two ways:

	git format-patch -o patches-output/ --full-diff v2.6.16.. crypto/

would give you one file a patch in patches-output directory,
numbered.

	git format-patch --stdout --full-diff v2.6.16.. crypto/

would give you a single stream of the above to the standard output.