The gregkh patch scripts

The gregkh patch scripts

[Alan Stern]

Greg:

Are the scripts you use for creating the various gregkh-*.patch files 
stored in 

http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/scripts/

?  Those files appear not to have been updated for quite some time.

Regardless, the way the current scripts work has got a problem.  As far
as I can tell, they take Linus's current tree, apply your various
accumulated quilt series files, and then compute the diff between the
end result and the starting point.  The output is labelled with name of
the starting tree.  (If my understanding is wrong then tell me so and
ignore the rest of this message.)

The problem is that Linus's current tree is not a good starting point.  
For example, right now several of the individual files such as
gregkh-07-usb-2.6.33.patch are completely empty.  This is because you
have sent all the pending changes upstream, so they are now included in
Linus's tree and hence they don't show up in a diff.

But...  The file is still named "gregkh-07-usb-2.6.33.patch", even
though it isn't a patch against 2.6.33 -- it's a patch against whatever
Linus's tree happened to contain at the time you ran the script.  
Since I don't know what tree that was (and I don't have git repository
for the kernel anyway), I can't use the patch file.  (Well, I _can_ use 
it because it's empty, but it doesn't do me any good -- the principle 
is still valid.)

As Linus has explained to quite a few people, this isn't a good way to
operate.  What your scripts do is essentially the same as rebasing a
publicly-exported git repository, which should be done relatively
infrequently.  It's okay to rebase against the actual releases and the
*-rcN trees, because they are well-defined intermediate points.  But
rebasing against a *-rcN-gitM should be quite rare, and rebasing
against anything else is basically forbidden.

This means the scripts shouldn't start with the vanilla
tree-of-the-moment.  They should start with the most recent release or
-rcN tag.  In rare cases they could use an rcN-gitM tag, but only if
there's a good reason.

So for example, the current gregkh-*-2.6.33.patch files should be quite
large, because they should contain all the accumulated development for
the last three months that hasn't gotten into 2.6.33.  When 2.6.34-rc1
is released then the files should get small, since they would then
contain only material that was submitted slightly before or during the
merge window.

Can the scripts be updated to work in this way?

Alan Stern

Re: The gregkh patch scripts

[Greg KH]

On Sat, Mar 06, 2010 at 11:19:20PM -0500, Alan Stern wrote:
> Greg:
> 
> Are the scripts you use for creating the various gregkh-*.patch files 
> stored in 
> 
> http://www.kernel.org/pub/linux/kernel/people/gregkh/gregkh-2.6/scripts/
> 
> ?  Those files appear not to have been updated for quite some time.

Until last week, yes, they were the most recent version.

However, last week I made some changes, which caused things to break in
different ways as you have noticed.

> Regardless, the way the current scripts work has got a problem.  As far
> as I can tell, they take Linus's current tree, apply your various
> accumulated quilt series files, and then compute the diff between the
> end result and the starting point.  The output is labelled with name of
> the starting tree.  (If my understanding is wrong then tell me so and
> ignore the rest of this message.)

No, that is correct.

I switched to using a git commit base to make it easier to handle the
merges in the linux-next tree due to problems that Stephen was having
when merging my trees into there.

> The problem is that Linus's current tree is not a good starting point.  
> For example, right now several of the individual files such as
> gregkh-07-usb-2.6.33.patch are completely empty.  This is because you
> have sent all the pending changes upstream, so they are now included in
> Linus's tree and hence they don't show up in a diff.

Yes that is true.

But I do think that Linus's tree is a good starting point, as that
reflects my current quilt patchset identically.

> But...  The file is still named "gregkh-07-usb-2.6.33.patch", even
> though it isn't a patch against 2.6.33 -- it's a patch against whatever
> Linus's tree happened to contain at the time you ran the script.  
> Since I don't know what tree that was (and I don't have git repository
> for the kernel anyway), I can't use the patch file.  (Well, I _can_ use 
> it because it's empty, but it doesn't do me any good -- the principle 
> is still valid.)

Yes, the issue is that the base tree is not labeled the same.  I should
rename the patch to be "gregkh-07-usb-SHA.patch" to be more refelective
of what it really is.

If you look at the quilt series that linux-next pulls from, those have
the correct marking with the # BEGIN markings that linux-next relies on.
It identifies the proper SHA1 to base the tree off of.

> As Linus has explained to quite a few people, this isn't a good way to
> operate.  What your scripts do is essentially the same as rebasing a
> publicly-exported git repository, which should be done relatively
> infrequently.  It's okay to rebase against the actual releases and the
> *-rcN trees, because they are well-defined intermediate points.  But
> rebasing against a *-rcN-gitM should be quite rare, and rebasing
> against anything else is basically forbidden.
> 
> This means the scripts shouldn't start with the vanilla
> tree-of-the-moment.  They should start with the most recent release or
> -rcN tag.  In rare cases they could use an rcN-gitM tag, but only if
> there's a good reason.
> 
> So for example, the current gregkh-*-2.6.33.patch files should be quite
> large, because they should contain all the accumulated development for
> the last three months that hasn't gotten into 2.6.33.  When 2.6.34-rc1
> is released then the files should get small, since they would then
> contain only material that was submitted slightly before or during the
> merge window.
> 
> Can the scripts be updated to work in this way?

Yes, I can switch back to going off of the -rcN-gitM markings that I was
using in the past.  If I do that does it make it easier for you?

And yes, you are right, this is a constant rebase, and that's wrong on
my part.

This makes me reconsider the fact that I'm using quilt for all of this.
Perhaps if I switch to a git tree, it would all be easier for people
like you?  Would you prefer that?

For stuff like my staging tree, I'm almost convinced that switching to
git makes more sense as keeping that many patches in quilt is a mess and
I don't ever end up rebasing anything or deleting any patches in the
middle anymore.

For USB, yeah, I think it's also time to switch over as well.  What do
you think?

thanks,

greg k-h

Re: The gregkh patch scripts

[Alan Stern]

On Sun, 7 Mar 2010, Greg KH wrote:

> Yes, I can switch back to going off of the -rcN-gitM markings that I was
> using in the past.  If I do that does it make it easier for you?
> 
> And yes, you are right, this is a constant rebase, and that's wrong on
> my part.

That's exactly the problem.  I don't mind rebasing for each -rcN; in
fact many of the early -rc kernels often contain important bug fixes so
I'm glad to have them.  But rebasing more often than that becomes
increasingly inconvenient.

> This makes me reconsider the fact that I'm using quilt for all of this.
> Perhaps if I switch to a git tree, it would all be easier for people
> like you?  Would you prefer that?

I'm happy either way (although by now I'm quite used to using quilt and
not so accustomed to git -- I haven't tried guilt).  Of course, other
people may feel differently.

> For stuff like my staging tree, I'm almost convinced that switching to
> git makes more sense as keeping that many patches in quilt is a mess and
> I don't ever end up rebasing anything or deleting any patches in the
> middle anymore.
> 
> For USB, yeah, I think it's also time to switch over as well.  What do
> you think?

Let's ask some other people on the USB list (CC'ed).  My main concern 
is to avoid rebasing too often, whichever source-code management system 
is used.

Alan Stern

Re: The gregkh patch scripts

[Felipe Balbi]

Hi,

On Sun, Mar 07, 2010 at 03:54:25PM -0500, Alan Stern wrote:
> Let's ask some other people on the USB list (CC'ed).  My main concern 
> is to avoid rebasing too often, whichever source-code management system 
> is used.

personally, I'm more used to git and find it difficult (or annoying) when
I need to rebase my own patches on top of Greg's quilt tree. git
quiltimport doesn't work always and I have to refresh patches on my own
and run git quiltimport later again.

Moving to a git tree would help me, but then again, I don't want to
become a burden. Whatever is better for Greg, I'll be adapating to ;-)

What I use currently to have a tree which people can pull from and still
be able to rebase on top of Greg's patches is something like:

$ git fetch linus
$ git reset --hard linus/master
$ git quiltimport --patches gregkh
$ git rebase greg my_patches
$ git merge -s ours master
$ git checkout master
$ git merge my_patches

it helps although rebuilding the tree always is a bit messy :-p

-- 
balbi

Re: The gregkh patch scripts

[Greg KH]

On Sun, Mar 07, 2010 at 03:54:25PM -0500, Alan Stern wrote:
> On Sun, 7 Mar 2010, Greg KH wrote:
> 
> > Yes, I can switch back to going off of the -rcN-gitM markings that I was
> > using in the past.  If I do that does it make it easier for you?
> > 
> > And yes, you are right, this is a constant rebase, and that's wrong on
> > my part.
> 
> That's exactly the problem.  I don't mind rebasing for each -rcN; in
> fact many of the early -rc kernels often contain important bug fixes so
> I'm glad to have them.  But rebasing more often than that becomes
> increasingly inconvenient.

Ok, I've moved back to the original way I was handling this, it should
all be back to normal now.

thanks,

greg k-h