[RFC] Xen 4.6 Acknowledgements

[RFC] Xen 4.6 Acknowledgements

[Lars Kurth]

Hi,

because we are now branching and opening master before the release, I have to make some changes to how I acknowledge Xen release contributions. 

In the past, I took the time-stamps of the two RELEASE tags for a release and counted contributions to xen.git and osstest.git (I didn't count qemu-trad)

However, we have no taken out mini-os, have raisin and some other repos. What I was planning to do was the following

1: split http://wiki.xenproject.org/wiki/Xen_Project_4.5_Acknowledgements - more correctly the 4.6 page - into a "Hypervisor only" and a "Hypervisor other" table

2: "Hypervisor only" will list contributions in xen.git between the release tags

3: "Hypervisor other" will list contributions in a number of related repos that are listed, but I would use the timestamps of the release tags. I was planning to include the following repos: osstest.git - can add friends also (as testing is important), raisin, mini-os (as it used to be in xen.git). 

I could also include qemu-traditional into (3), which I have not counted in the past. So I was thinking I'd not do this. Also including Linux, BSD, ... is hard as  pinpointing the xen-related parts are too hard.

Does this sounds sensible? Any objections?

Regards
Lars   

Re: [RFC] Xen 4.6 Acknowledgements

[Ian Campbell]

On Thu, 2015-10-08 at 18:04 +0100, Lars Kurth wrote:
> Hi,
> 
> because we are now branching and opening master before the release, I
> have to make some changes to how I acknowledge Xen release contributions.
> 
> In the past, I took the time-stamps of the two RELEASE tags for a release
> and counted contributions to xen.git and osstest.git (I didn't count qemu
> -trad)

How do the time-stamps get used? Are you doing git log --after=X -
-before=Y?

Why don't you just look at the commits between the two tags (i.e. git log
RELEASE-4.5.0..RELEASE-4.6.0 or whatever span you are using)?

The problem with using the time-stamp on a commit is that various git
operations (git rebase, git send-email + git am) can end up preserving the
original commit date in the authors tree, which can differ quite
significantly from the date something was committed to be pushed to the
repo.

For example see this 4 commit range:

$ git log --pretty=%ai\ %h\ %s 19c6d0784afafd9ce6e1549681f6a9eb429f6adf..4b405e6cff3c3fa7e3f5518f47b7477378b4410c
2014-09-08 15:24:59 +0100 4b405e6 tools/libxl: Infrastructure to convert a legacy stream
2015-06-12 18:19:00 +0100 bf4c9d2 tools/libxl: Infrastructure for reading a libxl migration v2 stream
2014-09-01 16:57:59 +0100 d3ea965 tools/libxl: Migration v2 stream format
2015-05-18 16:20:19 +0100 74a7c7a tools/python: Conversion utility for legacy migration streams

Those are 4 consecutive commits in the Xen history, but the author dates
(%ai) are obviously not.

Commit date (%ci) avoids this in some cases:
$ git log --pretty=%ci\ %h\ %s 19c6d0784afafd9ce6e1549681f6a9eb429f6adf..4b405e6cff3c3fa7e3f5518f47b7477378b4410c
2015-07-15 11:22:53 +0100 4b405e6 tools/libxl: Infrastructure to convert a legacy stream
2015-07-15 11:22:53 +0100 bf4c9d2 tools/libxl: Infrastructure for reading a libxl migration v2 stream
2015-07-15 11:22:53 +0100 d3ea965 tools/libxl: Migration v2 stream format
2015-07-15 11:22:53 +0100 74a7c7a tools/python: Conversion utility for legacy migration streams

But any contribution which arrived via a git pull rather than a git am
(which isn't too many in our current workflow) would potentially suffer the
same problem in the committer date too.

This probably doesn't hurt the data too much in the grand scheme of things,
but as shown above it would potentially exclude large projects which took a
long time to complete (exactly the sort of thing which should be credited).

All the above is pretty irrelevant to your actual proposal. Onto which:

> 3: "Hypervisor other" will list contributions in a number of related
> repos that are listed, but I would use the timestamps of the release
> tags. I was planning to include the following repos: osstest.git - can
> add friends also (as testing is important), raisin, mini-os (as it used
> to be in xen.git). 

FWIW some of the "other" repos to also gain their own tags corresponding to
the release (e.g. mini-os). Where it exists I suppose it makes sense to use
it?

> I could also include qemu-traditional into (3), which I have not counted
> in the past. So I was thinking I'd not do this. 

qemu-traditional doesn't get much traffic, but I don't see a reason to
exclude it. The separation/change you are proposing here sounds like an
ideal point to reintroduce it where it wasn't included before. 

> Also including Linux, BSD, ... is hard as  pinpointing the xen-related
> parts are too hard.

I suppose qemu-upstream falls into this bucket too? (Which I suppose mght
then be an argument for also excluding -trad?)

> Does this sounds sensible? Any objections?

The proposal looks fine to me, the above are pretty minor nits.

Ian.

Re: [RFC] Xen 4.6 Acknowledgements

[Lars Kurth]

> On 9 Oct 2015, at 10:09, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> 
> On Thu, 2015-10-08 at 18:04 +0100, Lars Kurth wrote:
>> Hi,
>> 
>> because we are now branching and opening master before the release, I
>> have to make some changes to how I acknowledge Xen release contributions.
>> 
>> In the past, I took the time-stamps of the two RELEASE tags for a release
>> and counted contributions to xen.git and osstest.git (I didn't count qemu
>> -trad)
> 
> How do the time-stamps get used? Are you doing git log --after=X -
> -before=Y?

I was using git log -p -M --since=x --until=y, with master checked out, which is what was recommended in GitDM when doing reporting. Mainly because most of the scripts I have are used to do monthly and yearly reporting.

> Why don't you just look at the commits between the two tags (i.e. git log
> RELEASE-4.5.0..RELEASE-4.6.0 or whatever span you are using)?

That's what I do for xen.git and was planning to do in future, but unfortunately RELEASE-4.6.0 are not applied consistently to all repos (aka OSSTEST, etc). 

> The problem with using the time-stamp on a commit is that various git
> operations (git rebase, git send-email + git am) can end up preserving the
> original commit date in the authors tree, which can differ quite
> significantly from the date something was committed to be pushed to the
> repo.

Does -M fix this issue? I can't find any docs for it.

>> 3: "Hypervisor other" will list contributions in a number of related
>> repos that are listed, but I would use the timestamps of the release
>> tags. I was planning to include the following repos: osstest.git - can
>> add friends also (as testing is important), raisin, mini-os (as it used
>> to be in xen.git). 
> 
> FWIW some of the "other" repos to also gain their own tags corresponding to
> the release (e.g. mini-os). Where it exists I suppose it makes sense to use
> it?

It is too painful to do this at least for this release, and some repos don't contain tags at all. The scripts I have, check out all the repos I want to measure (e.g. all XAPI, all Mirage, ... repos) into a directory and I use something like 

find $i/* -maxdepth 0 -type d | xargs -i git -C {} log $args >> $root/output/$i/$ext.gitlog

to aggregate all the logs for all the relevant repos.  

>> I could also include qemu-traditional into (3), which I have not counted
>> in the past. So I was thinking I'd not do this. 
> 
> qemu-traditional doesn't get much traffic, but I don't see a reason to
> exclude it. The separation/change you are proposing here sounds like an
> ideal point to reintroduce it where it wasn't included before. 

That's what I was thinking and why I left it out

>> Also including Linux, BSD, ... is hard as  pinpointing the xen-related
>> parts are too hard.
> 
> I suppose qemu-upstream falls into this bucket too? (Which I suppose mght
> then be an argument for also excluding -trad?)

Agreed, but the challenge is that it is near impossible to filter out KVM, Linux, ... contributions.

Regards
Lars

Re: [RFC] Xen 4.6 Acknowledgements

[Ian Campbell]

On Fri, 2015-10-09 at 10:57 +0100, Lars Kurth wrote:
> > On 9 Oct 2015, at 10:09, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > 
> > On Thu, 2015-10-08 at 18:04 +0100, Lars Kurth wrote:
> > > Hi,
> > > 
> > > because we are now branching and opening master before the release, I
> > > have to make some changes to how I acknowledge Xen release
> > > contributions.
> > > 
> > > In the past, I took the time-stamps of the two RELEASE tags for a
> > > release
> > > and counted contributions to xen.git and osstest.git (I didn't count
> > > qemu
> > > -trad)
> > 
> > How do the time-stamps get used? Are you doing git log --after=X -
> > -before=Y?
> 
> I was using git log -p -M --since=x --until=y, with master checked out,
> which is what was recommended in GitDM when doing reporting. Mainly
> because most of the scripts I have are used to do monthly and yearly
> reporting.

OK, I suppose if that's what GitDM says to do there's no reason to deviate.

> > Why don't you just look at the commits between the two tags (i.e. git log
> > RELEASE-4.5.0..RELEASE-4.6.0 or whatever span you are using)?
> 
> That's what I do for xen.git and was planning to do in future, but
> unfortunately RELEASE-4.6.0 are not applied consistently to all repos
> (aka OSSTEST, etc). 

Indeed, I only meant for repos which have the tags.

> > The problem with using the time-stamp on a commit is that various git
> > operations (git rebase, git send-email + git am) can end up preserving
> > the
> > original commit date in the authors tree, which can differ quite
> > significantly from the date something was committed to be pushed to the
> > repo.
> 
> Does -M fix this issue? I can't find any docs for it.

-M is "find renames", which makes the diff of code motion easier to follow,
so I don't think so.

> 
> > > 3: "Hypervisor other" will list contributions in a number of related
> > > repos that are listed, but I would use the timestamps of the release
> > > tags. I was planning to include the following repos: osstest.git - can
> > > add friends also (as testing is important), raisin, mini-os (as it used
> > > to be in xen.git). 
> > 
> > FWIW some of the "other" repos to also gain their own tags corresponding to
> > the release (e.g. mini-os). Where it exists I suppose it makes sense to use
> > it?
> 
> It is too painful to do this at least for this release,

Sure.

>  and some repos don't contain tags at all. The scripts I have, check out
> all the repos I want to measure (e.g. all XAPI, all Mirage, ... repos)
> into a directory and I use something like 
> 
> find $i/* -maxdepth 0 -type d | xargs -i git -C {} log $args >>
> $root/output/$i/$ext.gitlog
> 
> to aggregate all the logs for all the relevant repos.  
> 
> > > I could also include qemu-traditional into (3), which I have not
> > > counted
> > > in the past. So I was thinking I'd not do this. 
> > 
> > qemu-traditional doesn't get much traffic, but I don't see a reason to
> > exclude it. The separation/change you are proposing here sounds like an
> > ideal point to reintroduce it where it wasn't included before. 
> 
> That's what I was thinking and why I left it out

I was proposing leaving it in based on that reasoning though.

(did you mean this to be below the comment about upstream?).

> > > Also including Linux, BSD, ... is hard as  pinpointing the xen
> > > -related
> > > parts are too hard.
> > 
> > I suppose qemu-upstream falls into this bucket too? (Which I suppose
> > mght
> > then be an argument for also excluding -trad?)
> 
> Agreed, but the challenge is that it is near impossible to filter out
> KVM, Linux, ... contributions.

Right.

Ian.

Re: [RFC] Xen 4.6 Acknowledgements

[Lars Kurth]

> On 9 Oct 2015, at 11:08, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> 
> On Fri, 2015-10-09 at 10:57 +0100, Lars Kurth wrote:
>> 
>> 
>>>> I could also include qemu-traditional into (3), which I have not
>>>> counted
>>>> in the past. So I was thinking I'd not do this. 
>>> 
>>> qemu-traditional doesn't get much traffic, but I don't see a reason to
>>> exclude it. The separation/change you are proposing here sounds like an
>>> ideal point to reintroduce it where it wasn't included before. 
>> 
>> That's what I was thinking and why I left it out
> 
> I was proposing leaving it in based on that reasoning though.
> 
> (did you mean this to be below the comment about upstream?).

Correct

Lars