[cip-dev] [cip-kernel-sec 5/6] report_affected: add support for reporting on tags

[ben.hutchings@codethink.co.uk (Ben Hutchings)]

On Tue, 2019-06-25 at 12:26 +0900, Daniel Sangorrin wrote:
> Reporting on tags is useful for product engineers that
> have shipped a kernel with a specific tag and need to know
> which issues affect their product after some time.

I think this is a really important feature, but I'm not convinced about
this implementation.

> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> ?scripts/report_affected.py | 60 ++++++++++++++++++++++++++++++++------
> ?1 file changed, 51 insertions(+), 9 deletions(-)
> 
> diff --git a/scripts/report_affected.py b/scripts/report_affected.py
> index 7557dc8..32e9345 100755
> --- a/scripts/report_affected.py
> +++ b/scripts/report_affected.py
> @@ -9,7 +9,9 @@
> ?# Report issues affecting each stable branch.
> ?
> ?import argparse
> +import copy
> ?import subprocess
> +import re
> ?
> ?import kernel_sec.branch
> ?import kernel_sec.issue
> @@ -23,10 +25,26 @@ def main(git_repo, remotes,
> ?????????branches = []
> ?????????for branch in live_branches:
> ?????????????for name in branch_names:
> +????????????????# there could be multiple tags for the same branch
> +????????????????branch_copy = copy.deepcopy(branch)

This makes a lot of unnecessary copies!

> +????????????????if name[0] == 'v':
> +????????????????????# a stable tag, e.g. v4.4.92-cip11
> +????????????????????branch_copy['tag'] = name
> +????????????????????match = re.match(r'^v(\d+\.\d+).*', name)
> +????????????????????if not match:
> +????????????????????????raise ValueError('failed to parse tag %r' % name)
> +????????????????????if 'cip' in name:
> +????????????????????????name = 'linux-%s.y-cip' % match.group(1)
> +????????????????????else:
> +????????????????????????name = 'linux-%s.y' % match.group(1)

How about adding regexps for tags to the branch configuration?  Then
here we could match tags to branches with
re.matchall(branch['tag_regexp'], name).

> +????????????????if '/' in name:

'/' is valid in a branch or tag name, so how about using ':' which is
not?

> +????????????????????# a possibly custom tag, e.g. product-v1
> +????????????????????branch_copy['tag'] = name.split('/')[1]
> +????????????????????name = name.split('/')[0]

It would be more Pythonic to write this as a tuple assginment.

> ?????????????????if name[0].isdigit():
> ?????????????????????name = 'linux-%s.y' % name
> -????????????????if branch['short_name'] == name:
> -????????????????????branches.append(branch)
> +????????????????if branch_copy['short_name'] == name:
> +????????????????????branches.append(branch_copy)
> ?????????if not branches:
> ?????????????msg = "supplied branches didn't match any known branch"
> ?????????????raise argparse.ArgumentError(None, msg)
> @@ -40,6 +58,18 @@ def main(git_repo, remotes,
> ?
> ?????c_b_map = kernel_sec.branch.CommitBranchMap(git_repo, remotes, branches)
> ?
> +????# cache tag commits and set full_name to show the tag
> +????tag_commits = {}
> +????for branch in branches:
> +????????if 'tag' in branch:
> +????????????start = 'v' + branch['base_ver']
> +????????????end = branch['tag']
> +????????????for commit in kernel_sec.branch._get_commits(git_repo, end, start):

The leading '_' means private, so it shouldn't be called from here.  I
think it could be made public but it needs a better name, maybe
'iter_rev_list' matching the underlying command.

> +????????????????tag_commits.setdefault(end, []).append(commit)
[...]

Suppose I tried to query 'v4.4'; then I think we get start == end ==
'v4.4' and this loop doesn't execute at all.  tag_commits['v4.4'] won't
be defined at all, and the reporting loop will crash.

So this could be simply:

               tag_commits[end] = list(
                   kernel_sec.branch.iter_rev_list(git_repo, end, start))

But I think the collection type should also be changed from list to
set, so that tests for membership will be fast.

Ben.

-- 
Ben Hutchings, Software Developer                ?        Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom