[PATCH 00/10] Some more stable-rt-tools fixes

[PATCH 00/10] Some more stable-rt-tools fixes

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Hi Daniel,

I just did my first srt tools backport release for 3.18 - as promised,
here's the set of patches for the things I ran into.

Thanks,

Tom

Tom Zanussi (10):
  srt_commit: Fix --release-candidate arg misspelling
  srt_util: Make the 'configuration not found' message more informative
  srt_announce: Add an option to suppress git send-email auto-cc
  srt_announce: Allow rc releases to also reference release artifacts
  srt_create: Use old_tag.base, not old_tag, to create patch for rc
  announce-srt: Remove {major}.{minor} from upload paths
  srt_upload: Upload incremental update patch to prj_dir/incr, not
    prj_dir/older
  srt_announce: Add "new_tag_rc" for rc incremental releases
  announce-srt: Remove explicit v4.x from upload paths
  announce-srt-rc: Add tarball links

 announce-srt-rc.txt | 13 +++++++++++++
 announce-srt.txt    |  8 ++++----
 srt_announce.py     | 53 +++++++++++++++++++++++++++++++++--------------------
 srt_commit.py       |  4 ++--
 srt_create.py       |  2 +-
 srt_upload.py       | 11 ++++++++---
 srt_util.py         |  2 +-
 7 files changed, 62 insertions(+), 31 deletions(-)

-- 
2.14.1

[PATCH 01/10] srt_commit: Fix --release-candidate arg misspelling

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Fix misspelling of the current --release-canditate to make it the
intended --release-candidate.

Not because the misspelling itself is so bad, but so the user doesn't
have to remember to misspell it to use it ;-)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_commit.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/srt_commit.py b/srt_commit.py
index 9752a3f..8260513 100644
--- a/srt_commit.py
+++ b/srt_commit.py
@@ -114,7 +114,7 @@ def commit(config, rc):
 
 def add_argparser(parser):
     prs = parser.add_parser('commit')
-    prs.add_argument('--release-canditate', '-r',
+    prs.add_argument('--release-candidate', '-r',
                      default=None, metavar='N', type=int)
     return prs
 
@@ -123,4 +123,4 @@ def execute(args, config, ctx):
     if args.cmd != 'commit':
         return False
 
-    commit(config, args.release_canditate)
+    commit(config, args.release_candidate)
-- 
2.14.1

[PATCH 02/10] srt_util: Make the 'configuration not found' message more informative

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

When a command such as srt commit can't find an entry in srt.conf
corresponding to the current repo/branch, it errors out with:

  ERROR:root:Could not retrieve configuration

The message makes it sound like it couldn't find the srt.conf file,
but in reality that wasn't the problem - it couldn't find a group
matching repo/branch.  Add that detail to the message as such:

ERROR:root:Could not retrieve configuration
           stable-rt/linux-stable-rt/v3.18-rt-next from srt.conf

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_util.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/srt_util.py b/srt_util.py
index b572e03..c6099fd 100644
--- a/srt_util.py
+++ b/srt_util.py
@@ -97,7 +97,7 @@ def get_config():
         debug('Using configuration {0}'.format(config_name))
         config = read_config()[config_name]
     except:
-        error('Could not retrieve configuration')
+        error('Could not retrieve configuration {0} from srt.conf'.format(config_name))
         sys.exit(1)
 
     return config
-- 
2.14.1

[PATCH 03/10] srt_announce: Add an option to suppress git send-email auto-cc

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Sometimes it's useful to be able to do a 'real' dry run i.e. actually
sending out -rc patches and cover letter to a real e-mail address and
nowhere else.

Doing this with the default auto-cc defeats the purpose though, so add
an option to allow disabling it from the command line.

Example usage:

 srt -d announce -s v3.18.138-rt115 v3.18.138-rt116-rc1

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_announce.py | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/srt_announce.py b/srt_announce.py
index 3a89ef6..cc00412 100644
--- a/srt_announce.py
+++ b/srt_announce.py
@@ -71,8 +71,10 @@ def write_rc_cover_letter(config, ctx):
         f.write(coverletter)
 
 
-def send_rc_patches(config, ctx):
+def send_rc_patches(config, ctx, args):
     gmd = ['git', 'send-email', '--confirm=never']
+    if args.suppress_cc:
+        gmd += ['--suppress-cc=all']
     gmd += ['--to="{}"'.format(t) for t in config['MAIL_TO'].split(',')]
     gmd += [ctx.new_dir_mails]
 
@@ -84,15 +86,15 @@ def send_rc_patches(config, ctx):
         cmd(gmd)
 
 
-def announce_rc(config, ctx):
+def announce_rc(config, ctx, args):
     create_rc_patches(config, ctx)
     write_rc_cover_letter(config, ctx)
-    send_rc_patches(config, ctx)
+    send_rc_patches(config, ctx, args)
 
 
-def announce(config, ctx):
+def announce(config, ctx, args):
     if ctx.is_rc:
-        announce_rc(config, ctx)
+        announce_rc(config, ctx, args)
         return
 
     # rfc2822.html
@@ -134,6 +136,8 @@ def add_argparser(parser):
     prs = parser.add_parser('announce')
     prs.add_argument('OLD_TAG')
     prs.add_argument('NEW_TAG')
+    prs.add_argument('--suppress-cc', '-s', action="store_true", default=False,
+                     help='Don''t auto-cc anyone (for testing)')
     return prs
 
 
@@ -141,4 +145,4 @@ def execute(args, config, ctx):
     if args.cmd != 'announce':
         return False
 
-    announce(config, ctx)
+    announce(config, ctx, args)
-- 
2.14.1

[PATCH 04/10] srt_announce: Allow rc releases to also reference release artifacts

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

The code to generate the rc cover letter assumes there won't be any
references to release tarballs, which isn't the case for everyone, and
so doesn't pass in a lot of the format strings to format().

To change this, create a common map and have the cover-generating code
use that, after appending its own specific strings.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_announce.py | 36 ++++++++++++++++++++++--------------
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/srt_announce.py b/srt_announce.py
index cc00412..a1985cd 100644
--- a/srt_announce.py
+++ b/srt_announce.py
@@ -47,6 +47,16 @@ def create_rc_patches(config, ctx):
     cmd(['git', 'checkout', branch_name])
     cmd(['git', 'branch', '-D', 'next-tmp'])
 
+def cover_letter_replacements(config, ctx):
+    r = {"mail_to" : config['MAIL_TO'],
+         "major" : ctx.new_tag.major,
+         "minor" : ctx.new_tag.minor,
+         "patch" : ctx.new_tag.patch,
+         "new_version" : ctx.new_short_tag,
+         "old_version" : ctx.old_short_tag,
+         "prj_dir" : config['PRJ_DIR'],
+         "new_tag_rt" : ctx.new_tag.rt }
+    return r
 
 def write_rc_cover_letter(config, ctx):
     with open(ctx.new_dir_mails + '/0000-cover-letter.patch', 'r') as f:
@@ -59,10 +69,13 @@ def write_rc_cover_letter(config, ctx):
     with open(config['RC_TEXT'], 'r') as f:
         rc_text = f.read()
 
+    r = cover_letter_replacements(config, ctx)
+
     today = date.today()
     rc_date = today + timedelta(weeks=1)
-    rc_text = rc_text.format(new_version=ctx.new_tag,
-                             release_date=rc_date)
+    r["release_date"] = rc_date
+
+    rc_text = rc_text.format(**r)
 
     coverletter = coverletter.replace('*** BLURB HERE ***',
                                       rc_text)
@@ -105,18 +118,13 @@ def announce(config, ctx, args):
     with open(config['ANNOUNCE'], 'r') as f:
         stable_rt_text = f.read()
 
-    print(stable_rt_text.format(
-        date=timestamp,
-        mail_to=config['MAIL_TO'],
-        branch_name=get_remote_branch_name(),
-        branch_head=cmd(['git', 'rev-parse', 'HEAD']),
-        major=ctx.new_tag.major,
-        minor=ctx.new_tag.minor,
-        patch=ctx.new_tag.patch,
-        new_version=ctx.new_short_tag,
-        old_version=ctx.old_short_tag,
-        prj_dir=config['PRJ_DIR'],
-        new_tag_rt=ctx.new_tag.rt))
+    r = cover_letter_replacements(config, ctx)
+
+    r["date"] = timestamp
+    r["branch_name"] = get_remote_branch_name()
+    r["branch_head"] = cmd(['git', 'rev-parse', 'HEAD'])
+
+    print(stable_rt_text.format(**r))
 
     print(cmd(['git', '--no-pager', 'shortlog', '{0}..{1}'.
                format(ctx.old_tag, ctx.new_tag)]))
-- 
2.14.1

[PATCH 05/10] srt_create: Use old_tag.base, not old_tag, to create patch for rc

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Doing the diff between old_tag and new_tag just produces another
incremental diff instead of the expected rt patch plus including the
rc patches e.g.

  srt create v3.18.138-rt115 v3.18.138-rt116-rc1

should create the diff between the base v3.18.138 and
v3.18.138-rt116-rc1 and not between v3.18.138-rt115 and
v3.18.138-rt116-rc1.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_create.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/srt_create.py b/srt_create.py
index 90d1e1b..04833fe 100644
--- a/srt_create.py
+++ b/srt_create.py
@@ -64,7 +64,7 @@ def create(config, ctx):
             os.makedirs(d)
 
     if ctx.new_tag.is_rc:
-        create_patch_file(ctx.old_tag, str(ctx.new_tag), ctx.new_fln_patch)
+        create_patch_file(ctx.old_tag.base, str(ctx.new_tag), ctx.new_fln_patch)
         create_series(ctx.old_tag, ctx.new_tag, ctx.new_dir_series)
     else:
         create_patch_file(ctx.new_tag.base, str(ctx.new_tag), ctx.new_fln_patch)
-- 
2.14.1

[PATCH 06/10] announce-srt: Remove {major}.{minor} from upload paths

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Because PRJ_DIR already has the major.minor in the path, adding the
explicit {major}.{minor} after {prj_dir} duplicates that and creates
invalid links, so remove it.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 announce-srt.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/announce-srt.txt b/announce-srt.txt
index 631f942..912affe 100644
--- a/announce-srt.txt
+++ b/announce-srt.txt
@@ -20,12 +20,12 @@ Or to build {new_version} directly, the following patches should be applied:
 
   https://www.kernel.org/pub/linux/kernel/v4.x/patch-{major}.{minor}.{patch}.xz
 
-  https://www.kernel.org{prj_dir}/{major}.{minor}/patch-{new_version}.patch.xz
+  https://www.kernel.org{prj_dir}/patch-{new_version}.patch.xz
 
 
 You can also build from {old_version} by applying the incremental patch:
 
-  https://www.kernel.org{prj_dir}/{major}.{minor}/incr/patch-{old_version}-rt{new_tag_rt}.patch.xz
+  https://www.kernel.org{prj_dir}/incr/patch-{old_version}-rt{new_tag_rt}.patch.xz
 
 Enjoy!
    Daniel
-- 
2.14.1

[PATCH 07/10] srt_upload: Upload incremental update patch to prj_dir/incr, not prj_dir/older

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

The incremental patch should be uploaded to prj_dir/incr - currently
it's being uploaded to prj_dir/older, which also results in a broken
link in the generated cover letter.

This fix assumes the incremental file name is at position 2 in the
file list, which the other code in srt_upload also assumes.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_upload.py | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/srt_upload.py b/srt_upload.py
index 54da282..500f412 100644
--- a/srt_upload.py
+++ b/srt_upload.py
@@ -39,13 +39,18 @@ def upload(config, ctx):
 
     path = config['PRJ_DIR']
     older_path = path + '/' + 'older'
+    incr_path = path + '/' + 'incr'
 
     kup = ['kup']
 
-    for f in ctx.get_files():
+    for i, f in enumerate(ctx.get_files()):
         basename = os.path.splitext(f)[0]
-        kup.extend(['put', basename + '.xz', basename + '.sign',
-                    older_path + '/', '--'])
+        if i == 2:
+            kup.extend(['put', basename + '.xz', basename + '.sign',
+                        incr_path + '/', '--'])
+        else:
+            kup.extend(['put', basename + '.xz', basename + '.sign',
+                        older_path + '/', '--'])
 
     # skip incr_file
     for f in ctx.get_files()[:2]:
-- 
2.14.1

[PATCH 08/10] srt_announce: Add "new_tag_rc" for rc incremental releases

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

For rc releases, the filename generated for the incremental release
ends with -rt{new_tag_rt}-rc{new_tag_rc}.patch.xz rather than just
-rt{new_tag_rt}.patch.xz, so we need to make new_tag_rc available for
that.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 srt_announce.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/srt_announce.py b/srt_announce.py
index a1985cd..cbaa3f3 100644
--- a/srt_announce.py
+++ b/srt_announce.py
@@ -55,7 +55,8 @@ def cover_letter_replacements(config, ctx):
          "new_version" : ctx.new_short_tag,
          "old_version" : ctx.old_short_tag,
          "prj_dir" : config['PRJ_DIR'],
-         "new_tag_rt" : ctx.new_tag.rt }
+         "new_tag_rt" : ctx.new_tag.rt,
+         "new_tag_rc" : ctx.new_tag.rc }
     return r
 
 def write_rc_cover_letter(config, ctx):
-- 
2.14.1

[PATCH 09/10] announce-srt: Remove explicit v4.x from upload paths

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Other major kernel versions are still being maintained (3.x) or will
be soon maintained (5.x) so remove the explicit v4.x from the paths.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 announce-srt.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/announce-srt.txt b/announce-srt.txt
index 912affe..4b0950d 100644
--- a/announce-srt.txt
+++ b/announce-srt.txt
@@ -16,9 +16,9 @@ You can get this release via the git tree at:
 
 Or to build {new_version} directly, the following patches should be applied:
 
-  https://www.kernel.org/pub/linux/kernel/v4.x/linux-{major}.{minor}.tar.xz
+  https://www.kernel.org/pub/linux/kernel/v{major}.x/linux-{major}.{minor}.tar.xz
 
-  https://www.kernel.org/pub/linux/kernel/v4.x/patch-{major}.{minor}.{patch}.xz
+  https://www.kernel.org/pub/linux/kernel/v{major}.x/patch-{major}.{minor}.{patch}.xz
 
   https://www.kernel.org{prj_dir}/patch-{new_version}.patch.xz
 
-- 
2.14.1

[PATCH 10/10] announce-srt-rc: Add tarball links

[Tom Zanussi]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

There should be links in the cover letter pointing to the rc release
tarballs just as for any other release, so add them.

This also adds the -rc{new_tag_rc} suffix added by the previous patch
in the series.

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
---
 announce-srt-rc.txt | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/announce-srt-rc.txt b/announce-srt-rc.txt
index ca08c4c..89b69d9 100644
--- a/announce-srt-rc.txt
+++ b/announce-srt-rc.txt
@@ -15,6 +15,19 @@ final release is.
 If all goes well, this patch will be converted to the next main
 release on {release_date}.
 
+To build {new_version} directly, the following patches should be applied:
+
+  https://www.kernel.org/pub/linux/kernel/v{major}.x/linux-{major}.{minor}.tar.xz
+
+  https://www.kernel.org/pub/linux/kernel/v{major}.x/patch-{major}.{minor}.{patch}.xz
+
+  https://www.kernel.org{prj_dir}/patch-{new_version}.patch.xz
+
+
+You can also build from {old_version} by applying the incremental patch:
+
+  https://www.kernel.org{prj_dir}/incr/patch-{old_version}-rt{new_tag_rt}-rc{new_tag_rc}.patch.xz
+
 Enjoy,
 
 -- Daniel
-- 
2.14.1

Re: [PATCH 00/10] Some more stable-rt-tools fixes

[Daniel Wagner]

Hi Tom,

On 5/2/19 9:33 PM, Tom Zanussi wrote:
> I just did my first srt tools backport release for 3.18 - as promised,
> here's the set of patches for the things I ran into.

Thanks for the fixes. All patches applied. I updated the unit test to 
matches the changes. Everything looks good!

Thanks,
Daniel