[filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

[filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

[Markus Heidelberg]

SVN revision numbers from svn:externals property, which are a multiple
of 1024 (2^10), are transformed by SubGit to contain a binary suffix
("k", "m" and "g" have been checked) in .gitsvnextmodules file.
These aren't valid revision numbers in SVN either.

Examples:
  1024 -> 1k
  2048 -> 2k
  1048576 -> 1m
  1049600 -> 1025k
  1073741824 -> 1g

This led to the following error:
    svn_rev = int(parsed_config[section]['revision'])
ValueError: invalid literal for int() with base 10: '1k'

Signed-off-by: Markus Heidelberg <markus.heidelberg@web.de>
---
 contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals
index 0c81507..39ff288 100644
--- a/contrib/filter-repo-demos/convert-svnexternals
+++ b/contrib/filter-repo-demos/convert-svnexternals
@@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):

   return True, svnext_url

+def parse_revision_value(value):
+  """
+  Parse the value of key 'revision' from a .gitsvnextmodules file and return it
+  as integer.
+
+  Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit
+  instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...
+  """
+  suffix = value[-1]
+  if suffix in "kmg":
+    mult = {"k": 1024, "m": 1024**2, "g": 1024**3}
+    return int(value[0:-1]) * mult[suffix]
+  else:
+    return int(value)
+
 def add_submodule_tree_entry(commit, parsed_config, section):
   """
   Add a submodule entry to the tree of a Git commit.
@@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):

   # Get SVN revision
   if parsed_config.has_option(section, 'revision'):
-    svn_rev = int(parsed_config[section]['revision'])
+    svn_rev = parse_revision_value(parsed_config[section]['revision'])
   else:
     # TODO: revision has to be guessed according to commit timestamp, skip for now
     return False
--
2.40.0

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

[Elijah Newren]

On Wed, Apr 12, 2023 at 6:42 AM Markus Heidelberg
<markus.heidelberg@web.de> wrote:
>
> SVN revision numbers from svn:externals property, which are a multiple
> of 1024 (2^10), are transformed by SubGit to contain a binary suffix
> ("k", "m" and "g" have been checked) in .gitsvnextmodules file.
> These aren't valid revision numbers in SVN either.
>
> Examples:
>   1024 -> 1k
>   2048 -> 2k
>   1048576 -> 1m
>   1049600 -> 1025k
>   1073741824 -> 1g
>
> This led to the following error:
>     svn_rev = int(parsed_config[section]['revision'])
> ValueError: invalid literal for int() with base 10: '1k'
>
> Signed-off-by: Markus Heidelberg <markus.heidelberg@web.de>
> ---
>  contrib/filter-repo-demos/convert-svnexternals | 17 ++++++++++++++++-
>  1 file changed, 16 insertions(+), 1 deletion(-)
>
> diff --git a/contrib/filter-repo-demos/convert-svnexternals b/contrib/filter-repo-demos/convert-svnexternals
> index 0c81507..39ff288 100644
> --- a/contrib/filter-repo-demos/convert-svnexternals
> +++ b/contrib/filter-repo-demos/convert-svnexternals
> @@ -254,6 +254,21 @@ def get_absolute_svn_url(svnext_url, svn_root_url):
>
>    return True, svnext_url
>
> +def parse_revision_value(value):
> +  """
> +  Parse the value of key 'revision' from a .gitsvnextmodules file and return it
> +  as integer.
> +
> +  Used to handle non-numeric values like 1k, 2k, 3k etc. added by SubGit
> +  instead of 1024, 2048, 3072 etc., likewise 1m, 2m, ..., 1g, ...
> +  """
> +  suffix = value[-1]
> +  if suffix in "kmg":
> +    mult = {"k": 1024, "m": 1024**2, "g": 1024**3}
> +    return int(value[0:-1]) * mult[suffix]
> +  else:
> +    return int(value)
> +
>  def add_submodule_tree_entry(commit, parsed_config, section):
>    """
>    Add a submodule entry to the tree of a Git commit.
> @@ -271,7 +286,7 @@ def add_submodule_tree_entry(commit, parsed_config, section):
>
>    # Get SVN revision
>    if parsed_config.has_option(section, 'revision'):
> -    svn_rev = int(parsed_config[section]['revision'])
> +    svn_rev = parse_revision_value(parsed_config[section]['revision'])
>    else:
>      # TODO: revision has to be guessed according to commit timestamp, skip for now
>      return False
> --
> 2.40.0

Thanks for sending this in!  Applied.

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

[Junio C Hamano]

Elijah Newren <newren@gmail.com> writes:

> Thanks for sending this in!  Applied.

It may work around the issue, but sounds like it is SubGit that
needs fixing, at least seeing the problem description from the
sideline.

Re: [filter-repo PATCH] convert-svnexternals: fix parsing of wrongly transformed SVN revisions

[Markus Heidelberg]

Am Mittwoch, 12. April 2023, 21:26:32 CEST schrieb Junio C Hamano:

> It may work around the issue, but sounds like it is SubGit that
> needs fixing, at least seeing the problem description from the
> sideline.

Indeed the commit log should have been something like
  "work around wrongly transformed SVN revisions"
to make it clear that it wasn't a bug in the script.

Thanks to your comment I have now reported this SubGit issue in the
TMate Support Forum.