* [1.52][PATCH 1/3] tests/fetch.py: fix premirror test cases
2021-11-02 9:02 [1.52][PATCH 0/3] Review request Anuj Mittal
@ 2021-11-02 9:02 ` Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 2/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 3/3] fetch2: fix downloadfilename issue with premirror Anuj Mittal
2 siblings, 0 replies; 4+ messages in thread
From: Anuj Mittal @ 2021-11-02 9:02 UTC (permalink / raw)
To: bitbake-devel
From: Chen Qi <Qi.Chen@windriver.com>
When downloadfilename is specified, it is used to fetch from premirror.
So fix the test cases accordingly.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3b4d2e3b5024324058360a2a28f33c34114218d0)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
lib/bb/tests/fetch.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index 8ad1c8599..d06580db8 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -875,17 +875,17 @@ class FetcherNetworkTest(FetcherTest):
@skipIfNoNetwork()
def test_fetch_premirror_specify_downloadfilename_regex_uri(self):
self.d.setVar("PREMIRRORS", "http://.*/.* https://downloads.yoctoproject.org/releases/bitbake/")
- fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/1.0.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
fetcher.download()
- self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
@skipIfNoNetwork()
# BZ13039
def test_fetch_premirror_specify_downloadfilename_specific_uri(self):
self.d.setVar("PREMIRRORS", "http://invalid.yoctoproject.org/releases/bitbake https://downloads.yoctoproject.org/releases/bitbake")
- fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz;downloadfilename=bitbake-v1.0.0.tar.gz"], self.d)
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/1.0.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
fetcher.download()
- self.assertEqual(os.path.getsize(self.dldir + "/bitbake-v1.0.0.tar.gz"), 57749)
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
@skipIfNoNetwork()
def gitfetcher(self, url1, url2):
--
2.31.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [1.52][PATCH 2/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror
2021-11-02 9:02 [1.52][PATCH 0/3] Review request Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 1/3] tests/fetch.py: fix premirror test cases Anuj Mittal
@ 2021-11-02 9:02 ` Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 3/3] fetch2: fix downloadfilename issue with premirror Anuj Mittal
2 siblings, 0 replies; 4+ messages in thread
From: Anuj Mittal @ 2021-11-02 9:02 UTC (permalink / raw)
To: bitbake-devel
From: Chen Qi <Qi.Chen@windriver.com>
Add a test case test_fetch_premirror_use_downloadfilename_to_fetch to ensure
that 'downloadfilename' is used when fetching from premirror.
Although the other two previous test cases, test_fetch_premirror_specify_downloadfilename_regex_uri
and test_fetch_premirror_specify_downloadfilename_specific_uri already
implicitly contain such verification, we still need to add a very clear
case to ensure no regression.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 20aabc3d53f69949810ecf02295725db947ffef8)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
lib/bb/tests/fetch.py | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/lib/bb/tests/fetch.py b/lib/bb/tests/fetch.py
index d06580db8..a008e7cc5 100644
--- a/lib/bb/tests/fetch.py
+++ b/lib/bb/tests/fetch.py
@@ -887,6 +887,14 @@ class FetcherNetworkTest(FetcherTest):
fetcher.download()
self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
+ @skipIfNoNetwork()
+ def test_fetch_premirror_use_downloadfilename_to_fetch(self):
+ # Ensure downloadfilename is used when fetching from premirror.
+ self.d.setVar("PREMIRRORS", "http://.*/.* https://downloads.yoctoproject.org/releases/bitbake")
+ fetcher = bb.fetch.Fetch(["http://invalid.yoctoproject.org/releases/bitbake/bitbake-1.1.tar.gz;downloadfilename=bitbake-1.0.tar.gz"], self.d)
+ fetcher.download()
+ self.assertEqual(os.path.getsize(self.dldir + "/bitbake-1.0.tar.gz"), 57749)
+
@skipIfNoNetwork()
def gitfetcher(self, url1, url2):
def checkrevision(self, fetcher):
--
2.31.1
^ permalink raw reply related [flat|nested] 4+ messages in thread* [1.52][PATCH 3/3] fetch2: fix downloadfilename issue with premirror
2021-11-02 9:02 [1.52][PATCH 0/3] Review request Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 1/3] tests/fetch.py: fix premirror test cases Anuj Mittal
2021-11-02 9:02 ` [1.52][PATCH 2/3] tests/fetch.py: add test case to ensure downloadfilename is used for premirror Anuj Mittal
@ 2021-11-02 9:02 ` Anuj Mittal
2 siblings, 0 replies; 4+ messages in thread
From: Anuj Mittal @ 2021-11-02 9:02 UTC (permalink / raw)
To: bitbake-devel
From: Chen Qi <Qi.Chen@windriver.com>
The following commit to fix [Yocto #13039] causes regression of
the behavior of PREMIRRORS.
"bitbake: fetch2: fix premirror URI when downloadfilename defined"
Take meta-openembedded/meta-networking/recipes-protocols/freediameter/freediameter_1.4.0.bb
as an example.
SRC_URI = "\
http://www.freediameter.net/hg/${fd_pkgname}/archive/${PV}.tar.gz;downloadfilename=${fd_pkgname}-${PV}.tar.gz \
...
"
With the above commit, it now tries to fetch 1.4.0.tar.gz instead of
freeDiameter-1.4.0.tar.gz. This makes https://downloads.yoctoproject.org/mirror/sources
not work for freediameter, as it holds freeDiameter-1.4.0.tar.gz.
The commit above tries to avoid fetching from invalid url such as:
https://<some_mirror>/1.4.0.tar.gz/freeDiameter-1.4.0.tar.gz.
And its solution is to make basename to be 1.4.0.tar.gz, thus causing the
regression.
This patch fixes the above regression. For Yocto #13039, it now tries
to fetch from url: https://<some_mirror>/freeDiameter-1.4.0.tar.gz.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 96c30007dc0b32eee2b15771daec7948bc9bfd97)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
---
lib/bb/fetch2/__init__.py | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 259b2637a..8078991d7 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -466,9 +466,13 @@ def uri_replace(ud, uri_find, uri_replace, replacements, d, mirrortarball=None):
# Kill parameters, they make no sense for mirror tarballs
uri_decoded[5] = {}
elif ud.localpath and ud.method.supports_checksum(ud):
- basename = os.path.basename(uri_decoded[loc])
- if basename and not result_decoded[loc].endswith(basename):
- result_decoded[loc] = os.path.join(result_decoded[loc], basename)
+ basename = os.path.basename(ud.localpath)
+ if basename:
+ uri_basename = os.path.basename(uri_decoded[loc])
+ if basename != uri_basename and result_decoded[loc].endswith(uri_basename):
+ result_decoded[loc] = result_decoded[loc].replace(uri_basename, basename)
+ elif not result_decoded[loc].endswith(basename):
+ result_decoded[loc] = os.path.join(result_decoded[loc], basename)
else:
return None
result = encodeurl(result_decoded)
--
2.31.1
^ permalink raw reply related [flat|nested] 4+ messages in thread