* [PATCH 1/3] sstate.bbclass: kill MIRRORS and FILESPATH to speed up fetching
2013-06-10 20:46 [PATCH 0/3] sstate fetch improvements Christopher Larson
@ 2013-06-10 20:46 ` Christopher Larson
2013-06-10 20:50 ` Otavio Salvador
2013-06-10 20:46 ` [PATCH 2/3] sstate.bbclass: fetch archive+siginfo in a single pstaging_fetch Christopher Larson
2013-06-10 20:46 ` [PATCH 3/3] sstate.bbclass: brute force silence fetch errors Christopher Larson
2 siblings, 1 reply; 7+ messages in thread
From: Christopher Larson @ 2013-06-10 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Christopher Larson
From: Christopher Larson <chris_larson@mentor.com>
The default FILESPATH isn't really of use, as we don't expect to find sstate
archives buried in layer recipe directories, and the default MIRRORS is
intended for use for fetching SRC_URI, not sstate.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
meta/classes/sstate.bbclass | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index e74c642..3aa4371 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -508,6 +508,8 @@ def pstaging_fetch(sstatefetch, sstatepkg, d):
bb.mkdirhier(dldir)
+ localdata.delVar('MIRRORS')
+ localdata.delVar('FILESPATH')
localdata.setVar('DL_DIR', dldir)
localdata.setVar('PREMIRRORS', mirrors)
localdata.setVar('SRC_URI', srcuri)
--
1.8.3
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 2/3] sstate.bbclass: fetch archive+siginfo in a single pstaging_fetch
2013-06-10 20:46 [PATCH 0/3] sstate fetch improvements Christopher Larson
2013-06-10 20:46 ` [PATCH 1/3] sstate.bbclass: kill MIRRORS and FILESPATH to speed up fetching Christopher Larson
@ 2013-06-10 20:46 ` Christopher Larson
2013-06-10 20:52 ` Otavio Salvador
2013-06-10 20:46 ` [PATCH 3/3] sstate.bbclass: brute force silence fetch errors Christopher Larson
2 siblings, 1 reply; 7+ messages in thread
From: Christopher Larson @ 2013-06-10 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Christopher Larson
From: Christopher Larson <chris_larson@mentor.com>
This avoids unnecessary duplication of setup. The only visible change in
behavior will be the case if siginfo exists and the archive does not, in which
case it'll redownload both, but this doesn't seem unresasonable to me,
particularly since the archive is downloaded first, making this case
particularly unlikely.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
meta/classes/sstate.bbclass | 32 +++++++++++++++-----------------
1 file changed, 15 insertions(+), 17 deletions(-)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3aa4371..28dc312 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -224,8 +224,6 @@ def sstate_installpkg(ss, d):
if not os.path.exists(sstatepkg):
pstaging_fetch(sstatefetch, sstatepkg, d)
- if not os.path.exists(sstatepkg + ".siginfo"):
- pstaging_fetch(sstatefetch + ".siginfo", sstatepkg + ".siginfo", d)
if not os.path.isfile(sstatepkg):
bb.note("Staging package %s does not exist" % sstatepkg)
@@ -504,30 +502,30 @@ def pstaging_fetch(sstatefetch, sstatepkg, d):
bb.data.update_data(localdata)
dldir = localdata.expand("${SSTATE_DIR}")
- srcuri = "file://" + sstatefetch
-
bb.mkdirhier(dldir)
localdata.delVar('MIRRORS')
localdata.delVar('FILESPATH')
localdata.setVar('DL_DIR', dldir)
localdata.setVar('PREMIRRORS', mirrors)
- localdata.setVar('SRC_URI', srcuri)
# Try a fetch from the sstate mirror, if it fails just return and
# we will build the package
- try:
- fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
- fetcher.download()
-
- # Need to optimise this, if using file:// urls, the fetcher just changes the local path
- # For now work around by symlinking
- localpath = bb.data.expand(fetcher.localpath(srcuri), localdata)
- if localpath != sstatepkg and os.path.exists(localpath) and not os.path.exists(sstatepkg):
- os.symlink(localpath, sstatepkg)
-
- except bb.fetch2.BBFetchException:
- pass
+ for srcuri in ['file://{0}'.format(sstatefetch),
+ 'file://{0}.siginfo'.format(sstatefetch)]:
+ localdata.setVar('SRC_URI', srcuri)
+ try:
+ fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
+ fetcher.download()
+
+ # Need to optimise this, if using file:// urls, the fetcher just changes the local path
+ # For now work around by symlinking
+ localpath = bb.data.expand(fetcher.localpath(srcuri), localdata)
+ if localpath != sstatepkg and os.path.exists(localpath) and not os.path.exists(sstatepkg):
+ os.symlink(localpath, sstatepkg)
+
+ except bb.fetch2.BBFetchException:
+ break
def sstate_setscene(d):
shared_state = sstate_state_fromvars(d)
--
1.8.3
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH 3/3] sstate.bbclass: brute force silence fetch errors
2013-06-10 20:46 [PATCH 0/3] sstate fetch improvements Christopher Larson
2013-06-10 20:46 ` [PATCH 1/3] sstate.bbclass: kill MIRRORS and FILESPATH to speed up fetching Christopher Larson
2013-06-10 20:46 ` [PATCH 2/3] sstate.bbclass: fetch archive+siginfo in a single pstaging_fetch Christopher Larson
@ 2013-06-10 20:46 ` Christopher Larson
2013-06-10 20:53 ` Otavio Salvador
2 siblings, 1 reply; 7+ messages in thread
From: Christopher Larson @ 2013-06-10 20:46 UTC (permalink / raw)
To: openembedded-core; +Cc: Christopher Larson
From: Christopher Larson <chris_larson@mentor.com>
bb.fetch2.Fetcher.download() prints any fetch errors that come from the
underlying fetcher, rather than letting the caller do so, which means there's
no way to silence the fetch error messages without manipulating the loggers.
So we do so. The reason for doing this is that the user doesn't care if we
failed to fetch an sstate archive, the failure mode is just to rerun the real
task anyway.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
---
meta/classes/sstate.bbclass | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 28dc312..5b9d86e 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -491,6 +491,7 @@ def sstate_package(ss, d):
def pstaging_fetch(sstatefetch, sstatepkg, d):
import bb.fetch2
+ import logging
# Only try and fetch if the user has configured a mirror
mirrors = d.getVar('SSTATE_MIRRORS', True)
@@ -514,6 +515,8 @@ def pstaging_fetch(sstatefetch, sstatepkg, d):
for srcuri in ['file://{0}'.format(sstatefetch),
'file://{0}.siginfo'.format(sstatefetch)]:
localdata.setVar('SRC_URI', srcuri)
+ oldlevel = bb.fetch2.logger.level
+ bb.fetch2.logger.setLevel(logging.CRITICAL)
try:
fetcher = bb.fetch2.Fetch([srcuri], localdata, cache=False)
fetcher.download()
@@ -526,6 +529,8 @@ def pstaging_fetch(sstatefetch, sstatepkg, d):
except bb.fetch2.BBFetchException:
break
+ finally:
+ bb.fetch2.logger.setLevel(oldlevel)
def sstate_setscene(d):
shared_state = sstate_state_fromvars(d)
--
1.8.3
^ permalink raw reply related [flat|nested] 7+ messages in thread