All of lore.kernel.org
 help / color / mirror / Atom feed
From: Mike Crowe <mac@mcrowe.com>
To: openembedded-core@lists.openembedded.org
Cc: Mike Crowe <mac@mcrowe.com>
Subject: [PATCH] sstate: Truncate PV in sstate filenames that are too long
Date: Tue, 30 Jul 2019 12:01:11 +0100	[thread overview]
Message-ID: <20190730110111.5143-1-mac@mcrowe.com> (raw)

sstate filenames are generated by concatenating a variety of bits of
package metadata. Some of these parts could be long, which could cause
the filename to be longer than the 255 character maximum for ext4.

So, let's try to detect this situation and truncate the PV part of the
filename so that it will fit. If this happens, an ellipsis is added to
make it clear that the version number is incomplete.

SSTATE_PKG needs to be consistent for all tasks so that the hash
remains stable. This means that we need to make an assumption for the
maximum length of the task name. In this implementation, the task name
is limited to 27 characters.

This change also results in a sensible error message being emitted if
the resulting filename is still too long.

Signed-off-by: Mike Crowe <mac@mcrowe.com>

diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3342c5ef50..6313b1c538 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -8,6 +8,24 @@ def generate_sstatefn(spec, hash, d):
         hash = "INVALID"
     return hash[:2] + "/" + spec + hash
 
+def sstate_path(taskname, d):
+    max_filename_len = 245 # leave some room for ".siginfo"
+    max_addendum_len = 32 # '_' + taskname + '.tgz'
+    sstate_prefix = d.getVar('SSTATE_PKG')
+    excess = len(os.path.basename(sstate_prefix)) - (max_filename_len - max_addendum_len)
+    if excess > 0:
+        pv = d.getVar('PV')
+        if len(pv) >= excess and len(pv) >= 3:
+            short_pv = d.getVar('PV')[:-excess-3] + '...'
+            d2 = d.createCopy()
+            d2.setVar('PV', short_pv)
+            sstate_prefix = d2.getVar('SSTATE_PKG')
+
+    sstatepkg = sstate_prefix + '_'+ taskname + ".tgz"
+    if len(os.path.basename(sstatepkg)) > max_filename_len:
+        bb.error('Failed to shorten sstate filename')
+    return sstatepkg
+
 SSTATE_PKGARCH    = "${PACKAGE_ARCH}"
 SSTATE_PKGSPEC    = "sstate:${PN}:${PACKAGE_ARCH}${TARGET_VENDOR}-${TARGET_OS}:${PV}:${PR}:${SSTATE_PKGARCH}:${SSTATE_VERSION}:"
 SSTATE_SWSPEC     = "sstate:${PN}::${PV}:${PR}::${SSTATE_VERSION}:"
@@ -323,7 +341,7 @@ def sstate_installpkg(ss, d):
 
     sstateinst = d.expand("${WORKDIR}/sstate-install-%s/" % ss['task'])
     sstatefetch = d.getVar('SSTATE_PKGNAME') + '_' + ss['task'] + ".tgz"
-    sstatepkg = d.getVar('SSTATE_PKG') + '_' + ss['task'] + ".tgz"
+    sstatepkg = sstate_path(ss['task'], d)
 
     if not os.path.exists(sstatepkg):
         pstaging_fetch(sstatefetch, d)
@@ -617,7 +635,7 @@ def sstate_package(ss, d):
     tmpdir = d.getVar('TMPDIR')
 
     sstatebuild = d.expand("${WORKDIR}/sstate-build-%s/" % ss['task'])
-    sstatepkg = d.getVar('SSTATE_PKG') + '_'+ ss['task'] + ".tgz"
+    sstatepkg = sstate_path(ss['task'], d)
     bb.utils.remove(sstatebuild, recurse=True)
     bb.utils.mkdirhier(sstatebuild)
     bb.utils.mkdirhier(os.path.dirname(sstatepkg))
@@ -1084,8 +1102,8 @@ python sstate_eventhandler() {
         if taskname in ["fetch", "unpack", "patch", "populate_lic", "preconfigure"] and swspec:
             d.setVar("SSTATE_PKGSPEC", "${SSTATE_SWSPEC}")
             d.setVar("SSTATE_EXTRAPATH", "")
-        sstatepkg = d.getVar('SSTATE_PKG')
-        bb.siggen.dump_this_task(sstatepkg + '_' + taskname + ".tgz" ".siginfo", d)
+        sstateinfo = sstate_path(taskname, d) + ".siginfo"
+        bb.siggen.dump_this_task(sstateinfo, d)
 }
 
 SSTATE_PRUNE_OBSOLETEWORKDIR ?= "1"
-- 
2.20.1



             reply	other threads:[~2019-07-30 11:01 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-07-30 11:01 Mike Crowe [this message]
2019-07-30 13:25 ` [PATCH] sstate: Truncate PV in sstate filenames that are too long Mark Hatle
2019-07-30 13:49   ` Mike Crowe
2019-07-30 14:14     ` Mark Hatle
2019-07-31 11:13       ` Mike Crowe
2019-07-31 17:41       ` Mike Crowe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190730110111.5143-1-mac@mcrowe.com \
    --to=mac@mcrowe.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.