public inbox for openembedded-core@lists.openembedded.org
 help / color / mirror / Atom feed
From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate
Date: Wed, 20 Sep 2023 11:57:59 +0100	[thread overview]
Message-ID: <20230920105802.1008778-3-richard.purdie@linuxfoundation.org> (raw)
In-Reply-To: <20230920105802.1008778-1-richard.purdie@linuxfoundation.org>

When originally implemented, overlapping symlinks in DEPLOY_DIR were common. That
is no longer the case and these overlapping links are causing bugs in other areas
(e.g. bug 14123).

Therefore start showing errors for overlapping symlinks in shared areas. Whilst here,
fix a broken file reference in the grep command to match current file layouts and
update the message shown to users to match current times. Most of the message content
is obsolete now due to other advances and changes in the way the staging code
now works.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes-global/sstate.bbclass | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)

diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass
index 706c2ae9388..afcda2980b3 100644
--- a/meta/classes-global/sstate.bbclass
+++ b/meta/classes-global/sstate.bbclass
@@ -266,7 +266,7 @@ def sstate_install(ss, d):
     overlap_allowed = (d.getVar("SSTATE_ALLOW_OVERLAP_FILES") or "").split()
     match = []
     for f in sharedfiles:
-        if os.path.exists(f) and not os.path.islink(f):
+        if os.path.exists(f):
             f = os.path.normpath(f)
             realmatch = True
             for w in overlap_allowed:
@@ -276,36 +276,18 @@ def sstate_install(ss, d):
                     break
             if realmatch:
                 match.append(f)
-                sstate_search_cmd = "grep -rlF '%s' %s --exclude=master.list | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
+                sstate_search_cmd = "grep -rlF '%s' %s --exclude=index-* | sed -e 's:^.*/::'" % (f, d.expand("${SSTATE_MANIFESTS}"))
                 search_output = subprocess.Popen(sstate_search_cmd, shell=True, stdout=subprocess.PIPE).communicate()[0]
                 if search_output:
                     match.append("  (matched in %s)" % search_output.decode('utf-8').rstrip())
                 else:
                     match.append("  (not matched to any task)")
     if match:
-        bb.error("The recipe %s is trying to install files into a shared " \
-          "area when those files already exist. Those files and their manifest " \
-          "location are:\n  %s\nPlease verify which recipe should provide the " \
-          "above files.\n\nThe build has stopped, as continuing in this scenario WILL " \
-          "break things - if not now, possibly in the future (we've seen builds fail " \
-          "several months later). If the system knew how to recover from this " \
-          "automatically it would, however there are several different scenarios " \
-          "which can result in this and we don't know which one this is. It may be " \
-          "you have switched providers of something like virtual/kernel (e.g. from " \
-          "linux-yocto to linux-yocto-dev), in that case you need to execute the " \
-          "clean task for both recipes and it will resolve this error. It may be " \
-          "you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning " \
-          "those recipes should again resolve this error, however switching " \
-          "DISTRO_FEATURES on an existing build directory is not supported - you " \
-          "should really clean out tmp and rebuild (reusing sstate should be safe). " \
-          "It could be the overlapping files detected are harmless in which case " \
-          "adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could " \
-          "also be your build is including two different conflicting versions of " \
-          "things (e.g. bluez 4 and bluez 5 and the correct solution for that would " \
-          "be to resolve the conflict. If in doubt, please ask on the mailing list, " \
-          "sharing the error and filelist above." % \
+        bb.fatal("Recipe %s is trying to install files into a shared " \
+          "area when those files already exist. The files and the manifests listing " \
+          "them are:\n  %s\n"
+          "Please adjust the recipes so only one recipe provides a given file. " % \
           (d.getVar('PN'), "\n  ".join(match)))
-        bb.fatal("If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.")
 
     if ss['fixmedir'] and os.path.exists(ss['fixmedir'] + "/fixmepath.cmd"):
         sharedfiles.append(ss['fixmedir'] + "/fixmepath.cmd")
-- 
2.39.2



  parent reply	other threads:[~2023-09-20 10:58 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-09-20 10:57 [PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files Richard Purdie
2023-09-20 10:57 ` [PATCH 2/6 v2] create-spdx/sbom: Ensure files don't overlap between machines Richard Purdie
2023-09-20 10:57 ` Richard Purdie [this message]
2023-09-24  9:14   ` [OE-core] [PATCH 3/6] sstate: Stop allowing overlapping symlinks from sstate Martin Jansa
2023-09-24 10:10     ` Richard Purdie
2023-09-25 14:59     ` Richard Purdie
2023-09-25 17:19       ` Martin Jansa
2023-09-20 10:58 ` [PATCH 4/6] sstate: Fix nativesdk entry in SSTATE_ARCHS Richard Purdie
2023-09-20 10:58 ` [PATCH 5/6 v2] multilib: fix SSTATE_ARCHS for multilib usage Richard Purdie
2023-09-20 10:58 ` [PATCH 6/6] oeqa/selftest/bbtests: Improve and update test_non_gplv3 Richard Purdie
2023-12-19  1:11 ` [OE-core] [PATCH 1/6 v2] license/license_image: Fix license file layout to avoid overlapping files Russ Dill
2024-07-17 20:29   ` Livius

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230920105802.1008778-3-richard.purdie@linuxfoundation.org \
    --to=richard.purdie@linuxfoundation.org \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox