[PATCH] bitbake: Resolve symlink duplicates in layer paths

[PATCH] bitbake: Resolve symlink duplicates in layer paths

[Zk47T]

When BBLAYERS contains two paths pointing to the same directory via
symlinks, bitbake incorrectly reports duplicated BBFILE_COLLECTIONS.
Use os.path.realpath() to resolve symlinks before comparing layer
paths, and emit a warning instead of a fatal error.
Fixes [YOCTO #15591]

Signed-off-by: Zk47T <zizuzacker@gmail.com>
---
 bitbake/lib/bb/cookerdata.py   | 14 ++++++++++++++
 bitbake/lib/bb/utils.py        |  3 +++
 bitbake/lib/bblayers/action.py |  2 +-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/bb/cookerdata.py b/bitbake/lib/bb/cookerdata.py
index 0649e40995..823da5f4c4 100644
--- a/bitbake/lib/bb/cookerdata.py
+++ b/bitbake/lib/bb/cookerdata.py
@@ -370,6 +370,20 @@ class CookerDataBuilder(object):
             data = bb.data.createCopy(data)
             approved = bb.utils.approved_variables()
 
+            # Resolve symlinks in layer paths so that layers referenced
+            # via different paths (e.g. symlinks) are not treated as
+            # duplicates (see Bug 15591)
+            resolved_layers = []
+            seen_real = set()
+            for layer in layers:
+                real = os.path.realpath(layer)
+                if real not in seen_real:
+                    seen_real.add(real)
+                    resolved_layers.append(layer)
+                else:
+                    parselog.warning("Ignoring duplicate layer entry %s (resolves to %s which is already in BBLAYERS)", layer, real)
+            layers = resolved_layers
+
             # Check whether present layer directories exist
             for layer in layers:
                 if not os.path.isdir(layer):
diff --git a/bitbake/lib/bb/utils.py b/bitbake/lib/bb/utils.py
index 1b4fb93a30..1972c9864a 100644
--- a/bitbake/lib/bb/utils.py
+++ b/bitbake/lib/bb/utils.py
@@ -1441,6 +1441,9 @@ def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
         pth = remove_trailing_sep(pth)
         if 'HOME' in approved and '~' in pth:
             pth = os.path.expanduser(pth)
+        # Resolve symlinks for real paths, but not for glob patterns
+        if '*' not in pth and '?' not in pth:
+            pth = os.path.realpath(pth)
         return pth
 
     def layerlist_param(value):
diff --git a/bitbake/lib/bblayers/action.py b/bitbake/lib/bblayers/action.py
index a8f2699335..11cca74eb3 100644
--- a/bitbake/lib/bblayers/action.py
+++ b/bitbake/lib/bblayers/action.py
@@ -26,7 +26,7 @@ def plugin_init(plugins):
 class ActionPlugin(LayerPlugin):
     def do_add_layer(self, args):
         """Add one or more layers to bblayers.conf."""
-        layerdirs = [os.path.abspath(ldir) for ldir in args.layerdir]
+        layerdirs = [os.path.realpath(ldir) for ldir in args.layerdir]
 
         for layerdir in layerdirs:
             if not os.path.exists(layerdir):
-- 
2.34.1

Re: [bitbake-devel] [PATCH] bitbake: Resolve symlink duplicates in layer paths

[Richard Purdie]

On Fri, 2026-04-03 at 22:33 +0700, Zk47T via lists.openembedded.org wrote:
> When BBLAYERS contains two paths pointing to the same directory via
> symlinks, bitbake incorrectly reports duplicated BBFILE_COLLECTIONS.
> Use os.path.realpath() to resolve symlinks before comparing layer
> paths, and emit a warning instead of a fatal error.
> Fixes [YOCTO #15591]
> 
> Signed-off-by: Zk47T <zizuzacker@gmail.com>
> ---
>  bitbake/lib/bb/cookerdata.py   | 14 ++++++++++++++
>  bitbake/lib/bb/utils.py        |  3 +++
>  bitbake/lib/bblayers/action.py |  2 +-
>  3 files changed, 18 insertions(+), 1 deletion(-)

Why would you want to add the same layer twice, even if they are via
different paths?

I don't really see why this should be a warning...

Cheers,

Richard

Re: [PATCH] bitbake: Resolve symlink duplicates in layer paths

[Zk47T]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

The user doesn't intentionally add the same layer twice — this happens when the working directory uses a symlink path (e.g., /home/pokybuild → /srv/ pokybuild ). Running bitbake-layers add-layer /srv/pokybuild/.../meta-mingw resolves to a different string than what's already in bblayers.conf (which was set via the symlink path), causing ERROR: Found duplicated BBFILE_COLLECTIONS.

The main fix is using os.path.realpath() inste ad of os.path.abspath() in do_ add_layer() and canonicalise_ path() , which prevents the duplicate from being added in the first place.

Regarding the warning in cookerdata.py — I can remove it if you prefer silent deduplication. I added it for visibility when users have manually edited bblayers.conf with symlink paths. Would you prefer it to silently resolve duplicates instead?

This behavior as you shared in root ticket bugzilla https://bugzilla.yoctoproject. org/show_bug.cgi?id=15591 ( https://bugzilla.yoctoproject.org/show_bug.cgi?id=15591 )

I try to implement the fix

[-- Attachment #2: Type: text/html, Size: 4716 bytes --]