[PATCH v1 0/2] fs/squashfs: fix symlink load failure on large images

[PATCH v1 0/2] fs/squashfs: fix symlink load failure on large images

[Allan ELKAIM]

sqfsload fails to load a file through a symlink when the squashfs
image contains a large number of inodes (e.g. a rootfs that includes
the tzdata timezone database).

Root cause: sqfs_read_nest() resolves the symlink by calling itself
recursively without first freeing the parent directory's inode and
directory table buffers. This causes a temporary double allocation
that can exhaust the U-Boot heap. When malloc() subsequently fails
inside sqfs_read_directory_table(), the error goes undetected and
sqfs_search_dir() is called with a NULL pos_list pointer, leading to:

  Error: invalid inode reference to directory table.
  Failed to load '/boot/Image'

Patch 1 fixes the structural problem (temporary double allocation)
and plugs the silent NULL pointer path in sqfs_read_directory_table().
Patch 2 adds the missing return-value checks on sqfs_dir_offset() that
turn any residual lookup failure into a clean error propagation.

Both patches are independent and can be reviewed separately.

The bug was first observed on U-Boot v2024.01 and is still present
on v2026.04. The patches have been tested on a Raspberry Pi CM4
running U-Boot v2026.04 (Yocto Scarthgap 5.0.17) with a 325 MB
squashfs rootfs containing 22 517 inodes. The symlink
/boot/Image -> Image-6.6.63-v8 now resolves successfully.

This series addresses the bug reported at:
https://lists.denx.de/pipermail/u-boot/2026-May/618533.html


Allan ELKAIM (2):
  fs/squashfs: fix heap exhaustion during symlink resolution
  fs/squashfs: add sqfs_dir_offset() error checks

 fs/squashfs/sqfs.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

-- 
2.53.0

base-commit: 88dc2788777babfd6322fa655df549a019aa1e69

[PATCH v1 1/2] fs/squashfs: fix heap exhaustion during symlink resolution

[Allan ELKAIM]

When sqfs_read_nest() encounters a symlink it resolves it by calling
itself recursively. In the unfixed code this looks like:

  // dirsp is open: inode_table + dir_table still on heap
  resolved = sqfs_resolve_symlink(symlink, filename);
  ret = sqfs_read_nest(resolved, ...); // recursive: allocates a new
                                       // inode_table + dir_table pair
  free(resolved);
  goto out;
  // out: sqfs_closedir(dirsp) <- parent tables freed HERE, too late

There is no permanent leak: the parent's tables are freed at the
out: label once the recursive call returns. However, for the entire
duration of the recursive call both the parent's inode_table +
dir_table and the child's inode_table + dir_table are live on the
heap simultaneously. On large squashfs images these tables can be
significant in size, and this temporary double allocation may exhaust
the heap budget.

A superficial workaround would be to increase CONFIG_SYS_MALLOC_LEN,
but that wastes memory on all boards and does not address the
structural problem. The correct fix is to change the freeing order:
release the parent directory's resources before recursing. This way
only one set of inode and directory tables is live at any given time,
halving the peak heap usage during symlink resolution.

When heap exhaustion does occur and malloc returns NULL for dir_table
or pos_list inside sqfs_read_directory_table(), the failure is
currently silent and cascading:

  - metablks_count is not reset to -1 before the goto out, so the
    function returns a positive block count alongside a NULL pointer.
  - sqfs_opendir_nest() does not detect the failure (it only checks
    metablks_count < 1) and calls sqfs_search_dir() with m_list=NULL.
  - sqfs_dir_offset() iterates over m_list[0..n], reading from
    addresses 0x0, 0x4, 0x8, ... None of those values match the
    inode's start_block, so the function returns -EINVAL.
  - The error propagates up as a load failure with no indication
    that the root cause was heap exhaustion:

      Error: invalid inode reference to directory table.
      Failed to load '<symlink path>'

Two fixes:
1. In sqfs_read_directory_table(), set metablks_count = -1 whenever
   malloc fails after sqfs_count_metablks() returns a positive value,
   so that the caller's "metablks_count < 1" check correctly detects
   the failure and avoids calling sqfs_search_dir() with a NULL
   pos_list.
2. In sqfs_read_nest() and sqfs_size_nest(), call sqfs_closedir() on
   the parent dirsp before the recursive call so that the parent's
   inode and directory tables are freed before the child allocates
   its own. Only one set of tables is then live at any given time,
   halving peak heap usage during symlink resolution.

Link: https://lists.denx.de/pipermail/u-boot/2026-May/618533.html

Signed-off-by: Allan ELKAIM <allan.elkaim@gmail.com>
---

 fs/squashfs/sqfs.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)

diff --git a/fs/squashfs/sqfs.c b/fs/squashfs/sqfs.c
index 9cb8b4af..07e2bd82 100644
--- a/fs/squashfs/sqfs.c
+++ b/fs/squashfs/sqfs.c
@@ -853,12 +853,16 @@ static int sqfs_read_directory_table(unsigned char **dir_table, u32 **pos_list)
 		goto out;
 
 	*dir_table = malloc(metablks_count * SQFS_METADATA_BLOCK_SIZE);
-	if (!*dir_table)
+	if (!*dir_table) {
+		metablks_count = -1;
 		goto out;
+	}
 
 	*pos_list = malloc(metablks_count * sizeof(u32));
-	if (!*pos_list)
+	if (!*pos_list) {
+		metablks_count = -1;
 		goto out;
+	}
 
 	ret = sqfs_get_metablk_pos(*pos_list, dtb, table_offset,
 				   metablks_count);
@@ -1473,6 +1477,15 @@ static int sqfs_read_nest(const char *filename, void *buf, loff_t offset,
 
 		symlink = (struct squashfs_symlink_inode *)ipos;
 		resolved = sqfs_resolve_symlink(symlink, filename);
+		/*
+		 * Free the parent directory resources before recursing so that
+		 * the recursive call can allocate its own inode and directory
+		 * tables without exhausting the heap.
+		 */
+		free(dirs->entry);
+		dirs->entry = NULL;
+		sqfs_closedir(dirsp);
+		dirsp = NULL;
 		ret = sqfs_read_nest(resolved, buf, offset, len, actread);
 		free(resolved);
 		goto out;
@@ -1731,6 +1744,11 @@ static int sqfs_size_nest(const char *filename, loff_t *size)
 
 		symlink = (struct squashfs_symlink_inode *)ipos;
 		resolved = sqfs_resolve_symlink(symlink, filename);
+		/*
+		 * Free the parent directory resources before recursing.
+		 */
+		sqfs_closedir(dirsp);
+		dirsp = NULL;
 		ret = sqfs_size(resolved, size);
 		free(resolved);
 		break;
-- 
2.53.0

[PATCH v1 2/2] fs/squashfs: add sqfs_dir_offset() error checks

[Allan ELKAIM]

sqfs_dir_offset() returns a negative errno on failure, but three
call sites in sqfs_search_dir() use the return value as an array
index without checking for errors first. If the lookup fails,
dirs->table is set to an invalid address, leading to undefined
behavior.

Add negative-value guards after each sqfs_dir_offset() call so
that any lookup failure propagates cleanly as an error rather
than producing incorrect results.

Note: the corresponding sqfs_find_inode() NULL checks and the
heap exhaustion fix during symlink resolution are applied in
separate patches.

Signed-off-by: Allan ELKAIM <allan.elkaim@gmail.com>
---

 fs/squashfs/sqfs.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/fs/squashfs/sqfs.c b/fs/squashfs/sqfs.c
index 07e2bd82..430e9bac 100644
--- a/fs/squashfs/sqfs.c
+++ b/fs/squashfs/sqfs.c
@@ -496,6 +496,8 @@ static int sqfs_search_dir(struct squashfs_dir_stream *dirs, char **token_list,
 
 	/* get directory offset in directory table */
 	offset = sqfs_dir_offset(table, m_list, m_count);
+	if (offset < 0)
+		return offset;
 	dirs->table = &dirs->dir_table[offset];
 
 	/* Setup directory header */
@@ -627,6 +629,10 @@ static int sqfs_search_dir(struct squashfs_dir_stream *dirs, char **token_list,
 
 		/* Get dir. offset into the directory table */
 		offset = sqfs_dir_offset(table, m_list, m_count);
+		if (offset < 0) {
+			ret = offset;
+			goto out;
+		}
 		dirs->table = &dirs->dir_table[offset];
 
 		/* Copy directory header */
@@ -651,6 +657,10 @@ static int sqfs_search_dir(struct squashfs_dir_stream *dirs, char **token_list,
 	}
 
 	offset = sqfs_dir_offset(table, m_list, m_count);
+	if (offset < 0) {
+		ret = offset;
+		goto out;
+	}
 	dirs->table = &dirs->dir_table[offset];
 
 	if (get_unaligned_le16(&dir->inode_type) == SQFS_DIR_TYPE)
-- 
2.53.0