Re: [OE-core][scarthgap][PATCH] wic/engine: fix copying directories into wic image with ext* partition

[Daniel Dragomir <daniel.dragomir@windriver.com>]

On 2/20/26 11:41, Yoann Congal wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
> 
> On Fri Feb 20, 2026 at 10:19 AM CET, Daniel via lists.openembedded.org Dragomir wrote:
>>
>>
>> On 1/23/26 00:48, Daniel Dragomir wrote:
>>>
>>>
>>> On 10/3/25 19:43, Steve Sakoman wrote:
>>>> CAUTION: This email comes from a non Wind River email account!
>>>> Do not click links or open attachments unless you recognize the sender
>>>> and know the content is safe.
>>>>
>>>> On Fri, Oct 3, 2025 at 12:13 AM Dragomir, Daniel
>>>> <Daniel.Dragomir@windriver.com> wrote:
>>>>>
>>>>> Yes, this is an issue on master/master-next too.
>>>>> I tested and the same patch can be applied fine on both master and
>>>>> scarthgap branches.
>>>>
>>>> Thanks for checking.
>>>>
>>>> Please submit the patch for the master branch.  It would be helpful if
>>>> you would then ping me when you see that it has been accepted.  I'll
>>>> try to remember to watch for it, but a ping would ensure I don't miss
>>>> it :-)
>>>
>>> Hi Steve,
>>>
>>> Those changes were accepted in master branch. I created also a selftest
>>> for this scenario.
>>> Could you please back-port the commits to scarthgap branch? I tried
>>> locally and there is a minor merge conflict on one of the commits. If
>>> you want me to send again a scarthgap version of the commits on the
>>> mailing list, let me know.
>>>
>>> Commits from master branch are:
>>> 6de3d2602f oeqa/selftest/wic: test recursive dir copy on ext partitions
>>> 1ed38aff5f wic/engine: fix copying directories into wic image with ext*
>>> partition
>>>
>>
>> Hi Steve and Yoann,
> 
> Hello,
> 
>> Can you please help with a back-port on scarthgap for the commits bellow
>> which were accepted in master?
>>
>> 6de3d2602f oeqa/selftest/wic: test recursive dir copy on ext partitions
>> 1ed38aff5f wic/engine: fix copying directories into wic image with ext*
>> partition
> I reviewed those 2 and they will be in my next series.
> 
>> cb536737be wic/engine: error on old host debugfs for standalone directory copy
> For this one, can you send backport patches to scarthgap and whinlatter?
> 

Sure! Will send for both branches.
Thanks Yoann!

> 
>>
>> Thank you!
>> Daniel
>>
>>> Regards,
>>> Daniel
>>>
>>>>
>>>> Steve
>>>>
>>>>>
>>>>> Regards,
>>>>> Daniel
>>>>>
>>>>> ________________________________
>>>>> From: Steve Sakoman <steve@sakoman.com>
>>>>> Sent: Thursday, October 2, 2025 5:55 PM
>>>>> To: Dragomir, Daniel <Daniel.Dragomir@windriver.com>
>>>>> Cc: openembedded-core@lists.openembedded.org
>>>>> <openembedded-core@lists.openembedded.org>
>>>>> Subject: Re: [OE-core][scarthgap][PATCH] wic/engine: fix copying
>>>>> directories into wic image with ext* partition
>>>>>
>>>>> Is this also an issue on master?  If so, you will need to submit this
>>>>> patch for master before I can take it for scarthgap.  If not, can you
>>>>> explain why it isn't needed there?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Steve
>>>>>
>>>>> On Wed, Oct 1, 2025 at 2:25 PM Dragomir, Daniel via
>>>>> lists.openembedded.org
>>>>> <daniel.dragomir=windriver.com@lists.openembedded.org> wrote:
>>>>>>
>>>>>> wic uses debugfs to write on ext* partitions, but debugfs can only
>>>>>> write to the current working directory and it cannot copy complete
>>>>>> directory trees. Running 'wic ls' on a copied directory show this:
>>>>>>       -l: Ext2 inode is not a directory
>>>>>>
>>>>>> Fix this by creating a command list for debugfs (-f parameter) when
>>>>>> recursive parsing the host directory in order to create a similar
>>>>>> directory structure (mkdir) and copy files (write) on each level
>>>>>> into the destination directory from the wic's ext* partition.
>>>>>>
>>>>>> Signed-off-by: Daniel Dragomir <daniel.dragomir@windriver.com>
>>>>>> ---
>>>>>>    scripts/lib/wic/engine.py | 63
>>>>>> ++++++++++++++++++++++++++++++---------
>>>>>>    1 file changed, 49 insertions(+), 14 deletions(-)
>>>>>>
>>>>>> diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
>>>>>> index b9e60cbe4e..9d596be3a7 100644
>>>>>> --- a/scripts/lib/wic/engine.py
>>>>>> +++ b/scripts/lib/wic/engine.py
>>>>>> @@ -345,29 +345,64 @@ class Disk:
>>>>>>                                                       path))
>>>>>>
>>>>>>        def copy(self, src, dest):
>>>>>> -        """Copy partition image into wic image."""
>>>>>> -        pnum =  dest.part if isinstance(src, str) else src.part
>>>>>> +        """Copy files or directories to/from the vfat or ext*
>>>>>> partition."""
>>>>>> +        pnum = dest.part if isinstance(src, str) else src.part
>>>>>> +        partimg = self._get_part_image(pnum)
>>>>>>
>>>>>>            if self.partitions[pnum].fstype.startswith('ext'):
>>>>>> -            if isinstance(src, str):
>>>>>> -                cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\
>>>>>> -                      format(os.path.dirname(dest.path), src,
>>>>>> os.path.basename(src),
>>>>>> -                             self.debugfs, self._get_part_image(pnum))
>>>>>> -            else: # copy from wic
>>>>>> -                # run both dump and rdump to support both files and
>>>>>> directory
>>>>>> +            if isinstance(src, str): # host to image case
>>>>>> +                if os.path.isdir(src):
>>>>>> +                    base = os.path.abspath(src)
>>>>>> +                    base_parent = os.path.dirname(base)
>>>>>> +                    cmds = []
>>>>>> +                    made = set()
>>>>>> +
>>>>>> +                    for root, dirs, files in os.walk(base):
>>>>>> +                        for fname in files:
>>>>>> +                            host_file = os.path.join(root, fname)
>>>>>> +                            rel = os.path.relpath(host_file,
>>>>>> base_parent)
>>>>>> +                            dest_file = os.path.join(dest.path, rel)
>>>>>> +                            dest_dir = os.path.dirname(dest_file)
>>>>>> +
>>>>>> +                            # create dir structure (mkdir -p)
>>>>>> +                            parts = dest_dir.strip('/').split('/')
>>>>>> +                            cur = ''
>>>>>> +                            for p in parts:
>>>>>> +                                cur = cur + '/' + p
>>>>>> +                                if cur not in made:
>>>>>> +                                    cmds.append(f'mkdir "{cur}"')
>>>>>> +                                    made.add(cur)
>>>>>> +
>>>>>> +                            cmds.append(f'write "{host_file}"
>>>>>> "{dest_file}"')
>>>>>> +
>>>>>> +                    # write script to a temp file
>>>>>> +                    with tempfile.NamedTemporaryFile(mode='w',
>>>>>> delete=False,
>>>>>> +
>>>>>> prefix='wic-debugfs-') as tf:
>>>>>> +                        for line in cmds:
>>>>>> +                            tf.write(line + '\n')
>>>>>> +                        scriptname = tf.name
>>>>>> +
>>>>>> +                    cmd = f"{self.debugfs} -w -f {scriptname}
>>>>>> {partimg}"
>>>>>> +
>>>>>> +                else: # single file
>>>>>> +                    cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\
>>>>>> +                          format(os.path.dirname(dest.path), src,
>>>>>> +                                 os.path.basename(src),
>>>>>> self.debugfs, partimg)
>>>>>> +
>>>>>> +            else: # image to host case
>>>>>>                    cmd = "printf 'cd {}\ndump /{} {}\nrdump /{} {}\n'
>>>>>> | {} {}".\
>>>>>>                          format(os.path.dirname(src.path), src.path,
>>>>>> -                             dest, src.path, dest, self.debugfs,
>>>>>> -                             self._get_part_image(pnum))
>>>>>> +                             dest, src.path, dest, self.debugfs,
>>>>>> partimg)
>>>>>> +
>>>>>>            else: # fat
>>>>>>                if isinstance(src, str):
>>>>>>                    cmd = "{} -i {} -snop {} ::{}".format(self.mcopy,
>>>>>> -
>>>>>> self._get_part_image(pnum),
>>>>>> -                                                  src, dest.path)
>>>>>> +                                                      partimg,
>>>>>> +                                                      src, dest.path)
>>>>>>                else:
>>>>>>                    cmd = "{} -i {} -snop ::{} {}".format(self.mcopy,
>>>>>> -
>>>>>> self._get_part_image(pnum),
>>>>>> -                                                  src.path, dest)
>>>>>> +                                                      partimg,
>>>>>> +                                                      src.path, dest)
>>>>>>
>>>>>>            exec_cmd(cmd, as_shell=True)
>>>>>>            self._put_part_image(pnum)
>>>>>> --
>>>>>> 2.39.5
>>>>>>
>>>>>>
>>>>>>
>>>>>>
> 
> 
> --
> Yoann Congal
> Smile ECS
>