Re: [OE-core][scarthgap][PATCH] wic/engine: fix copying directories into wic image with ext* partition

[Daniel Dragomir <daniel.dragomir@windriver.com>]

On 1/23/26 00:48, Daniel Dragomir wrote:
> 
> 
> On 10/3/25 19:43, Steve Sakoman wrote:
>> CAUTION: This email comes from a non Wind River email account!
>> Do not click links or open attachments unless you recognize the sender 
>> and know the content is safe.
>>
>> On Fri, Oct 3, 2025 at 12:13 AM Dragomir, Daniel
>> <Daniel.Dragomir@windriver.com> wrote:
>>>
>>> Yes, this is an issue on master/master-next too.
>>> I tested and the same patch can be applied fine on both master and 
>>> scarthgap branches.
>>
>> Thanks for checking.
>>
>> Please submit the patch for the master branch.  It would be helpful if
>> you would then ping me when you see that it has been accepted.  I'll
>> try to remember to watch for it, but a ping would ensure I don't miss
>> it :-)
> 
> Hi Steve,
> 
> Those changes were accepted in master branch. I created also a selftest 
> for this scenario.
> Could you please back-port the commits to scarthgap branch? I tried 
> locally and there is a minor merge conflict on one of the commits. If 
> you want me to send again a scarthgap version of the commits on the 
> mailing list, let me know.
> 
> Commits from master branch are:
> 6de3d2602f oeqa/selftest/wic: test recursive dir copy on ext partitions
> 1ed38aff5f wic/engine: fix copying directories into wic image with ext* 
> partition
> 

Hi Steve and Yoann,

Can you please help with a back-port on scarthgap for the commits bellow 
which were accepted in master?

6de3d2602f oeqa/selftest/wic: test recursive dir copy on ext partitions
1ed38aff5f wic/engine: fix copying directories into wic image with ext* 
partition
cb536737be wic/engine: error on old host debugfs for standalone 
directory copy

Thank you!
Daniel

> Regards,
> Daniel
> 
>>
>> Steve
>>
>>>
>>> Regards,
>>> Daniel
>>>
>>> ________________________________
>>> From: Steve Sakoman <steve@sakoman.com>
>>> Sent: Thursday, October 2, 2025 5:55 PM
>>> To: Dragomir, Daniel <Daniel.Dragomir@windriver.com>
>>> Cc: openembedded-core@lists.openembedded.org 
>>> <openembedded-core@lists.openembedded.org>
>>> Subject: Re: [OE-core][scarthgap][PATCH] wic/engine: fix copying 
>>> directories into wic image with ext* partition
>>>
>>> Is this also an issue on master?  If so, you will need to submit this
>>> patch for master before I can take it for scarthgap.  If not, can you
>>> explain why it isn't needed there?
>>>
>>> Thanks,
>>>
>>> Steve
>>>
>>> On Wed, Oct 1, 2025 at 2:25 PM Dragomir, Daniel via
>>> lists.openembedded.org
>>> <daniel.dragomir=windriver.com@lists.openembedded.org> wrote:
>>>>
>>>> wic uses debugfs to write on ext* partitions, but debugfs can only
>>>> write to the current working directory and it cannot copy complete
>>>> directory trees. Running 'wic ls' on a copied directory show this:
>>>>      -l: Ext2 inode is not a directory
>>>>
>>>> Fix this by creating a command list for debugfs (-f parameter) when
>>>> recursive parsing the host directory in order to create a similar
>>>> directory structure (mkdir) and copy files (write) on each level
>>>> into the destination directory from the wic's ext* partition.
>>>>
>>>> Signed-off-by: Daniel Dragomir <daniel.dragomir@windriver.com>
>>>> ---
>>>>   scripts/lib/wic/engine.py | 63 
>>>> ++++++++++++++++++++++++++++++---------
>>>>   1 file changed, 49 insertions(+), 14 deletions(-)
>>>>
>>>> diff --git a/scripts/lib/wic/engine.py b/scripts/lib/wic/engine.py
>>>> index b9e60cbe4e..9d596be3a7 100644
>>>> --- a/scripts/lib/wic/engine.py
>>>> +++ b/scripts/lib/wic/engine.py
>>>> @@ -345,29 +345,64 @@ class Disk:
>>>>                                                      path))
>>>>
>>>>       def copy(self, src, dest):
>>>> -        """Copy partition image into wic image."""
>>>> -        pnum =  dest.part if isinstance(src, str) else src.part
>>>> +        """Copy files or directories to/from the vfat or ext* 
>>>> partition."""
>>>> +        pnum = dest.part if isinstance(src, str) else src.part
>>>> +        partimg = self._get_part_image(pnum)
>>>>
>>>>           if self.partitions[pnum].fstype.startswith('ext'):
>>>> -            if isinstance(src, str):
>>>> -                cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\
>>>> -                      format(os.path.dirname(dest.path), src, 
>>>> os.path.basename(src),
>>>> -                             self.debugfs, self._get_part_image(pnum))
>>>> -            else: # copy from wic
>>>> -                # run both dump and rdump to support both files and 
>>>> directory
>>>> +            if isinstance(src, str): # host to image case
>>>> +                if os.path.isdir(src):
>>>> +                    base = os.path.abspath(src)
>>>> +                    base_parent = os.path.dirname(base)
>>>> +                    cmds = []
>>>> +                    made = set()
>>>> +
>>>> +                    for root, dirs, files in os.walk(base):
>>>> +                        for fname in files:
>>>> +                            host_file = os.path.join(root, fname)
>>>> +                            rel = os.path.relpath(host_file, 
>>>> base_parent)
>>>> +                            dest_file = os.path.join(dest.path, rel)
>>>> +                            dest_dir = os.path.dirname(dest_file)
>>>> +
>>>> +                            # create dir structure (mkdir -p)
>>>> +                            parts = dest_dir.strip('/').split('/')
>>>> +                            cur = ''
>>>> +                            for p in parts:
>>>> +                                cur = cur + '/' + p
>>>> +                                if cur not in made:
>>>> +                                    cmds.append(f'mkdir "{cur}"')
>>>> +                                    made.add(cur)
>>>> +
>>>> +                            cmds.append(f'write "{host_file}" 
>>>> "{dest_file}"')
>>>> +
>>>> +                    # write script to a temp file
>>>> +                    with tempfile.NamedTemporaryFile(mode='w', 
>>>> delete=False,
>>>> +                                                     
>>>> prefix='wic-debugfs-') as tf:
>>>> +                        for line in cmds:
>>>> +                            tf.write(line + '\n')
>>>> +                        scriptname = tf.name
>>>> +
>>>> +                    cmd = f"{self.debugfs} -w -f {scriptname} 
>>>> {partimg}"
>>>> +
>>>> +                else: # single file
>>>> +                    cmd = "printf 'cd {}\nwrite {} {}\n' | {} -w {}".\
>>>> +                          format(os.path.dirname(dest.path), src,
>>>> +                                 os.path.basename(src), 
>>>> self.debugfs, partimg)
>>>> +
>>>> +            else: # image to host case
>>>>                   cmd = "printf 'cd {}\ndump /{} {}\nrdump /{} {}\n' 
>>>> | {} {}".\
>>>>                         format(os.path.dirname(src.path), src.path,
>>>> -                             dest, src.path, dest, self.debugfs,
>>>> -                             self._get_part_image(pnum))
>>>> +                             dest, src.path, dest, self.debugfs, 
>>>> partimg)
>>>> +
>>>>           else: # fat
>>>>               if isinstance(src, str):
>>>>                   cmd = "{} -i {} -snop {} ::{}".format(self.mcopy,
>>>> -                                                  
>>>> self._get_part_image(pnum),
>>>> -                                                  src, dest.path)
>>>> +                                                      partimg,
>>>> +                                                      src, dest.path)
>>>>               else:
>>>>                   cmd = "{} -i {} -snop ::{} {}".format(self.mcopy,
>>>> -                                                  
>>>> self._get_part_image(pnum),
>>>> -                                                  src.path, dest)
>>>> +                                                      partimg,
>>>> +                                                      src.path, dest)
>>>>
>>>>           exec_cmd(cmd, as_shell=True)
>>>>           self._put_part_image(pnum)
>>>> -- 
>>>> 2.39.5
>>>>
>>>>
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>> Links: You receive all messages sent to this group.
>>>> View/Reply Online (#224331): 
>>>> https://lists.openembedded.org/g/openembedded-core/message/224331
>>>> Mute This Topic: https://lists.openembedded.org/mt/115542019/3620601
>>>> Group Owner: openembedded-core+owner@lists.openembedded.org
>>>> Unsubscribe: 
>>>> https://lists.openembedded.org/g/openembedded-core/unsub 
>>>> [steve@sakoman.com]
>>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>>