U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

Hello,

We are using wi.bz2 as our image fstypes. The image gets created smoothly for the first time. Our wks file uses -sourceparam argument and mention the u-boot file

part u-boot --source rawcopy --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table --align 1

The issue comes when something is added to image recipe and it get rebuild. At that time somehow u-boot-mx6-solox.imx gets deleted from deploy folder and we get this error

Error: exec_cmd: du -Lbks /home/noor/SB/release_2015.12/build_soloX/tmp/deploy/images/mx6-solox/u-boot-mx6-solox.imx returned '1' instead of 0
Checking basic build environment...
Done.


I was able to reproduce the error by cleaning image recipe and rebuilding it. I got the same error. So something is being done in wic setup which removes the u-boot file. I need some help on this to resolve the issue of removal of u-boot.

Noor

[-- Attachment #2: Type: text/html, Size: 3158 bytes --]

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

[-- Attachment #1: Type: text/plain, Size: 1481 bytes --]

Hello Ed Bartosh,

Just to be clear the u-boot-mx6-solox.imx gets deleted when wic create command is executed. Kindly let us know what to do. We are stuck with our release.

Noor

From: openembedded-core-bounces@lists.openembedded.org [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Ahsan, Noor
Sent: Monday, October 19, 2015 8:42 PM
To: ed.bartosh@linux.intel.com
Cc: Krishnakar, Srikanth; Larson, Chris; openembedded-core@lists.openembedded.org
Subject: [OE-core] U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

Hello,

We are using wi.bz2 as our image fstypes. The image gets created smoothly for the first time. Our wks file uses -sourceparam argument and mention the u-boot file

part u-boot --source rawcopy --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table --align 1

The issue comes when something is added to image recipe and it get rebuild. At that time somehow u-boot-mx6-solox.imx gets deleted from deploy folder and we get this error

Error: exec_cmd: du -Lbks /home/noor/SB/release_2015.12/build_soloX/tmp/deploy/images/mx6-solox/u-boot-mx6-solox.imx returned '1' instead of 0
Checking basic build environment...
Done.


I was able to reproduce the error by cleaning image recipe and rebuilding it. I got the same error. So something is being done in wic setup which removes the u-boot file. I need some help on this to resolve the issue of removal of u-boot.

Noor

[-- Attachment #2: Type: text/html, Size: 4646 bytes --]

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

[-- Attachment #1: Type: text/plain, Size: 2129 bytes --]

Hello Ed,

Your help will be much appreciated. I am trying to locate the issue but no luck yet.

Noor

From: openembedded-core-bounces@lists.openembedded.org [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Ahsan, Noor
Sent: Tuesday, October 20, 2015 3:12 PM
To: ed.bartosh@linux.intel.com
Cc: openembedded-core@lists.openembedded.org; Larson, Chris; Krishnakar, Srikanth
Subject: Re: [OE-core] U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

Hello Ed Bartosh,

Just to be clear the u-boot-mx6-solox.imx gets deleted when wic create command is executed. Kindly let us know what to do. We are stuck with our release.

Noor

From: openembedded-core-bounces@lists.openembedded.org<mailto:openembedded-core-bounces@lists.openembedded.org> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf Of Ahsan, Noor
Sent: Monday, October 19, 2015 8:42 PM
To: ed.bartosh@linux.intel.com<mailto:ed.bartosh@linux.intel.com>
Cc: Krishnakar, Srikanth; Larson, Chris; openembedded-core@lists.openembedded.org<mailto:openembedded-core@lists.openembedded.org>
Subject: [OE-core] U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

Hello,

We are using wi.bz2 as our image fstypes. The image gets created smoothly for the first time. Our wks file uses -sourceparam argument and mention the u-boot file

part u-boot --source rawcopy --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table --align 1

The issue comes when something is added to image recipe and it get rebuild. At that time somehow u-boot-mx6-solox.imx gets deleted from deploy folder and we get this error

Error: exec_cmd: du -Lbks /home/noor/SB/release_2015.12/build_soloX/tmp/deploy/images/mx6-solox/u-boot-mx6-solox.imx returned '1' instead of 0
Checking basic build environment...
Done.


I was able to reproduce the error by cleaning image recipe and rebuilding it. I got the same error. So something is being done in wic setup which removes the u-boot file. I need some help on this to resolve the issue of removal of u-boot.

Noor

[-- Attachment #2: Type: text/html, Size: 6286 bytes --]

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Otavio Salvador]

On Wed, Oct 21, 2015 at 11:38 AM, Ahsan, Noor <Noor_Ahsan@mentor.com> wrote:
> Your help will be much appreciated. I am trying to locate the issue but no
> luck yet.

We reproduced this at O.S. Systems autobuilder today[1]. It seems very
close to what you see.

1. http://ci.ossystems.com.br/job/fsl-community-bsp-master-next_x11-imx6qsabresd/869/console

Can you take a look and confirm it is the same issue?

-- 
Otavio Salvador                             O.S. Systems
http://www.ossystems.com.br        http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854            Mobile: +1 (347) 903-9750

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ed Bartosh]

Hi Noor,

Sorry for delay. I missed your e-mail somehow.

Can you please show your recipe, wks file and the details of your build
configuration(MACHINE, bblayers, etc) ?

---
Regards,
Ed

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

[-- Attachment #1: Type: text/plain, Size: 723 bytes --]

Hello,

I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.

Noor

-----Original Message-----
From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
Sent: Tuesday, October 27, 2015 6:41 PM
To: Ahsan, Noor
Cc: openembedded-core@lists.openembedded.org
Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

Hi Noor,

Sorry for delay. I missed your e-mail somehow.

Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?

---
Regards,
Ed

[-- Attachment #2: Type: message/rfc822, Size: 5400 bytes --]

From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
To: "openembedded-core@lists.openembedded.org" <openembedded-core@lists.openembedded.org>
Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image creation build folder
Date: Thu, 22 Oct 2015 10:14:02 +0000
Message-ID: <365E1805BC95084CBE82381A0B86999401095FB9D1@EU-MBX-01.mgc.mentorg.com>



-----Original Message-----
From: Ahsan, Noor
Sent: Wednesday, October 21, 2015 9:04 PM
To: openembedded-core@lists.openembedded.org
Cc: Ahsan, Noor
Subject: [RFC] rawcopy.py: Copy source file in image creation build folder

From: Noor <noor_ahsan@mentor.com>

* When a file is given using --sourceparams then wic directly use that file
  instead of copying them to build folder. At time of assmebling it os.rename
  is called which rename all the files to name. In that process the original
  file get renamed. When image recipe is rebuilt then wic complains about
  missing file which renamed in previous build.

Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
---
 scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
index f0691ba..f17da27 100644
--- a/scripts/lib/wic/plugins/source/rawcopy.py
+++ b/scripts/lib/wic/plugins/source/rawcopy.py
@@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
             dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
                     (src, dst, source_params['skip'])
             exec_cmd(dd_cmd)
+        else:
+            dst = os.path.join(cr_workdir, source_params['file'])
+            dd_cmd = "cp src dst"
+            exec_cmd(dd_cmd)


dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.


         # get the size in the right units for kickstart (kB)
         du_cmd = "du -Lbks %s" % dst
--
1.7.9.5

--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ed Bartosh]

On Tue, Oct 27, 2015 at 02:28:02PM +0000, Ahsan, Noor wrote:
> Hello,
> 
> I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.

Can you point me out where the renaming you've mentioned is happening?

Anyway it would be good if I could reproduce the issue to better
understand it. Can you provide more details about your configuration?

Regards,
Ed


> 
> Noor
> 
> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
> Sent: Tuesday, October 27, 2015 6:41 PM
> To: Ahsan, Noor
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2
> 
> Hi Noor,
> 
> Sorry for delay. I missed your e-mail somehow.
> 
> Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?
> 
> ---
> Regards,
> Ed

> Date: Thu, 22 Oct 2015 10:14:02 +0000
> From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
> To: "openembedded-core@lists.openembedded.org"
>  <openembedded-core@lists.openembedded.org>
> Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image creation
>  build folder
> 
> 
> 
> -----Original Message-----
> From: Ahsan, Noor
> Sent: Wednesday, October 21, 2015 9:04 PM
> To: openembedded-core@lists.openembedded.org
> Cc: Ahsan, Noor
> Subject: [RFC] rawcopy.py: Copy source file in image creation build folder
> 
> From: Noor <noor_ahsan@mentor.com>
> 
> * When a file is given using --sourceparams then wic directly use that file
>   instead of copying them to build folder. At time of assmebling it os.rename
>   is called which rename all the files to name. In that process the original
>   file get renamed. When image recipe is rebuilt then wic complains about
>   missing file which renamed in previous build.
> 
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> ---
>  scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/scripts/lib/wic/plugins/source/rawcopy.py b/scripts/lib/wic/plugins/source/rawcopy.py
> index f0691ba..f17da27 100644
> --- a/scripts/lib/wic/plugins/source/rawcopy.py
> +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> @@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
>              dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
>                      (src, dst, source_params['skip'])
>              exec_cmd(dd_cmd)
> +        else:
> +            dst = os.path.join(cr_workdir, source_params['file'])
> +            dd_cmd = "cp src dst"
> +            exec_cmd(dd_cmd)
> 
> 
> dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.
> 
> 
>          # get the size in the right units for kickstart (kB)
>          du_cmd = "du -Lbks %s" % dst
> --
> 1.7.9.5
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


-- 
--
Regards,
Ed

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

This is my wks file 

part u-boot --source rawcopy --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table --align 1
part /boot --source bootimg-partition --ondisk mmcblk --fstype=vfat --label boot --active --align 4096 --size 8M --extra-space 0
part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4096

u-boot-mx6-solox.imx gets removed after wic is called.

It is in fsimage.py. see os.rename(source, image_file + '.p%d' % part['num']).


def assemble(self, image_file):
        msger.debug("Installing partitions")

        for part in self.partitions:
            source = part['source_file']
            if source:
                # install source_file contents into a partition
                cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
                      (source, image_file, self.sector_size,
                       part['start'], part['size'])
                exec_cmd(cmd)

                msger.debug("Installed %s in partition %d, sectors %d-%d, "
                            "size %d sectors" % \
                            (source, part['num'], part['start'],
                             part['start'] + part['size'] - 1, part['size']))

                os.rename(source, image_file + '.p%d' % part['num'])

-----Original Message-----
From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
Sent: Tuesday, October 27, 2015 8:27 PM
To: Ahsan, Noor
Cc: openembedded-core@lists.openembedded.org
Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

On Tue, Oct 27, 2015 at 02:28:02PM +0000, Ahsan, Noor wrote:
> Hello,
> 
> I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.

Can you point me out where the renaming you've mentioned is happening?

Anyway it would be good if I could reproduce the issue to better understand it. Can you provide more details about your configuration?

Regards,
Ed


> 
> Noor
> 
> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> Sent: Tuesday, October 27, 2015 6:41 PM
> To: Ahsan, Noor
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: U-boot file is being removed when image-recipe is rebuild 
> which uses wic.bz2
> 
> Hi Noor,
> 
> Sorry for delay. I missed your e-mail somehow.
> 
> Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?
> 
> ---
> Regards,
> Ed

> Date: Thu, 22 Oct 2015 10:14:02 +0000
> From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
> To: "openembedded-core@lists.openembedded.org"
>  <openembedded-core@lists.openembedded.org>
> Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image 
> creation  build folder
> 
> 
> 
> -----Original Message-----
> From: Ahsan, Noor
> Sent: Wednesday, October 21, 2015 9:04 PM
> To: openembedded-core@lists.openembedded.org
> Cc: Ahsan, Noor
> Subject: [RFC] rawcopy.py: Copy source file in image creation build 
> folder
> 
> From: Noor <noor_ahsan@mentor.com>
> 
> * When a file is given using --sourceparams then wic directly use that file
>   instead of copying them to build folder. At time of assmebling it os.rename
>   is called which rename all the files to name. In that process the original
>   file get renamed. When image recipe is rebuilt then wic complains about
>   missing file which renamed in previous build.
> 
> Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> ---
>  scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
> b/scripts/lib/wic/plugins/source/rawcopy.py
> index f0691ba..f17da27 100644
> --- a/scripts/lib/wic/plugins/source/rawcopy.py
> +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> @@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
>              dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
>                      (src, dst, source_params['skip'])
>              exec_cmd(dd_cmd)
> +        else:
> +            dst = os.path.join(cr_workdir, source_params['file'])
> +            dd_cmd = "cp src dst"
> +            exec_cmd(dd_cmd)
> 
> 
> dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.
> 
> 
>          # get the size in the right units for kickstart (kB)
>          du_cmd = "du -Lbks %s" % dst
> --
> 1.7.9.5
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


--
--
Regards,
Ed

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ed Bartosh]

On Tue, Oct 27, 2015 at 03:40:49PM +0000, Ahsan, Noor wrote:
> This is my wks file 
> 
> part u-boot --source rawcopy --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table --align 1
> part /boot --source bootimg-partition --ondisk mmcblk --fstype=vfat --label boot --active --align 4096 --size 8M --extra-space 0
> part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4096
> 
> u-boot-mx6-solox.imx gets removed after wic is called.
> 
> It is in fsimage.py. see os.rename(source, image_file + '.p%d' % part['num']).
> 
> 
> def assemble(self, image_file):
>         msger.debug("Installing partitions")
> 
>         for part in self.partitions:
>             source = part['source_file']
>             if source:
>                 # install source_file contents into a partition
>                 cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
>                       (source, image_file, self.sector_size,
>                        part['start'], part['size'])
>                 exec_cmd(cmd)
> 
>                 msger.debug("Installed %s in partition %d, sectors %d-%d, "
>                             "size %d sectors" % \
>                             (source, part['num'], part['start'],
>                              part['start'] + part['size'] - 1, part['size']))
> 
>                 os.rename(source, image_file + '.p%d' % part['num'])
> 

Thank you for the info! I'll try to reproduce the issue.

Wild guess: would copying a file instead of renaming it solve the issue?


Regards,
Ed

> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
> Sent: Tuesday, October 27, 2015 8:27 PM
> To: Ahsan, Noor
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2
> 
> On Tue, Oct 27, 2015 at 02:28:02PM +0000, Ahsan, Noor wrote:
> > Hello,
> > 
> > I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.
> 
> Can you point me out where the renaming you've mentioned is happening?
> 
> Anyway it would be good if I could reproduce the issue to better understand it. Can you provide more details about your configuration?
> 
> Regards,
> Ed
> 
> 
> > 
> > Noor
> > 
> > -----Original Message-----
> > From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> > Sent: Tuesday, October 27, 2015 6:41 PM
> > To: Ahsan, Noor
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: U-boot file is being removed when image-recipe is rebuild 
> > which uses wic.bz2
> > 
> > Hi Noor,
> > 
> > Sorry for delay. I missed your e-mail somehow.
> > 
> > Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?
> > 
> > ---
> > Regards,
> > Ed
> 
> > Date: Thu, 22 Oct 2015 10:14:02 +0000
> > From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
> > To: "openembedded-core@lists.openembedded.org"
> >  <openembedded-core@lists.openembedded.org>
> > Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image 
> > creation  build folder
> > 
> > 
> > 
> > -----Original Message-----
> > From: Ahsan, Noor
> > Sent: Wednesday, October 21, 2015 9:04 PM
> > To: openembedded-core@lists.openembedded.org
> > Cc: Ahsan, Noor
> > Subject: [RFC] rawcopy.py: Copy source file in image creation build 
> > folder
> > 
> > From: Noor <noor_ahsan@mentor.com>
> > 
> > * When a file is given using --sourceparams then wic directly use that file
> >   instead of copying them to build folder. At time of assmebling it os.rename
> >   is called which rename all the files to name. In that process the original
> >   file get renamed. When image recipe is rebuilt then wic complains about
> >   missing file which renamed in previous build.
> > 
> > Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> > ---
> >  scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/scripts/lib/wic/plugins/source/rawcopy.py 
> > b/scripts/lib/wic/plugins/source/rawcopy.py
> > index f0691ba..f17da27 100644
> > --- a/scripts/lib/wic/plugins/source/rawcopy.py
> > +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> > @@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
> >              dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
> >                      (src, dst, source_params['skip'])
> >              exec_cmd(dd_cmd)
> > +        else:
> > +            dst = os.path.join(cr_workdir, source_params['file'])
> > +            dd_cmd = "cp src dst"
> > +            exec_cmd(dd_cmd)
> > 
> > 
> > dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.
> > 
> > 
> >          # get the size in the right units for kickstart (kB)
> >          du_cmd = "du -Lbks %s" % dst
> > --
> > 1.7.9.5
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
> --
> --
> Regards,
> Ed

-- 
--
Regards,
Ed

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ahsan, Noor]

Ed,

Any update on this. 


I think os.rename should remain there. We should copy the file in build folder in rawcopy. Right now we don't copy it but in others we copy things in build folder.

Noor

-----Original Message-----
From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
Sent: Tuesday, October 27, 2015 10:42 PM
To: Ahsan, Noor
Cc: openembedded-core@lists.openembedded.org
Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

On Tue, Oct 27, 2015 at 03:40:49PM +0000, Ahsan, Noor wrote:
> This is my wks file
> 
> part u-boot --source rawcopy 
> --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table 
> --align 1 part /boot --source bootimg-partition --ondisk mmcblk 
> --fstype=vfat --label boot --active --align 4096 --size 8M 
> --extra-space 0 part / --source rootfs --ondisk mmcblk --fstype=ext4 
> --label root --align 4096
> 
> u-boot-mx6-solox.imx gets removed after wic is called.
> 
> It is in fsimage.py. see os.rename(source, image_file + '.p%d' % part['num']).
> 
> 
> def assemble(self, image_file):
>         msger.debug("Installing partitions")
> 
>         for part in self.partitions:
>             source = part['source_file']
>             if source:
>                 # install source_file contents into a partition
>                 cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
>                       (source, image_file, self.sector_size,
>                        part['start'], part['size'])
>                 exec_cmd(cmd)
> 
>                 msger.debug("Installed %s in partition %d, sectors %d-%d, "
>                             "size %d sectors" % \
>                             (source, part['num'], part['start'],
>                              part['start'] + part['size'] - 1, 
> part['size']))
> 
>                 os.rename(source, image_file + '.p%d' % part['num'])
> 

Thank you for the info! I'll try to reproduce the issue.

Wild guess: would copying a file instead of renaming it solve the issue?


Regards,
Ed

> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> Sent: Tuesday, October 27, 2015 8:27 PM
> To: Ahsan, Noor
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: U-boot file is being removed when image-recipe is rebuild 
> which uses wic.bz2
> 
> On Tue, Oct 27, 2015 at 02:28:02PM +0000, Ahsan, Noor wrote:
> > Hello,
> > 
> > I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.
> 
> Can you point me out where the renaming you've mentioned is happening?
> 
> Anyway it would be good if I could reproduce the issue to better understand it. Can you provide more details about your configuration?
> 
> Regards,
> Ed
> 
> 
> > 
> > Noor
> > 
> > -----Original Message-----
> > From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> > Sent: Tuesday, October 27, 2015 6:41 PM
> > To: Ahsan, Noor
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: U-boot file is being removed when image-recipe is 
> > rebuild which uses wic.bz2
> > 
> > Hi Noor,
> > 
> > Sorry for delay. I missed your e-mail somehow.
> > 
> > Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?
> > 
> > ---
> > Regards,
> > Ed
> 
> > Date: Thu, 22 Oct 2015 10:14:02 +0000
> > From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
> > To: "openembedded-core@lists.openembedded.org"
> >  <openembedded-core@lists.openembedded.org>
> > Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image 
> > creation  build folder
> > 
> > 
> > 
> > -----Original Message-----
> > From: Ahsan, Noor
> > Sent: Wednesday, October 21, 2015 9:04 PM
> > To: openembedded-core@lists.openembedded.org
> > Cc: Ahsan, Noor
> > Subject: [RFC] rawcopy.py: Copy source file in image creation build 
> > folder
> > 
> > From: Noor <noor_ahsan@mentor.com>
> > 
> > * When a file is given using --sourceparams then wic directly use that file
> >   instead of copying them to build folder. At time of assmebling it os.rename
> >   is called which rename all the files to name. In that process the original
> >   file get renamed. When image recipe is rebuilt then wic complains about
> >   missing file which renamed in previous build.
> > 
> > Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> > ---
> >  scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
> >  1 file changed, 4 insertions(+)
> > 
> > diff --git a/scripts/lib/wic/plugins/source/rawcopy.py
> > b/scripts/lib/wic/plugins/source/rawcopy.py
> > index f0691ba..f17da27 100644
> > --- a/scripts/lib/wic/plugins/source/rawcopy.py
> > +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> > @@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
> >              dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
> >                      (src, dst, source_params['skip'])
> >              exec_cmd(dd_cmd)
> > +        else:
> > +            dst = os.path.join(cr_workdir, source_params['file'])
> > +            dd_cmd = "cp src dst"
> > +            exec_cmd(dd_cmd)
> > 
> > 
> > dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.
> > 
> > 
> >          # get the size in the right units for kickstart (kB)
> >          du_cmd = "du -Lbks %s" % dst
> > --
> > 1.7.9.5
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 
> 
> --
> --
> Regards,
> Ed

--
--
Regards,
Ed

Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2

[Ed Bartosh]

On Thu, Oct 29, 2015 at 10:50:06AM +0000, Ahsan, Noor wrote:
> Ed,
> 
> Any update on this. 
>
> 
> I think os.rename should remain there. We should copy the file in build folder in rawcopy. Right now we don't copy it but in others we copy things in build folder.
>
I'm ok with the patch then if you update it according to Leonardo's
suggestion.



> Noor
> 
> -----Original Message-----
> From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com] 
> Sent: Tuesday, October 27, 2015 10:42 PM
> To: Ahsan, Noor
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: U-boot file is being removed when image-recipe is rebuild which uses wic.bz2
> 
> On Tue, Oct 27, 2015 at 03:40:49PM +0000, Ahsan, Noor wrote:
> > This is my wks file
> > 
> > part u-boot --source rawcopy 
> > --sourceparams="file=u-boot-mx6-solox.imx" --ondisk mmcblk --no-table 
> > --align 1 part /boot --source bootimg-partition --ondisk mmcblk 
> > --fstype=vfat --label boot --active --align 4096 --size 8M 
> > --extra-space 0 part / --source rootfs --ondisk mmcblk --fstype=ext4 
> > --label root --align 4096
> > 
> > u-boot-mx6-solox.imx gets removed after wic is called.
> > 
> > It is in fsimage.py. see os.rename(source, image_file + '.p%d' % part['num']).
> > 
> > 
> > def assemble(self, image_file):
> >         msger.debug("Installing partitions")
> > 
> >         for part in self.partitions:
> >             source = part['source_file']
> >             if source:
> >                 # install source_file contents into a partition
> >                 cmd = "dd if=%s of=%s bs=%d seek=%d count=%d conv=notrunc" % \
> >                       (source, image_file, self.sector_size,
> >                        part['start'], part['size'])
> >                 exec_cmd(cmd)
> > 
> >                 msger.debug("Installed %s in partition %d, sectors %d-%d, "
> >                             "size %d sectors" % \
> >                             (source, part['num'], part['start'],
> >                              part['start'] + part['size'] - 1, 
> > part['size']))
> > 
> >                 os.rename(source, image_file + '.p%d' % part['num'])
> > 
> 
> Thank you for the info! I'll try to reproduce the issue.
> 
> Wild guess: would copying a file instead of renaming it solve the issue?
> 
> 
> Regards,
> Ed
> 
> > -----Original Message-----
> > From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> > Sent: Tuesday, October 27, 2015 8:27 PM
> > To: Ahsan, Noor
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: U-boot file is being removed when image-recipe is rebuild 
> > which uses wic.bz2
> > 
> > On Tue, Oct 27, 2015 at 02:28:02PM +0000, Ahsan, Noor wrote:
> > > Hello,
> > > 
> > > I have identified the issue and sent and RFC on mailing list. One feedback I received was replace dd_cmd = "cp src dst" with dd_cmd = "cp %s %s" % (src, dst) which was and off sight from me. I need more detailed review of it.
> > 
> > Can you point me out where the renaming you've mentioned is happening?
> > 
> > Anyway it would be good if I could reproduce the issue to better understand it. Can you provide more details about your configuration?
> > 
> > Regards,
> > Ed
> > 
> > 
> > > 
> > > Noor
> > > 
> > > -----Original Message-----
> > > From: Ed Bartosh [mailto:ed.bartosh@linux.intel.com]
> > > Sent: Tuesday, October 27, 2015 6:41 PM
> > > To: Ahsan, Noor
> > > Cc: openembedded-core@lists.openembedded.org
> > > Subject: Re: U-boot file is being removed when image-recipe is 
> > > rebuild which uses wic.bz2
> > > 
> > > Hi Noor,
> > > 
> > > Sorry for delay. I missed your e-mail somehow.
> > > 
> > > Can you please show your recipe, wks file and the details of your build configuration(MACHINE, bblayers, etc) ?
> > > 
> > > ---
> > > Regards,
> > > Ed
> > 
> > > Date: Thu, 22 Oct 2015 10:14:02 +0000
> > > From: "Ahsan, Noor" <Noor_Ahsan@mentor.com>
> > > To: "openembedded-core@lists.openembedded.org"
> > >  <openembedded-core@lists.openembedded.org>
> > > Subject: Re: [OE-core] [RFC] rawcopy.py: Copy source file in image 
> > > creation  build folder
> > > 
> > > 
> > > 
> > > -----Original Message-----
> > > From: Ahsan, Noor
> > > Sent: Wednesday, October 21, 2015 9:04 PM
> > > To: openembedded-core@lists.openembedded.org
> > > Cc: Ahsan, Noor
> > > Subject: [RFC] rawcopy.py: Copy source file in image creation build 
> > > folder
> > > 
> > > From: Noor <noor_ahsan@mentor.com>
> > > 
> > > * When a file is given using --sourceparams then wic directly use that file
> > >   instead of copying them to build folder. At time of assmebling it os.rename
> > >   is called which rename all the files to name. In that process the original
> > >   file get renamed. When image recipe is rebuilt then wic complains about
> > >   missing file which renamed in previous build.
> > > 
> > > Signed-off-by: Noor Ahsan <noor_ahsan@mentor.com>
> > > ---
> > >  scripts/lib/wic/plugins/source/rawcopy.py |    4 ++++
> > >  1 file changed, 4 insertions(+)
> > > 
> > > diff --git a/scripts/lib/wic/plugins/source/rawcopy.py
> > > b/scripts/lib/wic/plugins/source/rawcopy.py
> > > index f0691ba..f17da27 100644
> > > --- a/scripts/lib/wic/plugins/source/rawcopy.py
> > > +++ b/scripts/lib/wic/plugins/source/rawcopy.py
> > > @@ -74,6 +74,10 @@ class RawCopyPlugin(SourcePlugin):
> > >              dd_cmd = "dd if=%s of=%s ibs=%s skip=1 conv=notrunc" % \
> > >                      (src, dst, source_params['skip'])
> > >              exec_cmd(dd_cmd)
> > > +        else:
> > > +            dst = os.path.join(cr_workdir, source_params['file'])
> > > +            dd_cmd = "cp src dst"
> > > +            exec_cmd(dd_cmd)
> > > 
> > > 
> > > dd_cmd will be replaced by cp_cmd but I need review of all impact of this so that I can send proper patch.
> > > 
> > > 
> > >          # get the size in the right units for kickstart (kB)
> > >          du_cmd = "du -Lbks %s" % dst
> > > --
> > > 1.7.9.5
> > > 
> > > --
> > > _______________________________________________
> > > Openembedded-core mailing list
> > > Openembedded-core@lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > 
> > 
> > --
> > --
> > Regards,
> > Ed
> 
> --
> --
> Regards,
> Ed

-- 
--
Regards,
Ed