[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

Hi, 

I've been working on a minimal root filesystem made with "deby". 
The code is here and it includes BSP meta layers for qemux86-64, Beaglebone black,
Renesas iwg20m board and CycloneV.

https://gitlab.com/cip-playground/project-x/tree/master/deby

The meta-cip-common layer is where we put the packages that will be included
for all targets. The BSP meta-layers contain settings related to the architecture of
the processor, and Linux/u-boot configuration.

Currently, I have tested qemux86-64 and Renesas iwg20m. Both of them worked fine
so in the next step I will try to run tests on them using B at D.
# Beaglebone should work fine, but I haven't tested it yet.

For Cyclone V, I need a kernel repository.
Koguchi-san: could you create it on https://gitlab.com/cip-playground/linux-cip-cyclonev

Finally, for U-boot I need repositories to fetch the sources.
Should we have separate repositories for each board? For example:
https://gitlab.com/cip-playground/u-boot-iwg20m
https://gitlab.com/cip-playground/u-boot-cyclonev

Thanks,
Daniel

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Chris Paterson]

Hello Daniel,

> From: Daniel Sangorrin [mailto:daniel.sangorrin at toshiba.co.jp]
> Sent: 07 July 2017 09:51
> 
> Hi,
> 
> I've been working on a minimal root filesystem made with "deby".
> The code is here and it includes BSP meta layers for qemux86-64, Beaglebone
> black, Renesas iwg20m board and CycloneV.
> 
> https://gitlab.com/cip-playground/project-x/tree/master/deby
> 
> The meta-cip-common layer is where we put the packages that will be
> included for all targets. The BSP meta-layers contain settings related to the
> architecture of the processor, and Linux/u-boot configuration.
> 
> Currently, I have tested qemux86-64 and Renesas iwg20m. Both of them
> worked fine so in the next step I will try to run tests on them using B at D.
> # Beaglebone should work fine, but I haven't tested it yet.

Great to see you're making some good progress.

> 
> For Cyclone V, I need a kernel repository.
> Koguchi-san: could you create it on https://gitlab.com/cip-playground/linux-
> cip-cyclonev
> 
> Finally, for U-boot I need repositories to fetch the sources.
> Should we have separate repositories for each board? For example:
> https://gitlab.com/cip-playground/u-boot-iwg20m
> https://gitlab.com/cip-playground/u-boot-cyclonev

If the consensus is to use different u-boot versions/repos for each platform, (rather than push vendors to add support upstream), then this approach makes sense.

However, perhaps we should try and use exiting repositories provided by the vendors, rather than CIP create and maintain them. If we start hosting these repositories we will have to continue to maintain them forever.

Of course, the downside of using externally maintained repositories is that support may end at any time. Perhaps bringing this ownership/maintenance into the CIP project is something we want to do/fund.


Kind regards, Chris

> 
> Thanks,
> Daniel
> 
> 

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Jan Kiszka]

On 2017-07-07 10:50, Daniel Sangorrin wrote:
> Hi, 
> 
> I've been working on a minimal root filesystem made with "deby". 
> The code is here and it includes BSP meta layers for qemux86-64, Beaglebone black,
> Renesas iwg20m board and CycloneV.
> 
> https://gitlab.com/cip-playground/project-x/tree/master/deby
> 
> The meta-cip-common layer is where we put the packages that will be included
> for all targets. The BSP meta-layers contain settings related to the architecture of
> the processor, and Linux/u-boot configuration.
> 
> Currently, I have tested qemux86-64 and Renesas iwg20m. Both of them worked fine
> so in the next step I will try to run tests on them using B at D.
> # Beaglebone should work fine, but I haven't tested it yet.
> 
> For Cyclone V, I need a kernel repository.
> Koguchi-san: could you create it on https://gitlab.com/cip-playground/linux-cip-cyclonev
> 
> Finally, for U-boot I need repositories to fetch the sources.
> Should we have separate repositories for each board? For example:
> https://gitlab.com/cip-playground/u-boot-iwg20m
> https://gitlab.com/cip-playground/u-boot-cyclonev

Nice work! Maybe I find the time to model a binary Debian variant as
well, using Isar [1] - ideally prior to DebConf.

Btw, instead of having yet another setup.sh, you may want to have a look
at kas [2]. There is no release yet (next week, hopefully), but it is
fairly mature. Moreover, you can build inside Docker, removing most of
the host dependencies. We are using this now with both Yocto and Isar
projects, often in layered ways [3] that enable reuse.

Jan

[1] https://github.com/ilbers/isar
[2] https://github.com/siemens/kas
[3]
https://github.com/siemens/meta-iot2000/commit/eb4c3df4a62d0e5c32c74082e4c5c467627b9a84

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

Hi Jan,

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> Sent: Friday, July 07, 2017 7:40 PM
> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> 
> On 2017-07-07 10:50, Daniel Sangorrin wrote:
> > Hi,
> >
> > I've been working on a minimal root filesystem made with "deby".
> > The code is here and it includes BSP meta layers for qemux86-64, Beaglebone black,
> > Renesas iwg20m board and CycloneV.
> >
> > https://gitlab.com/cip-playground/project-x/tree/master/deby
> >
> > The meta-cip-common layer is where we put the packages that will be included
> > for all targets. The BSP meta-layers contain settings related to the architecture of
> > the processor, and Linux/u-boot configuration.
> >
> > Currently, I have tested qemux86-64 and Renesas iwg20m. Both of them worked fine
> > so in the next step I will try to run tests on them using B at D.
> > # Beaglebone should work fine, but I haven't tested it yet.
> >
> > For Cyclone V, I need a kernel repository.
> > Koguchi-san: could you create it on https://gitlab.com/cip-playground/linux-cip-cyclonev
> >
> > Finally, for U-boot I need repositories to fetch the sources.
> > Should we have separate repositories for each board? For example:
> > https://gitlab.com/cip-playground/u-boot-iwg20m
> > https://gitlab.com/cip-playground/u-boot-cyclonev
> 
> Nice work! Maybe I find the time to model a binary Debian variant as
> well, using Isar [1] - ideally prior to DebConf.

Cool. By the way, I put all of the layers under /deby so that you could
create the isar version under the /isar folder.

> 
> Btw, instead of having yet another setup.sh, you may want to have a look
> at kas [2]. There is no release yet (next week, hopefully), but it is
> fairly mature. Moreover, you can build inside Docker, removing most of
> the host dependencies. We are using this now with both Yocto and Isar
> projects, often in layered ways [3] that enable reuse.

Yesterday I managed to build the qemux86-64 meta layer using the following
kas project configuration. Could you check it please?

https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Thanks,
Daniel


> 
> Jan
> 
> [1] https://github.com/ilbers/isar
> [2] https://github.com/siemens/kas
> [3]
> https://github.com/siemens/meta-iot2000/commit/eb4c3df4a62d0e5c32c74082e4c5c467627b9a84

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Jan Kiszka]

On 2017-07-11 01:36, Daniel Sangorrin wrote:
>> Btw, instead of having yet another setup.sh, you may want to have a look
>> at kas [2]. There is no release yet (next week, hopefully), but it is
>> fairly mature. Moreover, you can build inside Docker, removing most of
>> the host dependencies. We are using this now with both Yocto and Isar
>> projects, often in layered ways [3] that enable reuse.
> 
> Yesterday I managed to build the qemux86-64 meta layer using the following
> kas project configuration. Could you check it please?
> 
> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19

Looks good. You should just set version to 0.10 and use yesterday's
release. You can pull the corresponding docker image from
https://hub.docker.com/r/kasproject/, and there should be a pip package
as well soon.

Does it still make a difference to set BB_NUMBER_THREADS and
PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
looking for reasonable defaults we can set (?=) from kas automatically.
BB_DISKMON_DIRS should likely be included, I was adding it as well already.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

Hi Jan,

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> Sent: Tuesday, July 11, 2017 3:20 PM
> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> Cc: kas-devel
> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> 
> On 2017-07-11 01:36, Daniel Sangorrin wrote:
> >> Btw, instead of having yet another setup.sh, you may want to have a look
> >> at kas [2]. There is no release yet (next week, hopefully), but it is
> >> fairly mature. Moreover, you can build inside Docker, removing most of
> >> the host dependencies. We are using this now with both Yocto and Isar
> >> projects, often in layered ways [3] that enable reuse.
> >
> > Yesterday I managed to build the qemux86-64 meta layer using the following
> > kas project configuration. Could you check it please?
> >
> > https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
> 
> Looks good. You should just set version to 0.10 and use yesterday's
> release. You can pull the corresponding docker image from
> https://hub.docker.com/r/kasproject/, and there should be a pip package
> as well soon.

That was fast. I was using the next branch and built the Dockerfile included in kas.

I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
volume mount the project-x folder and manually set the proxy settings (for some reason using
build-args didn't work for me).
 
> Does it still make a difference to set BB_NUMBER_THREADS and
> PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
> looking for reasonable defaults we can set (?=) from kas automatically.
> BB_DISKMON_DIRS should likely be included, I was adding it as well already.

You know better than me ;)
I just checked the latest ref manual and apparently OE automatically sets them up.

Thanks,
Daniel

> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RDA ITP SES-DE
> Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Jan Kiszka]

On 2017-07-11 08:34, Daniel Sangorrin wrote:
> Hi Jan,
> 
>> -----Original Message-----
>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>> Sent: Tuesday, July 11, 2017 3:20 PM
>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>> Cc: kas-devel
>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>
>> On 2017-07-11 01:36, Daniel Sangorrin wrote:
>>>> Btw, instead of having yet another setup.sh, you may want to have a look
>>>> at kas [2]. There is no release yet (next week, hopefully), but it is
>>>> fairly mature. Moreover, you can build inside Docker, removing most of
>>>> the host dependencies. We are using this now with both Yocto and Isar
>>>> projects, often in layered ways [3] that enable reuse.
>>>
>>> Yesterday I managed to build the qemux86-64 meta layer using the following
>>> kas project configuration. Could you check it please?
>>>
>>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
>>
>> Looks good. You should just set version to 0.10 and use yesterday's
>> release. You can pull the corresponding docker image from
>> https://hub.docker.com/r/kasproject/, and there should be a pip package
>> as well soon.
> 
> That was fast. I was using the next branch and built the Dockerfile included in kas.
> 
> I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,

We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
it there a way to achieve this in an easier way? Still learning all this
docker stuff.

> volume mount the project-x folder and manually set the proxy settings (for some reason using
> build-args didn't work for me).

Hmm, they should, I'm using them for builds all the time (along with
corresponding settings for the docker daemon).

>  
>> Does it still make a difference to set BB_NUMBER_THREADS and
>> PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
>> looking for reasonable defaults we can set (?=) from kas automatically.
>> BB_DISKMON_DIRS should likely be included, I was adding it as well already.
> 
> You know better than me ;)
> I just checked the latest ref manual and apparently OE automatically sets them up.

Ah, good to know. The parallel things, or even also DISKMON?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> Sent: Tuesday, July 11, 2017 3:42 PM
> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> Cc: 'kas-devel'
> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> 
> On 2017-07-11 08:34, Daniel Sangorrin wrote:
> > Hi Jan,
> >
> >> -----Original Message-----
> >> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >> Sent: Tuesday, July 11, 2017 3:20 PM
> >> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >> Cc: kas-devel
> >> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>
> >> On 2017-07-11 01:36, Daniel Sangorrin wrote:
> >>>> Btw, instead of having yet another setup.sh, you may want to have a look
> >>>> at kas [2]. There is no release yet (next week, hopefully), but it is
> >>>> fairly mature. Moreover, you can build inside Docker, removing most of
> >>>> the host dependencies. We are using this now with both Yocto and Isar
> >>>> projects, often in layered ways [3] that enable reuse.
> >>>
> >>> Yesterday I managed to build the qemux86-64 meta layer using the following
> >>> kas project configuration. Could you check it please?
> >>>
> >>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
> >>
> >> Looks good. You should just set version to 0.10 and use yesterday's
> >> release. You can pull the corresponding docker image from
> >> https://hub.docker.com/r/kasproject/, and there should be a pip package
> >> as well soon.
> >
> > That was fast. I was using the next branch and built the Dockerfile included in kas.
> >
> > I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
> 
> We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
> it there a way to achieve this in an easier way? Still learning all this
> docker stuff.

I was using the -e flag:
  docker run -e USER_ID=`id -u $USER` -it kas sh

> > volume mount the project-x folder and manually set the proxy settings (for some reason using
> > build-args didn't work for me).
> 
> Hmm, they should, I'm using them for builds all the time (along with
> corresponding settings for the docker daemon).

Are you sure you didn't set them somewhere else in the docker configuration?
As far as I can see the Dockerfile is not using any ARG instruction at all.

> >> Does it still make a difference to set BB_NUMBER_THREADS and
> >> PARALLEL_MAKE? I'm seeing full CPU usage without it as well. I'm always
> >> looking for reasonable defaults we can set (?=) from kas automatically.
> >> BB_DISKMON_DIRS should likely be included, I was adding it as well already.
> >
> > You know better than me ;)
> > I just checked the latest ref manual and apparently OE automatically sets them up.
> 
> Ah, good to know. The parallel things, or even also DISKMON?

The parallel things.  BB_DISKMON_DIRS seems to be disabled by default.

Thanks,
Daniel

> 
> Jan
> 
> --
> Siemens AG, Corporate Technology, CT RDA ITP SES-DE
> Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Jan Kiszka]

On 2017-07-11 09:00, Daniel Sangorrin wrote:
>> -----Original Message-----
>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>> Sent: Tuesday, July 11, 2017 3:42 PM
>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>> Cc: 'kas-devel'
>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>
>> On 2017-07-11 08:34, Daniel Sangorrin wrote:
>>> Hi Jan,
>>>
>>>> -----Original Message-----
>>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>>>> Sent: Tuesday, July 11, 2017 3:20 PM
>>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>>>> Cc: kas-devel
>>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>>>
>>>> On 2017-07-11 01:36, Daniel Sangorrin wrote:
>>>>>> Btw, instead of having yet another setup.sh, you may want to have a look
>>>>>> at kas [2]. There is no release yet (next week, hopefully), but it is
>>>>>> fairly mature. Moreover, you can build inside Docker, removing most of
>>>>>> the host dependencies. We are using this now with both Yocto and Isar
>>>>>> projects, often in layered ways [3] that enable reuse.
>>>>>
>>>>> Yesterday I managed to build the qemux86-64 meta layer using the following
>>>>> kas project configuration. Could you check it please?
>>>>>
>>>>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
>>>>
>>>> Looks good. You should just set version to 0.10 and use yesterday's
>>>> release. You can pull the corresponding docker image from
>>>> https://hub.docker.com/r/kasproject/, and there should be a pip package
>>>> as well soon.
>>>
>>> That was fast. I was using the next branch and built the Dockerfile included in kas.
>>>
>>> I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
>>
>> We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
>> it there a way to achieve this in an easier way? Still learning all this
>> docker stuff.
> 
> I was using the -e flag:
>   docker run -e USER_ID=`id -u $USER` -it kas sh

Err, right, that's what I meant.

> 
>>> volume mount the project-x folder and manually set the proxy settings (for some reason using
>>> build-args didn't work for me).
>>
>> Hmm, they should, I'm using them for builds all the time (along with
>> corresponding settings for the docker daemon).
> 
> Are you sure you didn't set them somewhere else in the docker configuration?
> As far as I can see the Dockerfile is not using any ARG instruction at all.

Yes, I'm sure. I've just written the doc for a product SDK, and that
uses no further settings as well, see
https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

Hi Jan,

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> Sent: Wednesday, July 12, 2017 6:13 PM
> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> Cc: 'kas-devel'
> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> 
> On 2017-07-11 09:00, Daniel Sangorrin wrote:
> >> -----Original Message-----
> >> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >> Sent: Tuesday, July 11, 2017 3:42 PM
> >> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >> Cc: 'kas-devel'
> >> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>
> >> On 2017-07-11 08:34, Daniel Sangorrin wrote:
> >>> Hi Jan,
> >>>
> >>>> -----Original Message-----
> >>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >>>> Sent: Tuesday, July 11, 2017 3:20 PM
> >>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >>>> Cc: kas-devel
> >>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>>>
> >>>> On 2017-07-11 01:36, Daniel Sangorrin wrote:
> >>>>>> Btw, instead of having yet another setup.sh, you may want to have a look
> >>>>>> at kas [2]. There is no release yet (next week, hopefully), but it is
> >>>>>> fairly mature. Moreover, you can build inside Docker, removing most of
> >>>>>> the host dependencies. We are using this now with both Yocto and Isar
> >>>>>> projects, often in layered ways [3] that enable reuse.
> >>>>>
> >>>>> Yesterday I managed to build the qemux86-64 meta layer using the following
> >>>>> kas project configuration. Could you check it please?
> >>>>>
> >>>>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
> >>>>
> >>>> Looks good. You should just set version to 0.10 and use yesterday's
> >>>> release. You can pull the corresponding docker image from
> >>>> https://hub.docker.com/r/kasproject/, and there should be a pip package
> >>>> as well soon.
> >>>
> >>> That was fast. I was using the next branch and built the Dockerfile included in kas.
> >>>
> >>> I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
> >>
> >> We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
> >> it there a way to achieve this in an easier way? Still learning all this
> >> docker stuff.
> >
> > I was using the -e flag:
> >   docker run -e USER_ID=`id -u $USER` -it kas sh
> 
> Err, right, that's what I meant.
> 
> >
> >>> volume mount the project-x folder and manually set the proxy settings (for some reason using
> >>> build-args didn't work for me).
> >>
> >> Hmm, they should, I'm using them for builds all the time (along with
> >> corresponding settings for the docker daemon).
> >
> > Are you sure you didn't set them somewhere else in the docker configuration?
> > As far as I can see the Dockerfile is not using any ARG instruction at all.
> 
> Yes, I'm sure. I've just written the doc for a product SDK, and that
> uses no further settings as well, see
> https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.

hmmm, I think that you made a small mistake in the kas documentation [1]. You wrote that
http_proxy should be passed as a build-arg but as I said there is no ARG instruction in the
Dockerfile. 

In the meta-iot2000 link you are setting the proxy variables as environment variables using 
'docker run -e http_proxy=$http_proxy'  which does work indeed.

[1] https://kas.readthedocs.io/en/latest/devguide.html

Thanks,
Daniel

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Jan Kiszka]

On 2017-07-13 02:12, Daniel Sangorrin wrote:
> Hi Jan,
> 
>> -----Original Message-----
>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>> Sent: Wednesday, July 12, 2017 6:13 PM
>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>> Cc: 'kas-devel'
>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>
>> On 2017-07-11 09:00, Daniel Sangorrin wrote:
>>>> -----Original Message-----
>>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>>>> Sent: Tuesday, July 11, 2017 3:42 PM
>>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>>>> Cc: 'kas-devel'
>>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>>>
>>>> On 2017-07-11 08:34, Daniel Sangorrin wrote:
>>>>> Hi Jan,
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
>>>>>> Sent: Tuesday, July 11, 2017 3:20 PM
>>>>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
>>>>>> Cc: kas-devel
>>>>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
>>>>>>
>>>>>> On 2017-07-11 01:36, Daniel Sangorrin wrote:
>>>>>>>> Btw, instead of having yet another setup.sh, you may want to have a look
>>>>>>>> at kas [2]. There is no release yet (next week, hopefully), but it is
>>>>>>>> fairly mature. Moreover, you can build inside Docker, removing most of
>>>>>>>> the host dependencies. We are using this now with both Yocto and Isar
>>>>>>>> projects, often in layered ways [3] that enable reuse.
>>>>>>>
>>>>>>> Yesterday I managed to build the qemux86-64 meta layer using the following
>>>>>>> kas project configuration. Could you check it please?
>>>>>>>
>>>>>>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
>>>>>>
>>>>>> Looks good. You should just set version to 0.10 and use yesterday's
>>>>>> release. You can pull the corresponding docker image from
>>>>>> https://hub.docker.com/r/kasproject/, and there should be a pip package
>>>>>> as well soon.
>>>>>
>>>>> That was fast. I was using the next branch and built the Dockerfile included in kas.
>>>>>
>>>>> I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
>>>>
>>>> We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
>>>> it there a way to achieve this in an easier way? Still learning all this
>>>> docker stuff.
>>>
>>> I was using the -e flag:
>>>   docker run -e USER_ID=`id -u $USER` -it kas sh
>>
>> Err, right, that's what I meant.
>>
>>>
>>>>> volume mount the project-x folder and manually set the proxy settings (for some reason using
>>>>> build-args didn't work for me).
>>>>
>>>> Hmm, they should, I'm using them for builds all the time (along with
>>>> corresponding settings for the docker daemon).
>>>
>>> Are you sure you didn't set them somewhere else in the docker configuration?
>>> As far as I can see the Dockerfile is not using any ARG instruction at all.
>>
>> Yes, I'm sure. I've just written the doc for a product SDK, and that
>> uses no further settings as well, see
>> https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.
> 
> hmmm, I think that you made a small mistake in the kas documentation [1]. You wrote that
> http_proxy should be passed as a build-arg but as I said there is no ARG instruction in the
> Dockerfile. 

No, also that works: It won't leave the proxy settings behind in the
image, but it allows to build it behind a proxy.

We don't want to generate "proxified" images that will then only work in
a very specific environment. We are coming from there, it was a mess.
The image we have now can be fed into a CI build, and you only need to
configure the specific proxies at project level. Works nicely with our
corporate gitlab infrastructure.

Jan

> 
> In the meta-iot2000 link you are setting the proxy variables as environment variables using 
> 'docker run -e http_proxy=$http_proxy'  which does work indeed.>
> [1] https://kas.readthedocs.io/en/latest/devguide.html
> 
> Thanks,
> Daniel
> 
> 
> 
> 

-- 
Siemens AG, Corporate Technology, CT RDA ITP SES-DE
Corporate Competence Center Embedded Linux

[cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support

[Daniel Sangorrin]

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> Sent: Thursday, July 13, 2017 3:37 PM
> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> Cc: 'kas-devel'
> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> 
> On 2017-07-13 02:12, Daniel Sangorrin wrote:
> > Hi Jan,
> >
> >> -----Original Message-----
> >> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >> Sent: Wednesday, July 12, 2017 6:13 PM
> >> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >> Cc: 'kas-devel'
> >> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>
> >> On 2017-07-11 09:00, Daniel Sangorrin wrote:
> >>>> -----Original Message-----
> >>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >>>> Sent: Tuesday, July 11, 2017 3:42 PM
> >>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >>>> Cc: 'kas-devel'
> >>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>>>
> >>>> On 2017-07-11 08:34, Daniel Sangorrin wrote:
> >>>>> Hi Jan,
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Jan Kiszka [mailto:jan.kiszka at siemens.com]
> >>>>>> Sent: Tuesday, July 11, 2017 3:20 PM
> >>>>>> To: Daniel Sangorrin; cip-dev at lists.cip-project.org
> >>>>>> Cc: kas-devel
> >>>>>> Subject: Re: [cip-dev] Project-X (minimal root filesystem) : rewrite and renesas iwg20m board support
> >>>>>>
> >>>>>> On 2017-07-11 01:36, Daniel Sangorrin wrote:
> >>>>>>>> Btw, instead of having yet another setup.sh, you may want to have a look
> >>>>>>>> at kas [2]. There is no release yet (next week, hopefully), but it is
> >>>>>>>> fairly mature. Moreover, you can build inside Docker, removing most of
> >>>>>>>> the host dependencies. We are using this now with both Yocto and Isar
> >>>>>>>> projects, often in layered ways [3] that enable reuse.
> >>>>>>>
> >>>>>>> Yesterday I managed to build the qemux86-64 meta layer using the following
> >>>>>>> kas project configuration. Could you check it please?
> >>>>>>>
> >>>>>>> https://gitlab.com/cip-playground/project-x/commit/de611ae06040f6e426d3c5d1b77eb20b4af31a19
> >>>>>>
> >>>>>> Looks good. You should just set version to 0.10 and use yesterday's
> >>>>>> release. You can pull the corresponding docker image from
> >>>>>> https://hub.docker.com/r/kasproject/, and there should be a pip package
> >>>>>> as well soon.
> >>>>>
> >>>>> That was fast. I was using the next branch and built the Dockerfile included in kas.
> >>>>>
> >>>>> I had to create a daemon.json to set overlay2 as the storage driver, pass the USER_ID variable,
> >>>>
> >>>> We should probably suggest to call "USER_ID=$(id -u) docker run ..." -
> >>>> it there a way to achieve this in an easier way? Still learning all this
> >>>> docker stuff.
> >>>
> >>> I was using the -e flag:
> >>>   docker run -e USER_ID=`id -u $USER` -it kas sh
> >>
> >> Err, right, that's what I meant.
> >>
> >>>
> >>>>> volume mount the project-x folder and manually set the proxy settings (for some reason using
> >>>>> build-args didn't work for me).
> >>>>
> >>>> Hmm, they should, I'm using them for builds all the time (along with
> >>>> corresponding settings for the docker daemon).
> >>>
> >>> Are you sure you didn't set them somewhere else in the docker configuration?
> >>> As far as I can see the Dockerfile is not using any ARG instruction at all.
> >>
> >> Yes, I'm sure. I've just written the doc for a product SDK, and that
> >> uses no further settings as well, see
> >> https://github.com/siemens/meta-iot2000/blob/jan/staging/README.md#docker-build.
> >
> > hmmm, I think that you made a small mistake in the kas documentation [1]. You wrote that
> > http_proxy should be passed as a build-arg but as I said there is no ARG instruction in the
> > Dockerfile.
> 
> No, also that works: It won't leave the proxy settings behind in the
> image, but it allows to build it behind a proxy.
> 
> We don't want to generate "proxified" images that will then only work in
> a very specific environment. We are coming from there, it was a mess.
> The image we have now can be fed into a CI build, and you only need to
> configure the specific proxies at project level. Works nicely with our
> corporate gitlab infrastructure.

OK, I understand your approach.

By the way ARG does not leave any settings behind (ENV does).
I was just confused because I thought that all available build-args had
to be declared on the Dockerfile through the ARG instruction. 

In the past, not doing so would throw an error. From the current docker 
documentation [1], it seems that that has changed and now it should 
only throw a warning:
  "If a user specifies a build argument that was not defined in the Dockerfile, 
  the build outputs a warning. [Warning] One or more build-args [foo] were not consumed."

Strange enough, I didn't see any warnings either. I wonder if the documentation
is outdated. 

In any case, the kas documentation does not mention anything about setting the
variables on "docker run" so it might be a good idea to provide the information 
form the meta-iot2000's README

If you are building from within a proxy-restricted network, make sure the settings are available via the standard environment variables and add

    -e http_proxy=$http_proxy -e https_proxy=$https_proxy \
    -e ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy

Thanks,
Daniel 

[1] https://docs.docker.com/engine/reference/builder/#arg




Sorry I didn't know that you could use --build-arg
without defining an ARG instruction in the Dockerfile. It's not very clearly documented
on the docker website.

On the user's guide you mention "docker run -it kasproject/kas:<version> sh"

 or bind-mount the project into the container. See https://hub.docker.com/r/kasproject for all available images."
Still it might be helpful 

    -e http_proxy=$http_proxy -e https_proxy=$https_proxy \
    -e ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy




> 
> Jan
> 
> >
> > In the meta-iot2000 link you are setting the proxy variables as environment variables using
> > 'docker run -e http_proxy=$http_proxy'  which does work indeed.>
> > [1] https://kas.readthedocs.io/en/latest/devguide.html
> >
> > Thanks,
> > Daniel
> >
> >
> >
> >
> 
> --
> Siemens AG, Corporate Technology, CT RDA ITP SES-DE
> Corporate Competence Center Embedded Linux