[PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[Khem Raj]

Could fix issues where dirs with wrong permission is created with mkdir -p

Signed-off-by: Khem Raj <raj.khem@gmail.com>
---
 meta/classes/libc-package.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-package.bbclass
index 0b4c666a74..34025635ce 100644
--- a/meta/classes/libc-package.bbclass
+++ b/meta/classes/libc-package.bbclass
@@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
 do_prep_locale_tree() {
 	treedir=${WORKDIR}/locale-tree
 	rm -rf $treedir
-	mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
+	install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
 	tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
 	# unzip to avoid parsing errors
 	for i in $treedir/${datadir}/i18n/charmaps/*gz; do 
-- 
2.20.1

Re: [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org <openembedded-
> core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
> Sent: den 12 februari 2019 06:48
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> utility to create directories with right permissions
> 
> Could fix issues where dirs with wrong permission is created with mkdir -p
> 
> Signed-off-by: Khem Raj <raj.khem@gmail.com>
> ---
>  meta/classes/libc-package.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-
> package.bbclass
> index 0b4c666a74..34025635ce 100644
> --- a/meta/classes/libc-package.bbclass
> +++ b/meta/classes/libc-package.bbclass
> @@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
>  do_prep_locale_tree() {
>  	treedir=${WORKDIR}/locale-tree
>  	rm -rf $treedir
> -	mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> +	install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}

This does not change anything (install -d and mkdir -p does the same thing). 

>  	tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
>  	# unzip to avoid parsing errors
>  	for i in $treedir/${datadir}/i18n/charmaps/*gz; do
>		gunzip $i
>	done
>	tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}

This is where the problem happens. When the files are copied from 
${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that 
of ${LOCALETREESRC}${localedir}. However, I now realized that the problem 
may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES, 
which causes the above line to copy things it was never meant to copy, in 
this case ${LOCALETREESRC}${localedir}. I tried to just remove the line 
altogether and then it built just fine. But I do not know what the 
consequences are of removing it... I guess it is related to setting up an 
environment for qemu to run in, but since that seems to work fine without 
the files copied by the above line, maybe it is not needed? Or maybe it 
is just not needed when having usrmerge in DISTRO_FEATURES? I tried to 
backtrack it through the commit logs, but it was introduced with the 
do_prep_locale_tree() function, so no help there.

> --
> 2.20.1

//Peter

Re: [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[Khem Raj]

On Wed, Feb 13, 2019 at 10:05 PM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org <openembedded-
> > core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
> > Sent: den 12 februari 2019 06:48
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> > utility to create directories with right permissions
> >
> > Could fix issues where dirs with wrong permission is created with mkdir -p
> >
> > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > ---
> >  meta/classes/libc-package.bbclass | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-
> > package.bbclass
> > index 0b4c666a74..34025635ce 100644
> > --- a/meta/classes/libc-package.bbclass
> > +++ b/meta/classes/libc-package.bbclass
> > @@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
> >  do_prep_locale_tree() {
> >       treedir=${WORKDIR}/locale-tree
> >       rm -rf $treedir
> > -     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> > +     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
>
> This does not change anything (install -d and mkdir -p does the same thing).
>
> >       tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
> >       # unzip to avoid parsing errors
> >       for i in $treedir/${datadir}/i18n/charmaps/*gz; do
> >               gunzip $i
> >       done
> >       tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}
>
> This is where the problem happens. When the files are copied from
> ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed to that
> of ${LOCALETREESRC}${localedir}. However, I now realized that the problem
> may be due to the fact that I have usrmerge enabled in DISTRO_FEATURES,
> which causes the above line to copy things it was never meant to copy, in
> this case ${LOCALETREESRC}${localedir}. I tried to just remove the line
> altogether and then it built just fine. But I do not know what the
> consequences are of removing it... I guess it is related to setting up an
> environment for qemu to run in, but since that seems to work fine without
> the files copied by the above line, maybe it is not needed? Or maybe it
> is just not needed when having usrmerge in DISTRO_FEATURES? I tried to
> backtrack it through the commit logs, but it was introduced with the
> do_prep_locale_tree() function, so no help there.
>

I wonder if we should use --preserve option to tar -xf call here for
making it explicit.

> > --
> > 2.20.1
>
> //Peter
>

Re: [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[Peter Kjellerstedt]

> -----Original Message-----
> From: Khem Raj <raj.khem@gmail.com>
> Sent: den 14 februari 2019 16:56
> To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Cc: openembedded-core@lists.openembedded.org
> Subject: Re: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> utility to create directories with right permissions
> 
> On Wed, Feb 13, 2019 at 10:05 PM Peter Kjellerstedt
> <peter.kjellerstedt@axis.com> wrote:
> >
> > > -----Original Message-----
> > > From: openembedded-core-bounces@lists.openembedded.org
> <openembedded-
> > > core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
> > > Sent: den 12 februari 2019 06:48
> > > To: openembedded-core@lists.openembedded.org
> > > Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> > > utility to create directories with right permissions
> > >
> > > Could fix issues where dirs with wrong permission is created with mkdir -p
> > >
> > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > ---
> > >  meta/classes/libc-package.bbclass | 2 +-
> > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-package.bbclass
> > > index 0b4c666a74..34025635ce 100644
> > > --- a/meta/classes/libc-package.bbclass
> > > +++ b/meta/classes/libc-package.bbclass
> > > @@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
> > >  do_prep_locale_tree() {
> > >       treedir=${WORKDIR}/locale-tree
> > >       rm -rf $treedir
> > > -     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> > > +     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> >
> > This does not change anything (install -d and mkdir -p does the same
> > thing).
> >
> > >       tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
> > >       # unzip to avoid parsing errors
> > >       for i in $treedir/${datadir}/i18n/charmaps/*gz; do
> > >               gunzip $i
> > >       done
> > >       tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}
> >
> > This is where the problem happens. When the files are copied from
> > ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed 
> > to that of ${LOCALETREESRC}${localedir}. However, I now realized 
> > that the problem may be due to the fact that I have usrmerge enabled 
> > in DISTRO_FEATURES, which causes the above line to copy things it 
> > was never meant to copy, in this case ${LOCALETREESRC}${localedir}. 
> > I tried to just remove the line altogether and then it built just 
> > fine. But I do not know what the consequences are of removing it...

It turned out that the consequence is that if one actually does use qemu 
to generate the locale files, then it fails without that line. However, 
since qemu is not used by default for this since quite a long time ago, 
it is not needed in the normal case.
 
> > I guess it is related to setting up an environment for qemu to run 
> > in, but since that seems to work fine without the files copied by 
> > the above line, maybe it is not needed? Or maybe it is just not 
> > needed when having usrmerge in DISTRO_FEATURES? I tried to 
> > backtrack it through the commit logs, but it was introduced with the
> > do_prep_locale_tree() function, so no help there.
> 
> I wonder if we should use --preserve option to tar -xf call here for
> making it explicit.

I have added a pattern to the tar command that unpacks the files so 
only the files that are needed by qemu are unpacked (see the patch 
series I just sent). That should avoid the problem.

However, I wonder if anyone actually sets 
LOCALE_GENERATION_WITH_CROSS-LOCALEDEF to 0 (it is 1 by default), and 
uses qemu to generate the locale files? It takes quite a lot longer 
(four times on my machine, totaling nearly half an hour). Otherwise, 
if we removed that, the code could be simplified and avoid the copying 
of these files in the first place.

> > > --
> > > 2.20.1
> >
> > //Peter

//Peter

Re: [PATCH 2/3] libc-package.bbclass: Use install utility to create directories with right permissions

[Khem Raj]

On Sun, Feb 17, 2019 at 7:04 AM Peter Kjellerstedt
<peter.kjellerstedt@axis.com> wrote:
>
> > -----Original Message-----
> > From: Khem Raj <raj.khem@gmail.com>
> > Sent: den 14 februari 2019 16:56
> > To: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> > Cc: openembedded-core@lists.openembedded.org
> > Subject: Re: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> > utility to create directories with right permissions
> >
> > On Wed, Feb 13, 2019 at 10:05 PM Peter Kjellerstedt
> > <peter.kjellerstedt@axis.com> wrote:
> > >
> > > > -----Original Message-----
> > > > From: openembedded-core-bounces@lists.openembedded.org
> > <openembedded-
> > > > core-bounces@lists.openembedded.org> On Behalf Of Khem Raj
> > > > Sent: den 12 februari 2019 06:48
> > > > To: openembedded-core@lists.openembedded.org
> > > > Subject: [OE-core] [PATCH 2/3] libc-package.bbclass: Use install
> > > > utility to create directories with right permissions
> > > >
> > > > Could fix issues where dirs with wrong permission is created with mkdir -p
> > > >
> > > > Signed-off-by: Khem Raj <raj.khem@gmail.com>
> > > > ---
> > > >  meta/classes/libc-package.bbclass | 2 +-
> > > >  1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/meta/classes/libc-package.bbclass b/meta/classes/libc-package.bbclass
> > > > index 0b4c666a74..34025635ce 100644
> > > > --- a/meta/classes/libc-package.bbclass
> > > > +++ b/meta/classes/libc-package.bbclass
> > > > @@ -61,7 +61,7 @@ LOCALETREESRC ?= "${PKGD}"
> > > >  do_prep_locale_tree() {
> > > >       treedir=${WORKDIR}/locale-tree
> > > >       rm -rf $treedir
> > > > -     mkdir -p $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> > > > +     install -d $treedir/${base_bindir} $treedir/${base_libdir} $treedir/${datadir} $treedir/${localedir}
> > >
> > > This does not change anything (install -d and mkdir -p does the same
> > > thing).
> > >
> > > >       tar -cf - -C ${LOCALETREESRC}${datadir} -p i18n | tar -xf - -C $treedir/${datadir}
> > > >       # unzip to avoid parsing errors
> > > >       for i in $treedir/${datadir}/i18n/charmaps/*gz; do
> > > >               gunzip $i
> > > >       done
> > > >       tar -cf - -C ${LOCALETREESRC}${base_libdir} -p . | tar -xf - -C $treedir/${base_libdir}
> > >
> > > This is where the problem happens. When the files are copied from
> > > ${LOCALETREESRC}, the ownership of $treedir${localedir} is changed
> > > to that of ${LOCALETREESRC}${localedir}. However, I now realized
> > > that the problem may be due to the fact that I have usrmerge enabled
> > > in DISTRO_FEATURES, which causes the above line to copy things it
> > > was never meant to copy, in this case ${LOCALETREESRC}${localedir}.
> > > I tried to just remove the line altogether and then it built just
> > > fine. But I do not know what the consequences are of removing it...
>
> It turned out that the consequence is that if one actually does use qemu
> to generate the locale files, then it fails without that line. However,
> since qemu is not used by default for this since quite a long time ago,
> it is not needed in the normal case.
>
> > > I guess it is related to setting up an environment for qemu to run
> > > in, but since that seems to work fine without the files copied by
> > > the above line, maybe it is not needed? Or maybe it is just not
> > > needed when having usrmerge in DISTRO_FEATURES? I tried to
> > > backtrack it through the commit logs, but it was introduced with the
> > > do_prep_locale_tree() function, so no help there.
> >
> > I wonder if we should use --preserve option to tar -xf call here for
> > making it explicit.
>
> I have added a pattern to the tar command that unpacks the files so
> only the files that are needed by qemu are unpacked (see the patch
> series I just sent). That should avoid the problem.
>
> However, I wonder if anyone actually sets
> LOCALE_GENERATION_WITH_CROSS-LOCALEDEF to 0 (it is 1 by default), and
> uses qemu to generate the locale files? It takes quite a lot longer
> (four times on my machine, totaling nearly half an hour). Otherwise,
> if we removed that, the code could be simplified and avoid the copying
> of these files in the first place.

I think the option to not use cross-localedef should be kept alive,
even if its slower, its
helpful in cross validating issues when we might have them with cross localedef

>
> > > > --
> > > > 2.20.1
> > >
> > > //Peter
>
> //Peter
>