Openembedded Core Discussions
 help / color / mirror / Atom feed
* [PATCH 1/2] busybox: enable unicode support
@ 2019-07-17  9:08 Mikko Rapeli
  2019-07-17  9:08 ` [PATCH 2/2] cve-check.bbclass: initialize to_append Mikko Rapeli
  2019-07-17 11:44 ` [PATCH 1/2] busybox: enable unicode support Richard Purdie
  0 siblings, 2 replies; 6+ messages in thread
From: Mikko Rapeli @ 2019-07-17  9:08 UTC (permalink / raw)
  To: openembedded-core

While creating and deleting files with unicode or other
encodings works, it's annoying when ls and other core utils
show questionmarks instead of the unicode characters.
In 2019, it's quite common that users of embedded devices
based on yocto need unicode support. Debugging a box with
unicode encoded file names is a bit annoying when core utils
from busybox don't support them.

The unicode config fragment has the same config as Debian in their
deb and udeb builds of version 1:1.30.1-4.

If developers do not want this or other default yocto features in busybox,
or optimize the configuration for size, then they likely run a completely
custom configuration. Thus I think it's safe to enable unicode support
by default.

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 meta/recipes-core/busybox/busybox/unicode.cfg | 10 ++++++++++
 meta/recipes-core/busybox/busybox_1.31.0.bb   |  1 +
 2 files changed, 11 insertions(+)
 create mode 100644 meta/recipes-core/busybox/busybox/unicode.cfg

diff --git a/meta/recipes-core/busybox/busybox/unicode.cfg b/meta/recipes-core/busybox/busybox/unicode.cfg
new file mode 100644
index 0000000..ecb5a64
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/unicode.cfg
@@ -0,0 +1,10 @@
+CONFIG_UNICODE_SUPPORT=y
+# CONFIG_UNICODE_USING_LOCALE is not set
+CONFIG_FEATURE_CHECK_UNICODE_IN_ENV=y
+CONFIG_SUBST_WCHAR=63
+CONFIG_LAST_SUPPORTED_WCHAR=767
+CONFIG_UNICODE_COMBINING_WCHARS=y
+CONFIG_UNICODE_WIDE_WCHARS=y
+# CONFIG_UNICODE_BIDI_SUPPORT is not set
+# CONFIG_UNICODE_NEUTRAL_TABLE is not set
+# CONFIG_UNICODE_PRESERVE_BROKEN is not set
diff --git a/meta/recipes-core/busybox/busybox_1.31.0.bb b/meta/recipes-core/busybox/busybox_1.31.0.bb
index f018543..c1da372 100644
--- a/meta/recipes-core/busybox/busybox_1.31.0.bb
+++ b/meta/recipes-core/busybox/busybox_1.31.0.bb
@@ -37,6 +37,7 @@ SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
            ${@["", "file://init.cfg"][(d.getVar('VIRTUAL-RUNTIME_init_manager') == 'busybox')]} \
            ${@["", "file://mdev.cfg"][(d.getVar('VIRTUAL-RUNTIME_dev_manager') == 'busybox-mdev')]} \
            file://syslog.cfg \
+           file://unicode.cfg \
            file://inittab \
            file://rcS \
            file://rcK \
-- 
1.9.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread
* [PATCH 2/2] cve-check.bbclass: initialize to_append
  2019-07-17  9:08 [PATCH 1/2] busybox: enable unicode support Mikko Rapeli
@ 2019-07-17  9:08 ` Mikko Rapeli
  2019-07-17 11:44 ` [PATCH 1/2] busybox: enable unicode support Richard Purdie
  1 sibling, 0 replies; 6+ messages in thread
From: Mikko Rapeli @ 2019-07-17  9:08 UTC (permalink / raw)
  To: openembedded-core

Fixes build failure with core-image-minimal:

Exception: UnboundLocalError: local variable 'to_append' referenced before assignment

Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
---
 meta/classes/cve-check.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/cve-check.bbclass b/meta/classes/cve-check.bbclass
index ffd6243..9a31b3c 100644
--- a/meta/classes/cve-check.bbclass
+++ b/meta/classes/cve-check.bbclass
@@ -204,6 +204,7 @@ def check_cves(d, patched_cves):
             elif cve in patched_cves:
                 bb.note("%s has been patched" % (cve))
             else:
+                to_append = False
                 if (operator_start == '=' and pv == version_start):
                     cves_unpatched.append(cve)
                 else:
-- 
1.9.1



^ permalink raw reply related	[flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] busybox: enable unicode support
  2019-07-17  9:08 [PATCH 1/2] busybox: enable unicode support Mikko Rapeli
  2019-07-17  9:08 ` [PATCH 2/2] cve-check.bbclass: initialize to_append Mikko Rapeli
@ 2019-07-17 11:44 ` Richard Purdie
  2019-07-17 12:24   ` Mikko.Rapeli
  1 sibling, 1 reply; 6+ messages in thread
From: Richard Purdie @ 2019-07-17 11:44 UTC (permalink / raw)
  To: Mikko Rapeli, openembedded-core

On Wed, 2019-07-17 at 12:08 +0300, Mikko Rapeli wrote:
> While creating and deleting files with unicode or other
> encodings works, it's annoying when ls and other core utils
> show questionmarks instead of the unicode characters.
> In 2019, it's quite common that users of embedded devices
> based on yocto need unicode support. Debugging a box with
> unicode encoded file names is a bit annoying when core utils
> from busybox don't support them.
> 
> The unicode config fragment has the same config as Debian in their
> deb and udeb builds of version 1:1.30.1-4.
> 
> If developers do not want this or other default yocto features in
> busybox,
> or optimize the configuration for size, then they likely run a
> completely
> custom configuration. Thus I think it's safe to enable unicode
> support
> by default.
> 
> Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> ---
>  meta/recipes-core/busybox/busybox/unicode.cfg | 10 ++++++++++
>  meta/recipes-core/busybox/busybox_1.31.0.bb   |  1 +
>  2 files changed, 11 insertions(+)
>  create mode 100644 meta/recipes-core/busybox/busybox/unicode.cfg

Can you give an idea of the size implications of this change?

Cheers,

Richard



^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] busybox: enable unicode support
  2019-07-17 11:44 ` [PATCH 1/2] busybox: enable unicode support Richard Purdie
@ 2019-07-17 12:24   ` Mikko.Rapeli
  2019-07-17 13:26     ` Burton, Ross
  2019-07-20  2:31     ` Khem Raj
  0 siblings, 2 replies; 6+ messages in thread
From: Mikko.Rapeli @ 2019-07-17 12:24 UTC (permalink / raw)
  To: richard.purdie; +Cc: openembedded-core

On Wed, Jul 17, 2019 at 12:44:54PM +0100, Richard Purdie wrote:
> On Wed, 2019-07-17 at 12:08 +0300, Mikko Rapeli wrote:
> > While creating and deleting files with unicode or other
> > encodings works, it's annoying when ls and other core utils
> > show questionmarks instead of the unicode characters.
> > In 2019, it's quite common that users of embedded devices
> > based on yocto need unicode support. Debugging a box with
> > unicode encoded file names is a bit annoying when core utils
> > from busybox don't support them.
> > 
> > The unicode config fragment has the same config as Debian in their
> > deb and udeb builds of version 1:1.30.1-4.
> > 
> > If developers do not want this or other default yocto features in
> > busybox,
> > or optimize the configuration for size, then they likely run a
> > completely
> > custom configuration. Thus I think it's safe to enable unicode
> > support
> > by default.
> > 
> > Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> > ---
> >  meta/recipes-core/busybox/busybox/unicode.cfg | 10 ++++++++++
> >  meta/recipes-core/busybox/busybox_1.31.0.bb   |  1 +
> >  2 files changed, 11 insertions(+)
> >  create mode 100644 meta/recipes-core/busybox/busybox/unicode.cfg
> 
> Can you give an idea of the size implications of this change?

buildhistory isn't working for me, or maybe it is since the size difference
is zero bytes for busybox binaries and package. tmp has this patch,
tmp_master doesn't:

$ ls -l tmp*/deploy/images/qemux86/tmp/bin/busybox.*
-rwxr-xr-x 1 mcfrisk mcfrisk 612024 Jul 17 10:45 tmp/deploy/images/qemux86/tmp/bin/busybox.nosuid
-rwxr-xr-x 1 mcfrisk mcfrisk  54656 Jul 17 10:45 tmp/deploy/images/qemux86/tmp/bin/busybox.suid
-rwxr-xr-x 1 mcfrisk mcfrisk 612024 Jul 17 13:56 tmp_master/deploy/images/qemux86/tmp/bin/busybox.nosuid
-rwxr-xr-x 1 mcfrisk mcfrisk  54656 Jul 17 13:56 tmp_master/deploy/images/qemux86/tmp/bin/busybox.suid

Just to be sure, the files differ:

$ sha1sum tmp*/deploy/images/qemux86/tmp/bin/busybox.*
8866ced8ad8aa335bb5dfc8b229851b6e09397d7  tmp/deploy/images/qemux86/tmp/bin/busybox.nosuid
01cb28f1254f4200b005122ce03114b034b3aaec  tmp/deploy/images/qemux86/tmp/bin/busybox.suid
91ed5b26083c9b6f944344036b04ebf2aa5af6ff  tmp_master/deploy/images/qemux86/tmp/bin/busybox.nosuid
d4a3ee35246c0f42ba4f0a8a34cd96269b89e906  tmp_master/deploy/images/qemux86/tmp/bin/busybox.suid

buildhistory shows that busybox-dbg PKGSIZE increases from 4523596 to 4546396, 
busybox-ptest PKGSIZE decreases from 458949 to 458908 even when
/usr/lib/busybox/ptest/testsuite/tar.utf8.tar.bz2 is added, busybox-src
PKGSIZE increases from 4130952 to 4163274.

I found that hard to believe so I didn't trust buildhistory anymore.

-Mikko

^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] busybox: enable unicode support
  2019-07-17 12:24   ` Mikko.Rapeli
@ 2019-07-17 13:26     ` Burton, Ross
  2019-07-20  2:31     ` Khem Raj
  1 sibling, 0 replies; 6+ messages in thread
From: Burton, Ross @ 2019-07-17 13:26 UTC (permalink / raw)
  To: Mikko Rapeli; +Cc: OE-core

On Wed, 17 Jul 2019 at 13:25, <Mikko.Rapeli@bmw.de> wrote:

> buildhistory shows that busybox-dbg PKGSIZE increases from 4523596 to 4546396,
> busybox-ptest PKGSIZE decreases from 458949 to 458908 even when
> /usr/lib/busybox/ptest/testsuite/tar.utf8.tar.bz2 is added, busybox-src
> PKGSIZE increases from 4130952 to 4163274.

For me:

busybox-dbg:
-PKGSIZE = 5212120
+PKGSIZE = 5239440

busybox-ptest:
-PKGSIZE = 458881
+PKGSIZE = 458840

busybox-src:
-PKGSIZE = 4130692
+PKGSIZE = 4163014

busybox:
-PKGSIZE = 687247
+PKGSIZE = 691343

Oddly files-in-package doesn't appear to have been touched in the
repo.  I wonder if my sysroot changes broke that part of
buildhistory...

Ross


^ permalink raw reply	[flat|nested] 6+ messages in thread
* Re: [PATCH 1/2] busybox: enable unicode support
  2019-07-17 12:24   ` Mikko.Rapeli
  2019-07-17 13:26     ` Burton, Ross
@ 2019-07-20  2:31     ` Khem Raj
  1 sibling, 0 replies; 6+ messages in thread
From: Khem Raj @ 2019-07-20  2:31 UTC (permalink / raw)
  To: Mikko Rapeli; +Cc: Patches and discussions about the oe-core layer

On Wed, Jul 17, 2019 at 5:24 AM <Mikko.Rapeli@bmw.de> wrote:
>
> On Wed, Jul 17, 2019 at 12:44:54PM +0100, Richard Purdie wrote:
> > On Wed, 2019-07-17 at 12:08 +0300, Mikko Rapeli wrote:
> > > While creating and deleting files with unicode or other
> > > encodings works, it's annoying when ls and other core utils
> > > show questionmarks instead of the unicode characters.
> > > In 2019, it's quite common that users of embedded devices
> > > based on yocto need unicode support. Debugging a box with
> > > unicode encoded file names is a bit annoying when core utils
> > > from busybox don't support them.
> > >
> > > The unicode config fragment has the same config as Debian in their
> > > deb and udeb builds of version 1:1.30.1-4.
> > >
> > > If developers do not want this or other default yocto features in
> > > busybox,
> > > or optimize the configuration for size, then they likely run a
> > > completely
> > > custom configuration. Thus I think it's safe to enable unicode
> > > support
> > > by default.
> > >
> > > Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
> > > ---
> > >  meta/recipes-core/busybox/busybox/unicode.cfg | 10 ++++++++++
> > >  meta/recipes-core/busybox/busybox_1.31.0.bb   |  1 +
> > >  2 files changed, 11 insertions(+)
> > >  create mode 100644 meta/recipes-core/busybox/busybox/unicode.cfg
> >
> > Can you give an idea of the size implications of this change?
>
> buildhistory isn't working for me, or maybe it is since the size difference
> is zero bytes for busybox binaries and package. tmp has this patch,
> tmp_master doesn't:
>
> $ ls -l tmp*/deploy/images/qemux86/tmp/bin/busybox.*
> -rwxr-xr-x 1 mcfrisk mcfrisk 612024 Jul 17 10:45 tmp/deploy/images/qemux86/tmp/bin/busybox.nosuid
> -rwxr-xr-x 1 mcfrisk mcfrisk  54656 Jul 17 10:45 tmp/deploy/images/qemux86/tmp/bin/busybox.suid
> -rwxr-xr-x 1 mcfrisk mcfrisk 612024 Jul 17 13:56 tmp_master/deploy/images/qemux86/tmp/bin/busybox.nosuid
> -rwxr-xr-x 1 mcfrisk mcfrisk  54656 Jul 17 13:56 tmp_master/deploy/images/qemux86/tmp/bin/busybox.suid
>
> Just to be sure, the files differ:
>
> $ sha1sum tmp*/deploy/images/qemux86/tmp/bin/busybox.*
> 8866ced8ad8aa335bb5dfc8b229851b6e09397d7  tmp/deploy/images/qemux86/tmp/bin/busybox.nosuid
> 01cb28f1254f4200b005122ce03114b034b3aaec  tmp/deploy/images/qemux86/tmp/bin/busybox.suid
> 91ed5b26083c9b6f944344036b04ebf2aa5af6ff  tmp_master/deploy/images/qemux86/tmp/bin/busybox.nosuid
> d4a3ee35246c0f42ba4f0a8a34cd96269b89e906  tmp_master/deploy/images/qemux86/tmp/bin/busybox.suid
>
> buildhistory shows that busybox-dbg PKGSIZE increases from 4523596 to 4546396,
> busybox-ptest PKGSIZE decreases from 458949 to 458908 even when
> /usr/lib/busybox/ptest/testsuite/tar.utf8.tar.bz2 is added, busybox-src
> PKGSIZE increases from 4130952 to 4163274.
>
> I found that hard to believe so I didn't trust buildhistory anymore.
>

I think its fine to enable it, but we need to understand the size implications
maybe rename busybox build dir to something and with without so you
have two dirs to compare.

> -Mikko
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


^ permalink raw reply	[flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-07-20  2:31 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-07-17  9:08 [PATCH 1/2] busybox: enable unicode support Mikko Rapeli
2019-07-17  9:08 ` [PATCH 2/2] cve-check.bbclass: initialize to_append Mikko Rapeli
2019-07-17 11:44 ` [PATCH 1/2] busybox: enable unicode support Richard Purdie
2019-07-17 12:24   ` Mikko.Rapeli
2019-07-17 13:26     ` Burton, Ross
2019-07-20  2:31     ` Khem Raj

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox