[meta][dunfell][PATCH] busybox: Make high ASCII chars printable

[meta][dunfell][PATCH] busybox: Make high ASCII chars printable

[Nikhil R]

Currently busybox utilty "ls" fails to display filenames contains
UTF-8 characters and replaces with "?".

Steps to reproduce:
bin/busybox touch utf_test-Ü
bin/busybox ls utf_test-* > sample.log
bin/busybox cat sample.log
utf_test-??
bin/busybox hexdump -C sample.log | grep "c3 9c"

Therefore, above hexdump command is unable to find matching hex value
for the umlaute character.
Hence, change the libbb's printable_string() function to allow high ASCII
characters so that unicode filenames are displayed correctly.
This issue is not observed in upstream busybox.

Signed-off-by: Sana Kazi <Sana.Kazi@kpit.com>
Signed-off-by: Sana Kazi <sanakazisk19@gmail.com>
Signed-off-by: Nikhil R <nikhilar2410@gmail.com>
---
 .../270-libbb_make_unicode_printable.patch    | 39 +++++++++++++++++++
 meta/recipes-core/busybox/busybox_1.31.1.bb   |  1 +
 2 files changed, 40 insertions(+)
 create mode 100644 meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch

diff --git a/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
new file mode 100644
index 0000000000..001d2847fe
--- /dev/null
+++ b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
@@ -0,0 +1,39 @@
+busybox: make high ASCII chars printable (#7993)
+
+Currently busybox utils like "ls" fail to display filenames containing UTF-8
+characters, replacing any special characters with "?".
+
+Change libbb's printable_string() function to allow high ASCII characters so
+that unicode filenames are displayed correctls.
+
+Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
+Upstream-Status: Pending [https://git.openwrt.org/?p=openwrt/svn-archive/archive.git;a=patch;h=bbca780ca657d894fc457cac8eaaaec9d414bfc8]
+
+SVN-Revision: 43084
+---
+ .../270-libbb_make_unicode_printable.patch    | 20 +++++++++++++++++++
+ create mode 100644 package/utils/busybox/patches/270-libbb_make_unicode_printable.patch
+
+diff --git a/package/utils/busybox/patches/270-libbb_make_unicode_printable.patch b/package/utils/busybox/patches/270-libbb_make_unicode_printable.patch
+--- a/libbb/printable_string.c
++++ b/libbb/printable_string.c
+@@ -28,8 +28,6 @@
+ 		}
+ 		if (c < ' ')
+ 			break;
+-		if (c >= 0x7f)
+-			break;
+ 		s++;
+ 	}
+ 
+@@ -42,7 +40,7 @@
+ 			unsigned char c = *d;
+ 			if (c == '\0')
+ 				break;
+-			if (c < ' ' || c >= 0x7f)
++                        if (c < ' ')
+ 				*d = '?';
+ 			d++;
+ 		}
+--
+2.20.1
diff --git a/meta/recipes-core/busybox/busybox_1.31.1.bb b/meta/recipes-core/busybox/busybox_1.31.1.bb
index d062f0f7dd..3e0d888b05 100644
--- a/meta/recipes-core/busybox/busybox_1.31.1.bb
+++ b/meta/recipes-core/busybox/busybox_1.31.1.bb
@@ -57,6 +57,7 @@ SRC_URI = "https://busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \
            file://CVE-2021-423xx-awk.patch \
            file://0001-libbb-sockaddr2str-ensure-only-printable-characters-.patch \
            file://0002-nslookup-sanitize-all-printed-strings-with-printable.patch \
+           file://270-libbb_make_unicode_printable.patch \
            "
 SRC_URI_append_libc-musl = " file://musl.cfg "
 
-- 
2.25.1

Re: [OE-core] [meta][dunfell][PATCH] busybox: Make high ASCII chars printable

[Richard Purdie]

On Wed, 2023-03-29 at 12:17 +0530, nikhil wrote:
> Currently busybox utilty "ls" fails to display filenames contains
> UTF-8 characters and replaces with "?".
> 
> Steps to reproduce:
> bin/busybox touch utf_test-Ü
> bin/busybox ls utf_test-* > sample.log
> bin/busybox cat sample.log
> utf_test-??
> bin/busybox hexdump -C sample.log | grep "c3 9c"
> 
> Therefore, above hexdump command is unable to find matching hex value
> for the umlaute character.
> Hence, change the libbb's printable_string() function to allow high ASCII
> characters so that unicode filenames are displayed correctly.
> This issue is not observed in upstream busybox.
> 
> Signed-off-by: Sana Kazi <Sana.Kazi@kpit.com>
> Signed-off-by: Sana Kazi <sanakazisk19@gmail.com>
> Signed-off-by: Nikhil R <nikhilar2410@gmail.com>
> ---
>  .../270-libbb_make_unicode_printable.patch    | 39 +++++++++++++++++++
>  meta/recipes-core/busybox/busybox_1.31.1.bb   |  1 +
>  2 files changed, 40 insertions(+)
>  create mode 100644 meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> 
> diff --git a/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> new file mode 100644
> index 0000000000..001d2847fe
> --- /dev/null
> +++ b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> @@ -0,0 +1,39 @@
> +busybox: make high ASCII chars printable (#7993)
> +
> +Currently busybox utils like "ls" fail to display filenames containing UTF-8
> +characters, replacing any special characters with "?".
> +
> +Change libbb's printable_string() function to allow high ASCII characters so
> +that unicode filenames are displayed correctls.
> +
> +Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
> +Upstream-Status: Pending [https://git.openwrt.org/?p=openwrt/svn-archive/archive.git;a=patch;h=bbca780ca657d894fc457cac8eaaaec9d414bfc8]
> +

Has the patch been submitted to busybox? The upstream we refer to here
is really busybox. The link to openwrt is helpful but we should limit
to any submission to busybox itself for the purposes of upstream-
status.

Cheers,

Richard

Re: [OE-core] [meta][dunfell][PATCH] busybox: Make high ASCII chars printable

[Khem Raj]

On Wed, Mar 29, 2023 at 12:20 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Wed, 2023-03-29 at 12:17 +0530, nikhil wrote:
> > Currently busybox utilty "ls" fails to display filenames contains
> > UTF-8 characters and replaces with "?".
> >
> > Steps to reproduce:
> > bin/busybox touch utf_test-Ü
> > bin/busybox ls utf_test-* > sample.log
> > bin/busybox cat sample.log
> > utf_test-??
> > bin/busybox hexdump -C sample.log | grep "c3 9c"
> >
> > Therefore, above hexdump command is unable to find matching hex value
> > for the umlaute character.
> > Hence, change the libbb's printable_string() function to allow high ASCII
> > characters so that unicode filenames are displayed correctly.
> > This issue is not observed in upstream busybox.
> >
> > Signed-off-by: Sana Kazi <Sana.Kazi@kpit.com>
> > Signed-off-by: Sana Kazi <sanakazisk19@gmail.com>
> > Signed-off-by: Nikhil R <nikhilar2410@gmail.com>
> > ---
> >  .../270-libbb_make_unicode_printable.patch    | 39 +++++++++++++++++++
> >  meta/recipes-core/busybox/busybox_1.31.1.bb   |  1 +
> >  2 files changed, 40 insertions(+)
> >  create mode 100644 meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> >
> > diff --git a/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> > new file mode 100644
> > index 0000000000..001d2847fe
> > --- /dev/null
> > +++ b/meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch
> > @@ -0,0 +1,39 @@
> > +busybox: make high ASCII chars printable (#7993)
> > +
> > +Currently busybox utils like "ls" fail to display filenames containing UTF-8
> > +characters, replacing any special characters with "?".
> > +
> > +Change libbb's printable_string() function to allow high ASCII characters so
> > +that unicode filenames are displayed correctls.
> > +
> > +Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
> > +Upstream-Status: Pending [https://git.openwrt.org/?p=openwrt/svn-archive/archive.git;a=patch;h=bbca780ca657d894fc457cac8eaaaec9d414bfc8]
> > +
>
> Has the patch been submitted to busybox? The upstream we refer to here
> is really busybox. The link to openwrt is helpful but we should limit
> to any submission to busybox itself for the purposes of upstream-
> status.

yeah I think its worth a discussion on busybox ml.
>
> Cheers,
>
> Richard
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#179264): https://lists.openembedded.org/g/openembedded-core/message/179264
> Mute This Topic: https://lists.openembedded.org/mt/97923178/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [meta][dunfell][PATCH] busybox: Make high ASCII chars printable

[Michael Opdenacker]

Hi Nikhil,

On 29.03.23 at 08:47, nikhil wrote:
> Currently busybox utilty "ls" fails to display filenames contains
> UTF-8 characters and replaces with "?".
>
> Steps to reproduce:
> bin/busybox touch utf_test-Ü
> bin/busybox ls utf_test-* > sample.log
> bin/busybox cat sample.log
> utf_test-??
> bin/busybox hexdump -C sample.log | grep "c3 9c"
>
> Therefore, above hexdump command is unable to find matching hex value
> for the umlaute character.
> Hence, change the libbb's printable_string() function to allow high ASCII
> characters so that unicode filenames are displayed correctly.
> This issue is not observed in upstream busybox.
>
> Signed-off-by: Sana Kazi <Sana.Kazi@kpit.com>
> Signed-off-by: Sana Kazi <sanakazisk19@gmail.com>
> Signed-off-by: Nikhil R <nikhilar2410@gmail.com>
> ---
>   .../270-libbb_make_unicode_printable.patch    | 39 +++++++++++++++++++
>   meta/recipes-core/busybox/busybox_1.31.1.bb   |  1 +
>   2 files changed, 40 insertions(+)
>   create mode 100644 meta/recipes-core/busybox/busybox/270-libbb_make_unicode_printable.patch


Thanks for the patch!

There's an issue with your e-mail identity. You should add one thing to 
your git configuration, so that your patches are given an "Author" field 
which matches your "Signed-off-by" information. See 
https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Fixing_your_From_identity 
for details.

Thanks in advance
Cheers
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com