[Kirkstone] joe editor broken with current ncurses

[Kirkstone] joe editor broken with current ncurses

[tobias.jakobi]

[-- Attachment #1: Type: text/plain, Size: 1831 bytes --]

Hello,

I'm currently facing some problems with seem to originate from version bump of ncurses done in May this year. The problem manifests itself in applications using ncurses rendering garbage and "destroying" the terminal. I.e. you need to issue a terminal reset afterwards to make it usable again. The editor joe is affected, but according to other reports tmux as well.

It seems like this problem is known. E.g. the Gentoo bugtracker has some entries that analyse the problem, see here:
https://bugs.gentoo.org/904247
https://bugs.gentoo.org/904263

Gentoo currently provides two ncurses version in their repo, 6.4_p20230401 and 6.4_p20230527, where the latter one is masked because of this issue.

It appears that openembedded-core/kirkstone suffers from the same problem (even though only A 6.3 version of ncurses is used). I'm currently using commit 56503e3e80603de3b69acef2f6d32836bc9e5e5d of the layer (from end of October, so faily recent).

As a test I have reverted the following commits:
4d79b1cc4178ba88830bab59a45163bbddf586ce (ncurses: fix CVE-2023-29491)
862c1b109cf8f31522a250cc9ff4146fe526450c (ncurses: update to patchlevel 20220423)

This restore functionality of joe, but of course leaves the system vulnerable to the corresponding CVEs. I'm not sure on how to proceed here. joe is a commonly used editor on our systems and I really don't want to leave it in a broken state (as some developers in our team depend on it). At the same time I don't want to leave known CVEs unpatched.

I'm thinking about bumping ncurses to the (apparently unaffected?) 6.4 version that Gentoo currently ships and put the .bb in our custom layer for the time being. I don't know how well that would work though.

Are there any plans for a version bump in the Kirkstone branch?

With best wishes,
Tobias

[-- Attachment #2: Type: text/html, Size: 2075 bytes --]

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

I think bumping to 6.4 is the best option. Unfortunately stable branch
policy prevents that going into oe-core.

Note that the various date-versioned patchlevels are all development
snapshots. We used to think they're actual releases and ship them, but
it's finally been corrected in master.

Alex

On Tue, 21 Nov 2023 at 11:23, <tobias.jakobi@compleo-cs.com> wrote:
>
> Hello,
>
> I'm currently facing some problems with seem to originate from version bump of ncurses done in May this year. The problem manifests itself in applications using ncurses rendering garbage and "destroying" the terminal. I.e. you need to issue a terminal reset afterwards to make it usable again. The editor joe is affected, but according to other reports tmux as well.
>
> It seems like this problem is known. E.g. the Gentoo bugtracker has some entries that analyse the problem, see here:
> https://bugs.gentoo.org/904247
> https://bugs.gentoo.org/904263
>
> Gentoo currently provides two ncurses version in their repo, 6.4_p20230401 and 6.4_p20230527, where the latter one is masked because of this issue.
>
> It appears that openembedded-core/kirkstone suffers from the same problem (even though only A 6.3 version of ncurses is used). I'm currently using commit 56503e3e80603de3b69acef2f6d32836bc9e5e5d of the layer (from end of October, so faily recent).
>
> As a test I have reverted the following commits:
> 4d79b1cc4178ba88830bab59a45163bbddf586ce (ncurses: fix CVE-2023-29491)
> 862c1b109cf8f31522a250cc9ff4146fe526450c (ncurses: update to patchlevel 20220423)
>
> This restore functionality of joe, but of course leaves the system vulnerable to the corresponding CVEs. I'm not sure on how to proceed here. joe is a commonly used editor on our systems and I really don't want to leave it in a broken state (as some developers in our team depend on it). At the same time I don't want to leave known CVEs unpatched.
>
> I'm thinking about bumping ncurses to the (apparently unaffected?) 6.4 version that Gentoo currently ships and put the .bb in our custom layer for the time being. I don't know how well that would work though.
>
> Are there any plans for a version bump in the Kirkstone branch?
>
> With best wishes,
> Tobias
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#190962): https://lists.openembedded.org/g/openembedded-core/message/190962
> Mute This Topic: https://lists.openembedded.org/mt/102726054/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: Private: Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

It's not clear to me: do you also see the issues with master? Can you
try that please?

Please keep all responses on the mailing list.


Alex

On Wed, 22 Nov 2023 at 10:48, <tobias.jakobi@compleo-cs.com> wrote:
>
> Hello Alex,
>
> thanks for the suggestion. I've imported the ncurses recipe from oe-core/master, but the problem remained. It turns out that the CVE patch is what is causing the problems here. I read through the Gentoo bugreports again, and noticed that one user reported p20230918 to be working. Checking the ncurses commit log it seems like p20230918 includes a fix for the CVE, so additional patching is unnecessary.
>
> So my current approach is to use p20230918 (https://github.com/ThomasDickey/ncurses-snapshots/releases/tag/v6_4_20230918), but drop the CVE patch that the recipe is master applies. This seems to work, i.e. joe and tmux are both functional again.
>
> Sadly I have no idea which changes one would need to backport to the 6.3 version to fix the issue... :(

Re: [Kirkstone] joe editor broken with current ncurses

[tobias.jakobi]

[-- Attachment #1: Type: text/plain, Size: 895 bytes --]

Hello Alex,

thanks for the suggestion. I've imported the ncurses recipe from oe-core/master, but the problem remained. It turns out that the CVE patch is what is causing the problems here. I read through the Gentoo bugreports again, and noticed that one user reported p20230918 to be working. Checking the ncurses commit log it seems like p20230918 includes a fix for the CVE, so additional patching is unnecessary.

So my current approach is to use p20230918 (https://github.com/ThomasDickey/ncurses-snapshots/releases/tag/v6_4_20230918), but drop the CVE patch that the recipe is master applies. This seems to work, i.e. joe and tmux are both functional again.

Sadly I have no idea which changes one would need to backport to the 6.3 version to fix the issue... :(

P.S.: So oe-core/master does *not* work. master has ncurses 6.4 (no patchlevel), plus the patch for CVE-2023-29491.

[-- Attachment #2: Type: text/html, Size: 927 bytes --]

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

On Wed, 22 Nov 2023 at 11:10, <tobias.jakobi@compleo-cs.com> wrote:
> P.S.: So oe-core/master does *not* work. master has ncurses 6.4 (no patchlevel), plus the patch for CVE-2023-29491.

But did you specifically verify this or is this a conjecture based on
your backport? We would need to fix master before fixing any of the
release branches :-/

Alex

Re: [Kirkstone] joe editor broken with current ncurses

[tobias.jakobi]

[-- Attachment #1: Type: text/plain, Size: 490 bytes --]

I verified this by:
- copying the recipe from master verbatim into our layer
- building our image
- putting image on hardware
- firing up joe

Joe then shows the same broken behaviour that it does with the ncurses 6.3 snapshot in kirkstone.

Or do you want me to replace our kirkstone oe-core layer with master oe-core? I don't think this is going to work anyway, since some of our support layers (we are using a Toradex SoM powered by a NXP i.MX7) are not compatible with master.

[-- Attachment #2: Type: text/html, Size: 522 bytes --]

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

On Wed, 22 Nov 2023 at 11:20, <tobias.jakobi@compleo-cs.com> wrote:
>
> I verified this by:
> - copying the recipe from master verbatim into our layer
> - building our image
> - putting image on hardware
> - firing up joe
>
> Joe then shows the same broken behaviour that it does with the ncurses 6.3 snapshot in kirkstone.
>
> Or do you want me to replace our kirkstone oe-core layer with master oe-core? I don't think this is going to work anyway, since some of our support layers (we are using a Toradex SoM powered by a NXP i.MX7) are not compatible with master.

You then need to build master with something that works, perhaps
simply a qemu machine? No layers needed other than oe-core and
whatever provides ncurses apps you need to test.

Alex

Re: [Kirkstone] joe editor broken with current ncurses

[Tobias Jakobi]

Any pointers on how to setup such machine? Quick goggling gives me https://docs.yoctoproject.org/dev-manual/qemu.html, but would this be satisfactory for the test?

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

I think you can use
https://docs.yoctoproject.org/brief-yoctoprojectqs/index.html even.

Alex

On Wed, 22 Nov 2023 at 11:58, Tobias Jakobi
<tobias.jakobi@compleo-cs.com> wrote:
>
> Any pointers on how to setup such machine? Quick goggling gives me https://docs.yoctoproject.org/dev-manual/qemu.html, but would this be satisfactory for the test?
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#191059): https://lists.openembedded.org/g/openembedded-core/message/191059
> Mute This Topic: https://lists.openembedded.org/mt/102726054/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [Kirkstone] joe editor broken with current ncurses

[Tobias Jakobi]

OK, that was easier than expected. So I followed the guide, but I just build core-image-full-cmdline. I added the meta-oe layer (git master) to it and added joe to CORE_IMAGE_EXTRA_INSTALL. Same issue with this setup.

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Richard Purdie]

On Wed, 2023-11-22 at 07:20 -0800, Tobias Jakobi wrote:
> OK, that was easier than expected. So I followed the guide, but I
> just build core-image-full-cmdline. I added the meta-oe layer (git
> master) to it and added joe to CORE_IMAGE_EXTRA_INSTALL. Same issue
> with this setup.

Thanks for working through the testing.

It sounds like we as a project need to fix master. Once that happens,
there could be a case for backporting a different version if it fixes
both regressions and security issues.

It does depend on the new version not causing other problems though,
and us being able to be confident of that.

Cheers,

Richard

Re: [Kirkstone] joe editor broken with current ncurses

[Tobias Jakobi]

Yeah, the "causing other problems" bit is also what worries me at the moment. In particular because a lot of other packages depend on ncurses. Usually not on libncurses itself, but on libtinfo. Let me know if I can do further tests. I can't guarantee though how much more time I can spend on this. Our product owner is already asking me why stuff is taking so long :D

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Richard Purdie]

On Wed, 2023-11-22 at 07:46 -0800, Tobias Jakobi wrote:
> Yeah, the "causing other problems" bit is also what worries me at the
> moment. In particular because a lot of other packages depend on
> ncurses. Usually not on libncurses itself, but on libtinfo. Let me
> know if I can do further tests. I can't guarantee though how much
> more time I can spend on this. Our product owner is already asking me
> why stuff is taking so long :D

Someone looking into it and helping fix things properly does take time
but without it, the project doesn't work. You may be able to point out
all the fixes you haven't had to work on that come from the project and
that this is small by comparison! :)

Cheers,

Richard

Re: [OE-core] [Kirkstone] joe editor broken with current ncurses

[Alexander Kanavin]

I suppose you can check, on master:

- whether dropping the CVE fix addresses the problem
- whether updating to recent patchlevel addresses the problem
- bisect changes in the patchlevel down to what is needed to fix the
CVE and not regress

Given that you have a basic build, tweaking ncurses and rebuilding
isn't too time consuming.

Alex

On Wed, 22 Nov 2023 at 16:46, Tobias Jakobi
<tobias.jakobi@compleo-cs.com> wrote:
>
> Yeah, the "causing other problems" bit is also what worries me at the moment. In particular because a lot of other packages depend on ncurses. Usually not on libncurses itself, but on libtinfo. Let me know if I can do further tests. I can't guarantee though how much more time I can spend on this. Our product owner is already asking me why stuff is taking so long :D
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#191114): https://lists.openembedded.org/g/openembedded-core/message/191114
> Mute This Topic: https://lists.openembedded.org/mt/102726054/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>