* Re: [yocto] Y2038 proposal
2022-11-30 8:07 ` Y2038 proposal Alexander Kanavin
@ 2022-11-30 8:28 ` Lukasz Majewski
2022-11-30 9:07 ` Alexander Kanavin
2022-11-30 21:14 ` Alexander Kanavin
2022-11-30 11:02 ` Alexandre Belloni
` (3 subsequent siblings)
4 siblings, 2 replies; 27+ messages in thread
From: Lukasz Majewski @ 2022-11-30 8:28 UTC (permalink / raw)
To: Alexander Kanavin
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]
Hi Alexander,
> On Tue, 29 Nov 2022 at 16:45, Stephen Jolley
> <sjolley.yp.pm@gmail.com> wrote:
> > We’d welcome a proposal/series on how to move forward with the
> > Y2038 work for 32 bit platforms.
>
> I have the following proposal:
>
> 1. A branch is made where:
> a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> b. qemu is always started with "-rtc base=2040-01-01", simulating
> Y2038 actually occurring.
> c. an additional runtime test verifies that both RTC clock and system
> clock report 2040.
>
Please find a few comments:
1. There is already provided meta-y2038 [1] to test if 32 bit systems
correctly support Y2038 problem. It uses qemu machines from OE/Yocto
2. There are ptest available [2] to validate if the Y2038 problem works
correctly.
3. Support for running ptests mentioned in point 2. is already
available in the poky repository [3].
> 2. This branch is run through a-full on the autobuilder. Any uncovered
> issues are filed as bugs.
>
> 3. Once *all* of the bugs are addressed, repeat point 2.
>
> 4. Once there are no more open bugs, 1a is merged into master.
>
> Any fatal flaws in the plan?
>
> It's not hard to see that Y2038 problem is real and serious, e.g. on
> qemux86 core-image-full-cmdline built from master:
>
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# date -s "2040-01-01"
> Sun Jan 1 00:00:00 UTC 2040
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# ls /
> -sh: ls: command not found
>
> On qemux86_64 the same sequence works as expected, of course.
>
Yes, y2038 is an important issue.
I would be more than happy if we could reuse the previous work [1].
I've used OE/Yocto to validate the code during developing support for
'-D_TIME_BITS=64 ' flag in glibc.
It looks like the meta-y2038 can be used out of the box (after checking
if it still works with newest poky) when added to the Yocto Project
build/test infrastructure.
> Alex
Links:
[1] - https://github.com/lmajewski/meta-y2038
[2] - https://github.com/lmajewski/meta-y2038/blob/master/README#L201
[3] -
https://git.yoctoproject.org/poky/commit/?id=0e0c481a25f10f8f7ff1d69bda7f015186da0202
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [yocto] Y2038 proposal
2022-11-30 8:28 ` [yocto] " Lukasz Majewski
@ 2022-11-30 9:07 ` Alexander Kanavin
2022-11-30 9:40 ` Lukasz Majewski
2022-11-30 21:14 ` Alexander Kanavin
1 sibling, 1 reply; 27+ messages in thread
From: Alexander Kanavin @ 2022-11-30 9:07 UTC (permalink / raw)
To: Lukasz Majewski
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
On Wed, 30 Nov 2022 at 09:28, Lukasz Majewski <lukma@denx.de> wrote:
> Please find a few comments:
>
> 1. There is already provided meta-y2038 [1] to test if 32 bit systems
> correctly support Y2038 problem. It uses qemu machines from OE/Yocto
>
> 2. There are ptest available [2] to validate if the Y2038 problem works
> correctly.
>
> 3. Support for running ptests mentioned in point 2. is already
> available in the poky repository [3].
Thanks! So there should be a
d. 'glibc-tests-ptest' is executed across all architectures - probably
as a machine-specific selftest, and as well with qemu time set into
the future.
> It looks like the meta-y2038 can be used out of the box (after checking
> if it still works with newest poky) when added to the Yocto Project
> build/test infrastructure.
Unfortunately I do not think that layer can be easily added into the
test matrix. It has its own distro and images. No, we need to maintain
a poky branch where the same tweaks and fixes happen. Besides, those
fixes would need to be merged into oe-core proper eventually anyway.
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] Y2038 proposal
2022-11-30 9:07 ` Alexander Kanavin
@ 2022-11-30 9:40 ` Lukasz Majewski
2022-11-30 9:48 ` Alexander Kanavin
0 siblings, 1 reply; 27+ messages in thread
From: Lukasz Majewski @ 2022-11-30 9:40 UTC (permalink / raw)
To: Alexander Kanavin
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]
Hi Alexander,
> On Wed, 30 Nov 2022 at 09:28, Lukasz Majewski <lukma@denx.de> wrote:
> > Please find a few comments:
> >
> > 1. There is already provided meta-y2038 [1] to test if 32 bit
> > systems correctly support Y2038 problem. It uses qemu machines from
> > OE/Yocto
> >
> > 2. There are ptest available [2] to validate if the Y2038 problem
> > works correctly.
> >
> > 3. Support for running ptests mentioned in point 2. is already
> > available in the poky repository [3].
>
> Thanks! So there should be a
>
> d. 'glibc-tests-ptest' is executed across all architectures - probably
> as a machine-specific selftest, and as well with qemu time set into
> the future.
+1
>
> > It looks like the meta-y2038 can be used out of the box (after
> > checking if it still works with newest poky) when added to the
> > Yocto Project build/test infrastructure.
>
> Unfortunately I do not think that layer can be easily added into the
> test matrix. It has its own distro and images. No, we need to maintain
> a poky branch where the same tweaks and fixes happen. Besides, those
> fixes would need to be merged into oe-core proper eventually anyway.
>
It would be even better if the meta-y2038 could be dropped and _all_
its functionality could be merged to poky.
That would be _awesome_.
Please just be aware that this meta layer has some fixes for some
packages (for Y2038 ready glibc).
> Alex
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] Y2038 proposal
2022-11-30 9:40 ` Lukasz Majewski
@ 2022-11-30 9:48 ` Alexander Kanavin
0 siblings, 0 replies; 27+ messages in thread
From: Alexander Kanavin @ 2022-11-30 9:48 UTC (permalink / raw)
To: Lukasz Majewski
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
On Wed, 30 Nov 2022 at 10:40, Lukasz Majewski <lukma@denx.de> wrote:
> It would be even better if the meta-y2038 could be dropped and _all_
> its functionality could be merged to poky.
>
> That would be _awesome_.
>
> Please just be aware that this meta layer has some fixes for some
> packages (for Y2038 ready glibc).
Little by little. As long as we have a well-defined test plan, a
well-known list of issues, and follow a strict 'upstream first' policy
in addressing them (e.g. no fixes in layers that are 'forever
pending', un-upstreamable or have been rejected by upstreams), we'll
eventually get there.
And absolutely no promise to make this available anywhere except
master. If someone is using ancient yocto releases, they should run a
retro computing museum rather than critical infrastructure.
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] Y2038 proposal
2022-11-30 8:28 ` [yocto] " Lukasz Majewski
2022-11-30 9:07 ` Alexander Kanavin
@ 2022-11-30 21:14 ` Alexander Kanavin
2022-12-01 8:28 ` Lukasz Majewski
1 sibling, 1 reply; 27+ messages in thread
From: Alexander Kanavin @ 2022-11-30 21:14 UTC (permalink / raw)
To: Lukasz Majewski
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
On Wed, 30 Nov 2022 at 09:28, Lukasz Majewski <lukma@denx.de> wrote:
> 2. There are ptest available [2] to validate if the Y2038 problem works
> correctly.
>
> 3. Support for running ptests mentioned in point 2. is already
> available in the poky repository [3].
I just ran these tests in (32 bit) qemux86 on top of poky master (e.g.
no magic glibc flags), and they all passed. Do they need to be ran
after setting the date to the 'post-2038 future' to reveal the issues
and produce failures?
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] Y2038 proposal
2022-11-30 21:14 ` Alexander Kanavin
@ 2022-12-01 8:28 ` Lukasz Majewski
0 siblings, 0 replies; 27+ messages in thread
From: Lukasz Majewski @ 2022-12-01 8:28 UTC (permalink / raw)
To: Alexander Kanavin
Cc: openembedded-architecture, Yocto-mailing-list, OE-core,
Stephen Jolley, Richard Purdie
[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]
Hi Alexander,
> On Wed, 30 Nov 2022 at 09:28, Lukasz Majewski <lukma@denx.de> wrote:
> > 2. There are ptest available [2] to validate if the Y2038 problem
> > works correctly.
> >
> > 3. Support for running ptests mentioned in point 2. is already
> > available in the poky repository [3].
>
> I just ran these tests in (32 bit) qemux86 on top of poky master (e.g.
> no magic glibc flags), and they all passed. Do they need to be ran
> after setting the date to the 'post-2038 future' to reveal the issues
> and produce failures?
Yes. You need to run them with adjusted date:
date +'%Y-%m-%d %T' -s "2038-01-19 03:14:07"
ptest-runner glibc-tests
More info:
https://github.com/lmajewski/meta-y2038/blob/master/README#L201
>
> Alex
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] Y2038 proposal
2022-11-30 8:07 ` Y2038 proposal Alexander Kanavin
2022-11-30 8:28 ` [yocto] " Lukasz Majewski
@ 2022-11-30 11:02 ` Alexandre Belloni
2022-11-30 11:40 ` [OE-core] " Richard Purdie
2022-11-30 13:15 ` [Openembedded-architecture] " Richard Purdie
` (2 subsequent siblings)
4 siblings, 1 reply; 27+ messages in thread
From: Alexandre Belloni @ 2022-11-30 11:02 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-architecture, Yocto-mailing-list, OE-core
On 30/11/2022 09:07:50+0100, Alexander Kanavin wrote:
> On Tue, 29 Nov 2022 at 16:45, Stephen Jolley <sjolley.yp.pm@gmail.com> wrote:
> > We’d welcome a proposal/series on how to move forward with the Y2038 work for 32 bit platforms.
>
> I have the following proposal:
>
> 1. A branch is made where:
> a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> b. qemu is always started with "-rtc base=2040-01-01", simulating
> Y2038 actually occurring.
> c. an additional runtime test verifies that both RTC clock and system
> clock report 2040.
>
> 2. This branch is run through a-full on the autobuilder. Any uncovered
> issues are filed as bugs.
>
I ran a-full with "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" last week, it
didn't go too well gcc-sanitizer and pulseaudio being the main offenders
but buildtools needs to be investigated.
> 3. Once *all* of the bugs are addressed, repeat point 2.
>
> 4. Once there are no more open bugs, 1a is merged into master.
>
> Any fatal flaws in the plan?
>
> It's not hard to see that Y2038 problem is real and serious, e.g. on
> qemux86 core-image-full-cmdline built from master:
>
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# date -s "2040-01-01"
> Sun Jan 1 00:00:00 UTC 2040
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# ls /
> -sh: ls: command not found
>
> On qemux86_64 the same sequence works as expected, of course.
>
The main issue with the plan is that we are not running tests on 32
qemu anymore.
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [OE-core] [yocto] Y2038 proposal
2022-11-30 11:02 ` Alexandre Belloni
@ 2022-11-30 11:40 ` Richard Purdie
2022-11-30 12:07 ` Alexander Kanavin
0 siblings, 1 reply; 27+ messages in thread
From: Richard Purdie @ 2022-11-30 11:40 UTC (permalink / raw)
To: alexandre.belloni, Alexander Kanavin
Cc: openembedded-architecture, Yocto-mailing-list, OE-core
On Wed, 2022-11-30 at 12:02 +0100, Alexandre Belloni via
lists.openembedded.org wrote:
> On 30/11/2022 09:07:50+0100, Alexander Kanavin wrote:
> > On Tue, 29 Nov 2022 at 16:45, Stephen Jolley <sjolley.yp.pm@gmail.com> wrote:
> > > We’d welcome a proposal/series on how to move forward with the Y2038 work for 32 bit platforms.
> >
> > I have the following proposal:
> >
> > 1. A branch is made where:
> > a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> > b. qemu is always started with "-rtc base=2040-01-01", simulating
> > Y2038 actually occurring.
> > c. an additional runtime test verifies that both RTC clock and system
> > clock report 2040.
> >
> > 2. This branch is run through a-full on the autobuilder. Any uncovered
> > issues are filed as bugs.
> >
>
> I ran a-full with "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" last week, it
> didn't go too well gcc-sanitizer and pulseaudio being the main offenders
> but buildtools needs to be investigated.
What is the potential issue with builtools?
> > 3. Once *all* of the bugs are addressed, repeat point 2.
> >
> > 4. Once there are no more open bugs, 1a is merged into master.
> >
> > Any fatal flaws in the plan?
> >
> > It's not hard to see that Y2038 problem is real and serious, e.g. on
> > qemux86 core-image-full-cmdline built from master:
> >
> > root@qemux86:~# ls /
> > bin boot dev etc home lib lost+found media mnt proc
> > run sbin sys tmp usr var
> > root@qemux86:~# date -s "2040-01-01"
> > Sun Jan 1 00:00:00 UTC 2040
> > root@qemux86:~# ls /
> > bin boot dev etc home lib lost+found media mnt proc
> > run sbin sys tmp usr var
> > root@qemux86:~# ls /
> > -sh: ls: command not found
> >
> > On qemux86_64 the same sequence works as expected, of course.
> >
>
> The main issue with the plan is that we are not running tests on 32
> qemu anymore.
To be clear, we don't run ptests on 32 bit targets, only on qemux86-64
and qemuarm64 where we have KVM available. We do run image, sdk and
eSDK tests on our supported qemu targets, 32 and 64 bit.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [OE-core] [yocto] Y2038 proposal
2022-11-30 11:40 ` [OE-core] " Richard Purdie
@ 2022-11-30 12:07 ` Alexander Kanavin
2022-11-30 12:09 ` Lukasz Majewski
2022-11-30 12:11 ` Richard Purdie
0 siblings, 2 replies; 27+ messages in thread
From: Alexander Kanavin @ 2022-11-30 12:07 UTC (permalink / raw)
To: Richard Purdie
Cc: alexandre.belloni, openembedded-architecture, Yocto-mailing-list,
OE-core
On Wed, 30 Nov 2022 at 12:40, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> To be clear, we don't run ptests on 32 bit targets, only on qemux86-64
> and qemuarm64 where we have KVM available. We do run image, sdk and
> eSDK tests on our supported qemu targets, 32 and 64 bit.
I think kvm does allow 32 bit guest on a 64 bit host. But I can
imagine making full ptests work on 32 bit guests would be a struggle
for reasons unrelated to 2038, specifically lack of users outside of
embedded world.
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [OE-core] [yocto] Y2038 proposal
2022-11-30 12:07 ` Alexander Kanavin
@ 2022-11-30 12:09 ` Lukasz Majewski
2022-11-30 12:11 ` Richard Purdie
1 sibling, 0 replies; 27+ messages in thread
From: Lukasz Majewski @ 2022-11-30 12:09 UTC (permalink / raw)
To: Alexander Kanavin
Cc: Richard Purdie, alexandre.belloni, openembedded-architecture,
Yocto-mailing-list, OE-core
[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]
On Wed, 30 Nov 2022 13:07:27 +0100
"Alexander Kanavin" <alex.kanavin@gmail.com> wrote:
> On Wed, 30 Nov 2022 at 12:40, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>
> > To be clear, we don't run ptests on 32 bit targets, only on
> > qemux86-64 and qemuarm64 where we have KVM available. We do run
> > image, sdk and eSDK tests on our supported qemu targets, 32 and 64
> > bit.
>
> I think kvm does allow 32 bit guest on a 64 bit host.
+1
IIRC the MACH=qemux86 was working correctly...
> But I can
> imagine making full ptests work on 32 bit guests
Maybe only subset of ptests - i.e. those related to Y2038 could be run?
> would be a struggle
> for reasons unrelated to 2038, specifically lack of users outside of
> embedded world.
>
> Alex
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [OE-core] [yocto] Y2038 proposal
2022-11-30 12:07 ` Alexander Kanavin
2022-11-30 12:09 ` Lukasz Majewski
@ 2022-11-30 12:11 ` Richard Purdie
1 sibling, 0 replies; 27+ messages in thread
From: Richard Purdie @ 2022-11-30 12:11 UTC (permalink / raw)
To: Alexander Kanavin
Cc: alexandre.belloni, openembedded-architecture, Yocto-mailing-list,
OE-core
On Wed, 2022-11-30 at 13:07 +0100, Alexander Kanavin wrote:
> On Wed, 30 Nov 2022 at 12:40, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>
> > To be clear, we don't run ptests on 32 bit targets, only on qemux86-64
> > and qemuarm64 where we have KVM available. We do run image, sdk and
> > eSDK tests on our supported qemu targets, 32 and 64 bit.
>
> I think kvm does allow 32 bit guest on a 64 bit host.
For x86, yes. For arm, it varies and I know at least one of our arm
hosts doesn't support it, I don't know about the newer ones.
> But I can imagine making full ptests work on 32 bit guests would be a struggle
> for reasons unrelated to 2038, specifically lack of users outside of
> embedded world.
In general I think it shouldn't be too bad but we really do need to
test it and see.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [Openembedded-architecture] Y2038 proposal
2022-11-30 8:07 ` Y2038 proposal Alexander Kanavin
2022-11-30 8:28 ` [yocto] " Lukasz Majewski
2022-11-30 11:02 ` Alexandre Belloni
@ 2022-11-30 13:15 ` Richard Purdie
2022-11-30 13:36 ` [OE-core] " Lukasz Majewski
2022-12-01 10:27 ` Alexander Kanavin
2022-11-30 16:38 ` Khem Raj
2022-12-02 8:54 ` [OE-core] " Matt Johnston
4 siblings, 2 replies; 27+ messages in thread
From: Richard Purdie @ 2022-11-30 13:15 UTC (permalink / raw)
To: Alexander Kanavin, openembedded-architecture; +Cc: Yocto-mailing-list, OE-core
On Wed, 2022-11-30 at 09:07 +0100, Alexander Kanavin wrote:
> On Tue, 29 Nov 2022 at 16:45, Stephen Jolley <sjolley.yp.pm@gmail.com> wrote:
> > We’d welcome a proposal/series on how to move forward with the Y2038 work for 32 bit platforms.
>
> I have the following proposal:
>
> 1. A branch is made where:
> a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> b. qemu is always started with "-rtc base=2040-01-01", simulating
> Y2038 actually occurring.
> c. an additional runtime test verifies that both RTC clock and system
> clock report 2040.
>
> 2. This branch is run through a-full on the autobuilder. Any uncovered
> issues are filed as bugs.
>
> 3. Once *all* of the bugs are addressed, repeat point 2.
>
> 4. Once there are no more open bugs, 1a is merged into master.
>
> Any fatal flaws in the plan?
Others have made some good comments. My thoughts:
* We need to add some runtime tests to oeqa for this (in addition to
the ptests)
* We need to have a 32 bit ptest run on the autobuilder (qemux86 should
work, not sure we can make qemuarm fast). Whether this is manually
triggered, not sure. We could have a smaller set of ptests to run for
it?
* Could we optionally disable some of the glibc 32 bit function calls
to ensure they're not being used? We don't really want to diverge from
upstream glibc much though.
* We need to work out how to communicate this change happened and have
people "buy in" to it. The reason for that is that if someone has
existing binaries, there could be problems using them after the change.
We therefore need to be sure they are aware of it.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 13:15 ` [Openembedded-architecture] " Richard Purdie
@ 2022-11-30 13:36 ` Lukasz Majewski
2022-11-30 14:20 ` Richard Purdie
2022-12-01 10:27 ` Alexander Kanavin
1 sibling, 1 reply; 27+ messages in thread
From: Lukasz Majewski @ 2022-11-30 13:36 UTC (permalink / raw)
To: Richard Purdie
Cc: Alexander Kanavin, openembedded-architecture, Yocto-mailing-list,
OE-core
[-- Attachment #1: Type: text/plain, Size: 2743 bytes --]
Hi Richard,
> On Wed, 2022-11-30 at 09:07 +0100, Alexander Kanavin wrote:
> > On Tue, 29 Nov 2022 at 16:45, Stephen Jolley
> > <sjolley.yp.pm@gmail.com> wrote:
> > > We’d welcome a proposal/series on how to move forward with the
> > > Y2038 work for 32 bit platforms.
> >
> > I have the following proposal:
> >
> > 1. A branch is made where:
> > a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> > b. qemu is always started with "-rtc base=2040-01-01", simulating
> > Y2038 actually occurring.
> > c. an additional runtime test verifies that both RTC clock and
> > system clock report 2040.
> >
> > 2. This branch is run through a-full on the autobuilder. Any
> > uncovered issues are filed as bugs.
> >
> > 3. Once *all* of the bugs are addressed, repeat point 2.
> >
> > 4. Once there are no more open bugs, 1a is merged into master.
> >
> > Any fatal flaws in the plan?
>
> Others have made some good comments. My thoughts:
>
> * We need to add some runtime tests to oeqa for this (in addition to
> the ptests)
>
> * We need to have a 32 bit ptest run on the autobuilder (qemux86
> should work, not sure we can make qemuarm fast). Whether this is
> manually triggered, not sure. We could have a smaller set of ptests
> to run for it?
Y2038 ptests maybe?
Here is the list of integrated tests to ptests:
https://github.com/lmajewski/y2038-tests
>
> * Could we optionally disable some of the glibc 32 bit function calls
> to ensure they're not being used?
Could you be more specific here? Would you like to disable some
syscalls?
> We don't really want to diverge from
> upstream glibc much though.
Could you be more specific here? The glibc now supports the whole set
of syscalls as of 2.34 version?
To enable them one needs to pass -D_TIME_BITS=64 flag when compiling
programs.
This is now the official glibc ABI.
>
> * We need to work out how to communicate this change happened and have
> people "buy in" to it.
Ok.
> The reason for that is that if someone has
> existing binaries, there could be problems using them after the
> change.
The binary shall work without issues on glibc 2.34+ and 5.10+ kernel
without issues.
The only problem happens when new binaries with 64 bit time support are
run on glibc or kernel not supporting 64 bit time.
> We therefore need to be sure they are aware of it.
>
> Cheers,
>
> Richard
>
>
>
>
Best regards,
Lukasz Majewski
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 13:36 ` [OE-core] " Lukasz Majewski
@ 2022-11-30 14:20 ` Richard Purdie
2022-11-30 16:46 ` [yocto] " Ross Burton
0 siblings, 1 reply; 27+ messages in thread
From: Richard Purdie @ 2022-11-30 14:20 UTC (permalink / raw)
To: Lukasz Majewski
Cc: Alexander Kanavin, openembedded-architecture, Yocto-mailing-list,
OE-core
On Wed, 2022-11-30 at 14:36 +0100, Lukasz Majewski wrote:
> > On Wed, 2022-11-30 at 09:07 +0100, Alexander Kanavin wrote:
> > > On Tue, 29 Nov 2022 at 16:45, Stephen Jolley
> > > <sjolley.yp.pm@gmail.com> wrote:
> > > > We’d welcome a proposal/series on how to move forward with the
> > > > Y2038 work for 32 bit platforms.
> > >
> > > I have the following proposal:
> > >
> > > 1. A branch is made where:
> > > a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> > > b. qemu is always started with "-rtc base=2040-01-01", simulating
> > > Y2038 actually occurring.
> > > c. an additional runtime test verifies that both RTC clock and
> > > system clock report 2040.
> > >
> > > 2. This branch is run through a-full on the autobuilder. Any
> > > uncovered issues are filed as bugs.
> > >
> > > 3. Once *all* of the bugs are addressed, repeat point 2.
> > >
> > > 4. Once there are no more open bugs, 1a is merged into master.
> > >
> > > Any fatal flaws in the plan?
> >
> > Others have made some good comments. My thoughts:
> >
> > * We need to add some runtime tests to oeqa for this (in addition to
> > the ptests)
> >
> > * We need to have a 32 bit ptest run on the autobuilder (qemux86
> > should work, not sure we can make qemuarm fast). Whether this is
> > manually triggered, not sure. We could have a smaller set of ptests
> > to run for it?
>
> Y2038 ptests maybe?
>
> Here is the list of integrated tests to ptests:
> https://github.com/lmajewski/y2038-tests
Perhaps, yes.
> > * Could we optionally disable some of the glibc 32 bit function calls
> > to ensure they're not being used?
>
> Could you be more specific here? Would you like to disable some
> syscalls?
I'm meaning disabling the 32 bit glibc time functions.
> > We don't really want to diverge from
> > upstream glibc much though.
>
> Could you be more specific here? The glibc now supports the whole set
> of syscalls as of 2.34 version?
>
> To enable them one needs to pass -D_TIME_BITS=64 flag when compiling
> programs.
>
> This is now the official glibc ABI.
Right, but the 32 bit time functions/symbols are still available for
older binaries. My point is that anything using those older functions
is likely in need of attention so for Yocto Project/OE usage,
identifying those would be helpful. If we were to disable them, that
would make such usage very obvious.
>
> > The reason for that is that if someone has
> > existing binaries, there could be problems using them after the
> > change.
>
> The binary shall work without issues on glibc 2.34+ and 5.10+ kernel
> without issues.
Not necessarily. If it were a binary library, compiled with 32 bit
time_t, new binaries using it would use a different sized field.
> The only problem happens when new binaries with 64 bit time support are
> run on glibc or kernel not supporting 64 bit time.
That is definitely not the only problem. Some of the problems are
unlikely but we do need to consider them.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 14:20 ` Richard Purdie
@ 2022-11-30 16:46 ` Ross Burton
2022-11-30 16:56 ` Alexandre Belloni
0 siblings, 1 reply; 27+ messages in thread
From: Ross Burton @ 2022-11-30 16:46 UTC (permalink / raw)
To: Richard Purdie
Cc: Lukasz Majewski, Alexander Kanavin, openembedded-architecture,
Yocto-mailing-list, OE-core
On 30 Nov 2022, at 14:20, Richard Purdie via lists.yoctoproject.org <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
>>> * Could we optionally disable some of the glibc 32 bit function calls
>>> to ensure they're not being used?
>>
>> Could you be more specific here? Would you like to disable some
>> syscalls?
>
> I'm meaning disabling the 32 bit glibc time functions.
Some time ago I filed https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian has a nice sanity check where it warns if non-LFS glibc functions are used. I imagine the same logic could be used to check for 32-bit time_t use.
Ross
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 16:46 ` [yocto] " Ross Burton
@ 2022-11-30 16:56 ` Alexandre Belloni
2022-11-30 16:59 ` Richard Purdie
0 siblings, 1 reply; 27+ messages in thread
From: Alexandre Belloni @ 2022-11-30 16:56 UTC (permalink / raw)
To: Ross Burton
Cc: Richard Purdie, Lukasz Majewski, Alexander Kanavin,
openembedded-architecture, Yocto-mailing-list, OE-core
On 30/11/2022 16:46:17+0000, Ross Burton wrote:
> On 30 Nov 2022, at 14:20, Richard Purdie via lists.yoctoproject.org <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
> >>> * Could we optionally disable some of the glibc 32 bit function calls
> >>> to ensure they're not being used?
> >>
> >> Could you be more specific here? Would you like to disable some
> >> syscalls?
> >
> > I'm meaning disabling the 32 bit glibc time functions.
>
> Some time ago I filed https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian has a nice sanity check where it warns if non-LFS glibc functions are used. I imagine the same logic could be used to check for 32-bit time_t use.
>
We can simply disable COMPAT_32BIT_TIME in the kernel config.
> Ross
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#58682): https://lists.yoctoproject.org/g/yocto/message/58682
> Mute This Topic: https://lists.yoctoproject.org/mt/95357621/3617179
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 16:56 ` Alexandre Belloni
@ 2022-11-30 16:59 ` Richard Purdie
2022-12-05 10:00 ` Ola x Nilsson
0 siblings, 1 reply; 27+ messages in thread
From: Richard Purdie @ 2022-11-30 16:59 UTC (permalink / raw)
To: Alexandre Belloni, Ross Burton
Cc: Lukasz Majewski, Alexander Kanavin, openembedded-architecture,
Yocto-mailing-list, OE-core
On Wed, 2022-11-30 at 17:56 +0100, Alexandre Belloni wrote:
> On 30/11/2022 16:46:17+0000, Ross Burton wrote:
> > On 30 Nov 2022, at 14:20, Richard Purdie via lists.yoctoproject.org <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
> > > > > * Could we optionally disable some of the glibc 32 bit function calls
> > > > > to ensure they're not being used?
> > > >
> > > > Could you be more specific here? Would you like to disable some
> > > > syscalls?
> > >
> > > I'm meaning disabling the 32 bit glibc time functions.
> >
> > Some time ago I filed
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian
> > has a nice sanity check where it warns if non-LFS glibc functions
> > are used. I imagine the same logic could be used to check for 32-
> > bit time_t use.
That sounds interesting and something we should probably look into for
both issues...
> >
>
> We can simply disable COMPAT_32BIT_TIME in the kernel config.
That would cause runtime issues but not build time linking ones?
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-11-30 16:59 ` Richard Purdie
@ 2022-12-05 10:00 ` Ola x Nilsson
2022-12-05 11:04 ` Richard Purdie
0 siblings, 1 reply; 27+ messages in thread
From: Ola x Nilsson @ 2022-12-05 10:00 UTC (permalink / raw)
To: Richard Purdie
Cc: Alexandre Belloni, Ross Burton, Lukasz Majewski,
Alexander Kanavin, Yocto-mailing-list, OE-core,
openembedded-architecture
On Wed, Nov 30 2022, Richard Purdie wrote:
> On Wed, 2022-11-30 at 17:56 +0100, Alexandre Belloni wrote:
>> On 30/11/2022 16:46:17+0000, Ross Burton wrote:
>> > On 30 Nov 2022, at 14:20, Richard Purdie via
>> > lists.yoctoproject.org
>> > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
>> > > > > * Could we optionally disable some of the glibc 32 bit function calls
>> > > > > to ensure they're not being used?
>> > > >
>> > > > Could you be more specific here? Would you like to disable some
>> > > > syscalls?
>> > >
>> > > I'm meaning disabling the 32 bit glibc time functions.
>> >
>> > Some time ago I filed
>> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian
>> > has a nice sanity check where it warns if non-LFS glibc functions
>> > are used. I imagine the same logic could be used to check for 32-
>> > bit time_t use.
>
> That sounds interesting and something we should probably look into for
> both issues...
I have a working sanity checker that checks for any glibc functions
affected by -D_FILE_OFFSET_BITS=64 or -D_TIME_BITS=64.
The INSANE_SKIP functionality needs some more polish but I'd be happy to
contribute it.
Some libraries use both 32 and 64 bit APIs to glibc and needs exceptions
in the checker.
I have not run any world builds with this checker, I've focused on the
recipes we actually use so far so we could get to a testable system. My
biggest worry at the moment is rust, I know to little to know if it is
an actual problem and how to fix it.
I would like to be part of any "y2038 team" for Yocto.
--
Ola x Nilsson
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-12-05 10:00 ` Ola x Nilsson
@ 2022-12-05 11:04 ` Richard Purdie
2022-12-05 11:05 ` Ola x Nilsson
0 siblings, 1 reply; 27+ messages in thread
From: Richard Purdie @ 2022-12-05 11:04 UTC (permalink / raw)
To: Ola x Nilsson
Cc: Alexandre Belloni, Ross Burton, Lukasz Majewski,
Alexander Kanavin, Yocto-mailing-list, OE-core,
openembedded-architecture
On Mon, 2022-12-05 at 11:00 +0100, Ola x Nilsson wrote:
> On Wed, Nov 30 2022, Richard Purdie wrote:
>
> > On Wed, 2022-11-30 at 17:56 +0100, Alexandre Belloni wrote:
> > > On 30/11/2022 16:46:17+0000, Ross Burton wrote:
> > > > On 30 Nov 2022, at 14:20, Richard Purdie via
> > > > lists.yoctoproject.org
> > > > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
> > > > > > > * Could we optionally disable some of the glibc 32 bit function calls
> > > > > > > to ensure they're not being used?
> > > > > >
> > > > > > Could you be more specific here? Would you like to disable some
> > > > > > syscalls?
> > > > >
> > > > > I'm meaning disabling the 32 bit glibc time functions.
> > > >
> > > > Some time ago I filed
> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian
> > > > has a nice sanity check where it warns if non-LFS glibc functions
> > > > are used. I imagine the same logic could be used to check for 32-
> > > > bit time_t use.
> >
> > That sounds interesting and something we should probably look into for
> > both issues...
>
> I have a working sanity checker that checks for any glibc functions
> affected by -D_FILE_OFFSET_BITS=64 or -D_TIME_BITS=64.
> The INSANE_SKIP functionality needs some more polish but I'd be happy to
> contribute it.
>
> Some libraries use both 32 and 64 bit APIs to glibc and needs exceptions
> in the checker.
>
> I have not run any world builds with this checker, I've focused on the
> recipes we actually use so far so we could get to a testable system. My
> biggest worry at the moment is rust, I know to little to know if it is
> an actual problem and how to fix it.
>
> I would like to be part of any "y2038 team" for Yocto.
That does sound useful, perhaps sharing it as an RFC patch might be a
good place to start? We might be able to run one of the autobuilder
world targets against it, see how it looks for our core recipes?
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [yocto] [OE-core] [Openembedded-architecture] Y2038 proposal
2022-12-05 11:04 ` Richard Purdie
@ 2022-12-05 11:05 ` Ola x Nilsson
0 siblings, 0 replies; 27+ messages in thread
From: Ola x Nilsson @ 2022-12-05 11:05 UTC (permalink / raw)
To: Richard Purdie
Cc: Alexandre Belloni, Ross Burton, Lukasz Majewski,
Alexander Kanavin, Yocto-mailing-list, OE-core,
openembedded-architecture@lists.openembedded.org
On Mon, Dec 05 2022, Richard Purdie wrote:
> On Mon, 2022-12-05 at 11:00 +0100, Ola x Nilsson wrote:
>> On Wed, Nov 30 2022, Richard Purdie wrote:
>>
>> > On Wed, 2022-11-30 at 17:56 +0100, Alexandre Belloni wrote:
>> > > On 30/11/2022 16:46:17+0000, Ross Burton wrote:
>> > > > On 30 Nov 2022, at 14:20, Richard Purdie via
>> > > > lists.yoctoproject.org
>> > > > <richard.purdie=linuxfoundation.org@lists.yoctoproject.org> wrote:
>> > > > > > > * Could we optionally disable some of the glibc 32 bit function calls
>> > > > > > > to ensure they're not being used?
>> > > > > >
>> > > > > > Could you be more specific here? Would you like to disable some
>> > > > > > syscalls?
>> > > > >
>> > > > > I'm meaning disabling the 32 bit glibc time functions.
>> > > >
>> > > > Some time ago I filed
>> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=6803 as Debian
>> > > > has a nice sanity check where it warns if non-LFS glibc functions
>> > > > are used. I imagine the same logic could be used to check for 32-
>> > > > bit time_t use.
>> >
>> > That sounds interesting and something we should probably look into for
>> > both issues...
>>
>> I have a working sanity checker that checks for any glibc functions
>> affected by -D_FILE_OFFSET_BITS=64 or -D_TIME_BITS=64.
>> The INSANE_SKIP functionality needs some more polish but I'd be happy to
>> contribute it.
>>
>> Some libraries use both 32 and 64 bit APIs to glibc and needs exceptions
>> in the checker.
>>
>> I have not run any world builds with this checker, I've focused on the
>> recipes we actually use so far so we could get to a testable system. My
>> biggest worry at the moment is rust, I know to little to know if it is
>> an actual problem and how to fix it.
>>
>> I would like to be part of any "y2038 team" for Yocto.
>
> That does sound useful, perhaps sharing it as an RFC patch might be a
> good place to start? We might be able to run one of the autobuilder
> world targets against it, see how it looks for our core recipes?
That works for me. I've started preparing a patch for oe-core.
--
Ola x Nilsson
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [Openembedded-architecture] Y2038 proposal
2022-11-30 13:15 ` [Openembedded-architecture] " Richard Purdie
2022-11-30 13:36 ` [OE-core] " Lukasz Majewski
@ 2022-12-01 10:27 ` Alexander Kanavin
2022-12-01 10:36 ` Richard Purdie
1 sibling, 1 reply; 27+ messages in thread
From: Alexander Kanavin @ 2022-12-01 10:27 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-architecture, Yocto-mailing-list, OE-core
On Wed, 30 Nov 2022 at 14:15, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> * We need to have a 32 bit ptest run on the autobuilder (qemux86 should
> work, not sure we can make qemuarm fast). Whether this is manually
> triggered, not sure. We could have a smaller set of ptests to run for
> it?
I just ran qemux86 full ptest locally. It took 4h:10m (same as
qemuarm64 ptest on an arm worker). The fails were:
{'python3': ['test_deterministic_sets'],
'valgrind': ['gdbserver_tests/hgtls',
'gdbserver_tests/mcblocklistsearch',
'gdbserver_tests/mcbreak',
'gdbserver_tests/mcclean_after_fork',
'gdbserver_tests/mchelp',
'gdbserver_tests/mcinfcallRU',
'gdbserver_tests/mcinfcallWSRU',
'gdbserver_tests/mcinvokeRU',
'gdbserver_tests/mcinvokeWS',
'gdbserver_tests/mcleak',
'gdbserver_tests/mcmain_pic',
'gdbserver_tests/mcsignopass',
'gdbserver_tests/mcsigpass',
'gdbserver_tests/mcvabits',
'gdbserver_tests/mcwatchpoints',
'gdbserver_tests/mssnapshot',
'gdbserver_tests/nlcontrolc',
'gdbserver_tests/nlgone_abrt',
'gdbserver_tests/nlgone_exit',
'gdbserver_tests/nlgone_return',
'gdbserver_tests/nlpasssigalrm',
'gdbserver_tests/nlsigvgdb',
'gdbserver_tests/nlvgdbsigqueue',
'memcheck/tests/linux/memfd_create',
'memcheck/tests/linux/timerfd-syscall',
'memcheck/tests/origin5-bz2',
'massif/tests/mmapunmap']}
So I think we could as well fix these, and add full qemux86 ptest to
a-full? It is not heavy on the builder machine (mostly just runs a
single qemu thread), it's just long.
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [Openembedded-architecture] Y2038 proposal
2022-12-01 10:27 ` Alexander Kanavin
@ 2022-12-01 10:36 ` Richard Purdie
0 siblings, 0 replies; 27+ messages in thread
From: Richard Purdie @ 2022-12-01 10:36 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-architecture, Yocto-mailing-list, OE-core
On Thu, 2022-12-01 at 11:27 +0100, Alexander Kanavin wrote:
> On Wed, 30 Nov 2022 at 14:15, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > * We need to have a 32 bit ptest run on the autobuilder (qemux86 should
> > work, not sure we can make qemuarm fast). Whether this is manually
> > triggered, not sure. We could have a smaller set of ptests to run for
> > it?
>
> I just ran qemux86 full ptest locally. It took 4h:10m (same as
> qemuarm64 ptest on an arm worker). The fails were:
>
> {'python3': ['test_deterministic_sets'],
> 'valgrind': ['gdbserver_tests/hgtls',
> 'gdbserver_tests/mcblocklistsearch',
> 'gdbserver_tests/mcbreak',
> 'gdbserver_tests/mcclean_after_fork',
> 'gdbserver_tests/mchelp',
> 'gdbserver_tests/mcinfcallRU',
> 'gdbserver_tests/mcinfcallWSRU',
> 'gdbserver_tests/mcinvokeRU',
> 'gdbserver_tests/mcinvokeWS',
> 'gdbserver_tests/mcleak',
> 'gdbserver_tests/mcmain_pic',
> 'gdbserver_tests/mcsignopass',
> 'gdbserver_tests/mcsigpass',
> 'gdbserver_tests/mcvabits',
> 'gdbserver_tests/mcwatchpoints',
> 'gdbserver_tests/mssnapshot',
> 'gdbserver_tests/nlcontrolc',
> 'gdbserver_tests/nlgone_abrt',
> 'gdbserver_tests/nlgone_exit',
> 'gdbserver_tests/nlgone_return',
> 'gdbserver_tests/nlpasssigalrm',
> 'gdbserver_tests/nlsigvgdb',
> 'gdbserver_tests/nlvgdbsigqueue',
> 'memcheck/tests/linux/memfd_create',
> 'memcheck/tests/linux/timerfd-syscall',
> 'memcheck/tests/origin5-bz2',
> 'massif/tests/mmapunmap']}
>
> So I think we could as well fix these, and add full qemux86 ptest to
> a-full? It is not heavy on the builder machine (mostly just runs a
> single qemu thread), it's just long.
I think we should fix those and we should add the target to the
autobuilder but I'm reluctant to add a long test to a-full. The fact it
is relatively clean suggests it doesn't regress that often. We could do
something like a once a month trigger for it?
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [Openembedded-architecture] Y2038 proposal
2022-11-30 8:07 ` Y2038 proposal Alexander Kanavin
` (2 preceding siblings ...)
2022-11-30 13:15 ` [Openembedded-architecture] " Richard Purdie
@ 2022-11-30 16:38 ` Khem Raj
2022-12-02 8:54 ` [OE-core] " Matt Johnston
4 siblings, 0 replies; 27+ messages in thread
From: Khem Raj @ 2022-11-30 16:38 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: openembedded-architecture, Yocto-mailing-list, OE-core
On Wed, Nov 30, 2022 at 12:08 AM Alexander Kanavin
<alex.kanavin@gmail.com> wrote:
>
> On Tue, 29 Nov 2022 at 16:45, Stephen Jolley <sjolley.yp.pm@gmail.com> wrote:
> > We’d welcome a proposal/series on how to move forward with the Y2038 work for 32 bit platforms.
>
> I have the following proposal:
>
> 1. A branch is made where:
> a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
I have something like this on yoe/mut branch on contrib repo ( due to
musl removing the LFS hacks).
However there are packages which need to be fixed at build time.
> b. qemu is always started with "-rtc base=2040-01-01", simulating
> Y2038 actually occurring.
this is a good time machine :)
> c. an additional runtime test verifies that both RTC clock and system
> clock report 2040.
>
> 2. This branch is run through a-full on the autobuilder. Any uncovered
> issues are filed as bugs.
>
> 3. Once *all* of the bugs are addressed, repeat point 2.
>
> 4. Once there are no more open bugs, 1a is merged into master.
>
> Any fatal flaws in the plan?
>
Not much issues except that package fixes may need to be carried
locally for a while.
> It's not hard to see that Y2038 problem is real and serious, e.g. on
> qemux86 core-image-full-cmdline built from master:
>
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# date -s "2040-01-01"
> Sun Jan 1 00:00:00 UTC 2040
> root@qemux86:~# ls /
> bin boot dev etc home lib lost+found media mnt proc
> run sbin sys tmp usr var
> root@qemux86:~# ls /
> -sh: ls: command not found
>
> On qemux86_64 the same sequence works as expected, of course.
>
> Alex
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#1663): https://lists.openembedded.org/g/openembedded-architecture/message/1663
> Mute This Topic: https://lists.openembedded.org/mt/95354042/1997914
> Group Owner: openembedded-architecture+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-architecture/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [OE-core] Y2038 proposal
2022-11-30 8:07 ` Y2038 proposal Alexander Kanavin
` (3 preceding siblings ...)
2022-11-30 16:38 ` Khem Raj
@ 2022-12-02 8:54 ` Matt Johnston
2022-12-05 23:24 ` Richard Purdie
4 siblings, 1 reply; 27+ messages in thread
From: Matt Johnston @ 2022-12-02 8:54 UTC (permalink / raw)
To: Alexander Kanavin, openembedded-architecture; +Cc: Yocto-mailing-list, OE-core
On Wed, 2022-11-30 at 09:07 +0100, Alexander Kanavin wrote:
> > > > On Tue, 29 Nov 2022 at 16:45, Stephen Jolley
> > > > <sjolley.yp.pm@gmail.com> wrote:
> > > > > > > > We’d welcome a proposal/series on how to move forward with
> > > > > > > > the Y2038 work for 32 bit platforms.
> > > >
> > > > I have the following proposal:
> > > >
> > > > 1. A branch is made where:
> > > > a. "-D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64" is enabled globally.
> > > > b. qemu is always started with "-rtc base=2040-01-01", simulating
> > > > Y2038 actually occurring.
> > > > c. an additional runtime test verifies that both RTC clock and system
> > > > clock report 2040.
> > > >
> > > > 2. This branch is run through a-full on the autobuilder. Any
> > > > uncovered
> > > > issues are filed as bugs.
Your email prompted me to check my own software (Dropbear) and it showed a
few y2038 issues to fix. Those bugs wouldn't be noticed from a quick test -
it "only" prevented auth and idle timeouts from occurring.
gcc and clang are able to flag truncated conversions for 64-bit time_t with
-Wconversion, but that's very noisy. Comparing that against a 32-bit time_t
build, however, gives a pretty clean list of code that needs attention.
As an experiment I've built OpenBMC with and without 64-bit time_t,
https://github.com/mkj/yocto-y2038 has the results and a description. There
are a mix of false positives (particularly tv_usec/tv_nsec), but also some
real-looking things. As an example, busybox using a uint32_t to copy a dhcpd
lease expiry.
I'm not sure the best way to use these logs - they need manual review.
Expanding the list of packages should be easy, but there will be more that
need manual intervention to get rid of -Werror.
Cheers,
Matt
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [OE-core] Y2038 proposal
2022-12-02 8:54 ` [OE-core] " Matt Johnston
@ 2022-12-05 23:24 ` Richard Purdie
0 siblings, 0 replies; 27+ messages in thread
From: Richard Purdie @ 2022-12-05 23:24 UTC (permalink / raw)
To: Matt Johnston, Alexander Kanavin, openembedded-architecture
Cc: Yocto-mailing-list, OE-core
On Fri, 2022-12-02 at 16:54 +0800, Matt Johnston wrote:
> Your email prompted me to check my own software (Dropbear) and it showed a
> few y2038 issues to fix. Those bugs wouldn't be noticed from a quick test -
> it "only" prevented auth and idle timeouts from occurring.
>
> gcc and clang are able to flag truncated conversions for 64-bit time_t with
> -Wconversion, but that's very noisy. Comparing that against a 32-bit time_t
> build, however, gives a pretty clean list of code that needs attention.
>
> As an experiment I've built OpenBMC with and without 64-bit time_t,
> https://github.com/mkj/yocto-y2038 has the results and a description. There
> are a mix of false positives (particularly tv_usec/tv_nsec), but also some
> real-looking things. As an example, busybox using a uint32_t to copy a dhcpd
> lease expiry.
>
> I'm not sure the best way to use these logs - they need manual review.
> Expanding the list of packages should be easy, but there will be more that
> need manual intervention to get rid of -Werror.
That is really interesting data as it confirmed there are real world
issues which changing the compiler flags is going to break. Thanks for
sharing.
What you describe is relatively easy for a maintainer to do as a one
off check but not really something we can do at scale for all the
software we build. It worries me :/. I guess the one upside is that
whilst it did break some functionality, it didn't actually crash the
runtime if I understand what happened correctly.
I'm not sure this should stop our plan to switch the flags but it is
certainly something to think about and be aware of.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread