Should systemd be marked as incompatible with musl?

Should systemd be marked as incompatible with musl?

[Adrian Bunk]

systemd is fundamentally Linux-only and not portable to other kernels.

systemd upstream is using glibc extensions not present in other
C libraries.

systemd upstream is accepting technically correct patches that help 
building with musl, but there is no interest upstream in keeping systemd 
working with non-glibc C libraries.

The way things currently go, systemd/musl will require an ever-growing 
amount of not upstreamable patches - and this is not sustainable.

It is also not clear where the systemd/musl combination makes sense:
If an embedded system is so size limited that the size of glibc matters, 
then systemd is unlikely to be an option.

Other musl-using distribution also seem to favour more lightweight
init systems.

If someone has important usecases for using systemd with musl it might 
be possible to solve find solutions in an upstreamable way, but this 
would require a significant longterm commitment.

The current state is rather bad, some examples:

0001-Use-getenv-when-secure-versions-are-not-available.patch
looks like waiting for a CVE number.

0008-add-missing-FTW_-macros-for-musl.patch seems to break code in areas 
(rng, smack, selinux) where breakages can have security implications.

0002-don-t-use-glibc-specific-qsort_r.patch might introduce race 
conditions that cause data corruption.

0007-don-t-fail-if-GLOB_BRACE-and-GLOB_ALTDIRFUNC-is-not.patch makes 
some functions behave different from what is expected by callers.

0017-Do-not-disable-buffering-when-writing-to-oom_score_a.patch and
0001-do-not-disable-buffer-in-writing-files.patch seems like an issue
that would require debugging the root cause and then addressing it.

All this looks bad, and is expected to become worse.

In this situation it would be better to not claim that using systemd
with musl would be a supported option.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Burton, Ross]

On Thu, 23 May 2019 at 11:34, Adrian Bunk <bunk@stusta.de> wrote:
> systemd is fundamentally Linux-only and not portable to other kernels.
>
> systemd upstream is using glibc extensions not present in other
> C libraries.
>
> systemd upstream is accepting technically correct patches that help
> building with musl, but there is no interest upstream in keeping systemd
> working with non-glibc C libraries.
>
> The way things currently go, systemd/musl will require an ever-growing
> amount of not upstreamable patches - and this is not sustainable.

I think I have to agree with you: several of the extensions are for
security purposes, so we're potentially actively introducing issues.

Ross

Re: Should systemd be marked as incompatible with musl?

[ChenQi]

On 05/23/2019 08:22 PM, Burton, Ross wrote:
> On Thu, 23 May 2019 at 11:34, Adrian Bunk <bunk@stusta.de> wrote:
>> systemd is fundamentally Linux-only and not portable to other kernels.
>>
>> systemd upstream is using glibc extensions not present in other
>> C libraries.
>>
>> systemd upstream is accepting technically correct patches that help
>> building with musl, but there is no interest upstream in keeping systemd
>> working with non-glibc C libraries.
>>
>> The way things currently go, systemd/musl will require an ever-growing
>> amount of not upstreamable patches - and this is not sustainable.
> I think I have to agree with you: several of the extensions are for
> security purposes, so we're potentially actively introducing issues.
>
> Ross


I suggest we only apply these musl patches when the libc is musl.

SRC_URI += "${@d.getVar('SRC_URI_MUSL') if d.getVar('TCLIBC') == 'musl' 
else ''}"

In this way, these patches do not affect glibc users. People who 
actually use musl will test these patches and maybe improve them if they 
really care about it.

For example, the following patches are written by me.

0017-Do-not-disable-buffering-when-writing-to-oom_score_a.patch and
0001-do-not-disable-buffer-in-writing-files.patch

These two patches are to solve runtime problems in musl based system. I 
knew there's something different in buffering mechanism between glibc 
and musl that causes this problem, but I really did not want to spend 
time digging into it. And as systemd was using unbuffered way for 
writing files for a long time, I just disabled them.

As Adrian and Ross pointed out, these patches are introducing potential 
security problems. I agree that we should do something about it. At a 
minimum, the glibc based system should not be affected.

I'll send out patch and please help review it.

Best Regards,
Chen Qi

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On 5/23/19 6:45 PM, ChenQi wrote:
> On 05/23/2019 08:22 PM, Burton, Ross wrote:
>> On Thu, 23 May 2019 at 11:34, Adrian Bunk <bunk@stusta.de> wrote:
>>> systemd is fundamentally Linux-only and not portable to other kernels.
>>>
>>> systemd upstream is using glibc extensions not present in other
>>> C libraries.
>>>
>>> systemd upstream is accepting technically correct patches that help
>>> building with musl, but there is no interest upstream in keeping systemd
>>> working with non-glibc C libraries.
>>>
>>> The way things currently go, systemd/musl will require an ever-growing
>>> amount of not upstreamable patches - and this is not sustainable.
>> I think I have to agree with you: several of the extensions are for
>> security purposes, so we're potentially actively introducing issues.
>>
>> Ross
> 
> 
> I suggest we only apply these musl patches when the libc is musl.
> 
> SRC_URI += "${@d.getVar('SRC_URI_MUSL') if d.getVar('TCLIBC') == 'musl' 
> else ''}"
> 
> In this way, these patches do not affect glibc users. People who 
> actually use musl will test these patches and maybe improve them if they 
> really care about it.

thats acceptable approach. So far we have helped upstreams of apps to 
cleanup or add support for musl and it has worked reasonably well, 
systemd is a bit of odd man out since they explicitly seems to not care 
for non glibc/linux systems, still they do accept cleanups which are 
resulting from such ports so its not completely wasted excercise, but I 
think dropping systemd support completely from musl is not an option I 
would like to go with, there are cases where this makes sense. 
Especially when you have to cater to different set of devices from small 
to big, userspace remaining same is big advantage atleast in the world I 
am in.

> 
> For example, the following patches are written by me.
> 
> 0017-Do-not-disable-buffering-when-writing-to-oom_score_a.patch and
> 0001-do-not-disable-buffer-in-writing-files.patch
> 
> These two patches are to solve runtime problems in musl based system. I 
> knew there's something different in buffering mechanism between glibc 
> and musl that causes this problem, but I really did not want to spend 
> time digging into it. And as systemd was using unbuffered way for 
> writing files for a long time, I just disabled them.
> 
> As Adrian and Ross pointed out, these patches are introducing potential 
> security problems. I agree that we should do something about it. At a 
> minimum, the glibc based system should not be affected.
> 
> I'll send out patch and please help review it.
> 
> Best Regards,
> Chen Qi
> 

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
>...
> but I think dropping
> systemd support completely from musl is not an option I would like to go
> with, there are cases where this makes sense. Especially when you have to
> cater to different set of devices from small to big, userspace remaining
> same is big advantage atleast in the world I am in.
>...

That's a good point - when arguing against systemd as default init system.

systemd is bigger than glibc, therefore on very small systems where the 
size of the C library matters using systemd is usually not an option.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On 5/24/19 3:12 AM, Adrian Bunk wrote:
> On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
>> ...
>> but I think dropping
>> systemd support completely from musl is not an option I would like to go
>> with, there are cases where this makes sense. Especially when you have to
>> cater to different set of devices from small to big, userspace remaining
>> same is big advantage atleast in the world I am in.
>> ...
> 
> That's a good point - when arguing against systemd as default init system.
> 
> systemd is bigger than glibc, therefore on very small systems where the
> size of the C library matters using systemd is usually not an option.
> 

Yes, design-wise I concur, in practice, desktop distros rule the linux 
world and systemd is quite prevalent there, so sometimes you have to 
wear the shoes of same color, atleast thats what I see. apps want to run 
uniformly on both OE and non-OE systems

> cu
> Adrian
> 

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> 
> 
> On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > ...
> > > but I think dropping
> > > systemd support completely from musl is not an option I would like to go
> > > with, there are cases where this makes sense. Especially when you have to
> > > cater to different set of devices from small to big, userspace remaining
> > > same is big advantage atleast in the world I am in.
> > > ...
> > 
> > That's a good point - when arguing against systemd as default init system.
> > 
> > systemd is bigger than glibc, therefore on very small systems where the
> > size of the C library matters using systemd is usually not an option.
> 
> Yes, design-wise I concur, in practice, desktop distros rule the linux world
> and systemd is quite prevalent there,

Desktop distros don't use musl - it wouldn't make sense.

> so sometimes you have to wear the
> shoes of same color, atleast thats what I see.

Is any musl-using distribution supporting it with systemd?

After a cursory look around it doesn't seem that trying to support the 
weird combination of systemd and musl brings compatibility with anyone 
else.

> apps want to run uniformly on both OE and non-OE systems

This implies using glibc, which is the correct choice in most cases.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> >
> >
> > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > ...
> > > > but I think dropping
> > > > systemd support completely from musl is not an option I would like to go
> > > > with, there are cases where this makes sense. Especially when you have to
> > > > cater to different set of devices from small to big, userspace remaining
> > > > same is big advantage atleast in the world I am in.
> > > > ...
> > >
> > > That's a good point - when arguing against systemd as default init system.
> > >
> > > systemd is bigger than glibc, therefore on very small systems where the
> > > size of the C library matters using systemd is usually not an option.
> >
> > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > and systemd is quite prevalent there,
>
> Desktop distros don't use musl - it wouldn't make sense.
>

Yes,  what I was saying is that decisions on init systems and like that are
influenced by desktops too. Since the apps are being written for across the
board portfolio where some platforms are desktop/server driven some are
more embedded

> > so sometimes you have to wear the
> > shoes of same color, atleast thats what I see.
>
> Is any musl-using distribution supporting it with systemd?
>
> After a cursory look around it doesn't seem that trying to support the
> weird combination of systemd and musl brings compatibility with anyone
> else.
>
> > apps want to run uniformly on both OE and non-OE systems
>
> This implies using glibc, which is the correct choice in most cases.
>
> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Fri, May 24, 2019 at 10:31:25AM -0700, Khem Raj wrote:
> On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
> > On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> > > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > > ...
> > > > > but I think dropping
> > > > > systemd support completely from musl is not an option I would like to go
> > > > > with, there are cases where this makes sense. Especially when you have to
> > > > > cater to different set of devices from small to big, userspace remaining
> > > > > same is big advantage atleast in the world I am in.
> > > > > ...
> > > >
> > > > That's a good point - when arguing against systemd as default init system.
> > > >
> > > > systemd is bigger than glibc, therefore on very small systems where the
> > > > size of the C library matters using systemd is usually not an option.
> > >
> > > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > > and systemd is quite prevalent there,
> >
> > Desktop distros don't use musl - it wouldn't make sense.
> 
> Yes,  what I was saying is that decisions on init systems and like that are
> influenced by desktops too. Since the apps are being written for across the
> board portfolio where some platforms are desktop/server driven some are
> more embedded

The point is that most of the time using musl doesn't make sense.

Definitely not on a desktop, and also not with systemd as init system.

I haven't done exact measurements and it would also depend on the 
architecture, but the only real-world benefit of using musl instead
of glibc for an OE user is something like "3 MB smaller in an -Os build".

On many of todays embedded systems such a small size difference is 
irrelevant. In these cases the correct solution that stays compatible
with everything else is to use glibc.

And on really tiny systems where every single MB counts,
all other design choices also have a high emphasis on size.

Using systemd instead of busybox init (or some other small init system) 
would cost you much more space than what the C library choice gave.

And the current sad state of the systemd musl patching that makes it
compile but creates misbehaving functions and security vulnerabilities
makes it an even worse idea to use systemd with musl.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On Fri, May 24, 2019 at 10:59 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, May 24, 2019 at 10:31:25AM -0700, Khem Raj wrote:
> > On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> > > > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > > > ...
> > > > > > but I think dropping
> > > > > > systemd support completely from musl is not an option I would like to go
> > > > > > with, there are cases where this makes sense. Especially when you have to
> > > > > > cater to different set of devices from small to big, userspace remaining
> > > > > > same is big advantage atleast in the world I am in.
> > > > > > ...
> > > > >
> > > > > That's a good point - when arguing against systemd as default init system.
> > > > >
> > > > > systemd is bigger than glibc, therefore on very small systems where the
> > > > > size of the C library matters using systemd is usually not an option.
> > > >
> > > > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > > > and systemd is quite prevalent there,
> > >
> > > Desktop distros don't use musl - it wouldn't make sense.
> >
> > Yes,  what I was saying is that decisions on init systems and like that are
> > influenced by desktops too. Since the apps are being written for across the
> > board portfolio where some platforms are desktop/server driven some are
> > more embedded
>
> The point is that most of the time using musl doesn't make sense.
>
> Definitely not on a desktop, and also not with systemd as init system.
>
> I haven't done exact measurements and it would also depend on the
> architecture, but the only real-world benefit of using musl instead
> of glibc for an OE user is something like "3 MB smaller in an -Os build".
>
> On many of todays embedded systems such a small size difference is
> irrelevant. In these cases the correct solution that stays compatible
> with everything else is to use glibc.

No, there is DRAM use difference too.

>
> And on really tiny systems where every single MB counts,
> all other design choices also have a high emphasis on size.
>
> Using systemd instead of busybox init (or some other small init system)
> would cost you much more space than what the C library choice gave.
>
> And the current sad state of the systemd musl patching that makes it
> compile but creates misbehaving functions and security vulnerabilities
> makes it an even worse idea to use systemd with musl.
>

I think we should put in plan for 2.8 and define the scope, since we
are switching
poky defaults to systemd, this might have testing implications for
musl especially from
CI perpective

> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
> On Fri, May 24, 2019 at 10:59 AM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Fri, May 24, 2019 at 10:31:25AM -0700, Khem Raj wrote:
> > > On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > > On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> > > > > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > > > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > > > > ...
> > > > > > > but I think dropping
> > > > > > > systemd support completely from musl is not an option I would like to go
> > > > > > > with, there are cases where this makes sense. Especially when you have to
> > > > > > > cater to different set of devices from small to big, userspace remaining
> > > > > > > same is big advantage atleast in the world I am in.
> > > > > > > ...
> > > > > >
> > > > > > That's a good point - when arguing against systemd as default init system.
> > > > > >
> > > > > > systemd is bigger than glibc, therefore on very small systems where the
> > > > > > size of the C library matters using systemd is usually not an option.
> > > > >
> > > > > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > > > > and systemd is quite prevalent there,
> > > >
> > > > Desktop distros don't use musl - it wouldn't make sense.
> > >
> > > Yes,  what I was saying is that decisions on init systems and like that are
> > > influenced by desktops too. Since the apps are being written for across the
> > > board portfolio where some platforms are desktop/server driven some are
> > > more embedded
> >
> > The point is that most of the time using musl doesn't make sense.
> >
> > Definitely not on a desktop, and also not with systemd as init system.
> >
> > I haven't done exact measurements and it would also depend on the
> > architecture, but the only real-world benefit of using musl instead
> > of glibc for an OE user is something like "3 MB smaller in an -Os build".
> >
> > On many of todays embedded systems such a small size difference is
> > irrelevant. In these cases the correct solution that stays compatible
> > with everything else is to use glibc.
> 
> No, there is DRAM use difference too.

I didn't limit the size difference to the filesystem.

(But in practice RAM is usually a smaller problem than the filesystem.)

> > And on really tiny systems where every single MB counts,
> > all other design choices also have a high emphasis on size.
> >
> > Using systemd instead of busybox init (or some other small init system)
> > would cost you much more space than what the C library choice gave.
> >
> > And the current sad state of the systemd musl patching that makes it
> > compile but creates misbehaving functions and security vulnerabilities
> > makes it an even worse idea to use systemd with musl.
> >
> 
> I think we should put in plan for 2.8 and define the scope, since we
> are switching
> poky defaults to systemd,

Switching glibc builds to systemd as default is a reasonable decision.

Switching musl builds to systemd as default would be very bad.

Combining a tiny C library with a huge init system completely misses
the configurations where the tiny C library actually makes sense for 
users.

> this might have testing implications for
> musl especially from
> CI perpective

The CI should aim at testing what makes sense for users.

Offering musl becomes pointless if there is no small init system that 
is properly supported and tested with musl builds.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Andre McCurdy]

On Fri, May 24, 2019 at 11:46 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
> > On Fri, May 24, 2019 at 10:59 AM Adrian Bunk <bunk@stusta.de> wrote:
> > >
> > > On Fri, May 24, 2019 at 10:31:25AM -0700, Khem Raj wrote:
> > > > On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> > > > > > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > > > > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > > > > > ...
> > > > > > > > but I think dropping
> > > > > > > > systemd support completely from musl is not an option I would like to go
> > > > > > > > with, there are cases where this makes sense. Especially when you have to
> > > > > > > > cater to different set of devices from small to big, userspace remaining
> > > > > > > > same is big advantage atleast in the world I am in.
> > > > > > > > ...
> > > > > > >
> > > > > > > That's a good point - when arguing against systemd as default init system.
> > > > > > >
> > > > > > > systemd is bigger than glibc, therefore on very small systems where the
> > > > > > > size of the C library matters using systemd is usually not an option.
> > > > > >
> > > > > > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > > > > > and systemd is quite prevalent there,
> > > > >
> > > > > Desktop distros don't use musl - it wouldn't make sense.
> > > >
> > > > Yes,  what I was saying is that decisions on init systems and like that are
> > > > influenced by desktops too. Since the apps are being written for across the
> > > > board portfolio where some platforms are desktop/server driven some are
> > > > more embedded
> > >
> > > The point is that most of the time using musl doesn't make sense.
> > >
> > > Definitely not on a desktop, and also not with systemd as init system.
> > >
> > > I haven't done exact measurements and it would also depend on the
> > > architecture, but the only real-world benefit of using musl instead
> > > of glibc for an OE user is something like "3 MB smaller in an -Os build".
> > >
> > > On many of todays embedded systems such a small size difference is
> > > irrelevant. In these cases the correct solution that stays compatible
> > > with everything else is to use glibc.
> >
> > No, there is DRAM use difference too.
>
> I didn't limit the size difference to the filesystem.
>
> (But in practice RAM is usually a smaller problem than the filesystem.)
>
> > > And on really tiny systems where every single MB counts,
> > > all other design choices also have a high emphasis on size.
> > >
> > > Using systemd instead of busybox init (or some other small init system)
> > > would cost you much more space than what the C library choice gave.
> > >
> > > And the current sad state of the systemd musl patching that makes it
> > > compile but creates misbehaving functions and security vulnerabilities
> > > makes it an even worse idea to use systemd with musl.
> > >
> >
> > I think we should put in plan for 2.8 and define the scope, since we
> > are switching
> > poky defaults to systemd,
>
> Switching glibc builds to systemd as default is a reasonable decision.
>
> Switching musl builds to systemd as default would be very bad.
>
> Combining a tiny C library with a huge init system completely misses
> the configurations where the tiny C library actually makes sense for
> users.

For new projects yes. However I know of a project (a big project,
shipping millions of devices) which picked systemd and glibc long ago
and is now running out of space. They already have various solutions
to free up Flash (some apps switched to being runtime downloadable,
etc) but if/when more savings need to be found then switching from
glibc to musl might be a less invasive change than switching from
systemd to some other init system. We obviously shouldn't make
decisions for OE today based on the historical decisions of one
project, but I just want to make the point that real world projects
have a lifetime and may end up with a combination of systemd + musl
due on past decisions that may not make sense for a new project
starting today.

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On Fri, May 24, 2019 at 12:34 PM Andre McCurdy <armccurdy@gmail.com> wrote:
>
> On Fri, May 24, 2019 at 11:46 AM Adrian Bunk <bunk@stusta.de> wrote:
> >
> > On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
> > > On Fri, May 24, 2019 at 10:59 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > >
> > > > On Fri, May 24, 2019 at 10:31:25AM -0700, Khem Raj wrote:
> > > > > On Fri, May 24, 2019 at 10:27 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > > > > On Fri, May 24, 2019 at 09:13:08AM -0700, Khem Raj wrote:
> > > > > > > On 5/24/19 3:12 AM, Adrian Bunk wrote:
> > > > > > > > On Thu, May 23, 2019 at 07:16:53PM -0700, Khem Raj wrote:
> > > > > > > > > ...
> > > > > > > > > but I think dropping
> > > > > > > > > systemd support completely from musl is not an option I would like to go
> > > > > > > > > with, there are cases where this makes sense. Especially when you have to
> > > > > > > > > cater to different set of devices from small to big, userspace remaining
> > > > > > > > > same is big advantage atleast in the world I am in.
> > > > > > > > > ...
> > > > > > > >
> > > > > > > > That's a good point - when arguing against systemd as default init system.
> > > > > > > >
> > > > > > > > systemd is bigger than glibc, therefore on very small systems where the
> > > > > > > > size of the C library matters using systemd is usually not an option.
> > > > > > >
> > > > > > > Yes, design-wise I concur, in practice, desktop distros rule the linux world
> > > > > > > and systemd is quite prevalent there,
> > > > > >
> > > > > > Desktop distros don't use musl - it wouldn't make sense.
> > > > >
> > > > > Yes,  what I was saying is that decisions on init systems and like that are
> > > > > influenced by desktops too. Since the apps are being written for across the
> > > > > board portfolio where some platforms are desktop/server driven some are
> > > > > more embedded
> > > >
> > > > The point is that most of the time using musl doesn't make sense.
> > > >
> > > > Definitely not on a desktop, and also not with systemd as init system.
> > > >
> > > > I haven't done exact measurements and it would also depend on the
> > > > architecture, but the only real-world benefit of using musl instead
> > > > of glibc for an OE user is something like "3 MB smaller in an -Os build".
> > > >
> > > > On many of todays embedded systems such a small size difference is
> > > > irrelevant. In these cases the correct solution that stays compatible
> > > > with everything else is to use glibc.
> > >
> > > No, there is DRAM use difference too.
> >
> > I didn't limit the size difference to the filesystem.
> >
> > (But in practice RAM is usually a smaller problem than the filesystem.)
> >
> > > > And on really tiny systems where every single MB counts,
> > > > all other design choices also have a high emphasis on size.
> > > >
> > > > Using systemd instead of busybox init (or some other small init system)
> > > > would cost you much more space than what the C library choice gave.
> > > >
> > > > And the current sad state of the systemd musl patching that makes it
> > > > compile but creates misbehaving functions and security vulnerabilities
> > > > makes it an even worse idea to use systemd with musl.
> > > >
> > >
> > > I think we should put in plan for 2.8 and define the scope, since we
> > > are switching
> > > poky defaults to systemd,
> >
> > Switching glibc builds to systemd as default is a reasonable decision.
> >
> > Switching musl builds to systemd as default would be very bad.
> >
> > Combining a tiny C library with a huge init system completely misses
> > the configurations where the tiny C library actually makes sense for
> > users.
>
> For new projects yes. However I know of a project (a big project,
> shipping millions of devices) which picked systemd and glibc long ago
> and is now running out of space. They already have various solutions
> to free up Flash (some apps switched to being runtime downloadable,
> etc) but if/when more savings need to be found then switching from
> glibc to musl might be a less invasive change than switching from
> systemd to some other init system. We obviously shouldn't make
> decisions for OE today based on the historical decisions of one
> project, but I just want to make the point that real world projects
> have a lifetime and may end up with a combination of systemd + musl
> due on past decisions that may not make sense for a new project
> starting today.

rightly so, and it can also mean that project moves away from OE if headwind
is too strong.

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Fri, May 24, 2019 at 12:34:23PM -0700, Andre McCurdy wrote:
> On Fri, May 24, 2019 at 11:46 AM Adrian Bunk <bunk@stusta.de> wrote:
> > On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
>...
> > > I think we should put in plan for 2.8 and define the scope, since we
> > > are switching
> > > poky defaults to systemd,
> > 
> > Switching glibc builds to systemd as default is a reasonable decision.
> >
> > Switching musl builds to systemd as default would be very bad.
> >
> > Combining a tiny C library with a huge init system completely misses
> > the configurations where the tiny C library actually makes sense for
> > users.
> 
> For new projects yes. However I know of a project (a big project,
> shipping millions of devices) which picked systemd and glibc long ago
> and is now running out of space. They already have various solutions
> to free up Flash (some apps switched to being runtime downloadable,
> etc) but if/when more savings need to be found then switching from
> glibc to musl might be a less invasive change than switching from
> systemd to some other init system. We obviously shouldn't make
> decisions for OE today based on the historical decisions of one
> project, but I just want to make the point that real world projects
> have a lifetime and may end up with a combination of systemd + musl
> due on past decisions that may not make sense for a new project
> starting today.

I am feeling guilty that there are two only partially related
topics mixed in this discussion.

In this part of the discussion the topic was what the default 
(and CI-tested) init system for musl should be - it seems obvious
to me that systemd is not what users will usually want to use with musl.

But there is also the topic whether systemd on musl is
in a state that it should be made available at all.

Does any of these millions of devices have untrusted
users or an internet connection?

At that point my email that started this thread becomes relevant,
the fact that the systemd/musl patches in OE add new security 
vulnerabilities and other bugs - and none of the systemd-on-musl
proponents seems to consider this a problem they have to fix ASAP.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Andre McCurdy]

On Fri, May 24, 2019 at 1:28 PM Adrian Bunk <bunk@stusta.de> wrote:
> On Fri, May 24, 2019 at 12:34:23PM -0700, Andre McCurdy wrote:
> > On Fri, May 24, 2019 at 11:46 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
> >...
> > > > I think we should put in plan for 2.8 and define the scope, since we
> > > > are switching
> > > > poky defaults to systemd,
> > >
> > > Switching glibc builds to systemd as default is a reasonable decision.
> > >
> > > Switching musl builds to systemd as default would be very bad.
> > >
> > > Combining a tiny C library with a huge init system completely misses
> > > the configurations where the tiny C library actually makes sense for
> > > users.
> >
> > For new projects yes. However I know of a project (a big project,
> > shipping millions of devices) which picked systemd and glibc long ago
> > and is now running out of space. They already have various solutions
> > to free up Flash (some apps switched to being runtime downloadable,
> > etc) but if/when more savings need to be found then switching from
> > glibc to musl might be a less invasive change than switching from
> > systemd to some other init system. We obviously shouldn't make
> > decisions for OE today based on the historical decisions of one
> > project, but I just want to make the point that real world projects
> > have a lifetime and may end up with a combination of systemd + musl
> > due on past decisions that may not make sense for a new project
> > starting today.
>
> I am feeling guilty that there are two only partially related
> topics mixed in this discussion.
>
> In this part of the discussion the topic was what the default
> (and CI-tested) init system for musl should be - it seems obvious
> to me that systemd is not what users will usually want to use with musl.

It would be great to have an alternative init system option for
smaller devices in oe-core. I don't think collectively we have the
development capacity to support it though (it's probably far more work
than properly fixing all the currently known issues with systemd on
musl...).

> But there is also the topic whether systemd on musl is
> in a state that it should be made available at all.

I think it's wrong to remove stuff from oe-core just because it isn't
perfect or isn't CI tested. Having something in oe-core means there's
a common version to collaborate on. If it gets kicked out then
development efforts inevitably fragment. Users are generally smart
enough to understand the concept of an experimental feature - although
we could perhaps do a better job of identifying them. Maybe one day
when users can create a custom distro config by running "make
menuconfig" there will be an option to enable experimental features
and the combination of musl and systemd would be hidden behind that.
Until then, perhaps we could add a sanity check warning if musl and
systemd are enabled together?

> Does any of these millions of devices have untrusted
> users or an internet connection?

No local users (if I remember right, everything runs as root) but they
do all have broadband internet connections.

> At that point my email that started this thread becomes relevant,
> the fact that the systemd/musl patches in OE add new security
> vulnerabilities and other bugs - and none of the systemd-on-musl
> proponents seems to consider this a problem they have to fix ASAP.

It's certainly a problem and we should try to fix it. It's not at all
uncommon that patches to fix build issues with musl, clang, a new
version of gcc, etc have a life cycle... a first pass just to fix the
build and then updates as issues are found or better solutions get
merged upstream. It's the normal process. You could argue that we are
sometimes too quick to merge the first pass hacks and too slow to
review and update them, but unfortunately that's just a consequence of
limited developer resources (and it's always more fun to try to get
the latest version of something working than review and cleanup old
patches...).

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Fri, May 24, 2019 at 03:25:57PM -0700, Andre McCurdy wrote:
> On Fri, May 24, 2019 at 1:28 PM Adrian Bunk <bunk@stusta.de> wrote:
> > On Fri, May 24, 2019 at 12:34:23PM -0700, Andre McCurdy wrote:
> > > On Fri, May 24, 2019 at 11:46 AM Adrian Bunk <bunk@stusta.de> wrote:
> > > > On Fri, May 24, 2019 at 11:04:50AM -0700, Khem Raj wrote:
> > >...
> > > > > I think we should put in plan for 2.8 and define the scope, since we
> > > > > are switching
> > > > > poky defaults to systemd,
> > > >
> > > > Switching glibc builds to systemd as default is a reasonable decision.
> > > >
> > > > Switching musl builds to systemd as default would be very bad.
> > > >
> > > > Combining a tiny C library with a huge init system completely misses
> > > > the configurations where the tiny C library actually makes sense for
> > > > users.
> > >
> > > For new projects yes. However I know of a project (a big project,
> > > shipping millions of devices) which picked systemd and glibc long ago
> > > and is now running out of space. They already have various solutions
> > > to free up Flash (some apps switched to being runtime downloadable,
> > > etc) but if/when more savings need to be found then switching from
> > > glibc to musl might be a less invasive change than switching from
> > > systemd to some other init system. We obviously shouldn't make
> > > decisions for OE today based on the historical decisions of one
> > > project, but I just want to make the point that real world projects
> > > have a lifetime and may end up with a combination of systemd + musl
> > > due on past decisions that may not make sense for a new project
> > > starting today.
> >
> > I am feeling guilty that there are two only partially related
> > topics mixed in this discussion.
> >
> > In this part of the discussion the topic was what the default
> > (and CI-tested) init system for musl should be - it seems obvious
> > to me that systemd is not what users will usually want to use with musl.
> 
> It would be great to have an alternative init system option for
> smaller devices in oe-core. I don't think collectively we have the
> development capacity to support it though (it's probably far more work
> than properly fixing all the currently known issues with systemd on
> musl...).

Is there development capacity to support musl?

Supporting musl is a real pain across the board,
with new issues all the time.

For really tiny systems you need both a tiny C library and a tiny init 
system, so not properly supporting the combination of both forces users 
to use alternative options instead of OE.

Which minimizes the benefits gained by the pains of supporting musl.

> > But there is also the topic whether systemd on musl is
> > in a state that it should be made available at all.
> 
> I think it's wrong to remove stuff from oe-core just because it isn't
> perfect or isn't CI tested. Having something in oe-core means there's
> a common version to collaborate on. If it gets kicked out then
> development efforts inevitably fragment. Users are generally smart
> enough to understand the concept of an experimental feature - although
> we could perhaps do a better job of identifying them. Maybe one day
> when users can create a custom distro config by running "make
> menuconfig" there will be an option to enable experimental features
> and the combination of musl and systemd would be hidden behind that.
> Until then, perhaps we could add a sanity check warning if musl and
> systemd are enabled together?

That would be an improvement.

But it conflicts with the intention to make systemd the default
and CI-tested init system for musl.

>...
> > At that point my email that started this thread becomes relevant,
> > the fact that the systemd/musl patches in OE add new security
> > vulnerabilities and other bugs - and none of the systemd-on-musl
> > proponents seems to consider this a problem they have to fix ASAP.
> 
> It's certainly a problem and we should try to fix it. It's not at all
> uncommon that patches to fix build issues with musl, clang, a new
> version of gcc, etc have a life cycle... a first pass just to fix the
> build and then updates as issues are found or better solutions get
> merged upstream. It's the normal process. You could argue that we are
> sometimes too quick to merge the first pass hacks and too slow to
> review and update them, but unfortunately that's just a consequence of
> limited developer resources (and it's always more fun to try to get
> the latest version of something working than review and cleanup old
> patches...).

If upstreaming is possible at all.

With systemd on musl there is also the fundamental problem that 
neither of the upstreams is interested in compromising their 
software for the other.

Not to blame either of them, there is simply a fundamental conflict 
between the systemd "use all functionality available" and the musl
"be a tiny C library" and "follow standards and provide only some
GNU extensions".

One example:

systemd uses qsort_r.
musl upstream doesn't want to add qsort_r since it is nonstandard
and BSD and glibc ended up providing incompatible versions.
Most C libraries for Linux just follow glibc on that,
but musl upstream is not doing this with a reasonable
technical justification.

For 0002-don-t-use-glibc-specific-qsort_r.patch (which looks as if it 
does cause data corruption) upstream is therefore in practice POSIX,
unless you manage to convince systemd upstream to use something like
gnulib to provide all the functionality they use that is not provided
by musl. Which would also be a constant effort you have to makes since
systemd upstream would not care about musl when making changes to
their code.

Even harder are cases like on_exit(3) (not currently hit by systemd but 
by other Linux-only software), which might not be implementable as an 
external function outside the C library.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Andre McCurdy]

On Sat, May 25, 2019 at 12:25 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> Is there development capacity to support musl?

It would be shame if there wasn't enough developer capacity to support
musl but in the end it comes down to what contributors to OE want to
work on.

There's a danger that the areas which OE contributors want to work on
don't exactly match what the overall Embedded Linux community wants
and we alienate potential users (for example we ditched uClibc support
not because no one wants to use uClibc any more but because no
contributors to OE wanted to spend time on uClibc) but overall the
model seems to work OK. I think right now we have enough interest in
musl to keep it alive.

> Supporting musl is a real pain across the board,
> with new issues all the time.

There are always new issues and bugs to be solved in OE as a
consequence of trying to keep all packages up to date. Whether the
issues arising from musl are a real pain or a fun new set of problems
to solve is mostly a matter of perspective.

> For really tiny systems you need both a tiny C library and a tiny init> system, so not properly supporting the combination of both forces users
> to use alternative options instead of OE.
>
> Which minimizes the benefits gained by the pains of supporting musl.

A modern tiny init system would be nice to have, but it's not
essential or fair to say that musl is useless without one. Many
projects, especially tiny ones, manage fine with init scripts and
custom process management.

> > It's certainly a problem and we should try to fix it. It's not at all
> > uncommon that patches to fix build issues with musl, clang, a new
> > version of gcc, etc have a life cycle... a first pass just to fix the
> > build and then updates as issues are found or better solutions get
> > merged upstream. It's the normal process. You could argue that we are
> > sometimes too quick to merge the first pass hacks and too slow to
> > review and update them, but unfortunately that's just a consequence of
> > limited developer resources (and it's always more fun to try to get
> > the latest version of something working than review and cleanup old
> > patches...).
>
> If upstreaming is possible at all.
>
> With systemd on musl there is also the fundamental problem that
> neither of the upstreams is interested in compromising their
> software for the other.
>
> Not to blame either of them, there is simply a fundamental conflict
> between the systemd "use all functionality available" and the musl
> "be a tiny C library" and "follow standards and provide only some
> GNU extensions".
>
> One example:
>
> systemd uses qsort_r.
> musl upstream doesn't want to add qsort_r since it is nonstandard
> and BSD and glibc ended up providing incompatible versions.
> Most C libraries for Linux just follow glibc on that,
> but musl upstream is not doing this with a reasonable
> technical justification.
>
> For 0002-don-t-use-glibc-specific-qsort_r.patch (which looks as if it
> does cause data corruption) upstream is therefore in practice POSIX,
> unless you manage to convince systemd upstream to use something like
> gnulib to provide all the functionality they use that is not provided
> by musl. Which would also be a constant effort you have to makes since
> systemd upstream would not care about musl when making changes to
> their code.
>
> Even harder are cases like on_exit(3) (not currently hit by systemd but
> by other Linux-only software), which might not be implementable as an
> external function outside the C library.

Upstreamable solutions should always be preferred but if you can
accept that OE may need to carry patches which will never be
upstreamed (which isn't too much of a stretch given the age of some of
the patches we carry for gcc, busybox, etc) then none of the currently
known issues with systemd and musl seem like fundamental problems. A
few pragmatic patches applied by OE would go a long way to bridging
the conflicting goals of the two upstream projects. It's basically the
approach we've taken already - the question is just one of improving
the patches we already have (and maybe patching musl to add missing
functionality instead of only trying to patch systemd to not depend on
it).

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Tue, May 28, 2019 at 04:10:45PM -0700, Andre McCurdy wrote:
> On Sat, May 25, 2019 at 12:25 AM Adrian Bunk <bunk@stusta.de> wrote:
>...
> > Supporting musl is a real pain across the board,
> > with new issues all the time.
> 
> There are always new issues and bugs to be solved in OE as a
> consequence of trying to keep all packages up to date. Whether the
> issues arising from musl are a real pain or a fun new set of problems
> to solve is mostly a matter of perspective.

Usually someone submits a change, and later gets notified that the 
change was dropped from master-next due to a musl issue.

That's not fun.

And all these compile errors with musl due to header order are a real WTF,
every other library (not limited to C libraries) is now doing headers 
properly so that any order works. No fun in supporting a broken design.

> > For really tiny systems you need both a tiny C library and a tiny init> system, so not properly supporting the combination of both forces users
> > to use alternative options instead of OE.
> >
> > Which minimizes the benefits gained by the pains of supporting musl.
> 
> A modern tiny init system would be nice to have, but it's not
> essential or fair to say that musl is useless without one. Many
> projects, especially tiny ones, manage fine with init scripts and
> custom process management.

I was not asking for "modern".

If init scripts are not default and CI tested with musl,
then init scripts will soon become a broken part of OE.

>...
> A few pragmatic patches applied by OE would go a long way to bridging
> the conflicting goals of the two upstream projects. It's basically the
> approach we've taken already - the question is just one of improving
> the patches we already have (and maybe patching musl to add missing
> functionality instead of only trying to patch systemd to not depend on
> it).

I already tried patching musl in OE.
The change got reverted.

There are people who think that OE-specific patches to musl are not 
acceptable, and that it is better to force everyone in OE to patch 
individual packages all the time instead of adding a not upstreamable
patch to musl.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Khem Raj]

On Wed, May 29, 2019 at 9:31 AM Adrian Bunk <bunk@stusta.de> wrote:
>
> On Tue, May 28, 2019 at 04:10:45PM -0700, Andre McCurdy wrote:
> > On Sat, May 25, 2019 at 12:25 AM Adrian Bunk <bunk@stusta.de> wrote:
> >...
> > > Supporting musl is a real pain across the board,
> > > with new issues all the time.
> >
> > There are always new issues and bugs to be solved in OE as a
> > consequence of trying to keep all packages up to date. Whether the
> > issues arising from musl are a real pain or a fun new set of problems
> > to solve is mostly a matter of perspective.
>
> Usually someone submits a change, and later gets notified that the
> change was dropped from master-next due to a musl issue.
>
> That's not fun.

I have re-iterated many times that if your patch regresses musl and
you dont want to fix it
when I provide feedback on the patch, say it there. master-next is
merely an integration branch
it does not mean that if a patch shows up on master-next then its any
good or bad for that matter
its for CI purpose only. master-next queue gets re-aligned all the
time. My point to drop the patch
is that submitter would be interested in fixing it which most of the
times submitters are kind enough
but I will respect reluctance to fix musl of for that matter larger OE
interests that a developer might
not care. Many developers have chipped in to fix regressions in past
and I am hopeful they will
chime in future too.

OE is not a distro, its an infrastructure to build custom distros and
thats its USP, if we were to stick with
one distro then there are plenty out there which are far well supported.

>
> And all these compile errors with musl due to header order are a real WTF,
> every other library (not limited to C libraries) is now doing headers
> properly so that any order works. No fun in supporting a broken design.
>

I think these are concerns you must raise in musl community, OE is
downstream where we can experiment but not fix these problems, thy
should
be fixed in repsective upstreams as much as possible
upsttream might have answers or might benefit from this feedback

> > > For really tiny systems you need both a tiny C library and a tiny init> system, so not properly supporting the combination of both forces users
> > > to use alternative options instead of OE.
> > >
> > > Which minimizes the benefits gained by the pains of supporting musl.
> >
> > A modern tiny init system would be nice to have, but it's not
> > essential or fair to say that musl is useless without one. Many
> > projects, especially tiny ones, manage fine with init scripts and
> > custom process management.
>
> I was not asking for "modern".
>
> If init scripts are not default and CI tested with musl,
> then init scripts will soon become a broken part of OE.

Infact, there are CI system not exposed to community where its testing
sysvinit with musl

>
> >...
> > A few pragmatic patches applied by OE would go a long way to bridging
> > the conflicting goals of the two upstream projects. It's basically the
> > approach we've taken already - the question is just one of improving
> > the patches we already have (and maybe patching musl to add missing
> > functionality instead of only trying to patch systemd to not depend on
> > it).
>
> I already tried patching musl in OE.
> The change got reverted.
>

Its costly to change fundamental APIs in libc which are not accepted
upstream, especially in libc which will
go into SDKs and will become default API set solely provided by
OE thats a huge cost in time.  I suggested
you to submit the patch upstream musl, I still encourage you to do so.

> There are people who think that OE-specific patches to musl are not
> acceptable, and that it is better to force everyone in OE to patch
> individual packages all the time instead of adding a not upstreamable
> patch to musl.
>

I think you understanding is a bit different here, more than one
package using a specific non-standard API
does not legitimize it to become part of C library, Ask glibc
developer community they would tell you how much
libraries and functions they want to drop but can not.

> cu
> Adrian
>
> --
>
>        "Is there not promise of rain?" Ling Tan asked suddenly out
>         of the darkness. There had been need of rain for many days.
>        "Only a promise," Lao Er said.
>                                        Pearl S. Buck - Dragon Seed
>

Re: Should systemd be marked as incompatible with musl?

[Adrian Bunk]

On Wed, May 29, 2019 at 11:01:51AM +0200, Khem Raj wrote:
> On Wed, May 29, 2019 at 9:31 AM Adrian Bunk <bunk@stusta.de> wrote:
>...
> > And all these compile errors with musl due to header order are a real WTF,
> > every other library (not limited to C libraries) is now doing headers
> > properly so that any order works. No fun in supporting a broken design.
> 
> I think these are concerns you must raise in musl community, OE is
> downstream where we can experiment but not fix these problems, thy
> should
> be fixed in repsective upstreams as much as possible
> upsttream might have answers or might benefit from this feedback

This doesn't work when the root cause is an intentional design decision
by upstream, and everyone bringing up such a topic again will just be
considered annoying.

Just like asking musl upstream for a __MUSL__ define would not be 
successful, but would be required e.g. for making the musl patch
in webkitgtk upstreamable.

>...
> > > A few pragmatic patches applied by OE would go a long way to bridging
> > > the conflicting goals of the two upstream projects. It's basically the
> > > approach we've taken already - the question is just one of improving
> > > the patches we already have (and maybe patching musl to add missing
> > > functionality instead of only trying to patch systemd to not depend on
> > > it).
> >
> > I already tried patching musl in OE.
> > The change got reverted.
> 
> Its costly to change fundamental APIs in libc which are not accepted
> upstream, especially in libc which will
> go into SDKs and will become default API set solely provided by
> OE thats a huge cost in time.

A macro in a header does not change the ABI or fundamental APIs.

> I suggested
> you to submit the patch upstream musl, I still encourage you to do so.
>...

This was a dead end - musl upstream thinks that software shouldn't  
be doing "loop on EINTR".

At that point the only realistic options are to either patch 
TEMP_FAILURE_RETRY once into musl, or to patch it into all
current and future users.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Should systemd be marked as incompatible with musl?

[Andre McCurdy]

On Wed, May 29, 2019 at 12:31 AM Adrian Bunk <bunk@stusta.de> wrote:
> On Tue, May 28, 2019 at 04:10:45PM -0700, Andre McCurdy wrote:
> > On Sat, May 25, 2019 at 12:25 AM Adrian Bunk <bunk@stusta.de> wrote:
> >...
> > > Supporting musl is a real pain across the board,
> > > with new issues all the time.
> >
> > There are always new issues and bugs to be solved in OE as a
> > consequence of trying to keep all packages up to date. Whether the
> > issues arising from musl are a real pain or a fun new set of problems
> > to solve is mostly a matter of perspective.
>
> Usually someone submits a change, and later gets notified that the
> change was dropped from master-next due to a musl issue.
>
> That's not fun.

Maybe that happens sometimes but I'm not sure that's the "usual" case.
Patches to OE get dropped or ignored for many reasons. If your patch
gets as far as master-next and then gets dropped with some feedback
it's already done better than many others :-) If you're going to
contribute to OE you need to listen to the feedback... and be a little
persistent!

> And all these compile errors with musl due to header order are a real WTF,
> every other library (not limited to C libraries) is now doing headers
> properly so that any order works. No fun in supporting a broken design.

The musl header ordering issues (the potential for duplication of
definitions between low level kernel headers and libc headers) is
specific to C libraries. It's actually a perfect example of a problem
created by a hard line stance from upstream musl which could be
completely solved by a small pragmatic patch applied by OE.

> > > For really tiny systems you need both a tiny C library and a tiny init> system, so not properly supporting the combination of both forces users
> > > to use alternative options instead of OE.
> > >
> > > Which minimizes the benefits gained by the pains of supporting musl.
> >
> > A modern tiny init system would be nice to have, but it's not
> > essential or fair to say that musl is useless without one. Many
> > projects, especially tiny ones, manage fine with init scripts and
> > custom process management.
>
> I was not asking for "modern".
>
> If init scripts are not default and CI tested with musl,
> then init scripts will soon become a broken part of OE.

By that definition (ie anything which is not CI tested is broken) then
most of OE is broken. CI testing covers a very narrow set of
configurations and if you configure for a real world project you're
likely to stray away from them. A few releases back, openssh didn't
work for Thumb2 targets because of a conflict between binutils and
openssl but wasn't caught by CI testing because (at the time) CI
testing was ARMv5 only. In OE 2.7, sqlite3 is broken for big-endian
ARM targets (which aren't part of the CI testing). There are many
other examples. If you plan to use OE in a real product you'd better
plan to do your own CI testing and bug fixing. For many (most?) users
that's perfectly acceptable - it's far better to have functionality in
OE which is basically sound but may need some tweaks than to strip out
everything which hasn't been CI tested.

> >...
> > A few pragmatic patches applied by OE would go a long way to bridging
> > the conflicting goals of the two upstream projects. It's basically the
> > approach we've taken already - the question is just one of improving
> > the patches we already have (and maybe patching musl to add missing
> > functionality instead of only trying to patch systemd to not depend on
> > it).
>
> I already tried patching musl in OE.
> The change got reverted.
>
> There are people who think that OE-specific patches to musl are not
> acceptable, and that it is better to force everyone in OE to patch
> individual packages all the time instead of adding a not upstreamable
> patch to musl.

Yes, I saw that. The rules on what is and what is not acceptable in OE
are not particularly well defined (and then not consistently applied).
It's just the way it is...