[systemd] v248.2 and later won't boot 4.19 kernel series

[systemd] v248.2 and later won't boot 4.19 kernel series

[Philip Müller]

[-- Attachment #1.1: Type: text/plain, Size: 884 bytes --]

Hello all,

with commit "mount-util: shortcut things after generating top-level bind
mount" c2c331056a7c331a5478124b3cd6a34c9f539839
(5c5753b9ea5cc012586ae90d357d460dec4301a4 in master) systemd 248.2 and
later won't boot 4.19 kernel series. I tested 4.19.194 so far.

Lennart's  guess is that this is the mount table brokeness on old
kernels, that newer libmount worked around. i.e. /proc/self/mountinfo on
old kernels showed partially old data and partially new data, and
confused the heck out of everyone. "proc on 20" with mount options
"(rw,25)" doesn't look right at all.

If we look at 4.14 and 4.4 kernel series, those boot. 4.9 I've still to
test, as that one didn't boot either for me.

So an idea what is missing in 4.19 kernel series? Other kernels work.

Systemd issue is this one: https://github.com/systemd/systemd/issues/19926

-- 
Best, Philip


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [systemd] v248.2 and later won't boot 4.19 kernel series

[Greg Kroah-Hartman]

On Tue, Jun 15, 2021 at 04:59:27PM +0200, Philip Müller wrote:
> Hello all,
> 
> with commit "mount-util: shortcut things after generating top-level bind
> mount" c2c331056a7c331a5478124b3cd6a34c9f539839
> (5c5753b9ea5cc012586ae90d357d460dec4301a4 in master) systemd 248.2 and
> later won't boot 4.19 kernel series. I tested 4.19.194 so far.
> 
> Lennart's  guess is that this is the mount table brokeness on old
> kernels, that newer libmount worked around. i.e. /proc/self/mountinfo on
> old kernels showed partially old data and partially new data, and
> confused the heck out of everyone. "proc on 20" with mount options
> "(rw,25)" doesn't look right at all.
> 
> If we look at 4.14 and 4.4 kernel series, those boot. 4.9 I've still to
> test, as that one didn't boot either for me.
> 
> So an idea what is missing in 4.19 kernel series? Other kernels work.

Do newer kernels work?

Any chance you can run 'git bisect' to find the offending kernel patch?

4.19 is really old, is this a stable-update-issue, or a "something
changed since 4.19 was released" type of issue?

> Systemd issue is this one: https://github.com/systemd/systemd/issues/19926

I'm running systemd 248.3 just fine here on 5.13-rc6, so this seems
odd...

thanks,

greg k-h

Re: [systemd] v248.2 and later won't boot 4.19 kernel series

[Philip Müller]

[-- Attachment #1.1: Type: text/plain, Size: 479 bytes --]

On 15.06.21 17:39, Greg Kroah-Hartman wrote:
> Do newer kernels work?
> 
> Any chance you can run 'git bisect' to find the offending kernel patch?
> 
> 4.19 is really old, is this a stable-update-issue, or a "something
> changed since 4.19 was released" type of issue?

Older and newer kernel series work. So 4.4, 4.14, 5.10 and 5.13 I've
tested so far. No issues with 248.3 of Systemd. Have to see if any 4.19
kernel boot with that version ...

-- 
Best, Philip


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [systemd] v248.2 and later won't boot 4.19 kernel series

[Greg Kroah-Hartman]

On Tue, Jun 15, 2021 at 06:05:30PM +0200, Philip Müller wrote:
> On 15.06.21 17:39, Greg Kroah-Hartman wrote:
> > Do newer kernels work?
> > 
> > Any chance you can run 'git bisect' to find the offending kernel patch?
> > 
> > 4.19 is really old, is this a stable-update-issue, or a "something
> > changed since 4.19 was released" type of issue?
> 
> Older and newer kernel series work. So 4.4, 4.14, 5.10 and 5.13 I've
> tested so far. No issues with 248.3 of Systemd. Have to see if any 4.19
> kernel boot with that version ...

Please always be specific as to what .y version of the above kernels you
are testing, as there have been thousands of changes from the "first"
4.4.0 release to the release that is supported today (4.4.272).

And note that there is not a 5.13 kernel released yet :)

thanks,

greg k-h

Re: [systemd] v248.2 and later won't boot 4.19 kernel series

[Philip Müller]

[-- Attachment #1.1: Type: text/plain, Size: 577 bytes --]

On 15.06.21 18:14, Greg Kroah-Hartman wrote:
> Please always be specific as to what .y version of the above kernels you
> are testing, as there have been thousands of changes from the "first"

I tested the latest available point releases of each series.

Seems it boils down to an aufs4 patch, which was added by mistake 10
months ago on our end. After removing proc_mount.patch [1] it all worked.

Sorry for the noise.

[1]
https://gitlab.manjaro.org/packages/core/linux419/-/blob/1f7a4bbdfdc5962e80210042f57e1b22d7fa90eb/proc_mounts.patch

-- 
Best, Philip


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]