[Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

Bugfixes and updates.

https://github.com/containerd/containerd/releases/tag/v1.6.22

Signed-off-by: Christian Stewart <christian@aperture.us>
---
 package/containerd/containerd.hash | 2 +-
 package/containerd/containerd.mk   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/package/containerd/containerd.hash b/package/containerd/containerd.hash
index 345a7b16ff..8404675e30 100644
--- a/package/containerd/containerd.hash
+++ b/package/containerd/containerd.hash
@@ -1,3 +1,3 @@
 # Computed locally
-sha256  9452e95455d03a00d78ae0587595d0c18555bae7912068269efa25a724efe713  containerd-1.6.21.tar.gz
+sha256  b109aceacc814d7a637ed94ba5ade829cd2642841d03e06971ef124fa3b86899  containerd-1.6.22.tar.gz
 sha256  4bbe3b885e8cd1907ab4cf9a41e862e74e24b5422297a4f2fe524e6a30ada2b4  LICENSE
diff --git a/package/containerd/containerd.mk b/package/containerd/containerd.mk
index 83489a3abf..c4a6d854c3 100644
--- a/package/containerd/containerd.mk
+++ b/package/containerd/containerd.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-CONTAINERD_VERSION = 1.6.21
+CONTAINERD_VERSION = 1.6.22
 CONTAINERD_SITE = $(call github,containerd,containerd,v$(CONTAINERD_VERSION))
 CONTAINERD_LICENSE = Apache-2.0
 CONTAINERD_LICENSE_FILES = LICENSE
-- 
2.41.0

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Thomas Petazzoni via buildroot]

On Thu, 27 Jul 2023 16:22:35 -0700
Christian Stewart via buildroot <buildroot@buildroot.org> wrote:

> Bugfixes and updates.
> 
> https://github.com/containerd/containerd/releases/tag/v1.6.22
> 
> Signed-off-by: Christian Stewart <christian@aperture.us>
> ---
>  package/containerd/containerd.hash | 2 +-
>  package/containerd/containerd.mk   | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)

Applied to master, thanks.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni via buildroot <buildroot@buildroot.org> writes:

 > On Thu, 27 Jul 2023 16:22:35 -0700
 > Christian Stewart via buildroot <buildroot@buildroot.org> wrote:

 >> Bugfixes and updates.
 >> 
 >> https://github.com/containerd/containerd/releases/tag/v1.6.22
 >> 
 >> Signed-off-by: Christian Stewart <christian@aperture.us>
 >> ---
 >> package/containerd/containerd.hash | 2 +-
 >> package/containerd/containerd.mk   | 2 +-
 >> 2 files changed, 2 insertions(+), 2 deletions(-)

 > Applied to master, thanks.

Committed to 2023.02.x and 2023.05.x, thanks.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

Hi Christian,

It seems that this version hangs at startup, at least when built with
Buildroot 2023.02.4.

Typically my systemd services just timeout. When I manually start
containerd it just doesn't reach the message "containerd successfully
booted in", which older version did.

I've tested v1.6.23 as well, it shows the same behavior. For now I
reverted to v1.6.21.

Have you seen this behavior?

Btw, on master, where we use Docker 24.0.x, shouldn't containerd be
upgraded to the 1.7.x series?

--
Stefan


On 2023-07-28 01:22, Christian Stewart via buildroot wrote:
> Bugfixes and updates.
> 
> https://github.com/containerd/containerd/releases/tag/v1.6.22
> 
> Signed-off-by: Christian Stewart <christian@aperture.us>
> ---
>  package/containerd/containerd.hash | 2 +-
>  package/containerd/containerd.mk   | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/package/containerd/containerd.hash
> b/package/containerd/containerd.hash
> index 345a7b16ff..8404675e30 100644
> --- a/package/containerd/containerd.hash
> +++ b/package/containerd/containerd.hash
> @@ -1,3 +1,3 @@
>  # Computed locally
> -sha256 
> 9452e95455d03a00d78ae0587595d0c18555bae7912068269efa25a724efe713 
> containerd-1.6.21.tar.gz
> +sha256 
> b109aceacc814d7a637ed94ba5ade829cd2642841d03e06971ef124fa3b86899 
> containerd-1.6.22.tar.gz
>  sha256 
> 4bbe3b885e8cd1907ab4cf9a41e862e74e24b5422297a4f2fe524e6a30ada2b4 
> LICENSE
> diff --git a/package/containerd/containerd.mk b/package/containerd/containerd.mk
> index 83489a3abf..c4a6d854c3 100644
> --- a/package/containerd/containerd.mk
> +++ b/package/containerd/containerd.mk
> @@ -4,7 +4,7 @@
>  #
>
> ################################################################################
>  
> -CONTAINERD_VERSION = 1.6.21
> +CONTAINERD_VERSION = 1.6.22
>  CONTAINERD_SITE = $(call github,containerd,containerd,v$(CONTAINERD_VERSION))
>  CONTAINERD_LICENSE = Apache-2.0
>  CONTAINERD_LICENSE_FILES = LICENSE
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

Hi Stefan,

On Thu, Sep 7, 2023 at 4:23 PM Stefan Agner <stefan@agner.ch> wrote:
>
> Hi Christian,
>
> It seems that this version hangs at startup, at least when built with
> Buildroot 2023.02.4.

I am running 1.7.x on my systems so I can't confirm this currently.

> Btw, on master, where we use Docker 24.0.x, shouldn't containerd be
> upgraded to the 1.7.x series?

According to this 1.6.x is LTS and still getting updates:

https://containerd.io/releases/

According to these release notes, 1.6.x "has also become a long term
stable release for users who prefer releases with mostly stability
improvements and wish to wait a few releases for new functionality."

See: https://github.com/containerd/containerd/releases/tag/v1.7.0

Since Docker is supposed to be agnostic to different containerd
versions it made sense to use the LTS.

However, I am not against updating it to 1.7.x as I'm currently using
& testing that branch anyway.

Best,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

On 8 September 2023 01:28:36 CEST, Christian Stewart <christian@aperture.us> wrote:
>Hi Stefan,
>
>On Thu, Sep 7, 2023 at 4:23 PM Stefan Agner <stefan@agner.ch> wrote:
>>
>> Hi Christian,
>>
>> It seems that this version hangs at startup, at least when built with
>> Buildroot 2023.02.4.
>
>I am running 1.7.x on my systems so I can't confirm this currently.

I've just tested 1.7.3 as well, it shows the same behavior here. I'll need to do more testing. Ideally I like to see a stack trace (via signal or debugger), but not sure how to archive this with Buildroot and golang.

This is on aarch64 platforms.

>
>> Btw, on master, where we use Docker 24.0.x, shouldn't containerd be
>> upgraded to the 1.7.x series?
>
>According to this 1.6.x is LTS and still getting updates:
>
>https://containerd.io/releases/
>
>According to these release notes, 1.6.x "has also become a long term
>stable release for users who prefer releases with mostly stability
>improvements and wish to wait a few releases for new functionality."
>
>See: https://github.com/containerd/containerd/releases/tag/v1.7.0
>
>Since Docker is supposed to be agnostic to different containerd
>versions it made sense to use the LTS.
>
>However, I am not against updating it to 1.7.x as I'm currently using
>& testing that branch anyway.

Ok I see. Either way works for me.

Best regards,
Stefan

>
>Best,
>Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 1003 bytes --]

Hi Stefan,

On Thu, Sep 7, 2023, 4:51 PM Stefan Agner <stefan@agner.ch> wrote:

>
>
> On 8 September 2023 01:28:36 CEST, Christian Stewart <
> christian@aperture.us> wrote:
> >Hi Stefan,
> >
> >On Thu, Sep 7, 2023 at 4:23 PM Stefan Agner <stefan@agner.ch> wrote:
> >>
> >> Hi Christian,
> >>
> >> It seems that this version hangs at startup, at least when built with
> >> Buildroot 2023.02.4.
> >
> >I am running 1.7.x on my systems so I can't confirm this currently.
>
> I've just tested 1.7.3 as well, it shows the same behavior here. I'll need
> to do more testing. Ideally I like to see a stack trace (via signal or
> debugger), but not sure how to archive this with Buildroot and golang.
>
> This is on aarch64 platforms.
>

I hate to say "works on my machine," but with both versions on aarch64 it
works for me.

Maybe the filesystem you are using is different? I'm using ext4. Or is the
kernel config not set up quite right? Maybe old kernel?

Best,
Christian

[-- Attachment #1.2: Type: text/html, Size: 1677 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

On 2023-09-08 04:03, Christian Stewart wrote:

> Hi Stefan,
> 
> On Thu, Sep 7, 2023, 4:51 PM Stefan Agner <stefan@agner.ch> wrote: 
> 
>> On 8 September 2023 01:28:36 CEST, Christian Stewart <christian@aperture.us> wrote:
>>> Hi Stefan,
>>> 
>>> On Thu, Sep 7, 2023 at 4:23 PM Stefan Agner <stefan@agner.ch> wrote:
>>>> 
>>>> Hi Christian,
>>>> 
>>>> It seems that this version hangs at startup, at least when built with
>>>> Buildroot 2023.02.4.
>>> 
>>> I am running 1.7.x on my systems so I can't confirm this currently.
>> 
>> I've just tested 1.7.3 as well, it shows the same behavior here. I'll need to do more testing. Ideally I like to see a stack trace (via signal or debugger), but not sure how to archive this with Buildroot and golang.
>> 
>> This is on aarch64 platforms.
> 
> I hate to say "works on my machine," but with both versions on aarch64 it works for me.  
> 
> Maybe the filesystem you are using is different? I'm using ext4. Or is the kernel config not set up quite right? Maybe old kernel?  

The root file system is a squashfs, but the data is stored on ext4.

We are using the latest vanilla Linux kernel 6.1.52.

It seems to hang here:
WARN[2023-09-08T06:17:38.515218691Z] waiting for response from boltdb
open         plugin=bolt

From what I can tell it's some IPC mechanism or similar which is
hanging.

I can also reproduce it on x86-64. I guess it must be unique to our
operating system configuration in some way...

--
Stefan

> 
> Best, 
> Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

Hi Stefan,

Did you find a solution to this? Just following up.

On Thu, Sep 7, 2023 at 11:21 PM Stefan Agner <stefan@agner.ch> wrote:
> > I hate to say "works on my machine," but with both versions on aarch64 it works for me.
> >
> > Maybe the filesystem you are using is different? I'm using ext4. Or is the kernel config not set up quite right? Maybe old kernel?
>
> The root file system is a squashfs, but the data is stored on ext4.
>
> We are using the latest vanilla Linux kernel 6.1.52.
>
> It seems to hang here:
> WARN[2023-09-08T06:17:38.515218691Z] waiting for response from boltdb
> open         plugin=bolt
>
> From what I can tell it's some IPC mechanism or similar which is
> hanging.
>
> I can also reproduce it on x86-64. I guess it must be unique to our
> operating system configuration in some way...

Best,
Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

Hi Christian,

Finally came around looking into this a bit more closely. It seems that
containerd just didn't send the sd notification READY=1.

It seems that the problem is caused by the CRI plug-in. The plug-in
reports the following error:
time="2023-09-24T21:43:20.958509027Z" level=warning msg="failed to load
plugin io.containerd.grpc.v1.cri" error="failed to create CRI service:
failed to create cni conf monitor for default: failed to create the
parent of the cni conf dir=/etc/cni: mkdir /etc/cni: read-only

We are running containerd on a read-only squashfs. I guess this explains
also why you haven't seen this issue.

After disabling the plug-in via /etc/containerd/config.toml
disabled_plugins = [ "cri" ], the issue disappeared (containerd service
starts up correctly and prints the message "containerd successfully
booted in ...s"

Checking the changelog of v1.6.22 brings up this change:
https://github.com/containerd/containerd/pull/8826

The error above was already present before, so I didn't give it much
attention. But I guess the change to use atomicfile really changed the
behavior and now seems to cause the hang on startup.

In any case, from my point of view, nothing which needs to be addressed
in Buildroot.

Best regards,
Stefan


On 2023-09-15 23:48, Christian Stewart wrote:
> Hi Stefan,
> 
> Did you find a solution to this? Just following up.
> 
> On Thu, Sep 7, 2023 at 11:21 PM Stefan Agner <stefan@agner.ch> wrote:
>> > I hate to say "works on my machine," but with both versions on aarch64 it works for me.
>> >
>> > Maybe the filesystem you are using is different? I'm using ext4. Or is the kernel config not set up quite right? Maybe old kernel?
>>
>> The root file system is a squashfs, but the data is stored on ext4.
>>
>> We are using the latest vanilla Linux kernel 6.1.52.
>>
>> It seems to hang here:
>> WARN[2023-09-08T06:17:38.515218691Z] waiting for response from boltdb
>> open         plugin=bolt
>>
>> From what I can tell it's some IPC mechanism or similar which is
>> hanging.
>>
>> I can also reproduce it on x86-64. I guess it must be unique to our
>> operating system configuration in some way...
> 
> Best,
> Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Peter Korsgaard]

>>>>> "Stefan" == Stefan Agner <stefan@agner.ch> writes:

 > Hi Christian,
 > Finally came around looking into this a bit more closely. It seems that
 > containerd just didn't send the sd notification READY=1.

 > It seems that the problem is caused by the CRI plug-in. The plug-in
 > reports the following error:
 > time="2023-09-24T21:43:20.958509027Z" level=warning msg="failed to load
 > plugin io.containerd.grpc.v1.cri" error="failed to create CRI service:
 > failed to create cni conf monitor for default: failed to create the
 > parent of the cni conf dir=/etc/cni: mkdir /etc/cni: read-only

 > We are running containerd on a read-only squashfs. I guess this explains
 > also why you haven't seen this issue.

 > After disabling the plug-in via /etc/containerd/config.toml
 > disabled_plugins = [ "cri" ], the issue disappeared (containerd service
 > starts up correctly and prints the message "containerd successfully
 > booted in ...s"

 > Checking the changelog of v1.6.22 brings up this change:
 > https://github.com/containerd/containerd/pull/8826

 > The error above was already present before, so I didn't give it much
 > attention. But I guess the change to use atomicfile really changed the
 > behavior and now seems to cause the hang on startup.

 > In any case, from my point of view, nothing which needs to be addressed
 > in Buildroot.

If cri is only needed for when containerd is used with kubernetes, then
perhaps we should disable it by default so other people don't need to
debug the same issue.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

Hi Peter,

https://github.com/containerd/containerd/issues/9139#issuecomment-1736477534

On Sat, Sep 30, 2023 at 1:03 AM Peter Korsgaard <peter@korsgaard.com> wrote:
> If cri is only needed for when containerd is used with kubernetes, then
> perhaps we should disable it by default so other people don't need to
> debug the same issue.

I think we can safely disable cri on the default config.

Best,
Christian
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

On 2023-10-02 20:28, Christian Stewart wrote:
> Hi Peter,
> 
> https://github.com/containerd/containerd/issues/9139#issuecomment-1736477534
> 
> On Sat, Sep 30, 2023 at 1:03 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>> If cri is only needed for when containerd is used with kubernetes, then
>> perhaps we should disable it by default so other people don't need to
>> debug the same issue.
> 
> I think we can safely disable cri on the default config.

At that point we can also pass the no_cri build tag.

I'll create a PR to do just that.

--
Stefan
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 820 bytes --]

Stefan,

On Wed, Oct 4, 2023, 2:59 AM Stefan Agner <stefan@agner.ch> wrote:

> On 2023-10-02 20:28, Christian Stewart wrote:
> > Hi Peter,
> >
> >
> https://github.com/containerd/containerd/issues/9139#issuecomment-1736477534
> >
> > On Sat, Sep 30, 2023 at 1:03 AM Peter Korsgaard <peter@korsgaard.com>
> wrote:
> >> If cri is only needed for when containerd is used with kubernetes, then
> >> perhaps we should disable it by default so other people don't need to
> >> debug the same issue.
> >
> > I think we can safely disable cri on the default config.
>
> At that point we can also pass the no_cri build tag.
>
> I'll create a PR to do just that.
>

Maybe make it a adjustable config option, since I definitely want to use
kubernetes + cri w/ containerd in buildroot.

Best,
Christian

[-- Attachment #1.2: Type: text/html, Size: 1481 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v1 1/1] package/containerd: bump version to v1.6.22

[Stefan Agner]

On 2023-10-04 12:04, Christian Stewart wrote:

> Stefan,
> 
> On Wed, Oct 4, 2023, 2:59 AM Stefan Agner <stefan@agner.ch> wrote: 
> 
>> On 2023-10-02 20:28, Christian Stewart wrote:
>>> Hi Peter,
>>> 
>>> https://github.com/containerd/containerd/issues/9139#issuecomment-1736477534
>>> 
>>> On Sat, Sep 30, 2023 at 1:03 AM Peter Korsgaard <peter@korsgaard.com> wrote:
>>>> If cri is only needed for when containerd is used with kubernetes, then
>>>> perhaps we should disable it by default so other people don't need to
>>>> debug the same issue.
>>> 
>>> I think we can safely disable cri on the default config.
>> 
>> At that point we can also pass the no_cri build tag.
>> 
>> I'll create a PR to do just that.
> 
> Maybe make it a adjustable config option, since I definitely want to use kubernetes + cri w/ containerd in buildroot. 

Ok, I'll make a config for that.

That said, I would not disable it via config in that case as I would
expect someone who opts to build CRI will make sure that the
prerequisites (such as writable /etc as it seems) are taken care of.

--
Stefan
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot