All of lore.kernel.org
 help / color / mirror / Atom feed
From: Naveen N Rao <naveen@kernel.org>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	matoro <matoro_mailinglist_kernel@matoro.tk>
Cc: bpf@vger.kernel.org, Hari Bathini <hbathini@linux.ibm.com>,
	linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	ltp@lists.linux.it, stable@vger.kernel.org,
	Vitaly Chikunov <vt@altlinux.org>
Subject: Re: WARNING&Oops in v6.6.37 on ppc64lea - Trying to vfree() bad address (00000000453be747)
Date: Tue, 09 Jul 2024 15:02:13 +0530	[thread overview]
Message-ID: <1720516964.n61e0dnv80.naveen@kernel.org> (raw)
In-Reply-To: <2024070904-cod-bobcat-a0d0@gregkh>

Greg Kroah-Hartman wrote:
> On Mon, Jul 08, 2024 at 11:16:48PM -0400, matoro wrote:
>> On 2024-07-05 16:34, Vitaly Chikunov wrote:
>> > Hi,
>> > 
>> > There is new WARNING and Oops on ppc64le in v6.6.37 when running LTP tests:
>> > bpf_prog01, bpf_prog02, bpf_prog04, bpf_prog05, prctl04. Logs excerpt
>> > below. I
>> > see there is 1 commit in v6.6.36..v6.6.37 with call to
>> > bpf_jit_binary_pack_finalize, backported from 5 patch mainline patchset:
>> > 
>> >   f99feda5684a powerpc/bpf: use bpf_jit_binary_pack_[alloc|finalize|free]
>> > 

<snip>

>> > 
>> > And so on. Temporary build/test log is at
>> > https://git.altlinux.org/tasks/352218/build/100/ppc64le/log
>> > 
>> > Other stable/longterm branches or other architectures does not exhibit this.
>> > 
>> > Thanks,
>> 
>> Hi all - this just took down a production server for me, on POWER9 bare
>> metal.  Not running tests, just booting normally, before services even came
>> up.  Had to perform manual restoration, reverting to 6.6.36 worked.  Also
>> running 64k kernel, unsure if it's better on 4k kernel.
>> 
>> In case it's helpful, here's the log from my boot:
>> https://dpaste.org/Gyxxg/raw
> 
> Ok, this isn't good, something went wrong with my backports here.  Let
> me go revert them all and push out a new 6.6.y release right away.

I think the problem is that the series adding support for bpf prog_pack 
was partially backported. In particular, the below patches are missing 
from stable v6.6:
465cabc97b42 powerpc/code-patching: introduce patch_instructions()
033ffaf0af1f powerpc/bpf: implement bpf_arch_text_invalidate for bpf_prog_pack
6efc1675acb8 powerpc/bpf: implement bpf_arch_text_copy

It should be sufficient to revert commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) to allow the above to apply 
cleanly, followed by cherry picking commit 90d862f370b6 (powerpc/bpf: 
use bpf_jit_binary_pack_[alloc|finalize|free]) from upstream.

Alternately, commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) can be reverted.


- Naveen


WARNING: multiple messages have this Message-ID (diff)
From: Naveen N Rao <naveen@kernel.org>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	matoro <matoro_mailinglist_kernel@matoro.tk>
Cc: stable@vger.kernel.org, Hari Bathini <hbathini@linux.ibm.com>,
	bpf@vger.kernel.org, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	ltp@lists.linux.it
Subject: Re: [LTP] WARNING&Oops in v6.6.37 on ppc64lea - Trying to vfree() bad address (00000000453be747)
Date: Tue, 09 Jul 2024 15:02:13 +0530	[thread overview]
Message-ID: <1720516964.n61e0dnv80.naveen@kernel.org> (raw)
In-Reply-To: <2024070904-cod-bobcat-a0d0@gregkh>

Greg Kroah-Hartman wrote:
> On Mon, Jul 08, 2024 at 11:16:48PM -0400, matoro wrote:
>> On 2024-07-05 16:34, Vitaly Chikunov wrote:
>> > Hi,
>> > 
>> > There is new WARNING and Oops on ppc64le in v6.6.37 when running LTP tests:
>> > bpf_prog01, bpf_prog02, bpf_prog04, bpf_prog05, prctl04. Logs excerpt
>> > below. I
>> > see there is 1 commit in v6.6.36..v6.6.37 with call to
>> > bpf_jit_binary_pack_finalize, backported from 5 patch mainline patchset:
>> > 
>> >   f99feda5684a powerpc/bpf: use bpf_jit_binary_pack_[alloc|finalize|free]
>> > 

<snip>

>> > 
>> > And so on. Temporary build/test log is at
>> > https://git.altlinux.org/tasks/352218/build/100/ppc64le/log
>> > 
>> > Other stable/longterm branches or other architectures does not exhibit this.
>> > 
>> > Thanks,
>> 
>> Hi all - this just took down a production server for me, on POWER9 bare
>> metal.  Not running tests, just booting normally, before services even came
>> up.  Had to perform manual restoration, reverting to 6.6.36 worked.  Also
>> running 64k kernel, unsure if it's better on 4k kernel.
>> 
>> In case it's helpful, here's the log from my boot:
>> https://dpaste.org/Gyxxg/raw
> 
> Ok, this isn't good, something went wrong with my backports here.  Let
> me go revert them all and push out a new 6.6.y release right away.

I think the problem is that the series adding support for bpf prog_pack 
was partially backported. In particular, the below patches are missing 
from stable v6.6:
465cabc97b42 powerpc/code-patching: introduce patch_instructions()
033ffaf0af1f powerpc/bpf: implement bpf_arch_text_invalidate for bpf_prog_pack
6efc1675acb8 powerpc/bpf: implement bpf_arch_text_copy

It should be sufficient to revert commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) to allow the above to apply 
cleanly, followed by cherry picking commit 90d862f370b6 (powerpc/bpf: 
use bpf_jit_binary_pack_[alloc|finalize|free]) from upstream.

Alternately, commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) can be reverted.


- Naveen


-- 
Mailing list info: https://lists.linux.it/listinfo/ltp

WARNING: multiple messages have this Message-ID (diff)
From: Naveen N Rao <naveen@kernel.org>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	matoro <matoro_mailinglist_kernel@matoro.tk>
Cc: stable@vger.kernel.org, Vitaly Chikunov <vt@altlinux.org>,
	Hari Bathini <hbathini@linux.ibm.com>,
	bpf@vger.kernel.org, linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
	ltp@lists.linux.it
Subject: Re: WARNING&Oops in v6.6.37 on ppc64lea - Trying to vfree() bad address (00000000453be747)
Date: Tue, 09 Jul 2024 15:02:13 +0530	[thread overview]
Message-ID: <1720516964.n61e0dnv80.naveen@kernel.org> (raw)
In-Reply-To: <2024070904-cod-bobcat-a0d0@gregkh>

Greg Kroah-Hartman wrote:
> On Mon, Jul 08, 2024 at 11:16:48PM -0400, matoro wrote:
>> On 2024-07-05 16:34, Vitaly Chikunov wrote:
>> > Hi,
>> > 
>> > There is new WARNING and Oops on ppc64le in v6.6.37 when running LTP tests:
>> > bpf_prog01, bpf_prog02, bpf_prog04, bpf_prog05, prctl04. Logs excerpt
>> > below. I
>> > see there is 1 commit in v6.6.36..v6.6.37 with call to
>> > bpf_jit_binary_pack_finalize, backported from 5 patch mainline patchset:
>> > 
>> >   f99feda5684a powerpc/bpf: use bpf_jit_binary_pack_[alloc|finalize|free]
>> > 

<snip>

>> > 
>> > And so on. Temporary build/test log is at
>> > https://git.altlinux.org/tasks/352218/build/100/ppc64le/log
>> > 
>> > Other stable/longterm branches or other architectures does not exhibit this.
>> > 
>> > Thanks,
>> 
>> Hi all - this just took down a production server for me, on POWER9 bare
>> metal.  Not running tests, just booting normally, before services even came
>> up.  Had to perform manual restoration, reverting to 6.6.36 worked.  Also
>> running 64k kernel, unsure if it's better on 4k kernel.
>> 
>> In case it's helpful, here's the log from my boot:
>> https://dpaste.org/Gyxxg/raw
> 
> Ok, this isn't good, something went wrong with my backports here.  Let
> me go revert them all and push out a new 6.6.y release right away.

I think the problem is that the series adding support for bpf prog_pack 
was partially backported. In particular, the below patches are missing 
from stable v6.6:
465cabc97b42 powerpc/code-patching: introduce patch_instructions()
033ffaf0af1f powerpc/bpf: implement bpf_arch_text_invalidate for bpf_prog_pack
6efc1675acb8 powerpc/bpf: implement bpf_arch_text_copy

It should be sufficient to revert commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) to allow the above to apply 
cleanly, followed by cherry picking commit 90d862f370b6 (powerpc/bpf: 
use bpf_jit_binary_pack_[alloc|finalize|free]) from upstream.

Alternately, commit f99feda5684a (powerpc/bpf: use 
bpf_jit_binary_pack_[alloc|finalize|free]) can be reverted.


- Naveen


  reply	other threads:[~2024-07-09  9:34 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-07-05 20:34 WARNING&Oops in v6.6.37 on ppc64lea - Trying to vfree() bad address (00000000453be747) Vitaly Chikunov
2024-07-05 20:34 ` Vitaly Chikunov
2024-07-05 20:34 ` [LTP] " Vitaly Chikunov
2024-07-09  3:16 ` matoro
2024-07-09  3:16   ` [LTP] " matoro via ltp
2024-07-09  9:16   ` Greg Kroah-Hartman
2024-07-09  9:16     ` Greg Kroah-Hartman
2024-07-09  9:16     ` [LTP] " Greg Kroah-Hartman
2024-07-09  9:32     ` Naveen N Rao [this message]
2024-07-09  9:32       ` Naveen N Rao
2024-07-09  9:32       ` [LTP] " Naveen N Rao
2024-07-09  9:40       ` Greg Kroah-Hartman
2024-07-09  9:40         ` Greg Kroah-Hartman
2024-07-09  9:40         ` [LTP] " Greg Kroah-Hartman
2024-07-09 12:38         ` Michael Ellerman
2024-07-09 12:38           ` Michael Ellerman
2024-07-09 12:38           ` [LTP] " Michael Ellerman
2024-07-09 12:52           ` Michael Ellerman
2024-07-09 12:52             ` Michael Ellerman
2024-07-09 12:52             ` [LTP] " Michael Ellerman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1720516964.n61e0dnv80.naveen@kernel.org \
    --to=naveen@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hbathini@linux.ibm.com \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=ltp@lists.linux.it \
    --cc=matoro_mailinglist_kernel@matoro.tk \
    --cc=stable@vger.kernel.org \
    --cc=vt@altlinux.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.