KVM: PPC: Book3E: KVM HV host module doesn't work anymore

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

Hello,

The KVM HV host module doesn't work anymore since the PowerPC updates 
6.13-1 [1] and PowerPC MM updates [2].

We have tested it with our FSL P5020/P5040 boards [3].

The guest kernel doesn't boot. There is no output via the serial 
connection and no output via the graphics card. (QEMU)

It boots without the PowerPC updates 6.13-1 [1] and PowerPC MM updates [2].

Is the problem already known? Is there already a solution? I have seen 
many Book3S KVM HV updates. Have you perhaps forgotten Book3E?

Thanks,
Christian

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=42d9e8b7ccddee75989283cf7477305cfe3776ff
[2] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d
[3] http://wiki.amiga.org/index.php?title=X5000

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 18/12/24 12:10, Christian Zigotzky wrote:
> Hello,
>
> The KVM HV host module doesn't work anymore since the PowerPC updates 
> 6.13-1 [1] and PowerPC MM updates [2].
>
> We have tested it with our FSL P5020/P5040 boards [3].
>
> The guest kernel doesn't boot. There is no output via the serial 
> connection and no output via the graphics card. (QEMU)
>
> It boots without the PowerPC updates 6.13-1 [1] and PowerPC MM updates 
> [2].
>
> Is the problem already known? Is there already a solution? I have seen 
> many Book3S KVM HV updates. Have you perhaps forgotten Book3E?
>
> Thanks,
> Christian
>
> [1] 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=42d9e8b7ccddee75989283cf7477305cfe3776ff
> [2] 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d
> [3] http://wiki.amiga.org/index.php?title=X5000
>

I tried to bisect but without success. Link: 
https://github.com/chzigotzky/kernels/issues/4#issuecomment-2551460323

-- Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 18/12/24 15:41, Christian Zigotzky wrote:
> On 18/12/24 12:10, Christian Zigotzky wrote:
>> Hello,
>>
>> The KVM HV host module doesn't work anymore since the PowerPC updates 
>> 6.13-1 [1] and PowerPC MM updates [2].
>>
>> We have tested it with our FSL P5020/P5040 boards [3].
>>
>> The guest kernel doesn't boot. There is no output via the serial 
>> connection and no output via the graphics card. (QEMU)
>>
>> It boots without the PowerPC updates 6.13-1 [1] and PowerPC MM 
>> updates [2].
>>
>> Is the problem already known? Is there already a solution? I have 
>> seen many Book3S KVM HV updates. Have you perhaps forgotten Book3E?
>>
>> Thanks,
>> Christian
>>
>> [1] 
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=42d9e8b7ccddee75989283cf7477305cfe3776ff
>> [2] 
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d
>> [3] http://wiki.amiga.org/index.php?title=X5000
>>
>
> I tried to bisect but without success. Link: 
> https://github.com/chzigotzky/kernels/issues/4#issuecomment-2551460323
>
> -- Christian
>
It seems, that the PowerPC updates 6.13-1 aren't responsible for this 
issue. I had to test the PowerPC updates 6.13-1 during the bisect. [1]

Bisecting: 428 revisions left to test after this (roughly 9 steps)
[42d9e8b7ccddee75989283cf7477305cfe3776ff] Merge tag 'powerpc-6.13-1' of 
git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux

Result:  git bisect good

There aren't any changes in the PowerPC MM updates [2].

Do you have another idea? I like KVM HV on my FSL P5040 system and use 
it very often for testing. However, KVM HV is hardly used by our users, 
so we could remove it from our kernels if it were not fixed. It would 
save space to remove it.

-- Christian

[1] https://github.com/chzigotzky/kernels/issues/4
[2] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 20/12/24 13:08 Christian Zigotzky wrote:
> On 18/12/24 15:41, Christian Zigotzky wrote:
>> On 18/12/24 12:10, Christian Zigotzky wrote:
>>> Hello,
>>>
>>> The KVM HV host module doesn't work anymore since the PowerPC 
>>> updates 6.13-1 [1] and PowerPC MM updates [2].
>>>
>>> We have tested it with our FSL P5020/P5040 boards [3].
>>>
>>> The guest kernel doesn't boot. There is no output via the serial 
>>> connection and no output via the graphics card. (QEMU)
>>>
>>> It boots without the PowerPC updates 6.13-1 [1] and PowerPC MM 
>>> updates [2].
>>>
>>> Is the problem already known? Is there already a solution? I have 
>>> seen many Book3S KVM HV updates. Have you perhaps forgotten Book3E?
>>>
>>> Thanks,
>>> Christian
>>>
>>> [1] 
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=42d9e8b7ccddee75989283cf7477305cfe3776ff
>>> [2] 
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d
>>> [3] http://wiki.amiga.org/index.php?title=X5000
>>>
>>
>> I tried to bisect but without success. Link: 
>> https://github.com/chzigotzky/kernels/issues/4#issuecomment-2551460323
>>
>> -- Christian
>>
> It seems, that the PowerPC updates 6.13-1 aren't responsible for this 
> issue. I had to test the PowerPC updates 6.13-1 during the bisect. [1]
>
> Bisecting: 428 revisions left to test after this (roughly 9 steps)
> [42d9e8b7ccddee75989283cf7477305cfe3776ff] Merge tag 'powerpc-6.13-1' 
> of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
>
> Result:  git bisect good
>
> There aren't any changes in the PowerPC MM updates [2].
>
> Do you have another idea? I like KVM HV on my FSL P5040 system and use 
> it very often for testing. However, KVM HV is hardly used by our 
> users, so we could remove it from our kernels if it were not fixed. It 
> would save space to remove it.
>
> -- Christian
>
> [1] https://github.com/chzigotzky/kernels/issues/4
> [2] 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=5c00ff742bf5caf85f60e1c73999f99376fb865d
Maybe the kvm updates [1] from Paolo Bonzini are responsible for this issue.

+ Paolo Bonzini

I have found some changes in the e500_mmu_host.c file 
(a/arch/powerpc/kvm/e500_mmu_host.c) in the kvm updates [1].

-- Christian

[1] 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9f16d5e6f220661f73b36a4be1b21575651d8833

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

On 12/20/24 13:48, Christian Zigotzky wrote:
> Maybe the kvm updates [1] from Paolo Bonzini are responsible for this 
> issue.
> 
> + Paolo Bonzini
> 
> I have found some changes in the e500_mmu_host.c file (a/arch/powerpc/ 
> kvm/e500_mmu_host.c) in the kvm updates [1].
> 
> -- Christian
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ 
> commit/?id=9f16d5e6f220661f73b36a4be1b21575651d8833
> 

That's possible, there were very large treewide updates.  Can you bisect 
my merge?

Paolo

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 22/12/24 09:55, Paolo Bonzini wrote:
> On 20/12/24 13:48, Christian Zigotzky wrote:
>> Maybe the kvm updates [1] from Paolo Bonzini are responsible for this 
>> issue.
>>
>> + Paolo Bonzini
>>
>> I have found some changes in the e500_mmu_host.c file 
>> (a/arch/powerpc/ kvm/e500_mmu_host.c) in the kvm updates [1].
>>
>> -- Christian
>>
>> [1] 
>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ 
>> commit/?id=9f16d5e6f220661f73b36a4be1b21575651d8833
>>
>
> That's possible, there were very large treewide updates.  Can you 
> bisect my merge?
>
> Paolo
>

Hello Paolo,

I tried it but there is a host boot issue at the same time. Bisect log: 
https://github.com/chzigotzky/kernels/issues/4

Regards,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

On Sun, Dec 22, 2024 at 5:24 PM Christian Zigotzky
<chzigotzky@xenosoft.de> wrote:
> Hello Paolo,
>
> I tried it but there is a host boot issue at the same time. Bisect log:
> https://github.com/chzigotzky/kernels/issues/4

Thanks, I left there some instructions which I'll copy here for reference.

We need to bisect the host boot issue:

  git bisect start
  git checkout c9be85dabb376299504e0d391d15662c0edf8273
  # check that it does not boot
  git bisect bad
  git checkout adc218676eef25575469234709c2d87185ca223a
  # check that it boots
  git bisect good

If the two commits have the expected result, the first bisection step should be

  # Bisecting: a merge base must be tested
  # [e9001a382fa2c25...] Merge tag 'kvmarm-fixes-6.12-3' of ...

There are two possibilities.

## If e9001a382fa2c25 succeeds to boot ##

Then `git bisect good` and bisect to the patch that causes the host to
fail to boot. What's left seems to be all non-PPC KVM patches, which
is a bit weird but you never know. Once you have the result, we'll try
to fix that, and provide you with a fixed branch to bisect the actual
KVM-PPC failure.

A potential candidate, based on your skips, is commit eec1e5db464e
("KVM: Annotate that all paths in hva_to_pfn() might sleep").

Please also provide the messages for the boot failure (I haven't seen
it but I might be missing it).

## If e9001a382fa2c25 fails to boot ##

Then the boot regression was in the v6.12 tree. Fortunately the KVM
tree can be rebased without conflicts to v6.12; go on with

```
git checkout -b bisection-rebase b467ab82a9fde4b46c0cd2c299220857afb0e0d4
git rebase v6.12
# check that it boots. if it fails to boot, please report this. if it
succeds, continue:
git bisect start
git bisect bad bisection-rebase
git bisect good v6.12
```

Proceed to bisect looking for the failure to start guests. If you
never find a kernel that fails to boot, report the bisection log.

If you find a kernel that fails to boot, drop the bisection. Start
another bisection between v6.12 and bisection-rebase, this time
looking for the first patch that fails to boot, and report the
bisection log + the boot failure messages.

Paolo

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 22/12/24 18:49, Paolo Bonzini wrote:
 >
 > Go on with
 > git checkout -b bisection-rebase b467ab82a9fde4b46c0cd2c299220857afb0e0d4
 > git rebase v6.12
 > # check that it boots. if it fails to boot, please report this. if it 
succeds, continue:
 > git bisect start
 > git bisect bad bisection-rebase
 > git bisect good v6.12
 >
 > Proceed to bisect. If you never find a kernel that fails to boot, report.
 >
 > If you find a kernel that fails to boot, start another bisection 
between v6.12 and bisection-rebase, looking for the first patch that 
fails to boot.

Hi Bonzini,

Thank you for your answer.

On 22/12/24 18:49, Paolo Bonzini wrote:

     Thanks. Please try this

     git bisect start
     git checkout c9be85dabb376299504e0d391d15662c0edf8273

---

HEAD is now at c9be85dabb37 KVM: PPC: e500: Mark "struct page" dirty in 
kvmppc_e500_shadow_map()

On 22/12/24 18:49, Paolo Bonzini wrote:

     check that it does not boot

---

Yes, it doesn't boot

On 22/12/24 18:49, Paolo Bonzini wrote:

     git bisect bad
     git checkout adc218676eef25575469234709c2d87185ca223a

---

Previous HEAD position was c9be85dabb37 KVM: PPC: e500: Mark "struct 
page" dirty in kvmppc_e500_shadow_map() HEAD is now at adc218676eef 
Linux 6.12

On 22/12/24 18:49, Paolo Bonzini wrote:

     check that it boots

---

The host system boots and the guest kernel 6.13-rc3 boots in a KVM HV VM 
too.

git bisect good

Bisecting: a merge base must be tested
[e9001a382fa2c256229adc68d55212028b01d515] Merge tag 
'kvmarm-fixes-6.12-3' of 
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD

On 22/12/24 18:49, Paolo Bonzini wrote:

     There are two possibilities.
     If e9001a382fa2c256229adc68d55212028b01d515 succeeds to boot

     Then git bisect good and bisect to the patch that causes the host 
to fail to boot. What's left seems to be all non-PPC KVM patches, which 
is a bit weird but you never know. Once you have the result, we'll try 
to fix that, and provide you with a fixed branch to bisect the actual 
KVM-PPC failure.

---

The host system doesn't boot.

On 22/12/24 18:49, Paolo Bonzini wrote:

     A potential candidate, based on your skips, is commit eec1e5db464e 
("KVM: Annotate that all paths in hva_to_pfn() might sleep").

     Please also provide a log of the boot failure (I haven't seen it 
but I might be missing it).
     If e9001a382fa2c256229adc68d55212028b01d515 fails to boot

---

Unfortunately, there isn't a guest kernel boot log via serial log.

On 22/12/24 18:49, Paolo Bonzini wrote:

     Go on with

     git checkout -b bisection-rebase 
b467ab82a9fde4b46c0cd2c299220857afb0e0d4
     git rebase v6.12

---

Applying: Documentation: kvm: fix a few mistakes Applying: 
Documentation: kvm: replace section numbers with links
Applying: Documentation: kvm: reorganize introduction Applying: 
Documentation: KVM: fix malformed table
Applying: x86: KVM: Advertise CPUIDs for new instructions in Clearwater 
Forest
Applying: KVM: x86: expose MSR_PLATFORM_INFO as a feature MSR

On 22/12/24 18:49, Paolo Bonzini wrote:

     check that it boots. if it fails to boot, please report this.

---

The host kernel boots but the guest kernel doesn't boot so KVM HV 
doesn't work.

On 22/12/24 18:49, Paolo Bonzini wrote:

     git bisect start
     git bisect bad bisection-rebase
     git bisect good v6.12

---

Bisecting: 99 revisions left to test after this (roughly 7 steps) 
[e7f7cfdd90c5a8bff38e059ca885a668efd2311f] RISC-V: KVM: Save trap CSRs 
in kvm_riscv_vcpu_enter_exit()

git bisect bad (Host boots and guest doesn't boot)

Output:

Bisecting: 49 revisions left to test after this (roughly 6 steps) 
[4f126962ea9b0d267c4a46f6be0e12698e723210] KVM: Move x86's API to 
release a faultin page to common KVM

git bisect good (Host and guest boot)

Output:

Bisecting: 24 revisions left to test after this (roughly 5 steps) 
[2c56ca43bac9f741b9d1c45d292fd512cf22ce98] KVM: PPC: Use kvm_vcpu_map() 
to map guest memory to patch dcbz instructions

git bisect bad (Host boots and guest doesn't boot)

Output:

Bisecting: 12 revisions left to test after this (roughly 4 steps) 
[a585688ac3ca4a91f80179ffcadb33575d9539a9] KVM: PPC: Use 
__kvm_faultin_pfn() to handle page faults on Book3s Radix

git bisect bad (Host boots and guest doesn't boot)

Bisecting: 5 revisions left to test after this (roughly 3 steps) 
[9f9bae39c20dc83e9c6b0de0ab20260b37f1f48e] KVM: arm64: Mark "struct 
page" pfns accessed/dirty before dropping mmu_lock

git bisect bad (Host boots and guest doesn't boot)

Bisecting: 2 revisions left to test after this (roughly 2 steps) 
[16dd33b96e8c0fb5dc335efaf5ebc7f92feef1ae] KVM: PPC: e500: Mark "struct 
page" dirty in kvmppc_e500_shadow_map()

git bisect good (Host and guest boot)

Output:

Bisecting: 0 revisions left to test after this (roughly 1 step) 
[edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4] KVM: PPC: e500: Use 
__kvm_faultin_pfn() to handle page faults

git bisect bad (Host boots and guest doesn't boot)

Bisecting: 0 revisions left to test after this (roughly 0 steps) 
[a5608a81925937a230816652bc7b953880606e7c] KVM: PPC: e500: Mark "struct 
page" pfn accessed before dropping mmu_lock

git bisect good (Host and guest boot)

`edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 is the first bad commit
commit edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4
Author: Sean Christopherson seanjc@google.com
Date: Thu Oct 10 11:23:56 2024 -0700

KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults

Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(),
and continue the inexorable march towards the demise of
kvm_pfn_to_refcounted_page().

Signed-off-by: Sean Christopherson <seanjc@google.com>
Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-ID: <20241010182427.1434605-55-seanjc@google.com>

arch/powerpc/kvm/e500_mmu_host.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)`

Cheers,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

On Mon, Dec 23, 2024 at 3:41 PM Christian Zigotzky <chzigotzky@xenosoft.de> wrote:
> Bisecting: a merge base must be tested
> [e9001a382fa2c256229adc68d55212028b01d515] Merge tag
> 'kvmarm-fixes-6.12-3' of
> git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
>
> The host system doesn't boot.
> Unfortunately, there isn't a guest kernel boot log via serial log.

No big deal since the boot hang was fixed via rebase.

>      git checkout -b bisection-rebase
> b467ab82a9fde4b46c0cd2c299220857afb0e0d4
>      git rebase v6.12
>
> The host kernel boots but the guest kernel doesn't boot so KVM HV
> doesn't work.

As expected, so that's good.

>      git bisect start
>      git bisect bad bisection-rebase
>      git bisect good v6.12
>
> [skipping bisection process]
>
> `edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 is the first bad commit

Ok, that means that the culprit is commit 419cfb983ca93 (the above
is the rebased hash).  Thank you very much, your work was really
helpful!

Can you test this simple patch?

--------------- 8< ----------------
From: Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH] KVM: allow  NULL writable argument to __kvm_faultin_pfn

kvm_follow_pfn() is able to work with NULL in the .map_writable field
of the homonymous struct.  But __kvm_faultin_pfn() rejects the combo
despite KVM for e500 trying to use it.  Indeed .map_writable is not
particularly useful if the flags include FOLL_WRITE and readonly
guest memory is not supported, so add support to __kvm_faultin_pfn()
for this case.

Fixes: 1c7b627e9306 ("KVM: Add kvm_faultin_pfn() to specifically service guest page faults")
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index de2c11dae231..5177e56fdbd5 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2975,10 +2975,11 @@ kvm_pfn_t __kvm_faultin_pfn(const struct kvm_memory_slot *slot, gfn_t gfn,
  		.refcounted_page = refcounted_page,
  	};
  
-	if (WARN_ON_ONCE(!writable || !refcounted_page))
+	if (WARN_ON_ONCE(!refcounted_page))
  		return KVM_PFN_ERR_FAULT;
  
-	*writable = false;
+	if (writable)
+		*writable = false;
  	*refcounted_page = NULL;
  
  	return kvm_follow_pfn(&kfp);

Thanks,

Paolo

> commit edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4
> Author: Sean Christopherson seanjc@google.com
> Date: Thu Oct 10 11:23:56 2024 -0700
>
> KVM: PPC: e500: Use __kvm_faultin_pfn() to handle page faults
>
> Convert PPC e500 to use __kvm_faultin_pfn()+kvm_release_faultin_page(),
> and continue the inexorable march towards the demise of
> kvm_pfn_to_refcounted_page().
>
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> Tested-by: Dmitry Osipenko <dmitry.osipenko@collabora.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> Message-ID: <20241010182427.1434605-55-seanjc@google.com>
>
> arch/powerpc/kvm/e500_mmu_host.c | 8 +++-----
> 1 file changed, 3 insertions(+), 5 deletions(-)`
>
> Cheers,
> Christian
>
> --
> Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)
>

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 23/12/24 16:37, Paolo Bonzini wrote:
>
>
> On Mon, Dec 23, 2024 at 3:41 PM Christian Zigotzky 
> <chzigotzky@xenosoft.de> wrote:
>> Bisecting: a merge base must be tested
>> [e9001a382fa2c256229adc68d55212028b01d515] Merge tag
>> 'kvmarm-fixes-6.12-3' of
>> git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
>>
>> The host system doesn't boot.
>> Unfortunately, there isn't a guest kernel boot log via serial log.
>
> No big deal since the boot hang was fixed via rebase.
>
>>      git checkout -b bisection-rebase
>> b467ab82a9fde4b46c0cd2c299220857afb0e0d4
>>      git rebase v6.12
>>
>> The host kernel boots but the guest kernel doesn't boot so KVM HV
>> doesn't work.
>
> As expected, so that's good.
>
>>      git bisect start
>>      git bisect bad bisection-rebase
>>      git bisect good v6.12
>>
>> [skipping bisection process]
>>
>> `edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 is the first bad commit
>
> Ok, that means that the culprit is commit 419cfb983ca93 (the above
> is the rebased hash).  Thank you very much, your work was really
> helpful!
>
> Can you test this simple patch?
>
> --------------- 8< ----------------
> From: Paolo Bonzini <pbonzini@redhat.com>
> Subject: [PATCH] KVM: allow  NULL writable argument to __kvm_faultin_pfn
>
> kvm_follow_pfn() is able to work with NULL in the .map_writable field
> of the homonymous struct.  But __kvm_faultin_pfn() rejects the combo
> despite KVM for e500 trying to use it.  Indeed .map_writable is not
> particularly useful if the flags include FOLL_WRITE and readonly
> guest memory is not supported, so add support to __kvm_faultin_pfn()
> for this case.
>
> Fixes: 1c7b627e9306 ("KVM: Add kvm_faultin_pfn() to specifically 
> service guest page faults")
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index de2c11dae231..5177e56fdbd5 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2975,10 +2975,11 @@ kvm_pfn_t __kvm_faultin_pfn(const struct 
> kvm_memory_slot *slot, gfn_t gfn,
>          .refcounted_page = refcounted_page,
>      };
>
> -    if (WARN_ON_ONCE(!writable || !refcounted_page))
> +    if (WARN_ON_ONCE(!refcounted_page))
>          return KVM_PFN_ERR_FAULT;
>
> -    *writable = false;
> +    if (writable)
> +        *writable = false;
>      *refcounted_page = NULL;
>
>      return kvm_follow_pfn(&kfp);
>
> Thanks,
>
> Paolo

Hello Paolo,

Thanks a lot for your patch.

I tested it with the RC4 of kernel 6.13 today but unfortunately it 
doesn't solve the KVM HV issue. The guest system doesn't boot and I 
can't see any error messages. (console=ttyS0)

Regards,

Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

On 12/24/24 12:05, Christian Zigotzky wrote:
> Hello Paolo,
> 
> Thanks a lot for your patch.
> 
> I tested it with the RC4 of kernel 6.13 today but unfortunately it 
> doesn't solve the KVM HV issue. The guest system doesn't boot and I 
> can't see any error messages. (console=ttyS0)

But do you see anything on the host dmesg (without and with the patch), 
for example a WARN?

Also you might try edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 + the patch, 
to see if there are two bugs.

Paolo

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 24/12/24 12:42, Paolo Bonzini wrote:
> On 12/24/24 12:05, Christian Zigotzky wrote:
>> Hello Paolo,
>>
>> Thanks a lot for your patch.
>>
>> I tested it with the RC4 of kernel 6.13 today but unfortunately it 
>> doesn't solve the KVM HV issue. The guest system doesn't boot and I 
>> can't see any error messages. (console=ttyS0)
>
> But do you see anything on the host dmesg (without and with the 
> patch), for example a WARN?
Unfortunately no log during the start of KVM HV.
>
> Also you might try edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 + the 
> patch, to see if there are two bugs.
>
> Paolo
>
I think you mean 419cfb983ca93e75e905794521afefcfa07988bb because "git 
checkout edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4" doesn't work.

Error message: fatal: reference is not a tree: 
edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4

git checkout 419cfb983ca93e75e905794521afefcfa07988bb

Output:

Updating files: 100% (16656/16656), done.
Note: switching to '419cfb983ca93e75e905794521afefcfa07988bb'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
...
HEAD is now at 419cfb983ca9 KVM: PPC: e500: Use __kvm_faultin_pfn() to 
handle page faults

---

patch -p0 < kvm_main.patch // Your patch

Output:

patching file a/virt/kvm/kvm_main.c

Oh dear, the host kernel doesn't boot. :-(

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]

Il mer 25 dic 2024, 13:48 Christian Zigotzky <chzigotzky@xenosoft.de> ha
scritto:

> > But do you see anything on the host dmesg (without and with the
> > patch), for example a WARN?
> Unfortunately no log during the start of KVM HV.
>

Ok.

> Also you might try edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 + the
> > patch, to see if there are two bugs.
> >
> > Paolo
> >
> I think you mean 419cfb983ca93e75e905794521afefcfa07988bb because "git
> checkout edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4" doesn't work.
>
> Error message: fatal: reference is not a tree:
> edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4
>
> git checkout 419cfb983ca93e75e905794521afefcfa07988bb
>

That's the upstream commit that doesn't boot.
edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 was the rebased version of the
same commit, where your bisection finished.

Paolo

Output:
>
> Updating files: 100% (16656/16656), done.
> Note: switching to '419cfb983ca93e75e905794521afefcfa07988bb'.
>
> You are in 'detached HEAD' state. You can look around, make experimental
> changes and commit them, and you can discard any commits you make in this
> state without impacting any branches by switching back to a branch.
> ...
> HEAD is now at 419cfb983ca9 KVM: PPC: e500: Use __kvm_faultin_pfn() to
> handle page faults
>
> ---
>
> patch -p0 < kvm_main.patch // Your patch
>
> Output:
>
> patching file a/virt/kvm/kvm_main.c
>
> Oh dear, the host kernel doesn't boot. :-(
>
> --
> Sent with BrassMonkey 33.4.0 (
> https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)
>
>

[-- Attachment #2: Type: text/html, Size: 2755 bytes --]

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 26/12/24 08:00, Paolo Bonzini wrote:
>
>
> Il mer 25 dic 2024, 13:48 Christian Zigotzky <chzigotzky@xenosoft.de 
> <mailto:chzigotzky@xenosoft.de>> ha scritto:
>
>     > But do you see anything on the host dmesg (without and with the
>     > patch), for example a WARN?
>     Unfortunately no log during the start of KVM HV.
>
>
> Ok.
>
>     > Also you might try edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 + the
>     > patch, to see if there are two bugs.
>     >
>     > Paolo
>     >
>     I think you mean 419cfb983ca93e75e905794521afefcfa07988bb because
>     "git
>     checkout edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4" doesn't work.
>
>     Error message: fatal: reference is not a tree:
>     edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4
>
>     git checkout 419cfb983ca93e75e905794521afefcfa07988bb
>
>
> That's the upstream commit that doesn't boot. 
> edebc0a1b7ab5ff19b4f5cd011c93196fc34b3e4 was the rebased version of 
> the same commit, where your bisection finished.
>
> Paolo

Hi Paolo,

This has solved the issue:

cp linux-6.12.6/arch/powerpc/kvm/e500_mmu_host.c 
a/arch/powerpc/kvm/e500_mmu_host.c

cp linux-6.12.6/include/linux/kvm_host.h a/include/linux/kvm_host.h

cp linux-6.12.6/virt/kvm/kvm_main.c a/virt/kvm/kvm_main.c

cp linux-6.12.6/virt/kvm/kvm_mm.h a/virt/kvm/kvm_mm.h

I created a patch for reverting the code that leads to the issue.

Link: 
https://github.com/chzigotzky/kernels/blob/main/patches/X5000/kvm_ppc_revert.patch

Maybe you can find a hint to the issue in this revert patch.

I will further test KVM HV with the patched kernel.

Thanks,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

Re: KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Paolo Bonzini]

On Thu, Dec 26, 2024 at 3:26 PM Christian Zigotzky
<chzigotzky@xenosoft.de> wrote:
> This has solved the issue:
>
> cp linux-6.12.6/arch/powerpc/kvm/e500_mmu_host.c
> a/arch/powerpc/kvm/e500_mmu_host.c
> cp linux-6.12.6/include/linux/kvm_host.h a/include/linux/kvm_host.h
> cp linux-6.12.6/virt/kvm/kvm_main.c a/virt/kvm/kvm_main.c
> cp linux-6.12.6/virt/kvm/kvm_mm.h a/virt/kvm/kvm_mm.h

Yes, those are exactly the file touched in the bisection result and in
my patch. However, reverting blindly like this would break the build
for all other architectures, and reintroduce the bugs which were fixed
by the change.

The patch in https://github.com/user-attachments/files/18238781/kvm_main.patch
is incorrect (the "*writable = false" line was not removed, thus
leaving in the NULL pointer dereference).  Please test the
kvm-e500-bisection-rebase branch of
https://git.kernel.org/pub/scm/virt/kvm/kvm.git, which has the patch
applied already:

- first test commit 0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2, and
report if it fails

- then the entire branch, bisecting between
0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2 and the top if it fails.

Paolo

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 27/12/24 06:20, Paolo Bonzini wrote:
> Please test the kvm-e500-bisection-rebase branch of 
> https://git.kernel.org/pub/scm/virt/kvm/kvm.git, which has the patch 
> applied already:
> - first test commit 0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2, and report if it fails
>
> - then the entire branch, bisecting between 0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2 and the top if it fails.
>
> Paolo
>

Hello Paolo,

git clone https://git.kernel.org/pub/scm/virt/kvm/kvm.git -b 
kvm-e500-bisection-rebase a

git checkout 0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2

Output:

... HEAD is now at 0096d3855a5e KVM: PPC: e500: Use __kvm_faultin_pfn() 
to handle page faults

make -j8 CROSS_COMPILE=powerpc-linux-gnu- ARCH=powerpc oldconfig

make -j8 CROSS_COMPILE=powerpc-linux-gnu- ARCH=powerpc uImage

The host kernel and the guest kernel boots with "-enable-kvm" so KVM HV 
works.

Paolo wrote:

then the entire branch, bisecting between 
0096d3855a5e80b9d4e43e5a291e8c79aa4abbc2 and the top if it fails.

---

git log

commit 15ac24da3251e47fe3a63ea52d8c5ecb0b26b02c (HEAD -> 
kvm-e500-bisection-rebase, origin/kvm-e500-bisection-rebase)
Author: Paolo Bonzini <pbonzini@redhat.com>
Date:   Wed Nov 13 12:39:45 2024 +0100

I tested the top additionally and the host and guest boot.

BTW, I also tested KVM PR on my P.A. Semi Nemo board yesterday. KVM PR 
isn’t affected. PA6T-1682M is a Book3S SoC with AltiVec support.

Thanks,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

Hi All,

I found out which area is responsible for the KVM HV issue while 
reducing the revert patch.

If I replace the following line in the file 
'a/arch/powerpc/kvm/e500_mmu_host.c' then KVM HV works again.

-        pfn = __kvm_faultin_pfn(slot, gfn, FOLL_WRITE, NULL, &page);
+        pfn = gfn_to_pfn_memslot(slot, gfn);


On 28/12/24 15:55, Paolo Bonzini wrote:
 > The fix is already found, I will send it out shortly but everybody is on
 > vacation now

Thanks a lot! Could you please post the patch on this list? I would like 
to patch the RC5 of kernel 6.13 next week.

Thanks,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

Hi All,

I have compiled the RC5 of kernel 6.13 [1] with the latest revert patch. [2]

After that, I successfully tested the following QEMU/KVM HV components.

- Guest performance: good
- Guest boot time: short
- KVM HV works (-enable-kvm)
- SMP works (-smp 4)
- QEMU User Networking (SLIRP) via VirtIO_net works (Tested it with SSH 
and with a web browser)
- QEMU USB Audio works (Tested it with some audio files in the guest system)
- VirtIO GPU works
- Virtual mouse and keyboard controller works (VirtIO input)
- 9p mount of a host folder via VirtIO works in the guest system
- Serial output from the guest works (console=ttyS0,115200)
- Shutdown and reboot works

The issue still exists but I hope Paolo's patch can fix the issue. [3]

Cheers,
Christian

[1] https://github.com/chzigotzky/kernels/releases/tag/v6.13.0-rc5
[2] 
https://github.com/chzigotzky/kernels/blob/main/patches/X5000/kvm_ppc_revert.patch
[3] https://github.com/chzigotzky/kernels/issues/4

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

Hello Paolo,

First, I wish you a Happy New Year. Thanks a lot for your patch! I 
patched the RC5 of kernel 6.13 with your patch and KVM HV works! :-)

kvm_hv_final.patch:

virt/kvm/kvm_main.c | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index de2c11dae231..5177e56fdbd5 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2975,10 +2975,11 @@ kvm_pfn_t __kvm_faultin_pfn(const struct 
kvm_memory_slot *slot, gfn_t gfn,
                 .refcounted_page = refcounted_page,
         };

-       if (WARN_ON_ONCE(!writable || !refcounted_page))
+       if (WARN_ON_ONCE(!refcounted_page))
                 return KVM_PFN_ERR_FAULT;

-       *writable = false;
+       if (writable)
+               *writable = false;
         *refcounted_page = NULL;

         return kvm_follow_pfn(&kfp);
---

patch -p0 < kvm_hv_final.patch

patching file a/virt/kvm/kvm_main.c

Now, we have a good patch after 56 test kernels!!!!!!!!!

I will test all QEMU/KVM HV components and report, if they work.

Cheers,
Christian

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)

KVM: PPC: Book3E: KVM HV host module doesn't work anymore

[Christian Zigotzky]

On 01/01/25 14:30, Christian Zigotzky wrote:
> Hello Paolo,
>
> First, I wish you a Happy New Year. Thanks a lot for your patch! I 
> patched the RC5 of kernel 6.13 with your patch and KVM HV works! :-)
>
> kvm_hv_final.patch:
>
> virt/kvm/kvm_main.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index de2c11dae231..5177e56fdbd5 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -2975,10 +2975,11 @@ kvm_pfn_t __kvm_faultin_pfn(const struct 
> kvm_memory_slot *slot, gfn_t gfn,
>                 .refcounted_page = refcounted_page,
>         };
>
> -       if (WARN_ON_ONCE(!writable || !refcounted_page))
> +       if (WARN_ON_ONCE(!refcounted_page))
>                 return KVM_PFN_ERR_FAULT;
>
> -       *writable = false;
> +       if (writable)
> +               *writable = false;
>         *refcounted_page = NULL;
>
>         return kvm_follow_pfn(&kfp);
> ---
>
> patch -p0 < kvm_hv_final.patch
>
> patching file a/virt/kvm/kvm_main.c
>
> Now, we have a good patch after 56 test kernels!!!!!!!!!
>
> I will test all QEMU/KVM HV components and report, if they work.
>
> Cheers,
> Christian
>
Test results:

- Guest performance: good
- Guest boot time: short
- KVM HV works (-enable-kvm)
- SMP works (-smp 4)
- QEMU User Networking (SLIRP) via VirtIO_net works (Tested it with SSH 
and with a web browser)
- QEMU USB Audio works (Tested it with some audio files in the guest system)
- VirtIO GPU works
- Virtual mouse and keyboard controller work (VirtIO input)
- 9p mount of a host folder via VirtIO works in the guest system
- Serial output from the guest works (console=ttyS0,115200)
- Shutdown and reboot work

Final result: I have not noticed any changes compared to KVM HV with 
kernel 6.12.

@Paolo
There has been a lot of work in KVM for kernel 6.13.
Could you please just explain in a few words the news about KVM? I would 
like to share the KVM news with our users.

Thanks

-- 
Sent with BrassMonkey 33.4.0 (https://github.com/wicknix/brass-monkey/releases/tag/33.4.0)