All of lore.kernel.org
 help / color / mirror / Atom feed
From: Narayana Murty N <nnmlinux@linux.ibm.com>
To: Anushree Mathur <anushree.mathur@linux.ibm.com>,
	qemu-ppc@nongnu.org, qemu-devel@nongnu.org
Cc: npiggin@gmail.com, harshpb@linux.ibm.com, mahesh@linux.ibm.com,
	ganeshgr@linux.ibm.com, sbhat@linux.ibm.com,
	vaibhav@linux.ibm.com, anushree.mathur@linux.vnet.ibm.com,
	clg@redhat.com, pierrick.bouvier@oss.qualcomm.com,
	philmd@linaro.org
Subject: Re: [PATCH v3 0/6] ppc/spapr: Add RTAS error injection support for VFIO EEH
Date: Tue, 9 Jun 2026 14:25:52 +0530	[thread overview]
Message-ID: <f85ea418-f285-4d12-8612-4bb641e1888b@linux.ibm.com> (raw)
In-Reply-To: <c705eb45-18d9-4239-8e99-b964048f946c@linux.ibm.com>



On 08/06/26 11:01 PM, Anushree Mathur wrote:
> Hi Narayana,
> I tested this patch and saw a qemu crash just after triggering error 
> injection on guest after applying the patch series.
> Here is my full analysis:
>
> 1) Started the guest
> 2) Attached an NVME backplane device to the guest.
> 3) Triggered the error injection on guest console for the attached 
> NVME device.
>
> It came to the following message and the guest crashed:
>
> Injecting an ioa-bus-error...
>
> Following is the qemu logs after crash:
>
>
> 2026-06-08T06:25:16.420015Z qemu-system-ppc64: warning: kernel_irqchip 
> allowed but unavailable: IRQ_XIVE capability must be present for KVM
> Falling back to kernel-irqchip=off
> 2026-06-08T06:25:16.495864Z RTAS: Read 236 bytes from device-tree
> 2026-06-08T06:28:43.769276Z qemu-system-ppc64: 0213:60:00.0 BAR 0: 
> failed to create dma-buf: PCI BAR IOMMU mappings may fail: Invalid 
> argument
> 2026-06-08T13:21:00.218278Z RTAS: Read 236 bytes from device-tree
> 2026-06-08T13:21:05.658991Z qemu-system-ppc64: warning: kernel_irqchip 
> allowed but unavailable: IRQ_XIVE capability must be present for KVM
> Falling back to kernel-irqchip=off
> 2026-06-08T13:21:05.794879Z RTAS: Read 236 bytes from device-tree
> 2026-06-08T13:41:59.736480Z RTAS: Read 236 bytes from device-tree
> 2026-06-08T13:42:13.712991Z qemu-system-ppc64: warning: kernel_irqchip 
> allowed but unavailable: IRQ_XIVE capability must be present for KVM
> Falling back to kernel-irqchip=off
> 2026-06-08T13:42:13.848406Z RTAS: Read 236 bytes from device-tree
> 2026-06-08T14:24:42.012714Z qemu-system-ppc64: 0213:60:00.0 BAR 0: 
> failed to create dma-buf: PCI BAR IOMMU mappings may fail: Invalid 
> argument
> 2026-06-08 14:34:09.570+0000: shutting down, reason=crashed
>
Hi Anushree,

Thank you very much for the detailed crash report and thorough testing!
This is extremely valuable feedback that helped identify a critical bug in
the error injection implementation.

I'll prepare a v4 of the patch series with this fix. Could you please 
test it
again once I send it out? I'd particularly appreciate testing with:
- Multiple PHBs in the system
- Devices attached to different PHBs
- Both IOA bus errors and other error types (if possible)

Thanks again for the excellent bug report!

Best regards,
Narayana Murty N
>
> Thank you,
> Anushree Mathur
>
> On 20/05/26 3:24 PM, Narayana Murty N wrote:
>> This patch series implements comprehensive RTAS-based error injection
>> support for VFIO EEH (Enhanced Error Handling) on PowerPC sPAPR 
>> platforms.
>> The implementation enables guest-initiated PCI error injection for 
>> improved
>> testing and diagnostics of EEH recovery mechanisms.
>>
>> Background
>> ----------
>> EEH is a critical feature on PowerPC platforms that provides error 
>> detection,
>> isolation, and recovery for PCI devices. Testing EEH recovery paths 
>> requires
>> the ability to inject various types of errors into the system. While 
>> physical
>> hardware supports error injection through firmware interfaces, QEMU's 
>> VFIO
>> implementation previously lacked this capability.
>>
>> This series bridges that gap by implementing the IBM RTAS error 
>> injection
>> interface, allowing guests to inject PCI errors through the same 
>> firmware
>> calls used on physical hardware. This enables comprehensive testing 
>> of device
>> drivers' EEH recovery code paths in virtualized environments.
>>
>> Implementation Overview
>> -----------------------
>> The patch series introduces three new RTAS calls:
>>    - ibm,open-errinjct:  Opens an error injection session
>>    - ibm,errinjct:       Injects a specific error type
>>    - ibm,close-errinjct: Closes the error injection session
>>
>> The implementation supports multiple error types including:
>>    - IOA bus errors (32-bit and 64-bit addressing)
>>    - Memory/IO/Config space load/store errors
>>    - DMA read/write errors
>>    - Cache and TLB corruption scenarios
>>    - Special recovery events
>>
>> Tesed on pseries and powernv hosts on kvm guest with errinjct tool.
>>
>> Patch Organization
>> ------------------
>> Patch 1: Adds the VFIO backend for error injection
>> Patch 2: Implements the ibm,errinjct RTAS call handler
>> Patch 3: Adds session management (open/close) RTAS calls
>> Patch 4: Advertises capabilities via device tree properties
>> Patch 5: Refactors EEH specific code/stubs to new files.
>> Patch 6: Updates MAINTAINERS file
>>
>> Changelog:
>> ----------
>> v3:
>>    - Fixed the build failure reported at 
>> https://github.com/p-b-o/qemu-ci/actions/runs/26094993976
>>    - Also fixed a gitlab CI breakage in patch 2 (qemu_log_mask 
>> LOG_UNIMP)
>> v2: Addressed refactor suggestions from Cedric, Pierrick
>> v1: 
>> https://lore.kernel.org/all/20260512071112.9675-1-nnmlinux@linux.ibm.com/ 
>>
>>
>>
>> Narayana Murty N (6):
>>    ppc/spapr: Add VFIO EEH error injection backend
>>    ppc/spapr: Add ibm,errinjct RTAS call handler
>>    ppc/spapr: Add support for 'ibm, open-errinjct' and 'ibm,
>>      close-errinjct'
>>    ppc/spapr: Advertise RTAS error injection call support via FDT
>>      property
>>    ppc/spapr: Split VFIO code and refactor EEH interface
>>    MAINTAINERS: Add entry for sPAPR PCI VFIO EEH support
>>
>>   MAINTAINERS                  |   6 +
>>   hw/ppc/Kconfig               |   2 +-
>>   hw/ppc/meson.build           |   1 +
>>   hw/ppc/spapr.c               | 104 +++++++++++
>>   hw/ppc/spapr_pci.c           | 219 ++++++++++++++++++++++
>>   hw/ppc/spapr_pci_vfio.c      | 314 +------------------------------
>>   hw/ppc/spapr_pci_vfio_eeh.c  | 346 +++++++++++++++++++++++++++++++++++
>>   include/hw/pci-host/spapr.h  |  37 +---
>>   include/hw/ppc/spapr.h       |  57 +++++-
>>   include/hw/ppc/spapr_vfio.h  |  28 +++
>>   stubs/meson.build            |   1 +
>>   stubs/spapr_phb_vfio-stubs.c |  52 ++++++
>>   12 files changed, 816 insertions(+), 351 deletions(-)
>>   create mode 100644 hw/ppc/spapr_pci_vfio_eeh.c
>>   create mode 100644 include/hw/ppc/spapr_vfio.h
>>   create mode 100644 stubs/spapr_phb_vfio-stubs.c 



      reply	other threads:[~2026-06-09  8:56 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-05-20  9:54 [PATCH v3 0/6] ppc/spapr: Add RTAS error injection support for VFIO EEH Narayana Murty N
2026-05-20  9:54 ` [PATCH v3 1/6] ppc/spapr: Add VFIO EEH error injection backend Narayana Murty N
2026-08-13 16:02   ` Nikhil Kumar Singh
2026-08-13 16:35   ` Nikhil Kumar Singh
2026-05-20  9:54 ` [PATCH v3 2/6] ppc/spapr: Add ibm,errinjct RTAS call handler Narayana Murty N
2026-08-13 16:11   ` Nikhil Kumar Singh
2026-05-20  9:54 ` [PATCH v3 3/6] ppc/spapr: Add support for 'ibm, open-errinjct' and 'ibm, close-errinjct' Narayana Murty N
2026-08-13 16:39   ` Nikhil Kumar Singh
2026-05-20  9:54 ` [PATCH v3 4/6] ppc/spapr: Advertise RTAS error injection call support via FDT property Narayana Murty N
2026-05-22  9:07   ` Shivaprasad G Bhat
2026-06-01  6:05     ` Narayana Murty N
2026-08-13 16:49   ` Nikhil Kumar Singh
2026-05-20  9:54 ` [PATCH v3 5/6] ppc/spapr: Split VFIO code and refactor EEH interface Narayana Murty N
2026-05-22 16:07   ` Pierrick Bouvier
2026-05-22 16:29     ` Narayana Murty N
2026-08-14  6:26   ` Harsh Prateek Bora
2026-05-20  9:54 ` [PATCH v3 6/6] MAINTAINERS: Add entry for sPAPR PCI VFIO EEH support Narayana Murty N
2026-06-08 17:31 ` [PATCH v3 0/6] ppc/spapr: Add RTAS error injection support for VFIO EEH Anushree Mathur
2026-06-09  8:55   ` Narayana Murty N [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f85ea418-f285-4d12-8612-4bb641e1888b@linux.ibm.com \
    --to=nnmlinux@linux.ibm.com \
    --cc=anushree.mathur@linux.ibm.com \
    --cc=anushree.mathur@linux.vnet.ibm.com \
    --cc=clg@redhat.com \
    --cc=ganeshgr@linux.ibm.com \
    --cc=harshpb@linux.ibm.com \
    --cc=mahesh@linux.ibm.com \
    --cc=npiggin@gmail.com \
    --cc=philmd@linaro.org \
    --cc=pierrick.bouvier@oss.qualcomm.com \
    --cc=qemu-devel@nongnu.org \
    --cc=qemu-ppc@nongnu.org \
    --cc=sbhat@linux.ibm.com \
    --cc=vaibhav@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.