Memory failure handling of VFIO-pinned THP

[Vikram Sethi <vsethi@nvidia.com>]

[-- Attachment #1: Type: text/plain, Size: 1950 bytes --]

Hello,

I was looking at memory_failure handling of pinned transparent hugepages (specifically pinned by VFIO for a VM with physical I/O).

AFAICT, on the initial memory error detected interrupt call memory_failure won't be able to split the THP because it is pinned, and will return -EBUSY without actually unmapping any processes with mappings to the THP with uncorrected memory error.

 

Later, when the VM does a load to the bad location (consumes poison), looking at the firmware first path on ARM64, the SEA exception will be forwarded by Firmware to host kernel, where the GHES code will queue work for memory_failure, where again memory_failure will exit early for the pinned THP, and userspace won't get the SIGBUS with Action Required code to be able to inject the error into the VM.

 

Discussing with James, we were wondering why the pinned THP isn't treated like hugetlbfs memory failure, marking the entire hugepage with hw_poison flag, and unmapping of mapped processes when the error is detected (memory_failure_hugetlb calling hwpoison_user_mappings)? If that were done, when the VM later tries to load the bad location, the resulting VM fault will get the appropriate VM_FAULT_HWPOISON code, which will trigger KVM to send the SIGBUS with Action Required code to userspace, which can then inject to the VM?

I do understand that the page is pinned so that DMAs can happen from the VM's I/O devices without I/O faults, but since the hw_poison flag would be set for the page on the initial "error detected" interrupt by memory_failure, the kernel wouldn't reallocate the page anyway. And any interim DMA writes that hit the bad page wouldn't be corrupting anyone else, and DMA reads would be getting poison back/completer abort. 

 

Am I missing something, or is this currently broken for VFIO and VM THP pages with memory failure (at least as far as signaling user space goes)?

 

Thanks,

Vikram


[-- Attachment #2: Type: text/html, Size: 3353 bytes --]