From: Naman Jain <namjain@linux.microsoft.com>
To: Sahil Chandna <sahilchandna@linux.microsoft.com>,
kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, longli@microsoft.com, lpieralisi@kernel.org,
kwilczynski@kernel.org, mani@kernel.org, robh@kernel.org,
bhelgaas@google.com, linux-hyperv@vger.kernel.org,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] PCI: hv: Warn when wait_for_response() waits indefinitely
Date: Tue, 25 Aug 2026 12:16:06 +0530 [thread overview]
Message-ID: <736091cf-8d00-415e-a890-e2dfa6739325@linux.microsoft.com> (raw)
In-Reply-To: <20260825051850.2438816-1-sahilchandna@linux.microsoft.com>
On 8/25/2026 10:47 AM, Sahil Chandna wrote:
> A guest can wait indefinitely in wait_for_response() for the host to
> send either a rescind message or a packet completion. If the
> host does not send either, the guest can remain blocked with no
> diagnostic indicating a reason.
> This was observed during a guest kernel upgrade in which the
> host-side application handling the PCI channel faulted, causing the
> guest to never receive the completion request.
> Add a periodic warning in wait_for_response() when the wait exceeds
> a timeout so that such a hang is visible in the guest's kernel log
> and can be correlated with host-side state.
>
> Suggested-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
> Signed-off-by: Sahil Chandna <sahilchandna@linux.microsoft.com>
> ---
> This was sent earlier upstream [1]
> [1] https://lore.kernel.org/linux-hyperv/20260612174010.2598695-1-hamzamahfooz@linux.microsoft.com/
> ---
> drivers/pci/controller/pci-hyperv.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index cfc8fa403dad..c4fba0039164 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -1040,11 +1040,18 @@ static void put_pcichild(struct hv_pci_dev *hpdev)
>
> /*
> * There is no good way to get notified from vmbus_onoffer_rescind(),
> - * so let's use polling here, since this is not a hot path.
> + * so let's use polling here, since this is not a hot path. If
> + * wait_for_response() has been polling for PCI_RESPONSE_HANG_TIMEOUT_SEC
> + * without either a rescind or completion, add a periodic warning.
> */
> +#define PCI_RESPONSE_HANG_TIMEOUT_SEC 300
> +
> static int wait_for_response(struct hv_device *hdev,
> struct completion *comp)
> {
> + unsigned long delay = secs_to_jiffies(PCI_RESPONSE_HANG_TIMEOUT_SEC);
> + u64 timeout = get_jiffies_64() + delay;
> +
> while (true) {
> if (hdev->channel->rescind) {
> dev_warn_once(&hdev->device, "The device is gone.\n");
> @@ -1053,6 +1060,11 @@ static int wait_for_response(struct hv_device *hdev,
>
> if (wait_for_completion_timeout(comp, HZ / 10))
> break;
> +
> + if (time_after64(get_jiffies_64(), timeout)) {
> + dev_warn(&hdev->device, "PCI stuck waiting for response.\n");
> + timeout = get_jiffies_64() + delay;
> + }
> }
There can be some enhancements in above patch to address these problems:
1. Logging forever every 5 minutes in case of no completion or rescind.
2. If we now print warning once, not knowing if completion ever arrived.
3. On solving pt. 1 and 2 by adding a print for completion, one should
avoid adding a print by default for regular timely completions.
Basically something like this:
#define PCI_RESPONSE_WARN_TIMEOUT_SEC 300
static int wait_for_response(struct hv_device *hdev,
struct completion *comp)
{
unsigned long warn_at =
jiffies + secs_to_jiffies(PCI_RESPONSE_WARN_TIMEOUT_SEC);
bool warned = false;
while (true) {
if (hdev->channel->rescind) {
dev_warn_once(&hdev->device, "The device is gone.\n");
return -ENODEV;
}
if (wait_for_completion_timeout(comp, HZ / 10)) {
if (warned || time_after_eq(jiffies, warn_at))
dev_warn(&hdev->device,
"PCI response received after prolonged wait.\n");
return 0;
}
if (!warned && time_after_eq(jiffies, warn_at)) {
dev_warn(&hdev->device,
"PCI still waiting for response.\n");
warned = true;
}
}
}
Regards,
Naman
next prev parent reply other threads:[~2026-08-25 6:46 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-25 5:17 [PATCH] PCI: hv: Warn when wait_for_response() waits indefinitely Sahil Chandna
2026-08-25 5:30 ` sashiko-bot
2026-08-25 12:03 ` Sahil Chandna
2026-08-25 6:46 ` Naman Jain [this message]
2026-08-25 12:00 ` Sahil Chandna
2026-08-25 17:00 ` Long Li
2026-08-27 16:07 ` Michael Kelley
2026-08-27 17:34 ` Saurabh Singh Sengar
2026-08-28 5:40 ` Naman Jain
2026-08-28 14:59 ` Hamza Mahfooz
2026-09-01 11:14 ` Sahil Chandna
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=736091cf-8d00-415e-a890-e2dfa6739325@linux.microsoft.com \
--to=namjain@linux.microsoft.com \
--cc=bhelgaas@google.com \
--cc=decui@microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=kwilczynski@kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=robh@kernel.org \
--cc=sahilchandna@linux.microsoft.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.