From: Sahil Chandna <sahilchandna@linux.microsoft.com>
To: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, longli@microsoft.com, lpieralisi@kernel.org,
kwilczynski@kernel.org, mani@kernel.org, robh@kernel.org,
bhelgaas@google.com, linux-hyperv@vger.kernel.org,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
mhklinux@outlook.com
Subject: [PATCH v3] PCI: hv: Warn when wait_for_response() waits indefinitely
Date: Tue, 8 Sep 2026 04:30:06 -0700 [thread overview]
Message-ID: <20260908113009.3005964-1-sahilchandna@linux.microsoft.com> (raw)
A guest can wait indefinitely in wait_for_response() for the host to
send either a rescind message or a packet completion. If the
host does not send either, the guest can remain blocked with no
diagnostic indicating a reason.
This was observed during a guest kernel upgrade in which the
host-side application handling the PCI channel faulted, causing the
guest to never receive the completion request.
Add a warning in wait_for_response() when the wait exceeds
a timeout so that such a hang is visible in the guest's kernel log
and can be correlated with host-side state.
Suggested-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Sahil Chandna <sahilchandna@linux.microsoft.com>
---
Changes since v2:
- Add counter based timeout instead of introducing another timer
Link to v2: https://lore.kernel.org/all/20260902115854.2629164-1-sahilchandna@linux.microsoft.com/
Changes since v1:
- Removed periodic warning to one time warning in 2 minutes
- Include vmbus relid and stuck PCI msg.
Link to v1: https://lore.kernel.org/all/20260825051850.2438816-1-sahilchandna@linux.microsoft.com/
drivers/pci/controller/pci-hyperv.c | 46 ++++++++++++++++++++++-------
1 file changed, 36 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 89816a2bd7cd..bd07402c47ed 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -1040,19 +1040,38 @@ static void put_pcichild(struct hv_pci_dev *hpdev)
/*
* There is no good way to get notified from vmbus_onoffer_rescind(),
- * so let's use polling here, since this is not a hot path.
+ * so let's use polling here, since this is not a hot path. If
+ * wait_for_response() has been polling for 2 minutes
+ * without either a rescind or completion, add a warning.
*/
+#define PCI_RESPONSE_HANG_TICKS 1200
+
static int wait_for_response(struct hv_device *hdev,
- struct completion *comp)
+ struct completion *comp,
+ const char *msg_type)
{
+ u64 counter = 0;
+
while (true) {
if (hdev->channel->rescind) {
dev_warn_once(&hdev->device, "The device is gone.\n");
return -ENODEV;
}
- if (wait_for_completion_timeout(comp, HZ / 10))
+ counter++;
+
+ if (wait_for_completion_timeout(comp, HZ / 10)) {
+ if (counter > PCI_RESPONSE_HANG_TICKS)
+ dev_warn(&hdev->device,
+ "Late %s completion arrived.\n", msg_type);
break;
+ }
+
+ if (counter == PCI_RESPONSE_HANG_TICKS) {
+ dev_err(&hdev->device,
+ "%s stuck waiting for response, relid = %u\n",
+ msg_type, hdev->channel->offermsg.child_relid);
+ }
}
return 0;
@@ -1518,7 +1537,8 @@ static int hv_read_config_block(struct pci_dev *pdev, void *buf,
if (ret)
return ret;
- ret = wait_for_response(hbus->hdev, &comp_pkt.comp_pkt.host_event);
+ ret = wait_for_response(hbus->hdev, &comp_pkt.comp_pkt.host_event,
+ "PCI_READ_BLOCK");
if (ret)
return ret;
@@ -1607,7 +1627,8 @@ static int hv_write_config_block(struct pci_dev *pdev, void *buf,
if (ret)
return ret;
- ret = wait_for_response(hbus->hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hbus->hdev, &comp_pkt.host_event,
+ "PCI_WRITE_BLOCK");
if (ret)
return ret;
@@ -2624,7 +2645,8 @@ static struct hv_pci_dev *new_pcichild_device(struct hv_pcibus_device *hbus,
if (ret)
goto error;
- if (wait_for_response(hbus->hdev, &comp_pkt.host_event))
+ if (wait_for_response(hbus->hdev, &comp_pkt.host_event,
+ "PCI_QUERY_RESOURCE_REQUIREMENTS"))
goto error;
hpdev->desc = *desc;
@@ -3256,7 +3278,8 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev,
(unsigned long)pkt, VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_QUERY_PROTOCOL_VERSION");
if (ret) {
dev_err(&hdev->device,
@@ -3476,7 +3499,8 @@ static int hv_pci_enter_d0(struct hv_device *hdev)
(unsigned long)pkt, VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_BUS_D0ENTRY");
if (ret)
goto exit;
@@ -3553,7 +3577,8 @@ static int hv_pci_query_relations(struct hv_device *hdev)
ret = vmbus_sendpacket(hdev->channel, &message, sizeof(message),
0, VM_PKT_DATA_INBAND, 0);
if (!ret)
- ret = wait_for_response(hdev, &comp);
+ ret = wait_for_response(hdev, &comp,
+ "PCI_QUERY_BUS_RELATIONS");
/*
* In the case of fast device addition/removal, it's possible that
@@ -3644,7 +3669,8 @@ static int hv_send_resources_allocated(struct hv_device *hdev)
VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_RESOURCE_ASSIGNED");
if (ret)
break;
--
2.53.0
next reply other threads:[~2026-09-08 11:30 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-08 11:30 Sahil Chandna [this message]
2026-09-08 11:38 ` [PATCH v3] PCI: hv: Warn when wait_for_response() waits indefinitely sashiko-bot
2026-09-09 9:31 ` Naman Jain
2026-09-11 10:29 ` Manivannan Sadhasivam
2026-09-13 22:39 ` Wei Liu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260908113009.3005964-1-sahilchandna@linux.microsoft.com \
--to=sahilchandna@linux.microsoft.com \
--cc=bhelgaas@google.com \
--cc=decui@microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=kwilczynski@kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=mhklinux@outlook.com \
--cc=robh@kernel.org \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.