From: Sahil Chandna <sahilchandna@linux.microsoft.com>
To: kys@microsoft.com, haiyangz@microsoft.com,
namjain@linux.microsoft.com, mhklinux@outlook.com,
hamzamahfooz@linux.microsoft.com, wei.liu@kernel.org,
decui@microsoft.com, longli@microsoft.com, lpieralisi@kernel.org,
kwilczynski@kernel.org, mani@kernel.org, robh@kernel.org,
bhelgaas@google.com, linux-hyperv@vger.kernel.org,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH v2] PCI: hv: warn when wait_for_response() waits indefinitely
Date: Wed, 2 Sep 2026 04:58:51 -0700 [thread overview]
Message-ID: <20260902115854.2629164-1-sahilchandna@linux.microsoft.com> (raw)
A guest can wait indefinitely in wait_for_response() for the host to
send either a rescind message or a packet completion. If the
host does not send either, the guest can remain blocked with no
diagnostic indicating a reason.
This was observed during a guest kernel upgrade in which the
host-side application handling the PCI channel faulted, causing the
guest to never receive the completion request.
Add a warning in wait_for_response() when the wait exceeds
a timeout so that such a hang is visible in the guest's kernel log
and can be correlated with host-side state.
Suggested-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Suggested-by: Naman Jain <namjain@linux.microsoft.com>
Suggested-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Sahil Chandna <sahilchandna@linux.microsoft.com>
---
Changes since v1:
- Removed periodic warning to one time warning in 2 minutes
- Include vmbus relid and stuck PCI msg.
Link to v1: https://lore.kernel.org/all/20260825051850.2438816-1-sahilchandna@linux.microsoft.com/
---
drivers/pci/controller/pci-hyperv.c | 48 +++++++++++++++++++++++------
1 file changed, 38 insertions(+), 10 deletions(-)
diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
index 89816a2bd7cd..ca9d8efd748c 100644
--- a/drivers/pci/controller/pci-hyperv.c
+++ b/drivers/pci/controller/pci-hyperv.c
@@ -1040,19 +1040,40 @@ static void put_pcichild(struct hv_pci_dev *hpdev)
/*
* There is no good way to get notified from vmbus_onoffer_rescind(),
- * so let's use polling here, since this is not a hot path.
+ * so let's use polling here, since this is not a hot path. If
+ * wait_for_response() has been polling for PCI_RESPONSE_HANG_TIMEOUT_SEC
+ * without either a rescind or completion, add a warning.
*/
+#define PCI_RESPONSE_HANG_TIMEOUT_SEC 120
+
static int wait_for_response(struct hv_device *hdev,
- struct completion *comp)
+ struct completion *comp,
+ const char *msg_type)
{
+ unsigned long delay = secs_to_jiffies(PCI_RESPONSE_HANG_TIMEOUT_SEC);
+ u64 timeout = get_jiffies_64() + delay;
+ bool warned = false;
+
while (true) {
if (hdev->channel->rescind) {
dev_warn_once(&hdev->device, "The device is gone.\n");
return -ENODEV;
}
- if (wait_for_completion_timeout(comp, HZ / 10))
+ if (wait_for_completion_timeout(comp, HZ / 10)) {
+ if (warned)
+ dev_warn(&hdev->device,
+ "Late %s completion arrived.\n", msg_type);
break;
+ }
+
+ if (!warned && time_after64(get_jiffies_64(), timeout)) {
+ dev_err(&hdev->device,
+ "%s stuck waiting for response, relid = %u\n",
+ msg_type, hdev->channel->offermsg.child_relid);
+
+ warned = true;
+ }
}
return 0;
@@ -1518,7 +1539,8 @@ static int hv_read_config_block(struct pci_dev *pdev, void *buf,
if (ret)
return ret;
- ret = wait_for_response(hbus->hdev, &comp_pkt.comp_pkt.host_event);
+ ret = wait_for_response(hbus->hdev, &comp_pkt.comp_pkt.host_event,
+ "PCI_READ_BLOCK");
if (ret)
return ret;
@@ -1607,7 +1629,8 @@ static int hv_write_config_block(struct pci_dev *pdev, void *buf,
if (ret)
return ret;
- ret = wait_for_response(hbus->hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hbus->hdev, &comp_pkt.host_event,
+ "PCI_WRITE_BLOCK");
if (ret)
return ret;
@@ -2624,7 +2647,8 @@ static struct hv_pci_dev *new_pcichild_device(struct hv_pcibus_device *hbus,
if (ret)
goto error;
- if (wait_for_response(hbus->hdev, &comp_pkt.host_event))
+ if (wait_for_response(hbus->hdev, &comp_pkt.host_event,
+ "PCI_QUERY_RESOURCE_REQUIREMENTS"))
goto error;
hpdev->desc = *desc;
@@ -3256,7 +3280,8 @@ static int hv_pci_protocol_negotiation(struct hv_device *hdev,
(unsigned long)pkt, VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_QUERY_PROTOCOL_VERSION");
if (ret) {
dev_err(&hdev->device,
@@ -3476,7 +3501,8 @@ static int hv_pci_enter_d0(struct hv_device *hdev)
(unsigned long)pkt, VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_BUS_D0ENTRY");
if (ret)
goto exit;
@@ -3553,7 +3579,8 @@ static int hv_pci_query_relations(struct hv_device *hdev)
ret = vmbus_sendpacket(hdev->channel, &message, sizeof(message),
0, VM_PKT_DATA_INBAND, 0);
if (!ret)
- ret = wait_for_response(hdev, &comp);
+ ret = wait_for_response(hdev, &comp,
+ "PCI_QUERY_BUS_RELATIONS");
/*
* In the case of fast device addition/removal, it's possible that
@@ -3644,7 +3671,8 @@ static int hv_send_resources_allocated(struct hv_device *hdev)
VM_PKT_DATA_INBAND,
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
if (!ret)
- ret = wait_for_response(hdev, &comp_pkt.host_event);
+ ret = wait_for_response(hdev, &comp_pkt.host_event,
+ "PCI_RESOURCE_ASSIGNED");
if (ret)
break;
--
2.53.0
next reply other threads:[~2026-09-02 11:59 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-02 11:58 Sahil Chandna [this message]
2026-09-02 12:09 ` [PATCH v2] PCI: hv: warn when wait_for_response() waits indefinitely sashiko-bot
2026-09-03 9:01 ` Naman Jain
2026-09-03 9:25 ` Naman Jain
2026-09-03 16:48 ` Michael Kelley
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260902115854.2629164-1-sahilchandna@linux.microsoft.com \
--to=sahilchandna@linux.microsoft.com \
--cc=bhelgaas@google.com \
--cc=decui@microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=hamzamahfooz@linux.microsoft.com \
--cc=kwilczynski@kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=longli@microsoft.com \
--cc=lpieralisi@kernel.org \
--cc=mani@kernel.org \
--cc=mhklinux@outlook.com \
--cc=namjain@linux.microsoft.com \
--cc=robh@kernel.org \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox