The Linux Kernel Mailing List
 help / color / mirror / Atom feed
From: Naman Jain <namjain@linux.microsoft.com>
To: Sahil Chandna <sahilchandna@linux.microsoft.com>,
	kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
	decui@microsoft.com, longli@microsoft.com, lpieralisi@kernel.org,
	kwilczynski@kernel.org, mani@kernel.org, robh@kernel.org,
	bhelgaas@google.com, linux-hyperv@vger.kernel.org,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] PCI: hv: Warn when wait_for_response() waits indefinitely
Date: Tue, 25 Aug 2026 12:16:06 +0530	[thread overview]
Message-ID: <736091cf-8d00-415e-a890-e2dfa6739325@linux.microsoft.com> (raw)
In-Reply-To: <20260825051850.2438816-1-sahilchandna@linux.microsoft.com>



On 8/25/2026 10:47 AM, Sahil Chandna wrote:
> A guest can wait indefinitely in wait_for_response() for the host to
> send either a rescind message or a packet completion. If the
> host does not send either, the guest can remain blocked with no
> diagnostic indicating a reason.
> This was observed during a guest kernel upgrade in which the
> host-side application handling the PCI channel faulted, causing the
> guest to never receive the completion request.
> Add a periodic warning in wait_for_response() when the wait exceeds
> a timeout so that such a hang is visible in the guest's kernel log
> and can be correlated with host-side state.
> 
> Suggested-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
> Signed-off-by: Sahil Chandna <sahilchandna@linux.microsoft.com>
> ---
> This was sent earlier upstream [1]
> [1] https://lore.kernel.org/linux-hyperv/20260612174010.2598695-1-hamzamahfooz@linux.microsoft.com/
> ---
>   drivers/pci/controller/pci-hyperv.c | 14 +++++++++++++-
>   1 file changed, 13 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c
> index cfc8fa403dad..c4fba0039164 100644
> --- a/drivers/pci/controller/pci-hyperv.c
> +++ b/drivers/pci/controller/pci-hyperv.c
> @@ -1040,11 +1040,18 @@ static void put_pcichild(struct hv_pci_dev *hpdev)
> 
>   /*
>    * There is no good way to get notified from vmbus_onoffer_rescind(),
> - * so let's use polling here, since this is not a hot path.
> + * so let's use polling here, since this is not a hot path. If
> + * wait_for_response() has been polling for PCI_RESPONSE_HANG_TIMEOUT_SEC
> + * without either a rescind or completion, add a periodic warning.
>    */
> +#define PCI_RESPONSE_HANG_TIMEOUT_SEC 300
> +
>   static int wait_for_response(struct hv_device *hdev,
>   			     struct completion *comp)
>   {
> +	unsigned long delay = secs_to_jiffies(PCI_RESPONSE_HANG_TIMEOUT_SEC);
> +	u64 timeout = get_jiffies_64() + delay;
> +
>   	while (true) {
>   		if (hdev->channel->rescind) {
>   			dev_warn_once(&hdev->device, "The device is gone.\n");
> @@ -1053,6 +1060,11 @@ static int wait_for_response(struct hv_device *hdev,
> 
>   		if (wait_for_completion_timeout(comp, HZ / 10))
>   			break;
> +
> +		if (time_after64(get_jiffies_64(), timeout)) {
> +			dev_warn(&hdev->device, "PCI stuck waiting for response.\n");
> +			timeout = get_jiffies_64() + delay;
> +		}
>   	}

There can be some enhancements in above patch to address these problems:
1. Logging forever every 5 minutes in case of no completion or rescind.
2. If we now print warning once, not knowing if completion ever arrived.
3. On solving pt. 1 and 2 by adding a print for completion, one should 
avoid adding a print by default for regular timely completions.

Basically something like this:

#define PCI_RESPONSE_WARN_TIMEOUT_SEC 300

static int wait_for_response(struct hv_device *hdev,
                  struct completion *comp)
{
     unsigned long warn_at =
         jiffies + secs_to_jiffies(PCI_RESPONSE_WARN_TIMEOUT_SEC);
     bool warned = false;

     while (true) {
         if (hdev->channel->rescind) {
             dev_warn_once(&hdev->device, "The device is gone.\n");
             return -ENODEV;
         }

         if (wait_for_completion_timeout(comp, HZ / 10)) {
             if (warned || time_after_eq(jiffies, warn_at))
                 dev_warn(&hdev->device,
                      "PCI response received after prolonged wait.\n");
             return 0;
         }

         if (!warned && time_after_eq(jiffies, warn_at)) {
             dev_warn(&hdev->device,
                  "PCI still waiting for response.\n");
             warned = true;
         }
     }
}

Regards,
Naman

  reply	other threads:[~2026-08-25  6:46 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-25  5:17 [PATCH] PCI: hv: Warn when wait_for_response() waits indefinitely Sahil Chandna
2026-08-25  6:46 ` Naman Jain [this message]
2026-08-25 12:00   ` Sahil Chandna
2026-08-25 17:00   ` Long Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=736091cf-8d00-415e-a890-e2dfa6739325@linux.microsoft.com \
    --to=namjain@linux.microsoft.com \
    --cc=bhelgaas@google.com \
    --cc=decui@microsoft.com \
    --cc=haiyangz@microsoft.com \
    --cc=kwilczynski@kernel.org \
    --cc=kys@microsoft.com \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=longli@microsoft.com \
    --cc=lpieralisi@kernel.org \
    --cc=mani@kernel.org \
    --cc=robh@kernel.org \
    --cc=sahilchandna@linux.microsoft.com \
    --cc=wei.liu@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox