[PATCH] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[PATCH] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

If there's a persistent error in the hypervisor, the scsi warning for
failed IO can flood the kernel log and max out CPU utilization,
preventing troubleshooting from the VM side. Ratelimit the warning so
it doesn't DOS the VM.

Closes: https://github.com/microsoft/WSL/issues/9173
Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
---
If there's a persistent error in the hypervisor, the scsi warning for
failed IO can overwhelm the kernel log and max out CPU utilization,
preventing troubleshooting. Ratelimit the warning so it doesn't DOS the
VM.

This is not super critical and can be deferred to an 6.14 rc since it
mostly occurs when there's a problem with the host disk or filesystem.
---
 drivers/scsi/storvsc_drv.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index 45665d4aca925a10ce4293edf40cebdc4d4997f2..5a101ac06c478aad9b01072e7a6c5af48219f5c8 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -171,6 +171,12 @@ do {								\
 		dev_warn(&(dev)->device, fmt, ##__VA_ARGS__);	\
 } while (0)
 
+#define storvsc_log_ratelimited(dev, level, fmt, ...)				\
+do {										\
+	if (do_logging(level))							\
+		dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__);	\
+} while (0)
+
 struct vmscsi_request {
 	u16 length;
 	u8 srb_status;
@@ -1176,7 +1182,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device,
 		int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ?
 			STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR;
 
-		storvsc_log(device, loglevel,
+		storvsc_log_ratelimited(device, loglevel,
 			"tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n",
 			scsi_cmd_to_rq(request->cmd)->tag,
 			stor_pkt->vm_srb.cdb[0],

---
base-commit: 4e16367cfe0ce395f29d0482b78970cce8e1db73
change-id: 20241023-eahariha-ratelimit-storvsc-b3fe53f02a58

Best regards,
-- 
Easwar Hariharan <eahariha@linux.microsoft.com>

RE: [PATCH] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Michael Kelley]

From: Easwar Hariharan <eahariha@linux.microsoft.com> Sent: Tuesday, January 7, 2025 9:29 AM
> 
> If there's a persistent error in the hypervisor, the scsi warning for
> failed IO can flood the kernel log and max out CPU utilization,
> preventing troubleshooting from the VM side. Ratelimit the warning so
> it doesn't DOS the VM.
> 
> Closes:
> https://github.com/microsoft/WSL/issues/9173
> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> ---
> If there's a persistent error in the hypervisor, the scsi warning for
> failed IO can overwhelm the kernel log and max out CPU utilization,
> preventing troubleshooting. Ratelimit the warning so it doesn't DOS the
> VM.
> 
> This is not super critical and can be deferred to an 6.14 rc since it
> mostly occurs when there's a problem with the host disk or filesystem.
> ---
>  drivers/scsi/storvsc_drv.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> index
> 45665d4aca925a10ce4293edf40cebdc4d4997f2..5a101ac06c478aad9b01072e7a6c
> 5af48219f5c8 100644
> --- a/drivers/scsi/storvsc_drv.c
> +++ b/drivers/scsi/storvsc_drv.c
> @@ -171,6 +171,12 @@ do {
> 	\
>  		dev_warn(&(dev)->device, fmt, ##__VA_ARGS__);	\
>  } while (0)
> 
> +#define storvsc_log_ratelimited(dev, level, fmt, ...)				\
> +do {										\
> +	if (do_logging(level))							\
> +		dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__);	\
> +} while (0)
> +
>  struct vmscsi_request {
>  	u16 length;
>  	u8 srb_status;
> @@ -1176,7 +1182,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device,
>  		int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ?
>  			STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR;
> 
> -		storvsc_log(device, loglevel,
> +		storvsc_log_ratelimited(device, loglevel,
>  			"tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n",
>  			scsi_cmd_to_rq(request->cmd)->tag,
>  			stor_pkt->vm_srb.cdb[0],
> 
> ---
> base-commit: 4e16367cfe0ce395f29d0482b78970cce8e1db73
> change-id: 20241023-eahariha-ratelimit-storvsc-b3fe53f02a58
> 
> Best regards,
> --
> Easwar Hariharan <eahariha@linux.microsoft.com>
> 

Reviewed-by: Michael Kelley <mhklinux@outlook.com>

Re: [PATCH] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Martin K. Petersen]

Easwar,

> If there's a persistent error in the hypervisor, the scsi warning for
> failed IO can flood the kernel log and max out CPU utilization,
> preventing troubleshooting from the VM side. Ratelimit the warning so
> it doesn't DOS the VM.

Applied to 6.14/scsi-staging, thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering