[PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream

If there's a persistent error in the hypervisor, the SCSI warning for
failed I/O can flood the kernel log and max out CPU utilization,
preventing troubleshooting from the VM side. Ratelimit the warning so
it doesn't DoS the VM.

Closes: https://github.com/microsoft/WSL/issues/9173
Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
---
 drivers/scsi/storvsc_drv.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index d0b55c1fa908..b3c588b102d9 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -171,6 +171,12 @@ do {								\
 		dev_warn(&(dev)->device, fmt, ##__VA_ARGS__);	\
 } while (0)
 
+#define storvsc_log_ratelimited(dev, level, fmt, ...)				\
+do {										\
+	if (do_logging(level))							\
+		dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__);	\
+} while (0)
+
 struct vmscsi_request {
 	u16 length;
 	u8 srb_status;
@@ -1177,7 +1183,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device,
 		int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ?
 			STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR;
 
-		storvsc_log(device, loglevel,
+		storvsc_log_ratelimited(device, loglevel,
 			"tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n",
 			scsi_cmd_to_rq(request->cmd)->tag,
 			stor_pkt->vm_srb.cdb[0],
-- 
2.43.0

Re: [PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Greg KH]

On Mon, Jan 27, 2025 at 06:29:54PM +0000, Easwar Hariharan wrote:
> commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
> 
> If there's a persistent error in the hypervisor, the SCSI warning for
> failed I/O can flood the kernel log and max out CPU utilization,
> preventing troubleshooting from the VM side. Ratelimit the warning so
> it doesn't DoS the VM.
> 
> Closes: https://github.com/microsoft/WSL/issues/9173
> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> ---
>  drivers/scsi/storvsc_drv.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)

What about 6.13.y?

Re: [PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Sasha Levin]

[ Sasha's backport helper bot ]

Hi,

The upstream commit SHA1 provided is correct: d2138eab8cde61e0e6f62d0713e45202e8457d6d


Status in newer kernel trees:
6.13.y | Branch not found
6.12.y | Not found

Note: The patch differs from the upstream commit:
---
1:  d2138eab8cde6 ! 1:  6ee86d3429520 scsi: storvsc: Ratelimit warning logs to prevent VM denial of service
    @@ Metadata
      ## Commit message ##
         scsi: storvsc: Ratelimit warning logs to prevent VM denial of service
     
    +    commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
    +
         If there's a persistent error in the hypervisor, the SCSI warning for
         failed I/O can flood the kernel log and max out CPU utilization,
         preventing troubleshooting from the VM side. Ratelimit the warning so
    @@ Commit message
         Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
         Reviewed-by: Michael Kelley <mhklinux@outlook.com>
         Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
    +    Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
     
      ## drivers/scsi/storvsc_drv.c ##
     @@ drivers/scsi/storvsc_drv.c: do {								\
---

Results of testing on various branches:

| Branch                    | Patch Apply | Build Test |
|---------------------------|-------------|------------|
| stable/linux-6.12.y       |  Success    |  Success   |

Re: [PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

On 1/27/2025 9:35 PM, Greg KH wrote:
> On Mon, Jan 27, 2025 at 06:29:54PM +0000, Easwar Hariharan wrote:
>> commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
>>
>> If there's a persistent error in the hypervisor, the SCSI warning for
>> failed I/O can flood the kernel log and max out CPU utilization,
>> preventing troubleshooting from the VM side. Ratelimit the warning so
>> it doesn't DoS the VM.
>>
>> Closes: https://github.com/microsoft/WSL/issues/9173
>> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
>> Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
>> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
>> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
>> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
>> ---
>>  drivers/scsi/storvsc_drv.c | 8 +++++++-
>>  1 file changed, 7 insertions(+), 1 deletion(-)
> 
> What about 6.13.y?
> 

Yes, needs a backport to that too, another miss on my part. :(

Would you rather I send it after rc1 is tagged, or now for completeness?

- Easwar (he/him)

Re: [PATCH 6.12.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Greg KH]

On Tue, Jan 28, 2025 at 09:08:42AM -0800, Easwar Hariharan wrote:
> On 1/27/2025 9:35 PM, Greg KH wrote:
> > On Mon, Jan 27, 2025 at 06:29:54PM +0000, Easwar Hariharan wrote:
> >> commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
> >>
> >> If there's a persistent error in the hypervisor, the SCSI warning for
> >> failed I/O can flood the kernel log and max out CPU utilization,
> >> preventing troubleshooting from the VM side. Ratelimit the warning so
> >> it doesn't DoS the VM.
> >>
> >> Closes: https://github.com/microsoft/WSL/issues/9173
> >> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> >> Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
> >> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> >> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
> >> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> >> ---
> >>  drivers/scsi/storvsc_drv.c | 8 +++++++-
> >>  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> > What about 6.13.y?
> > 
> 
> Yes, needs a backport to that too, another miss on my part. :(
> 
> Would you rather I send it after rc1 is tagged, or now for completeness?

Whenever you have it, we can't apply any of these until -rc1 is out.

thanks,

greg k-h