[PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream

If there's a persistent error in the hypervisor, the SCSI warning for
failed I/O can flood the kernel log and max out CPU utilization,
preventing troubleshooting from the VM side. Ratelimit the warning so
it doesn't DoS the VM.

Closes: https://github.com/microsoft/WSL/issues/9173
Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
---
 drivers/scsi/storvsc_drv.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index d0b55c1fa908..b3c588b102d9 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -171,6 +171,12 @@ do {								\
 		dev_warn(&(dev)->device, fmt, ##__VA_ARGS__);	\
 } while (0)
 
+#define storvsc_log_ratelimited(dev, level, fmt, ...)				\
+do {										\
+	if (do_logging(level))							\
+		dev_warn_ratelimited(&(dev)->device, fmt, ##__VA_ARGS__);	\
+} while (0)
+
 struct vmscsi_request {
 	u16 length;
 	u8 srb_status;
@@ -1177,7 +1183,7 @@ static void storvsc_on_io_completion(struct storvsc_device *stor_device,
 		int loglevel = (stor_pkt->vm_srb.cdb[0] == TEST_UNIT_READY) ?
 			STORVSC_LOGGING_WARN : STORVSC_LOGGING_ERROR;
 
-		storvsc_log(device, loglevel,
+		storvsc_log_ratelimited(device, loglevel,
 			"tag#%d cmd 0x%x status: scsi 0x%x srb 0x%x hv 0x%x\n",
 			scsi_cmd_to_rq(request->cmd)->tag,
 			stor_pkt->vm_srb.cdb[0],
-- 
2.43.0

Re: [PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

On 1/27/2025 10:29 AM, Easwar Hariharan wrote:
> commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
> 
> If there's a persistent error in the hypervisor, the SCSI warning for
> failed I/O can flood the kernel log and max out CPU utilization,
> preventing troubleshooting from the VM side. Ratelimit the warning so
> it doesn't DoS the VM.
> 
> Closes: https://github.com/microsoft/WSL/issues/9173
> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> ---
>  drivers/scsi/storvsc_drv.c | 8 +++++++-
>  1 file changed, 7 insertions(+), 1 deletion(-)
> 

I just remembered that we should wait for Linus to tag the rc before
sending backports, so apologies for sending this (and its 6.1 and 6.12
friends) out before rc1 was tagged.

- Easwar (he/him)

Re: [PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Greg KH]

On Mon, Jan 27, 2025 at 11:25:17AM -0800, Easwar Hariharan wrote:
> On 1/27/2025 10:29 AM, Easwar Hariharan wrote:
> > commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
> > 
> > If there's a persistent error in the hypervisor, the SCSI warning for
> > failed I/O can flood the kernel log and max out CPU utilization,
> > preventing troubleshooting from the VM side. Ratelimit the warning so
> > it doesn't DoS the VM.
> > 
> > Closes: https://github.com/microsoft/WSL/issues/9173
> > Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> > Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
> > Reviewed-by: Michael Kelley <mhklinux@outlook.com>
> > Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
> > Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
> > ---
> >  drivers/scsi/storvsc_drv.c | 8 +++++++-
> >  1 file changed, 7 insertions(+), 1 deletion(-)
> > 
> 
> I just remembered that we should wait for Linus to tag the rc before
> sending backports, so apologies for sending this (and its 6.1 and 6.12
> friends) out before rc1 was tagged.

Why was this not tagged for stable in the first place?

thanks,

greg k-h

Re: [PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Sasha Levin]

[ Sasha's backport helper bot ]

Hi,

The upstream commit SHA1 provided is correct: d2138eab8cde61e0e6f62d0713e45202e8457d6d


Status in newer kernel trees:
6.13.y | Branch not found
6.12.y | Not found
6.6.y | Not found

Note: The patch differs from the upstream commit:
---
1:  d2138eab8cde6 ! 1:  eff28b745a790 scsi: storvsc: Ratelimit warning logs to prevent VM denial of service
    @@ Metadata
      ## Commit message ##
         scsi: storvsc: Ratelimit warning logs to prevent VM denial of service
     
    +    commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
    +
         If there's a persistent error in the hypervisor, the SCSI warning for
         failed I/O can flood the kernel log and max out CPU utilization,
         preventing troubleshooting from the VM side. Ratelimit the warning so
    @@ Commit message
         Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
         Reviewed-by: Michael Kelley <mhklinux@outlook.com>
         Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
    +    Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
     
      ## drivers/scsi/storvsc_drv.c ##
     @@ drivers/scsi/storvsc_drv.c: do {								\
---

Results of testing on various branches:

| Branch                    | Patch Apply | Build Test |
|---------------------------|-------------|------------|
| stable/linux-6.6.y        |  Success    |  Success   |

Re: [PATCH 6.6.y] scsi: storvsc: Ratelimit warning logs to prevent VM denial of service

[Easwar Hariharan]

On 1/27/2025 9:34 PM, Greg KH wrote:
> On Mon, Jan 27, 2025 at 11:25:17AM -0800, Easwar Hariharan wrote:
>> On 1/27/2025 10:29 AM, Easwar Hariharan wrote:
>>> commit d2138eab8cde61e0e6f62d0713e45202e8457d6d upstream
>>>
>>> If there's a persistent error in the hypervisor, the SCSI warning for
>>> failed I/O can flood the kernel log and max out CPU utilization,
>>> preventing troubleshooting from the VM side. Ratelimit the warning so
>>> it doesn't DoS the VM.
>>>
>>> Closes: https://github.com/microsoft/WSL/issues/9173
>>> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
>>> Link: https://lore.kernel.org/r/20250107-eahariha-ratelimit-storvsc-v1-1-7fc193d1f2b0@linux.microsoft.com
>>> Reviewed-by: Michael Kelley <mhklinux@outlook.com>
>>> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
>>> Signed-off-by: Easwar Hariharan <eahariha@linux.microsoft.com>
>>> ---
>>>  drivers/scsi/storvsc_drv.c | 8 +++++++-
>>>  1 file changed, 7 insertions(+), 1 deletion(-)
>>>
>>
>> I just remembered that we should wait for Linus to tag the rc before
>> sending backports, so apologies for sending this (and its 6.1 and 6.12
>> friends) out before rc1 was tagged.
> 
> Why was this not tagged for stable in the first place?
> 
> thanks,
> 
> greg k-h

An oversight when I sent the original patch.

- Easwar (he/him)