[PATCH] mm/vmscan: make do_shrink_slab more robust.

[PATCH] mm/vmscan: make do_shrink_slab more robust.

[Jiang Biao]

When running ltp stress test for 7*24 hours, the kernel occasionally
complains the following warning continuously,

mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222526086287711848
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222420761333860545
mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
nr=-9222287677544280360
...

The tracing result shows the freeable(mb_cache_shrink_scan returns)
is -1, which causes the continuous accumulation and overflow of
total_scan.

This patch make do_shrink_slab more robust when
shrinker->count_objects return negative freeable.

Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
---
 mm/vmscan.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/vmscan.c b/mm/vmscan.c
index eb2f031..3ea28f0 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
 	long scanned = 0, next_deferred;
 
 	freeable = shrinker->count_objects(shrinker, shrinkctl);
-	if (freeable == 0)
+	if (freeable <= 0)
 		return 0;
 
 	/*
-- 
2.7.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[Minchan Kim]

Hello,

On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:
> When running ltp stress test for 7*24 hours, the kernel occasionally
> complains the following warning continuously,
> 
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222526086287711848
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222420761333860545
> mb_cache_shrink_scan+0x0/0x3f0 negative objects to delete
> nr=-9222287677544280360
> ...
> 
> The tracing result shows the freeable(mb_cache_shrink_scan returns)
> is -1, which causes the continuous accumulation and overflow of
> total_scan.

Good catch.

> 
> This patch make do_shrink_slab more robust when
> shrinker->count_objects return negative freeable.

Shrinker.h says count_objects should return 0 if there are no
freeable objects, not -1.

So if something returns -1, changing it with returning 0 would
be more proper fix.

Thanks.


> 
> Signed-off-by: Jiang Biao <jiang.biao2@zte.com.cn>
> ---
>  mm/vmscan.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/vmscan.c b/mm/vmscan.c
> index eb2f031..3ea28f0 100644
> --- a/mm/vmscan.c
> +++ b/mm/vmscan.c
> @@ -323,7 +323,7 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl,
>  	long scanned = 0, next_deferred;
>  
>  	freeable = shrinker->count_objects(shrinker, shrinkctl);
> -	if (freeable == 0)
> +	if (freeable <= 0)
>  		return 0;
>  
>  	/*
> -- 
> 2.7.4
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[jiang.biao2]

[-- Attachment #1.1: Type: text/plain, Size: 666 bytes --]

On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > This patch make do_shrink_slab more robust when
> > shrinker->count_objects return negative freeable.
> 
> Shrinker.h says count_objects should return 0 if there are no
> freeable objects, not -1.
> 
> So if something returns -1, changing it with returning 0 would
> be more proper fix.
> 
Hi,
Indeed it's not a bug of vmscan, but there are many shrinkers
out there, which may return negative value unwillingly(in some 
rare cases, such as decreasing cocurrently). It's unlikely and 
should be avioded, but not impossible, this patch may make it 
more robust and could not hurt :).
Thanks.

Regards,

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[Minchan Kim]

On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > This patch make do_shrink_slab more robust when
> > > shrinker->count_objects return negative freeable.
> > 
> > Shrinker.h says count_objects should return 0 if there are no
> > freeable objects, not -1.
> > 
> > So if something returns -1, changing it with returning 0 would
> > be more proper fix.
> > 
> Hi,
> Indeed it's not a bug of vmscan, but there are many shrinkers
> out there, which may return negative value unwillingly(in some 
> rare cases, such as decreasing cocurrently). It's unlikely and 
> should be avioded, but not impossible, this patch may make it 
> more robust and could not hurt :).

Yub, I'm not strong against of your claim. However, let's think
from different point of view.

API says it should return 0 unless shrinker cannot find freeable
object any more but with your change, implmentation handles
although a shrinker return minus value by mistake.

In future, MM guys might want to extend count_objects returning
-ERR_SOMETHING to propagate error, for example but we cannot.
Because some of shrinkers already rely on the implementation so
if we start to support minus value return, some of shrinker might
be broken.

Yes, it's the imaginary scenario but wanted why such changes
makes us trouble in future, API PoV.

Other way is you can change the description so that count_scan
API can return any value if it's less or equal to zero but not
sure it's worth. Anyway, maintainer will judge but my opinion
is not worth to do at this moment. We have been happy for a
long time without that.

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[jiang.biao2]

[-- Attachment #1.1: Type: text/plain, Size: 2048 bytes --]

> On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:> > On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > > This patch make do_shrink_slab more robust when
> > > > shrinker->count_objects return negative freeable.
> > >
> > > Shrinker.h says count_objects should return 0 if there are no
> > > freeable objects, not -1.
> > >
> > > So if something returns -1, changing it with returning 0 would
> > be more proper fix.
> > >
> > Hi,
> > Indeed it's not a bug of vmscan, but there are many shrinkers
> > out there, which may return negative value unwillingly(in some
> > rare cases, such as decreasing cocurrently). It's unlikely and
> > should be avioded, but not impossible, this patch may make it
> > more robust and could not hurt :).
> 
> Yub, I'm not strong against of your claim. However, let's think
> from different point of view.
> 
> API says it should return 0 unless shrinker cannot find freeable
> object any more but with your change, implmentation handles
> although a shrinker return minus value by mistake.
> 
> In future, MM guys might want to extend count_objects returning
> -ERR_SOMETHING to propagate error, for example but we cannot.
> Because some of shrinkers already rely on the implementation so
> if we start to support minus value return, some of shrinker might
> be broken.
> 
> Yes, it's the imaginary scenario but wanted why such changes
> makes us trouble in future, API PoV.
I agree with your concern.  How about we take another way by 
adding some warning in such case? such as,
        freeable = shrinker->count_objects(shrinker, shrinkctl);
+       if (unlikely(freeable < 0)) {
+               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
+                       shrinker->scan_objects, freeable);
+               freeable = 0;  //maybe not needed?
+       }
        if (freeable == 0)
                return 0;
In this way, we would not break the API, but could alert user exception 
with message, and make it more robust in such case.

Thanks.

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[Minchan Kim]

On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:
> > On Mon, Nov 27, 2017 at 12:46:27PM +0800, jiang.biao2@zte.com.cn wrote:> > On Mon, Nov 27, 2017 at 09:37:30AM +0800, Jiang Biao wrote:> >
> > > > > This patch make do_shrink_slab more robust when
> > > > > shrinker->count_objects return negative freeable.
> > > >
> > > > Shrinker.h says count_objects should return 0 if there are no
> > > > freeable objects, not -1.
> > > >
> > > > So if something returns -1, changing it with returning 0 would
> > > be more proper fix.
> > > >
> > > Hi,
> > > Indeed it's not a bug of vmscan, but there are many shrinkers
> > > out there, which may return negative value unwillingly(in some
> > > rare cases, such as decreasing cocurrently). It's unlikely and
> > > should be avioded, but not impossible, this patch may make it
> > > more robust and could not hurt :).
> > 
> > Yub, I'm not strong against of your claim. However, let's think
> > from different point of view.
> > 
> > API says it should return 0 unless shrinker cannot find freeable
> > object any more but with your change, implmentation handles
> > although a shrinker return minus value by mistake.
> > 
> > In future, MM guys might want to extend count_objects returning
> > -ERR_SOMETHING to propagate error, for example but we cannot.
> > Because some of shrinkers already rely on the implementation so
> > if we start to support minus value return, some of shrinker might
> > be broken.
> > 
> > Yes, it's the imaginary scenario but wanted why such changes
> > makes us trouble in future, API PoV.
> I agree with your concern.  How about we take another way by 
> adding some warning in such case? such as,
>         freeable = shrinker->count_objects(shrinker, shrinkctl);
> +       if (unlikely(freeable < 0)) {
> +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> +                       shrinker->scan_objects, freeable);
> +               freeable = 0;  //maybe not needed?
> +       }
>         if (freeable == 0)
>                 return 0;
> In this way, we would not break the API, but could alert user exception 
> with message, and make it more robust in such case.

True but it would be a problem robust vs. effectivess tradeoff.
Think about that everyone want to make thier code robust.
It means they start to dump lots of defensive code so code start
to look like complicated as well as binary bloating.
So, whenever we add some more, we should think how effective
the code I am putting?

In this case, I'm skeptical, Sorry. But others might have different
opinions. :)

Thanks.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[jiang.biao2]

[-- Attachment #1.1: Type: text/plain, Size: 1680 bytes --]

On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:> > I agree with your concern.  How about we take another way by
> > adding some warning in such case? such as,
> >         freeable = shrinker->count_objects(shrinker, shrinkctl);
> > +       if (unlikely(freeable < 0)) {
> > +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> > +                       shrinker->scan_objects, freeable);
> > +               freeable = 0;  //maybe not needed?
> > +       }
> >         if (freeable == 0)
> >                 return 0;
> > In this way, we would not break the API, but could alert user exception
> > with message, and make it more robust in such case.
>
> True but it would be a problem robust vs. effectivess tradeoff.
> Think about that everyone want to make thier code robust.
> It means they start to dump lots of defensive code so code start
> to look like complicated as well as binary bloating.
> So, whenever we add some more, we should think how effective
> the code I am putting?
> 
> In this case, I'm skeptical, Sorry. But others might have different
> opinions. :)

With all due respect. I still think the robustness is more important than 
effectiveness in this case. :)
First, the minus value could break all the following procedure in 
do_shrink_slab which may lead to long time looping on 
*tight on memory* situations.
Second, there is *unlikely* macro to minimize the cost.
Third, there is already similar procedure for *total_scan < 0*.
Last, it did happen in my scenario and maybe happen in other 
unexpected situations.

Just my personal option, thanks a lot for you reply and explanation. :)

Regards,

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[Michal Hocko]

On Mon 27-11-17 15:26:54, jiang.biao2@zte.com.cn wrote:
> On Mon, Nov 27, 2017 at 02:27:20PM +0800, jiang.biao2@zte.com.cn wrote:> > I agree with your concern.  How about we take another way by
> > > adding some warning in such case? such as,
> > >         freeable = shrinker->count_objects(shrinker, shrinkctl);
> > > +       if (unlikely(freeable < 0)) {
> > > +               pr_err("shrink_slab: %pF negative objects returned. freeable=%ld\n",
> > > +                       shrinker->scan_objects, freeable);
> > > +               freeable = 0;  //maybe not needed?
> > > +       }
> > >         if (freeable == 0)
> > >                 return 0;
> > > In this way, we would not break the API, but could alert user exception
> > > with message, and make it more robust in such case.
> >
> > True but it would be a problem robust vs. effectivess tradeoff.
> > Think about that everyone want to make thier code robust.
> > It means they start to dump lots of defensive code so code start
> > to look like complicated as well as binary bloating.
> > So, whenever we add some more, we should think how effective
> > the code I am putting?
> > 
> > In this case, I'm skeptical, Sorry. But others might have different
> > opinions. :)
> 
> With all due respect. I still think the robustness is more important than 
> effectiveness in this case. :)

This is a slow path so I wouldn't worry about the performance much. On
the other hand I agree that the API is well documented so adding a
warning is too defensive. We simply assume that the kernel running in
the kernel is reasonable. So I would say, fix your code.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] mm/vmscan: make do_shrink_slab more robust.

[jiang.biao2]

[-- Attachment #1.1: Type: text/plain, Size: 565 bytes --]

On Mon 27-11-17 15:26:54, jiang.biao2@zte.com.cn wrote:
> >> > With all due respect. I still think the robustness is more important than
> > effectiveness in this case. :)
> 
> This is a slow path so I wouldn't worry about the performance much. On
> the other hand I agree that the API is well documented so adding a
> warning is too defensive. We simply assume that the kernel running in
> the kernel is reasonable. So I would say, fix your code.
Ok, already send another patch for mbache shrinker to ensure non-minus 
return.
Thank you for your reply :)

Regards,