[PATCH] staging: rtl8712: reduce stack usage

[PATCH] staging: rtl8712: reduce stack usage

[Arnd Bergmann]

The "translate_scan" function in rtl8712 uses a lot of stack, and
gets inlined into its single caller, r8711_wx_get_scan, which
in some configurations now blows the 1024 byte stack warning
limit:

drivers/staging/rtl8712/rtl871x_ioctl_linux.c: In function 'r8711_wx_get_scan':
drivers/staging/rtl8712/rtl871x_ioctl_linux.c:1227:1: error: the frame size of 1032 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]

This somewhat reduces the stack usage by moving the translate_scan
function out of line with the noinline_for_stack annotation.
It might be possible to modify translate_scan() a little further
to reduce the stack usage, but with this patch, we can build without
the warning, the the call chain to get here is rather predictable
(sys_ioctl->vfs_ioctl->sock_ioctl->dev_ioctl->wext_ioctl->
r8711_wx_get_scan).

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
 drivers/staging/rtl8712/rtl871x_ioctl_linux.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
index db2e31bcdd77..a15f3ce70223 100644
--- a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
+++ b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
@@ -137,7 +137,7 @@ static inline void handle_group_key(struct ieee_param *param,
 	}
 }
 
-static inline char *translate_scan(struct _adapter *padapter,
+static noinline_for_stack char *translate_scan(struct _adapter *padapter,
 				   struct iw_request_info *info,
 				   struct wlan_network *pnetwork,
 				   char *start, char *stop)
-- 
2.7.0

[PATCH] staging: rtl8712: reduce stack usage

[Larry Finger]

On 02/17/2016 02:32 AM, Arnd Bergmann wrote:
> The "translate_scan" function in rtl8712 uses a lot of stack, and
> gets inlined into its single caller, r8711_wx_get_scan, which
> in some configurations now blows the 1024 byte stack warning
> limit:
>
> drivers/staging/rtl8712/rtl871x_ioctl_linux.c: In function 'r8711_wx_get_scan':
> drivers/staging/rtl8712/rtl871x_ioctl_linux.c:1227:1: error: the frame size of 1032 bytes is larger than 1024 bytes [-Werror=frame-larger-than=]
>
> This somewhat reduces the stack usage by moving the translate_scan
> function out of line with the noinline_for_stack annotation.
> It might be possible to modify translate_scan() a little further
> to reduce the stack usage, but with this patch, we can build without
> the warning, the the call chain to get here is rather predictable
> (sys_ioctl->vfs_ioctl->sock_ioctl->dev_ioctl->wext_ioctl->
> r8711_wx_get_scan).
>
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
>   drivers/staging/rtl8712/rtl871x_ioctl_linux.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)

As there is no reason for translate_scan() to be inlined, this patch seems 
reasonable The largest user of stack likely comes from the line "struct iw_event 
iwe". Changing that to a pointer, and using kalloc to acquire the space would 
likely clear the compile message, but that would require many more changes.

Acked-by: Larry Finger <Larry.Finger@lwfinger.net>

Larry

>
> diff --git a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> index db2e31bcdd77..a15f3ce70223 100644
> --- a/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> +++ b/drivers/staging/rtl8712/rtl871x_ioctl_linux.c
> @@ -137,7 +137,7 @@ static inline void handle_group_key(struct ieee_param *param,
>   	}
>   }
>
> -static inline char *translate_scan(struct _adapter *padapter,
> +static noinline_for_stack char *translate_scan(struct _adapter *padapter,
>   				   struct iw_request_info *info,
>   				   struct wlan_network *pnetwork,
>   				   char *start, char *stop)
>

[PATCH] staging: rtl8712: reduce stack usage

[Arnd Bergmann]

On Wednesday 17 February 2016 20:20:15 Larry Finger wrote:
> > ---
> >   drivers/staging/rtl8712/rtl871x_ioctl_linux.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> As there is no reason for translate_scan() to be inlined, this patch seems 
> reasonable The largest user of stack likely comes from the line "struct iw_event 
> iwe". Changing that to a pointer, and using kalloc to acquire the space would 
> likely clear the compile message, but that would require many more changes.
> 
> Acked-by: Larry Finger <Larry.Finger@lwfinger.net>
> 
> 

Thanks!

I believe that the problem is the combination of "struct iw_event"
and the two buffers "u8 wpa_ie[255], rsn_ie[255];" and "u8 wps_ie[512];".
Just to document this better: if we ever decide to use a dynamic allocation,
I would allocate all of the above together.

	Arnd

[PATCH] staging: rtl8712: reduce stack usage

[Dan Carpenter]

Of course, we still reach almost the same maximum stack usage as before
because we're going to put the function on stack in the end anyway.

regards,
dan carpenter