[PATCH] iscsi: fall back to sendmsg for slab pages

[PATCH] iscsi: fall back to sendmsg for slab pages

[Vasily Averin]

In "XFS over network block device" scenario XFS can create IO requests
with slab-based XFS metadata. During processing such requests
tcp_sendpage() can merge skb fragments with neighbour slab objects.

If receiving side is located on the same host tcp_recvmsg() can trigger
BUG_ON in hardening check and crash the host with following message:

usercopy: kernel memory exposure attempt detected
		from XXXXXXXX (kmalloc-512) (1024 bytes)

This patch redirect such requests from sednpage to sendmsg path.
The problem is similar to one described in recent commit 7e241f647dc7
("libceph: fall back to sendmsg for slab pages")

Signed-off-by: Vasily Averin <vvs-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org>
---
 drivers/scsi/libiscsi_tcp.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index 8a6b1b3f8277..66d97d3bef5a 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -129,12 +129,17 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
 	BUG_ON(sg->length == 0);
 
 	/*
+	 * We always map for the recv path.
+	 *
 	 * If the page count is greater than one it is ok to send
 	 * to the network layer's zero copy send path. If not we
-	 * have to go the slow sendmsg path. We always map for the
-	 * recv path.
+	 * have to go the slow sendmsg path.
+	 *
+	 * Same goes for slab pages: skb_can_coalesce() allows
+	 * coalescing neighboring slab objects into a single frag which
+	 * triggers one of hardened usercopy checks.
 	 */
-	if (page_count(sg_page(sg)) >= 1 && !recv)
+	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
 		return;
 
 	if (recv) {
-- 
2.17.1

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi: fall back to sendmsg for slab pages

[Vasily Averin]

James, Martin,
noone replied 2 weeks,
could you please pick up this patch?

According to Network guru sendpage must not be called for Slab objects.
Unfortunately this happen in real life, for example when XFS send metadata via network block device.
Some of such cases -- drbd and ceph -- already have PageSlab() check, however iscsi still lacks it.

It was triggered host to crash during internal OpenVZ tests,
fixed kernel passed this test successfully.

This patch forces iscsi_tcp_segment_map() to set up segment->data for Slab pages
and it switches iscsi_sw_tcp_xmit_segment() to use sendmsg instead of sendpage. 

Thank you,
	Vasily Averin

On 2/21/19 6:23 PM, Vasily Averin wrote:
> In "XFS over network block device" scenario XFS can create IO requests
> with slab-based XFS metadata. During processing such requests
> tcp_sendpage() can merge skb fragments with neighbour slab objects.
> 
> If receiving side is located on the same host tcp_recvmsg() can trigger
> BUG_ON in hardening check and crash the host with following message:
> 
> usercopy: kernel memory exposure attempt detected
> 		from XXXXXXXX (kmalloc-512) (1024 bytes)
> 
> This patch redirect such requests from sednpage to sendmsg path.
> The problem is similar to one described in recent commit 7e241f647dc7
> ("libceph: fall back to sendmsg for slab pages")
> 
> Signed-off-by: Vasily Averin <vvs-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org>
> ---
>  drivers/scsi/libiscsi_tcp.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
> index 8a6b1b3f8277..66d97d3bef5a 100644
> --- a/drivers/scsi/libiscsi_tcp.c
> +++ b/drivers/scsi/libiscsi_tcp.c
> @@ -129,12 +129,17 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
>  	BUG_ON(sg->length == 0);
>  
>  	/*
> +	 * We always map for the recv path.
> +	 *
>  	 * If the page count is greater than one it is ok to send
>  	 * to the network layer's zero copy send path. If not we
> -	 * have to go the slow sendmsg path. We always map for the
> -	 * recv path.
> +	 * have to go the slow sendmsg path.
> +	 *
> +	 * Same goes for slab pages: skb_can_coalesce() allows
> +	 * coalescing neighboring slab objects into a single frag which
> +	 * triggers one of hardened usercopy checks.
>  	 */
> -	if (page_count(sg_page(sg)) >= 1 && !recv)
> +	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
>  		return;
>  
>  	if (recv) {
> 

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi: fall back to sendmsg for slab pages

[Martin K. Petersen]

Vasily,

> James, Martin, noone replied 2 weeks, could you please pick up this
> patch?

No objections from me wrt. to the change. However, I am awaiting a
review from the iSCSI maintainers Lee and Chris.

-- 
Martin K. Petersen	Oracle Linux Engineering

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi: fall back to sendmsg for slab pages

[Lee Duncan]

On 3/6/19 3:33 AM, Vasily Averin wrote:
> James, Martin,
> noone replied 2 weeks,
> could you please pick up this patch?
> 
> According to Network guru sendpage must not be called for Slab objects.
> Unfortunately this happen in real life, for example when XFS send metadata via network block device.
> Some of such cases -- drbd and ceph -- already have PageSlab() check, however iscsi still lacks it.
> 
> It was triggered host to crash during internal OpenVZ tests,
> fixed kernel passed this test successfully.
> 
> This patch forces iscsi_tcp_segment_map() to set up segment->data for Slab pages
> and it switches iscsi_sw_tcp_xmit_segment() to use sendmsg instead of sendpage. 
> 
> Thank you,
> 	Vasily Averin
> 
> On 2/21/19 6:23 PM, Vasily Averin wrote:
>> In "XFS over network block device" scenario XFS can create IO requests
>> with slab-based XFS metadata. During processing such requests
>> tcp_sendpage() can merge skb fragments with neighbour slab objects.
>>
>> If receiving side is located on the same host tcp_recvmsg() can trigger
>> BUG_ON in hardening check and crash the host with following message:
>>
>> usercopy: kernel memory exposure attempt detected
>> 		from XXXXXXXX (kmalloc-512) (1024 bytes)
>>
>> This patch redirect such requests from sednpage to sendmsg path.
>> The problem is similar to one described in recent commit 7e241f647dc7
>> ("libceph: fall back to sendmsg for slab pages")
>>
>> Signed-off-by: Vasily Averin <vvs-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org>
>> ---
>>  drivers/scsi/libiscsi_tcp.c | 11 ++++++++---
>>  1 file changed, 8 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
>> index 8a6b1b3f8277..66d97d3bef5a 100644
>> --- a/drivers/scsi/libiscsi_tcp.c
>> +++ b/drivers/scsi/libiscsi_tcp.c
>> @@ -129,12 +129,17 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
>>  	BUG_ON(sg->length == 0);
>>  
>>  	/*
>> +	 * We always map for the recv path.
>> +	 *
>>  	 * If the page count is greater than one it is ok to send
>>  	 * to the network layer's zero copy send path. If not we
>> -	 * have to go the slow sendmsg path. We always map for the
>> -	 * recv path.
>> +	 * have to go the slow sendmsg path.
>> +	 *
>> +	 * Same goes for slab pages: skb_can_coalesce() allows
>> +	 * coalescing neighboring slab objects into a single frag which
>> +	 * triggers one of hardened usercopy checks.
>>  	 */
>> -	if (page_count(sg_page(sg)) >= 1 && !recv)
>> +	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
>>  		return;
>>  
>>  	if (recv) {
>>
> 

Reviewed-by: Lee Duncan <lduncan-IBi9RG/b67k@public.gmane.org>

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi: fall back to sendmsg for slab pages

[Chris Leech]

On Thu, Feb 21, 2019 at 06:23:17PM +0300, Vasily Averin wrote:
> In "XFS over network block device" scenario XFS can create IO requests
> with slab-based XFS metadata. During processing such requests
> tcp_sendpage() can merge skb fragments with neighbour slab objects.
> 
> If receiving side is located on the same host tcp_recvmsg() can trigger
> BUG_ON in hardening check and crash the host with following message:
> 
> usercopy: kernel memory exposure attempt detected
> 		from XXXXXXXX (kmalloc-512) (1024 bytes)
> 
> This patch redirect such requests from sednpage to sendmsg path.
> The problem is similar to one described in recent commit 7e241f647dc7
> ("libceph: fall back to sendmsg for slab pages")
> 
> Signed-off-by: Vasily Averin <vvs-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org>

This seems reasonable to me, I'm relying on Vasily's testing for now but
it seems right.

Acked-by: Chris Leech <cleech-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>

> ---
>  drivers/scsi/libiscsi_tcp.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
> index 8a6b1b3f8277..66d97d3bef5a 100644
> --- a/drivers/scsi/libiscsi_tcp.c
> +++ b/drivers/scsi/libiscsi_tcp.c
> @@ -129,12 +129,17 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
>  	BUG_ON(sg->length == 0);
>  
>  	/*
> +	 * We always map for the recv path.
> +	 *
>  	 * If the page count is greater than one it is ok to send
>  	 * to the network layer's zero copy send path. If not we
> -	 * have to go the slow sendmsg path. We always map for the
> -	 * recv path.
> +	 * have to go the slow sendmsg path.
> +	 *
> +	 * Same goes for slab pages: skb_can_coalesce() allows
> +	 * coalescing neighboring slab objects into a single frag which
> +	 * triggers one of hardened usercopy checks.
>  	 */
> -	if (page_count(sg_page(sg)) >= 1 && !recv)
> +	if (!recv && page_count(sg_page(sg)) >= 1 && !PageSlab(sg_page(sg)))
>  		return;
>  
>  	if (recv) {
> -- 
> 2.17.1


-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.

Re: [PATCH] iscsi: fall back to sendmsg for slab pages

[Martin K. Petersen]

Vasily,

> In "XFS over network block device" scenario XFS can create IO requests
> with slab-based XFS metadata. During processing such requests
> tcp_sendpage() can merge skb fragments with neighbour slab objects.

Applied to 5.1/scsi-queue, thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

-- 
You received this message because you are subscribed to the Google Groups "open-iscsi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-iscsi+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
To post to this group, send email to open-iscsi-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org
Visit this group at https://groups.google.com/group/open-iscsi.
For more options, visit https://groups.google.com/d/optout.