[PATCH v3 40/55] iscsi: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage

[PATCH v3 40/55] iscsi: Use sendmsg(MSG_SPLICE_PAGES) rather than sendpage

[David Howells]

Use sendmsg() with MSG_SPLICE_PAGES rather than sendpage.  This allows
multiple pages and multipage folios to be passed through.

TODO: iscsit_fe_sendpage_sg() should perhaps set up a bio_vec array for the
entire set of pages it's going to transfer plus two for the header and
trailer and page fragments to hold the header and trailer - and then call
sendmsg once for the entire message.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: "Martin K. Petersen" <martin.petersen@oracle.com>
cc: "David S. Miller" <davem@davemloft.net>
cc: Eric Dumazet <edumazet@google.com>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Jens Axboe <axboe@kernel.dk>
cc: Matthew Wilcox <willy@infradead.org>
cc: linux-scsi@vger.kernel.org
cc: target-devel@vger.kernel.org
cc: netdev@vger.kernel.org
---
 drivers/scsi/iscsi_tcp.c                 | 31 ++++++++++++------------
 drivers/scsi/iscsi_tcp.h                 |  2 +-
 drivers/target/iscsi/iscsi_target_util.c | 14 ++++++-----
 3 files changed, 24 insertions(+), 23 deletions(-)

diff --git a/drivers/scsi/iscsi_tcp.c b/drivers/scsi/iscsi_tcp.c
index c76f82fb8b63..cf3eb55d2a76 100644
--- a/drivers/scsi/iscsi_tcp.c
+++ b/drivers/scsi/iscsi_tcp.c
@@ -301,35 +301,37 @@ static int iscsi_sw_tcp_xmit_segment(struct iscsi_tcp_conn *tcp_conn,
 
 	while (!iscsi_tcp_segment_done(tcp_conn, segment, 0, r)) {
 		struct scatterlist *sg;
+		struct msghdr msg = {};
+		union {
+			struct kvec kv;
+			struct bio_vec bv;
+		} vec;
 		unsigned int offset, copy;
-		int flags = 0;
 
 		r = 0;
 		offset = segment->copied;
 		copy = segment->size - offset;
 
 		if (segment->total_copied + segment->size < segment->total_size)
-			flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
+			msg.msg_flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
 
 		if (tcp_sw_conn->queue_recv)
-			flags |= MSG_DONTWAIT;
+			msg.msg_flags |= MSG_DONTWAIT;
 
-		/* Use sendpage if we can; else fall back to sendmsg */
 		if (!segment->data) {
+			if (tcp_conn->iscsi_conn->datadgst_en)
+				msg.msg_flags |= MSG_SPLICE_PAGES;
 			sg = segment->sg;
 			offset += segment->sg_offset + sg->offset;
-			r = tcp_sw_conn->sendpage(sk, sg_page(sg), offset,
-						  copy, flags);
+			bvec_set_page(&vec.bv, sg_page(sg), copy, offset);
+			iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &vec.bv, 1, copy);
 		} else {
-			struct msghdr msg = { .msg_flags = flags };
-			struct kvec iov = {
-				.iov_base = segment->data + offset,
-				.iov_len = copy
-			};
-
-			r = kernel_sendmsg(sk, &msg, &iov, 1, copy);
+			vec.kv.iov_base = segment->data + offset;
+			vec.kv.iov_len  = copy;
+			iov_iter_kvec(&msg.msg_iter, ITER_SOURCE, &vec.kv, 1, copy);
 		}
 
+		r = sock_sendmsg(sk, &msg);
 		if (r < 0) {
 			iscsi_tcp_segment_unmap(segment);
 			return r;
@@ -746,7 +748,6 @@ iscsi_sw_tcp_conn_bind(struct iscsi_cls_session *cls_session,
 	sock_no_linger(sk);
 
 	iscsi_sw_tcp_conn_set_callbacks(conn);
-	tcp_sw_conn->sendpage = tcp_sw_conn->sock->ops->sendpage;
 	/*
 	 * set receive state machine into initial state
 	 */
@@ -778,8 +779,6 @@ static int iscsi_sw_tcp_conn_set_param(struct iscsi_cls_conn *cls_conn,
 			mutex_unlock(&tcp_sw_conn->sock_lock);
 			return -ENOTCONN;
 		}
-		tcp_sw_conn->sendpage = conn->datadgst_en ?
-			sock_no_sendpage : tcp_sw_conn->sock->ops->sendpage;
 		mutex_unlock(&tcp_sw_conn->sock_lock);
 		break;
 	case ISCSI_PARAM_MAX_R2T:
diff --git a/drivers/scsi/iscsi_tcp.h b/drivers/scsi/iscsi_tcp.h
index 68e14a344904..d6ec08d7eb63 100644
--- a/drivers/scsi/iscsi_tcp.h
+++ b/drivers/scsi/iscsi_tcp.h
@@ -48,7 +48,7 @@ struct iscsi_sw_tcp_conn {
 	uint32_t		sendpage_failures_cnt;
 	uint32_t		discontiguous_hdr_cnt;
 
-	ssize_t (*sendpage)(struct socket *, struct page *, int, size_t, int);
+	bool			can_splice_to_tcp;
 };
 
 struct iscsi_sw_tcp_host {
diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c
index 26dc8ed3045b..c7d58e41ac3b 100644
--- a/drivers/target/iscsi/iscsi_target_util.c
+++ b/drivers/target/iscsi/iscsi_target_util.c
@@ -1078,6 +1078,8 @@ int iscsit_fe_sendpage_sg(
 	struct iscsit_conn *conn)
 {
 	struct scatterlist *sg = cmd->first_data_sg;
+	struct bio_vec bvec;
+	struct msghdr msghdr = { .msg_flags = MSG_SPLICE_PAGES,	};
 	struct kvec iov;
 	u32 tx_hdr_size, data_len;
 	u32 offset = cmd->first_data_sg_off;
@@ -1121,17 +1123,17 @@ int iscsit_fe_sendpage_sg(
 		u32 space = (sg->length - offset);
 		u32 sub_len = min_t(u32, data_len, space);
 send_pg:
-		tx_sent = conn->sock->ops->sendpage(conn->sock,
-					sg_page(sg), sg->offset + offset, sub_len, 0);
+		bvec_set_page(&bvec, sg_page(sg), sub_len, sg->offset + offset);
+		iov_iter_bvec(&msghdr.msg_iter, ITER_SOURCE, &bvec, 1, sub_len);
+
+		tx_sent = conn->sock->ops->sendmsg(conn->sock, &msghdr, sub_len);
 		if (tx_sent != sub_len) {
 			if (tx_sent == -EAGAIN) {
-				pr_err("tcp_sendpage() returned"
-						" -EAGAIN\n");
+				pr_err("sendmsg/splice returned -EAGAIN\n");
 				goto send_pg;
 			}
 
-			pr_err("tcp_sendpage() failure: %d\n",
-					tx_sent);
+			pr_err("sendmsg/splice failure: %d\n", tx_sent);
 			return -1;
 		}
 

[PATCH v3 41/55] iscsi: Assume "sendpage" is okay in iscsi_tcp_segment_map()

[David Howells]

As iscsi is now using sendmsg() with MSG_SPLICE_PAGES rather than sendpage,
assume that sendpage_ok() will return true in iscsi_tcp_segment_map() and
leave it to TCP to copy the data if not.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: "Martin K. Petersen" <martin.petersen@oracle.com>
cc: "David S. Miller" <davem@davemloft.net>
cc: Eric Dumazet <edumazet@google.com>
cc: Jakub Kicinski <kuba@kernel.org>
cc: Paolo Abeni <pabeni@redhat.com>
cc: Jens Axboe <axboe@kernel.dk>
cc: Matthew Wilcox <willy@infradead.org>
cc: linux-scsi@vger.kernel.org
cc: target-devel@vger.kernel.org
cc: netdev@vger.kernel.org
---
 drivers/scsi/libiscsi_tcp.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
index c182aa83f2c9..07ba0d864820 100644
--- a/drivers/scsi/libiscsi_tcp.c
+++ b/drivers/scsi/libiscsi_tcp.c
@@ -128,18 +128,11 @@ static void iscsi_tcp_segment_map(struct iscsi_segment *segment, int recv)
 	 * coalescing neighboring slab objects into a single frag which
 	 * triggers one of hardened usercopy checks.
 	 */
-	if (!recv && sendpage_ok(sg_page(sg)))
+	if (!recv)
 		return;
 
-	if (recv) {
-		segment->atomic_mapped = true;
-		segment->sg_mapped = kmap_atomic(sg_page(sg));
-	} else {
-		segment->atomic_mapped = false;
-		/* the xmit path can sleep with the page mapped so use kmap */
-		segment->sg_mapped = kmap(sg_page(sg));
-	}
-
+	segment->atomic_mapped = true;
+	segment->sg_mapped = kmap_atomic(sg_page(sg));
 	segment->data = segment->sg_mapped + sg->offset + segment->sg_offset;
 }
 

Re: [PATCH v3 41/55] iscsi: Assume "sendpage" is okay in iscsi_tcp_segment_map()

[Fabio M. De Francesco]

On venerdì 31 marzo 2023 18:09:00 CEST David Howells wrote:
> As iscsi is now using sendmsg() with MSG_SPLICE_PAGES rather than sendpage,
> assume that sendpage_ok() will return true in iscsi_tcp_segment_map() and
> leave it to TCP to copy the data if not.
> 
> Signed-off-by: David Howells <dhowells@redhat.com>
> cc: "Martin K. Petersen" <martin.petersen@oracle.com>
> cc: "David S. Miller" <davem@davemloft.net>
> cc: Eric Dumazet <edumazet@google.com>
> cc: Jakub Kicinski <kuba@kernel.org>
> cc: Paolo Abeni <pabeni@redhat.com>
> cc: Jens Axboe <axboe@kernel.dk>
> cc: Matthew Wilcox <willy@infradead.org>
> cc: linux-scsi@vger.kernel.org
> cc: target-devel@vger.kernel.org
> cc: netdev@vger.kernel.org
> ---
>  drivers/scsi/libiscsi_tcp.c | 13 +++----------
>  1 file changed, 3 insertions(+), 10 deletions(-)
> 
> diff --git a/drivers/scsi/libiscsi_tcp.c b/drivers/scsi/libiscsi_tcp.c
> index c182aa83f2c9..07ba0d864820 100644
> --- a/drivers/scsi/libiscsi_tcp.c
> +++ b/drivers/scsi/libiscsi_tcp.c
> @@ -128,18 +128,11 @@ static void iscsi_tcp_segment_map(struct iscsi_segment
> *segment, int recv) * coalescing neighboring slab objects into a single frag
> which
>  	 * triggers one of hardened usercopy checks.
>  	 */
> -	if (!recv && sendpage_ok(sg_page(sg)))
> +	if (!recv)
>  		return;
> 
> -	if (recv) {
> -		segment->atomic_mapped = true;
> -		segment->sg_mapped = kmap_atomic(sg_page(sg));
> -	} else {
> -		segment->atomic_mapped = false;
> -		/* the xmit path can sleep with the page mapped so use 
kmap */
> -		segment->sg_mapped = kmap(sg_page(sg));
> -	}
> -
> +	segment->atomic_mapped = true;
> +	segment->sg_mapped = kmap_atomic(sg_page(sg));

As you probably know, kmap_atomic() is deprecated.

I must admit that I'm not an expert of this code, however, it looks like the 
mapping has no need to rely on the side effects of kmap_atomic() (i.e., 
pagefault_disable() and preempt_disable() - but I'm not entirely sure about 
the possibility that preemption should be explicitly disabled along with the 
replacement with kmap_local_page()). 

Last year I've been working on several conversions from kmap{,_atomic}() to 
kmap_local_page(), however I'm still not sure to understand what's happening 
here...

Am I missing any important details? Can you please explain why we still need 
that kmap_atomic() instead of kmap_local_page()? 

Thanks in advance,

Fabio

>  	segment->data = segment->sg_mapped + sg->offset + segment-
>sg_offset;
>  }

Re: [PATCH v3 41/55] iscsi: Assume "sendpage" is okay in iscsi_tcp_segment_map()

[David Howells]

Fabio M. De Francesco <fmdefrancesco@gmail.com> wrote:

> > -	if (recv) {
> > -		segment->atomic_mapped = true;
> > -		segment->sg_mapped = kmap_atomic(sg_page(sg));
> > -	} else {
> > -		segment->atomic_mapped = false;
> > -		/* the xmit path can sleep with the page mapped so use 
> kmap */
> > -		segment->sg_mapped = kmap(sg_page(sg));
> > -	}
> > -
> > +	segment->atomic_mapped = true;
> > +	segment->sg_mapped = kmap_atomic(sg_page(sg));
> 
> As you probably know, kmap_atomic() is deprecated.
> 
> I must admit that I'm not an expert of this code, however, it looks like the 
> mapping has no need to rely on the side effects of kmap_atomic() (i.e., 
> pagefault_disable() and preempt_disable() - but I'm not entirely sure about 
> the possibility that preemption should be explicitly disabled along with the 
> replacement with kmap_local_page()). 
> 
> Last year I've been working on several conversions from kmap{,_atomic}() to 
> kmap_local_page(), however I'm still not sure to understand what's happening 
> here...
> 
> Am I missing any important details? Can you please explain why we still need 
> that kmap_atomic() instead of kmap_local_page()? 

Actually, it might be worth dropping segment->sg_mapped and segment->data and
only doing the kmap_local when necessary.

And this:

			struct msghdr msg = { .msg_flags = flags };
			struct kvec iov = {
				.iov_base = segment->data + offset,
				.iov_len = copy
			};

			r = kernel_sendmsg(sk, &msg, &iov, 1, copy);

should really be using struct bvec, not struct kvec - then the mapping isn't
necessary.  It looks like this might be the only place the mapping is used,
but I'm not 100% certain.

David

Re: [PATCH v3 41/55] iscsi: Assume "sendpage" is okay in iscsi_tcp_segment_map()

[Fabio M. De Francesco]

On martedì 25 aprile 2023 10:30:30 CEST David Howells wrote:
> Fabio M. De Francesco <fmdefrancesco@gmail.com> wrote:
> > > -	if (recv) {
> > > -		segment->atomic_mapped = true;
> > > -		segment->sg_mapped = kmap_atomic(sg_page(sg));
> > > -	} else {
> > > -		segment->atomic_mapped = false;
> > > -		/* the xmit path can sleep with the page mapped so use
> > 
> > kmap */
> > 
> > > -		segment->sg_mapped = kmap(sg_page(sg));
> > > -	}
> > > -
> > > +	segment->atomic_mapped = true;
> > > +	segment->sg_mapped = kmap_atomic(sg_page(sg));
> > 
> > As you probably know, kmap_atomic() is deprecated.
> > 
> > I must admit that I'm not an expert of this code, however, it looks like 
the
> > mapping has no need to rely on the side effects of kmap_atomic() (i.e.,
> > pagefault_disable() and preempt_disable() - but I'm not entirely sure 
about
> > the possibility that preemption should be explicitly disabled along with 
the
> > replacement with kmap_local_page()).
> > 
> > Last year I've been working on several conversions from kmap{,_atomic}() 
to
> > kmap_local_page(), however I'm still not sure to understand what's 
happening
> > here...
> > 
> > Am I missing any important details? Can you please explain why we still 
need
> > that kmap_atomic() instead of kmap_local_page()?
> 
> Actually, it might be worth dropping segment->sg_mapped and segment->data 
and
> only doing the kmap_local when necessary.
> 
> And this:
> 
> 			struct msghdr msg = { .msg_flags = flags };
> 			struct kvec iov = {
> 				.iov_base = segment->data + offset,
> 				.iov_len = copy
> 			};
> 
> 			r = kernel_sendmsg(sk, &msg, &iov, 1, copy);
> 
> should really be using struct bvec, not struct kvec - then the mapping isn't
> necessary.

FWIW, struct bvec looks better suited (despite I have very little knowledge of 
this code).

I assume that you noticed that we also have the unmapping counterpart 
(iscsi_tcp_segment_unmap()) which should also be addressed accordingly.

> It looks like this might be the only place the mapping is used,
> but I'm not 100% certain.

It seems that kmap_atomic() (as well as kmap(), which you deleted) is only 
called by iscsi_tcp_segment_map(), which in turn is called only by  
iscsi_tcp_segment_done(). I can't see any other places where the mapping is 
used.

I hope that this dialogue may help you somehow to choose the best suited way 
to get rid of that deprecated kmap_atomic().

Thanks for taking time to address questions from newcomers :-) 

Fabio

> 
> David