* Re: [PATCH v4 0/5] vsock/virtio: optimizations to increase the throughput
From: Stefano Garzarella @ 2019-07-30 15:38 UTC (permalink / raw)
To: Jason Wang
Cc: Michael S. Tsirkin, Stefan Hajnoczi, David S. Miller, netdev,
linux-kernel, virtualization, kvm
In-Reply-To: <fc568e3d-7af5-5895-89e8-32e35b0f9af4@redhat.com>
On Tue, Jul 30, 2019 at 06:03:24PM +0800, Jason Wang wrote:
>
> On 2019/7/30 下午5:40, Stefano Garzarella wrote:
> > On Mon, Jul 29, 2019 at 09:59:23AM -0400, Michael S. Tsirkin wrote:
> > > On Wed, Jul 17, 2019 at 01:30:25PM +0200, Stefano Garzarella wrote:
> > > > This series tries to increase the throughput of virtio-vsock with slight
> > > > changes.
> > > > While I was testing the v2 of this series I discovered an huge use of memory,
> > > > so I added patch 1 to mitigate this issue. I put it in this series in order
> > > > to better track the performance trends.
> > > Series:
> > >
> > > Acked-by: Michael S. Tsirkin <mst@redhat.com>
> > >
> > > Can this go into net-next?
> > >
> > I think so.
> > Michael, Stefan thanks to ack the series!
> >
> > Should I resend it with net-next tag?
> >
> > Thanks,
> > Stefano
>
>
> I think so.
Okay, I'll resend it!
Thanks,
Stefano
^ permalink raw reply
* [PATCH net-next v5 0/5] vsock/virtio: optimizations to increase the throughput
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
This series tries to increase the throughput of virtio-vsock with slight
changes.
While I was testing the v2 of this series I discovered an huge use of memory,
so I added patch 1 to mitigate this issue. I put it in this series in order
to better track the performance trends.
v5:
- rebased all patches on net-next
- added Stefan's R-b and Michael's A-b
v4: https://patchwork.kernel.org/cover/11047717
v3: https://patchwork.kernel.org/cover/10970145
v2: https://patchwork.kernel.org/cover/10938743
v1: https://patchwork.kernel.org/cover/10885431
Below are the benchmarks step by step. I used iperf3 [1] modified with VSOCK
support. As Michael suggested in the v1, I booted host and guest with 'nosmap'.
A brief description of patches:
- Patches 1: limit the memory usage with an extra copy for small packets
- Patches 2+3: reduce the number of credit update messages sent to the
transmitter
- Patches 4+5: allow the host to split packets on multiple buffers and use
VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max packet size allowed
host -> guest [Gbps]
pkt_size before opt p 1 p 2+3 p 4+5
32 0.032 0.030 0.048 0.051
64 0.061 0.059 0.108 0.117
128 0.122 0.112 0.227 0.234
256 0.244 0.241 0.418 0.415
512 0.459 0.466 0.847 0.865
1K 0.927 0.919 1.657 1.641
2K 1.884 1.813 3.262 3.269
4K 3.378 3.326 6.044 6.195
8K 5.637 5.676 10.141 11.287
16K 8.250 8.402 15.976 16.736
32K 13.327 13.204 19.013 20.515
64K 21.241 21.341 20.973 21.879
128K 21.851 22.354 21.816 23.203
256K 21.408 21.693 21.846 24.088
512K 21.600 21.899 21.921 24.106
guest -> host [Gbps]
pkt_size before opt p 1 p 2+3 p 4+5
32 0.045 0.046 0.057 0.057
64 0.089 0.091 0.103 0.104
128 0.170 0.179 0.192 0.200
256 0.364 0.351 0.361 0.379
512 0.709 0.699 0.731 0.790
1K 1.399 1.407 1.395 1.427
2K 2.670 2.684 2.745 2.835
4K 5.171 5.199 5.305 5.451
8K 8.442 8.500 10.083 9.941
16K 12.305 12.259 13.519 15.385
32K 11.418 11.150 11.988 24.680
64K 10.778 10.659 11.589 35.273
128K 10.421 10.339 10.939 40.338
256K 10.300 9.719 10.508 36.562
512K 9.833 9.808 10.612 35.979
As Stefan suggested in the v1, I measured also the efficiency in this way:
efficiency = Mbps / (%CPU_Host + %CPU_Guest)
The '%CPU_Guest' is taken inside the VM. I know that it is not the best way,
but it's provided for free from iperf3 and could be an indication.
host -> guest efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
pkt_size before opt p 1 p 2+3 p 4+5
32 0.35 0.45 0.79 1.02
64 0.56 0.80 1.41 1.54
128 1.11 1.52 3.03 3.12
256 2.20 2.16 5.44 5.58
512 4.17 4.18 10.96 11.46
1K 8.30 8.26 20.99 20.89
2K 16.82 16.31 39.76 39.73
4K 30.89 30.79 74.07 75.73
8K 53.74 54.49 124.24 148.91
16K 80.68 83.63 200.21 232.79
32K 132.27 132.52 260.81 357.07
64K 229.82 230.40 300.19 444.18
128K 332.60 329.78 331.51 492.28
256K 331.06 337.22 339.59 511.59
512K 335.58 328.50 331.56 504.56
guest -> host efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
pkt_size before opt p 1 p 2+3 p 4+5
32 0.43 0.43 0.53 0.56
64 0.85 0.86 1.04 1.10
128 1.63 1.71 2.07 2.13
256 3.48 3.35 4.02 4.22
512 6.80 6.67 7.97 8.63
1K 13.32 13.31 15.72 15.94
2K 25.79 25.92 30.84 30.98
4K 50.37 50.48 58.79 59.69
8K 95.90 96.15 107.04 110.33
16K 145.80 145.43 143.97 174.70
32K 147.06 144.74 146.02 282.48
64K 145.25 143.99 141.62 406.40
128K 149.34 146.96 147.49 489.34
256K 156.35 149.81 152.21 536.37
512K 151.65 150.74 151.52 519.93
[1] https://github.com/stefano-garzarella/iperf/
Stefano Garzarella (5):
vsock/virtio: limit the memory used per-socket
vsock/virtio: reduce credit update messages
vsock/virtio: fix locking in virtio_transport_inc_tx_pkt()
vhost/vsock: split packets to send using multiple buffers
vsock/virtio: change the maximum packet size allowed
drivers/vhost/vsock.c | 68 ++++++++++++-----
include/linux/virtio_vsock.h | 4 +-
net/vmw_vsock/virtio_transport.c | 1 +
net/vmw_vsock/virtio_transport_common.c | 99 ++++++++++++++++++++-----
4 files changed, 134 insertions(+), 38 deletions(-)
--
2.20.1
^ permalink raw reply
* [PATCH net-next v5 1/5] vsock/virtio: limit the memory used per-socket
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
Since virtio-vsock was introduced, the buffers filled by the host
and pushed to the guest using the vring, are directly queued in
a per-socket list. These buffers are preallocated by the guest
with a fixed size (4 KB).
The maximum amount of memory used by each socket should be
controlled by the credit mechanism.
The default credit available per-socket is 256 KB, but if we use
only 1 byte per packet, the guest can queue up to 262144 of 4 KB
buffers, using up to 1 GB of memory per-socket. In addition, the
guest will continue to fill the vring with new 4 KB free buffers
to avoid starvation of other sockets.
This patch mitigates this issue copying the payload of small
packets (< 128 bytes) into the buffer of last packet queued, in
order to avoid wasting memory.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/vhost/vsock.c | 2 +
include/linux/virtio_vsock.h | 1 +
net/vmw_vsock/virtio_transport.c | 1 +
net/vmw_vsock/virtio_transport_common.c | 60 +++++++++++++++++++++----
4 files changed, 55 insertions(+), 9 deletions(-)
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index 6a50e1d0529c..6c8390a2af52 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -329,6 +329,8 @@ vhost_vsock_alloc_pkt(struct vhost_virtqueue *vq,
return NULL;
}
+ pkt->buf_len = pkt->len;
+
nbytes = copy_from_iter(pkt->buf, pkt->len, &iov_iter);
if (nbytes != pkt->len) {
vq_err(vq, "Expected %u byte payload, got %zu bytes\n",
diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h
index e223e2632edd..7d973903f52e 100644
--- a/include/linux/virtio_vsock.h
+++ b/include/linux/virtio_vsock.h
@@ -52,6 +52,7 @@ struct virtio_vsock_pkt {
/* socket refcnt not held, only use for cancellation */
struct vsock_sock *vsk;
void *buf;
+ u32 buf_len;
u32 len;
u32 off;
bool reply;
diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c
index 0815d1357861..082a30936690 100644
--- a/net/vmw_vsock/virtio_transport.c
+++ b/net/vmw_vsock/virtio_transport.c
@@ -307,6 +307,7 @@ static void virtio_vsock_rx_fill(struct virtio_vsock *vsock)
break;
}
+ pkt->buf_len = buf_len;
pkt->len = buf_len;
sg_init_one(&hdr, &pkt->hdr, sizeof(pkt->hdr));
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 6f1a8aff65c5..095221f94786 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -26,6 +26,9 @@
/* How long to wait for graceful shutdown of a connection */
#define VSOCK_CLOSE_TIMEOUT (8 * HZ)
+/* Threshold for detecting small packets to copy */
+#define GOOD_COPY_LEN 128
+
static const struct virtio_transport *virtio_transport_get_ops(void)
{
const struct vsock_transport *t = vsock_core_get_transport();
@@ -64,6 +67,9 @@ virtio_transport_alloc_pkt(struct virtio_vsock_pkt_info *info,
pkt->buf = kmalloc(len, GFP_KERNEL);
if (!pkt->buf)
goto out_pkt;
+
+ pkt->buf_len = len;
+
err = memcpy_from_msg(pkt->buf, info->msg, len);
if (err)
goto out;
@@ -841,24 +847,60 @@ virtio_transport_recv_connecting(struct sock *sk,
return err;
}
+static void
+virtio_transport_recv_enqueue(struct vsock_sock *vsk,
+ struct virtio_vsock_pkt *pkt)
+{
+ struct virtio_vsock_sock *vvs = vsk->trans;
+ bool free_pkt = false;
+
+ pkt->len = le32_to_cpu(pkt->hdr.len);
+ pkt->off = 0;
+
+ spin_lock_bh(&vvs->rx_lock);
+
+ virtio_transport_inc_rx_pkt(vvs, pkt);
+
+ /* Try to copy small packets into the buffer of last packet queued,
+ * to avoid wasting memory queueing the entire buffer with a small
+ * payload.
+ */
+ if (pkt->len <= GOOD_COPY_LEN && !list_empty(&vvs->rx_queue)) {
+ struct virtio_vsock_pkt *last_pkt;
+
+ last_pkt = list_last_entry(&vvs->rx_queue,
+ struct virtio_vsock_pkt, list);
+
+ /* If there is space in the last packet queued, we copy the
+ * new packet in its buffer.
+ */
+ if (pkt->len <= last_pkt->buf_len - last_pkt->len) {
+ memcpy(last_pkt->buf + last_pkt->len, pkt->buf,
+ pkt->len);
+ last_pkt->len += pkt->len;
+ free_pkt = true;
+ goto out;
+ }
+ }
+
+ list_add_tail(&pkt->list, &vvs->rx_queue);
+
+out:
+ spin_unlock_bh(&vvs->rx_lock);
+ if (free_pkt)
+ virtio_transport_free_pkt(pkt);
+}
+
static int
virtio_transport_recv_connected(struct sock *sk,
struct virtio_vsock_pkt *pkt)
{
struct vsock_sock *vsk = vsock_sk(sk);
- struct virtio_vsock_sock *vvs = vsk->trans;
int err = 0;
switch (le16_to_cpu(pkt->hdr.op)) {
case VIRTIO_VSOCK_OP_RW:
- pkt->len = le32_to_cpu(pkt->hdr.len);
- pkt->off = 0;
-
- spin_lock_bh(&vvs->rx_lock);
- virtio_transport_inc_rx_pkt(vvs, pkt);
- list_add_tail(&pkt->list, &vvs->rx_queue);
- spin_unlock_bh(&vvs->rx_lock);
-
+ virtio_transport_recv_enqueue(vsk, pkt);
sk->sk_data_ready(sk);
return err;
case VIRTIO_VSOCK_OP_CREDIT_UPDATE:
--
2.20.1
^ permalink raw reply related
* [PATCH net-next v5 3/5] vsock/virtio: fix locking in virtio_transport_inc_tx_pkt()
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
fwd_cnt and last_fwd_cnt are protected by rx_lock, so we should use
the same spinlock also if we are in the TX path.
Move also buf_alloc under the same lock.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
include/linux/virtio_vsock.h | 2 +-
net/vmw_vsock/virtio_transport_common.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h
index 49fc9d20bc43..4c7781f4b29b 100644
--- a/include/linux/virtio_vsock.h
+++ b/include/linux/virtio_vsock.h
@@ -35,7 +35,6 @@ struct virtio_vsock_sock {
/* Protected by tx_lock */
u32 tx_cnt;
- u32 buf_alloc;
u32 peer_fwd_cnt;
u32 peer_buf_alloc;
@@ -43,6 +42,7 @@ struct virtio_vsock_sock {
u32 fwd_cnt;
u32 last_fwd_cnt;
u32 rx_bytes;
+ u32 buf_alloc;
struct list_head rx_queue;
};
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index a85559d4d974..34a2b42313b7 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -210,11 +210,11 @@ static void virtio_transport_dec_rx_pkt(struct virtio_vsock_sock *vvs,
void virtio_transport_inc_tx_pkt(struct virtio_vsock_sock *vvs, struct virtio_vsock_pkt *pkt)
{
- spin_lock_bh(&vvs->tx_lock);
+ spin_lock_bh(&vvs->rx_lock);
vvs->last_fwd_cnt = vvs->fwd_cnt;
pkt->hdr.fwd_cnt = cpu_to_le32(vvs->fwd_cnt);
pkt->hdr.buf_alloc = cpu_to_le32(vvs->buf_alloc);
- spin_unlock_bh(&vvs->tx_lock);
+ spin_unlock_bh(&vvs->rx_lock);
}
EXPORT_SYMBOL_GPL(virtio_transport_inc_tx_pkt);
--
2.20.1
^ permalink raw reply related
* [PATCH net-next v5 2/5] vsock/virtio: reduce credit update messages
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
In order to reduce the number of credit update messages,
we send them only when the space available seen by the
transmitter is less than VIRTIO_VSOCK_MAX_PKT_BUF_SIZE.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
include/linux/virtio_vsock.h | 1 +
net/vmw_vsock/virtio_transport_common.c | 16 +++++++++++++---
2 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/include/linux/virtio_vsock.h b/include/linux/virtio_vsock.h
index 7d973903f52e..49fc9d20bc43 100644
--- a/include/linux/virtio_vsock.h
+++ b/include/linux/virtio_vsock.h
@@ -41,6 +41,7 @@ struct virtio_vsock_sock {
/* Protected by rx_lock */
u32 fwd_cnt;
+ u32 last_fwd_cnt;
u32 rx_bytes;
struct list_head rx_queue;
};
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 095221f94786..a85559d4d974 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -211,6 +211,7 @@ static void virtio_transport_dec_rx_pkt(struct virtio_vsock_sock *vvs,
void virtio_transport_inc_tx_pkt(struct virtio_vsock_sock *vvs, struct virtio_vsock_pkt *pkt)
{
spin_lock_bh(&vvs->tx_lock);
+ vvs->last_fwd_cnt = vvs->fwd_cnt;
pkt->hdr.fwd_cnt = cpu_to_le32(vvs->fwd_cnt);
pkt->hdr.buf_alloc = cpu_to_le32(vvs->buf_alloc);
spin_unlock_bh(&vvs->tx_lock);
@@ -261,6 +262,7 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
struct virtio_vsock_sock *vvs = vsk->trans;
struct virtio_vsock_pkt *pkt;
size_t bytes, total = 0;
+ u32 free_space;
int err = -EFAULT;
spin_lock_bh(&vvs->rx_lock);
@@ -291,11 +293,19 @@ virtio_transport_stream_do_dequeue(struct vsock_sock *vsk,
virtio_transport_free_pkt(pkt);
}
}
+
+ free_space = vvs->buf_alloc - (vvs->fwd_cnt - vvs->last_fwd_cnt);
+
spin_unlock_bh(&vvs->rx_lock);
- /* Send a credit pkt to peer */
- virtio_transport_send_credit_update(vsk, VIRTIO_VSOCK_TYPE_STREAM,
- NULL);
+ /* We send a credit update only when the space available seen
+ * by the transmitter is less than VIRTIO_VSOCK_MAX_PKT_BUF_SIZE
+ */
+ if (free_space < VIRTIO_VSOCK_MAX_PKT_BUF_SIZE) {
+ virtio_transport_send_credit_update(vsk,
+ VIRTIO_VSOCK_TYPE_STREAM,
+ NULL);
+ }
return total;
--
2.20.1
^ permalink raw reply related
* [PATCH net-next v5 4/5] vhost/vsock: split packets to send using multiple buffers
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
If the packets to sent to the guest are bigger than the buffer
available, we can split them, using multiple buffers and fixing
the length in the packet header.
This is safe since virtio-vsock supports only stream sockets.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/vhost/vsock.c | 66 ++++++++++++++++++-------
net/vmw_vsock/virtio_transport_common.c | 15 ++++--
2 files changed, 60 insertions(+), 21 deletions(-)
diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c
index 6c8390a2af52..9f57736fe15e 100644
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@ -102,7 +102,7 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock,
struct iov_iter iov_iter;
unsigned out, in;
size_t nbytes;
- size_t len;
+ size_t iov_len, payload_len;
int head;
spin_lock_bh(&vsock->send_pkt_list_lock);
@@ -147,8 +147,24 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock,
break;
}
- len = iov_length(&vq->iov[out], in);
- iov_iter_init(&iov_iter, READ, &vq->iov[out], in, len);
+ iov_len = iov_length(&vq->iov[out], in);
+ if (iov_len < sizeof(pkt->hdr)) {
+ virtio_transport_free_pkt(pkt);
+ vq_err(vq, "Buffer len [%zu] too small\n", iov_len);
+ break;
+ }
+
+ iov_iter_init(&iov_iter, READ, &vq->iov[out], in, iov_len);
+ payload_len = pkt->len - pkt->off;
+
+ /* If the packet is greater than the space available in the
+ * buffer, we split it using multiple buffers.
+ */
+ if (payload_len > iov_len - sizeof(pkt->hdr))
+ payload_len = iov_len - sizeof(pkt->hdr);
+
+ /* Set the correct length in the header */
+ pkt->hdr.len = cpu_to_le32(payload_len);
nbytes = copy_to_iter(&pkt->hdr, sizeof(pkt->hdr), &iov_iter);
if (nbytes != sizeof(pkt->hdr)) {
@@ -157,33 +173,47 @@ vhost_transport_do_send_pkt(struct vhost_vsock *vsock,
break;
}
- nbytes = copy_to_iter(pkt->buf, pkt->len, &iov_iter);
- if (nbytes != pkt->len) {
+ nbytes = copy_to_iter(pkt->buf + pkt->off, payload_len,
+ &iov_iter);
+ if (nbytes != payload_len) {
virtio_transport_free_pkt(pkt);
vq_err(vq, "Faulted on copying pkt buf\n");
break;
}
- vhost_add_used(vq, head, sizeof(pkt->hdr) + pkt->len);
+ vhost_add_used(vq, head, sizeof(pkt->hdr) + payload_len);
added = true;
- if (pkt->reply) {
- int val;
-
- val = atomic_dec_return(&vsock->queued_replies);
-
- /* Do we have resources to resume tx processing? */
- if (val + 1 == tx_vq->num)
- restart_tx = true;
- }
-
/* Deliver to monitoring devices all correctly transmitted
* packets.
*/
virtio_transport_deliver_tap_pkt(pkt);
- total_len += pkt->len;
- virtio_transport_free_pkt(pkt);
+ pkt->off += payload_len;
+ total_len += payload_len;
+
+ /* If we didn't send all the payload we can requeue the packet
+ * to send it with the next available buffer.
+ */
+ if (pkt->off < pkt->len) {
+ spin_lock_bh(&vsock->send_pkt_list_lock);
+ list_add(&pkt->list, &vsock->send_pkt_list);
+ spin_unlock_bh(&vsock->send_pkt_list_lock);
+ } else {
+ if (pkt->reply) {
+ int val;
+
+ val = atomic_dec_return(&vsock->queued_replies);
+
+ /* Do we have resources to resume tx
+ * processing?
+ */
+ if (val + 1 == tx_vq->num)
+ restart_tx = true;
+ }
+
+ virtio_transport_free_pkt(pkt);
+ }
} while(likely(!vhost_exceeds_weight(vq, ++pkts, total_len)));
if (added)
vhost_signal(&vsock->dev, vq);
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 34a2b42313b7..56fab3f03d0e 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -97,8 +97,17 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque)
struct virtio_vsock_pkt *pkt = opaque;
struct af_vsockmon_hdr *hdr;
struct sk_buff *skb;
+ size_t payload_len;
+ void *payload_buf;
- skb = alloc_skb(sizeof(*hdr) + sizeof(pkt->hdr) + pkt->len,
+ /* A packet could be split to fit the RX buffer, so we can retrieve
+ * the payload length from the header and the buffer pointer taking
+ * care of the offset in the original packet.
+ */
+ payload_len = le32_to_cpu(pkt->hdr.len);
+ payload_buf = pkt->buf + pkt->off;
+
+ skb = alloc_skb(sizeof(*hdr) + sizeof(pkt->hdr) + payload_len,
GFP_ATOMIC);
if (!skb)
return NULL;
@@ -138,8 +147,8 @@ static struct sk_buff *virtio_transport_build_skb(void *opaque)
skb_put_data(skb, &pkt->hdr, sizeof(pkt->hdr));
- if (pkt->len) {
- skb_put_data(skb, pkt->buf, pkt->len);
+ if (payload_len) {
+ skb_put_data(skb, payload_buf, payload_len);
}
return skb;
--
2.20.1
^ permalink raw reply related
* [PATCH net-next v5 5/5] vsock/virtio: change the maximum packet size allowed
From: Stefano Garzarella @ 2019-07-30 15:43 UTC (permalink / raw)
To: netdev
Cc: virtualization, linux-kernel, Jason Wang, kvm, Michael S. Tsirkin,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
Since now we are able to split packets, we can avoid limiting
their sizes to VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE.
Instead, we can use VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max
packet size.
Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
---
net/vmw_vsock/virtio_transport_common.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/vmw_vsock/virtio_transport_common.c b/net/vmw_vsock/virtio_transport_common.c
index 56fab3f03d0e..94cc0fa3e848 100644
--- a/net/vmw_vsock/virtio_transport_common.c
+++ b/net/vmw_vsock/virtio_transport_common.c
@@ -181,8 +181,8 @@ static int virtio_transport_send_pkt_info(struct vsock_sock *vsk,
vvs = vsk->trans;
/* we can send less than pkt_len bytes */
- if (pkt_len > VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE)
- pkt_len = VIRTIO_VSOCK_DEFAULT_RX_BUF_SIZE;
+ if (pkt_len > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE)
+ pkt_len = VIRTIO_VSOCK_MAX_PKT_BUF_SIZE;
/* virtio_transport_get_credit might return less than pkt_len credit */
pkt_len = virtio_transport_get_credit(vvs, pkt_len);
--
2.20.1
^ permalink raw reply related
* Re: [PATCH 1/2] net: dsa: mv88e6xxx: add support to setup led-control register through device-tree
From: Hubert Feurstein @ 2019-07-30 15:50 UTC (permalink / raw)
To: Andrew Lunn
Cc: netdev, linux-kernel, Vivien Didelot, Florian Fainelli,
David S. Miller
In-Reply-To: <20190730140930.GM28552@lunn.ch>
Am Di., 30. Juli 2019 um 16:09 Uhr schrieb Andrew Lunn <andrew@lunn.ch>:
[...]
> Sorry, but this is not going to be accepted. There is an ongoing
> discussion about PHY LEDs and how they should be configured. Switch
> LEDs are no different from PHY LEDs. So they should use the same basic
> concept.
>
> Please take a look at the discussion around:
>
> [RFC] dt-bindings: net: phy: Add subnode for LED configuration
>
> Marvell designers have made this more difficult than it should be by
> moving the registers out of the PHY address space and into the switch
> address space. So we are going to have to implement this code twice
> :-(
Ok, good to know. I'll wait for the first implementation and take it
as a reference.
Hubert
^ permalink raw reply
* Re: [net-next 01/13] net/mlx5e: Print a warning when LRO feature is dropped or not allowed
From: Willem de Bruijn @ 2019-07-30 15:52 UTC (permalink / raw)
To: Saeed Mahameed; +Cc: David S. Miller, netdev@vger.kernel.org, Huy Nguyen
In-Reply-To: <20190729234934.23595-2-saeedm@mellanox.com>
On Mon, Jul 29, 2019 at 7:50 PM Saeed Mahameed <saeedm@mellanox.com> wrote:
>
> From: Huy Nguyen <huyn@mellanox.com>
>
> When user enables LRO via ethtool and if the RQ mode is legacy,
> mlx5e_fix_features drops the request without any explanation.
> Add netdev_warn to cover this case.
>
> Fixes: 6c3a823e1e9c ("net/mlx5e: RX, Remove HW LRO support in legacy RQ")
> Signed-off-by: Huy Nguyen <huyn@mellanox.com>
> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
> ---
> drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> index 47eea6b3a1c3..776eb46d263d 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> @@ -3788,9 +3788,10 @@ static netdev_features_t mlx5e_fix_features(struct net_device *netdev,
> netdev_warn(netdev, "Dropping C-tag vlan stripping offload due to S-tag vlan\n");
> }
> if (!MLX5E_GET_PFLAG(params, MLX5E_PFLAG_RX_STRIDING_RQ)) {
> - features &= ~NETIF_F_LRO;
> - if (params->lro_en)
> + if (features & NETIF_F_LRO) {
> netdev_warn(netdev, "Disabling LRO, not supported in legacy RQ\n");
This warns about "Disabling LRO" on an enable request?
More fundamentally, it appears that the device does not advertise
the feature as configurable in netdev_hw_features as of commit
6c3a823e1e9c ("net/mlx5e: RX, Remove HW LRO support in
legacy RQ"), so shouldn't this be caught by the device driver
independent ethtool code?
^ permalink raw reply
* Re: [PATCH net-next v5 0/5] vsock/virtio: optimizations to increase the throughput
From: Michael S. Tsirkin @ 2019-07-30 15:54 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, virtualization, linux-kernel, Jason Wang, kvm,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730154334.237789-1-sgarzare@redhat.com>
On Tue, Jul 30, 2019 at 05:43:29PM +0200, Stefano Garzarella wrote:
> This series tries to increase the throughput of virtio-vsock with slight
> changes.
> While I was testing the v2 of this series I discovered an huge use of memory,
> so I added patch 1 to mitigate this issue. I put it in this series in order
> to better track the performance trends.
>
> v5:
> - rebased all patches on net-next
> - added Stefan's R-b and Michael's A-b
This doesn't solve all issues around allocation - as I mentioned I think
we will need to improve accounting for that,
and maybe add pre-allocation.
But it's a great series of steps in the right direction!
> v4: https://patchwork.kernel.org/cover/11047717
> v3: https://patchwork.kernel.org/cover/10970145
> v2: https://patchwork.kernel.org/cover/10938743
> v1: https://patchwork.kernel.org/cover/10885431
>
> Below are the benchmarks step by step. I used iperf3 [1] modified with VSOCK
> support. As Michael suggested in the v1, I booted host and guest with 'nosmap'.
>
> A brief description of patches:
> - Patches 1: limit the memory usage with an extra copy for small packets
> - Patches 2+3: reduce the number of credit update messages sent to the
> transmitter
> - Patches 4+5: allow the host to split packets on multiple buffers and use
> VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max packet size allowed
>
> host -> guest [Gbps]
> pkt_size before opt p 1 p 2+3 p 4+5
>
> 32 0.032 0.030 0.048 0.051
> 64 0.061 0.059 0.108 0.117
> 128 0.122 0.112 0.227 0.234
> 256 0.244 0.241 0.418 0.415
> 512 0.459 0.466 0.847 0.865
> 1K 0.927 0.919 1.657 1.641
> 2K 1.884 1.813 3.262 3.269
> 4K 3.378 3.326 6.044 6.195
> 8K 5.637 5.676 10.141 11.287
> 16K 8.250 8.402 15.976 16.736
> 32K 13.327 13.204 19.013 20.515
> 64K 21.241 21.341 20.973 21.879
> 128K 21.851 22.354 21.816 23.203
> 256K 21.408 21.693 21.846 24.088
> 512K 21.600 21.899 21.921 24.106
>
> guest -> host [Gbps]
> pkt_size before opt p 1 p 2+3 p 4+5
>
> 32 0.045 0.046 0.057 0.057
> 64 0.089 0.091 0.103 0.104
> 128 0.170 0.179 0.192 0.200
> 256 0.364 0.351 0.361 0.379
> 512 0.709 0.699 0.731 0.790
> 1K 1.399 1.407 1.395 1.427
> 2K 2.670 2.684 2.745 2.835
> 4K 5.171 5.199 5.305 5.451
> 8K 8.442 8.500 10.083 9.941
> 16K 12.305 12.259 13.519 15.385
> 32K 11.418 11.150 11.988 24.680
> 64K 10.778 10.659 11.589 35.273
> 128K 10.421 10.339 10.939 40.338
> 256K 10.300 9.719 10.508 36.562
> 512K 9.833 9.808 10.612 35.979
>
> As Stefan suggested in the v1, I measured also the efficiency in this way:
> efficiency = Mbps / (%CPU_Host + %CPU_Guest)
>
> The '%CPU_Guest' is taken inside the VM. I know that it is not the best way,
> but it's provided for free from iperf3 and could be an indication.
>
> host -> guest efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> pkt_size before opt p 1 p 2+3 p 4+5
>
> 32 0.35 0.45 0.79 1.02
> 64 0.56 0.80 1.41 1.54
> 128 1.11 1.52 3.03 3.12
> 256 2.20 2.16 5.44 5.58
> 512 4.17 4.18 10.96 11.46
> 1K 8.30 8.26 20.99 20.89
> 2K 16.82 16.31 39.76 39.73
> 4K 30.89 30.79 74.07 75.73
> 8K 53.74 54.49 124.24 148.91
> 16K 80.68 83.63 200.21 232.79
> 32K 132.27 132.52 260.81 357.07
> 64K 229.82 230.40 300.19 444.18
> 128K 332.60 329.78 331.51 492.28
> 256K 331.06 337.22 339.59 511.59
> 512K 335.58 328.50 331.56 504.56
>
> guest -> host efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> pkt_size before opt p 1 p 2+3 p 4+5
>
> 32 0.43 0.43 0.53 0.56
> 64 0.85 0.86 1.04 1.10
> 128 1.63 1.71 2.07 2.13
> 256 3.48 3.35 4.02 4.22
> 512 6.80 6.67 7.97 8.63
> 1K 13.32 13.31 15.72 15.94
> 2K 25.79 25.92 30.84 30.98
> 4K 50.37 50.48 58.79 59.69
> 8K 95.90 96.15 107.04 110.33
> 16K 145.80 145.43 143.97 174.70
> 32K 147.06 144.74 146.02 282.48
> 64K 145.25 143.99 141.62 406.40
> 128K 149.34 146.96 147.49 489.34
> 256K 156.35 149.81 152.21 536.37
> 512K 151.65 150.74 151.52 519.93
>
> [1] https://github.com/stefano-garzarella/iperf/
>
> Stefano Garzarella (5):
> vsock/virtio: limit the memory used per-socket
> vsock/virtio: reduce credit update messages
> vsock/virtio: fix locking in virtio_transport_inc_tx_pkt()
> vhost/vsock: split packets to send using multiple buffers
> vsock/virtio: change the maximum packet size allowed
>
> drivers/vhost/vsock.c | 68 ++++++++++++-----
> include/linux/virtio_vsock.h | 4 +-
> net/vmw_vsock/virtio_transport.c | 1 +
> net/vmw_vsock/virtio_transport_common.c | 99 ++++++++++++++++++++-----
> 4 files changed, 134 insertions(+), 38 deletions(-)
>
> --
> 2.20.1
^ permalink raw reply
* Re: [PATCH net-next v5 0/5] vsock/virtio: optimizations to increase the throughput
From: Michael S. Tsirkin @ 2019-07-30 15:55 UTC (permalink / raw)
To: Stefano Garzarella
Cc: netdev, virtualization, linux-kernel, Jason Wang, kvm,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730115339-mutt-send-email-mst@kernel.org>
On Tue, Jul 30, 2019 at 11:54:53AM -0400, Michael S. Tsirkin wrote:
> On Tue, Jul 30, 2019 at 05:43:29PM +0200, Stefano Garzarella wrote:
> > This series tries to increase the throughput of virtio-vsock with slight
> > changes.
> > While I was testing the v2 of this series I discovered an huge use of memory,
> > so I added patch 1 to mitigate this issue. I put it in this series in order
> > to better track the performance trends.
> >
> > v5:
> > - rebased all patches on net-next
> > - added Stefan's R-b and Michael's A-b
>
> This doesn't solve all issues around allocation - as I mentioned I think
> we will need to improve accounting for that,
> and maybe add pre-allocation.
> But it's a great series of steps in the right direction!
>
So
Acked-by: Michael S. Tsirkin <mst@redhat.com>
>
> > v4: https://patchwork.kernel.org/cover/11047717
> > v3: https://patchwork.kernel.org/cover/10970145
> > v2: https://patchwork.kernel.org/cover/10938743
> > v1: https://patchwork.kernel.org/cover/10885431
> >
> > Below are the benchmarks step by step. I used iperf3 [1] modified with VSOCK
> > support. As Michael suggested in the v1, I booted host and guest with 'nosmap'.
> >
> > A brief description of patches:
> > - Patches 1: limit the memory usage with an extra copy for small packets
> > - Patches 2+3: reduce the number of credit update messages sent to the
> > transmitter
> > - Patches 4+5: allow the host to split packets on multiple buffers and use
> > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max packet size allowed
> >
> > host -> guest [Gbps]
> > pkt_size before opt p 1 p 2+3 p 4+5
> >
> > 32 0.032 0.030 0.048 0.051
> > 64 0.061 0.059 0.108 0.117
> > 128 0.122 0.112 0.227 0.234
> > 256 0.244 0.241 0.418 0.415
> > 512 0.459 0.466 0.847 0.865
> > 1K 0.927 0.919 1.657 1.641
> > 2K 1.884 1.813 3.262 3.269
> > 4K 3.378 3.326 6.044 6.195
> > 8K 5.637 5.676 10.141 11.287
> > 16K 8.250 8.402 15.976 16.736
> > 32K 13.327 13.204 19.013 20.515
> > 64K 21.241 21.341 20.973 21.879
> > 128K 21.851 22.354 21.816 23.203
> > 256K 21.408 21.693 21.846 24.088
> > 512K 21.600 21.899 21.921 24.106
> >
> > guest -> host [Gbps]
> > pkt_size before opt p 1 p 2+3 p 4+5
> >
> > 32 0.045 0.046 0.057 0.057
> > 64 0.089 0.091 0.103 0.104
> > 128 0.170 0.179 0.192 0.200
> > 256 0.364 0.351 0.361 0.379
> > 512 0.709 0.699 0.731 0.790
> > 1K 1.399 1.407 1.395 1.427
> > 2K 2.670 2.684 2.745 2.835
> > 4K 5.171 5.199 5.305 5.451
> > 8K 8.442 8.500 10.083 9.941
> > 16K 12.305 12.259 13.519 15.385
> > 32K 11.418 11.150 11.988 24.680
> > 64K 10.778 10.659 11.589 35.273
> > 128K 10.421 10.339 10.939 40.338
> > 256K 10.300 9.719 10.508 36.562
> > 512K 9.833 9.808 10.612 35.979
> >
> > As Stefan suggested in the v1, I measured also the efficiency in this way:
> > efficiency = Mbps / (%CPU_Host + %CPU_Guest)
> >
> > The '%CPU_Guest' is taken inside the VM. I know that it is not the best way,
> > but it's provided for free from iperf3 and could be an indication.
> >
> > host -> guest efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> > pkt_size before opt p 1 p 2+3 p 4+5
> >
> > 32 0.35 0.45 0.79 1.02
> > 64 0.56 0.80 1.41 1.54
> > 128 1.11 1.52 3.03 3.12
> > 256 2.20 2.16 5.44 5.58
> > 512 4.17 4.18 10.96 11.46
> > 1K 8.30 8.26 20.99 20.89
> > 2K 16.82 16.31 39.76 39.73
> > 4K 30.89 30.79 74.07 75.73
> > 8K 53.74 54.49 124.24 148.91
> > 16K 80.68 83.63 200.21 232.79
> > 32K 132.27 132.52 260.81 357.07
> > 64K 229.82 230.40 300.19 444.18
> > 128K 332.60 329.78 331.51 492.28
> > 256K 331.06 337.22 339.59 511.59
> > 512K 335.58 328.50 331.56 504.56
> >
> > guest -> host efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> > pkt_size before opt p 1 p 2+3 p 4+5
> >
> > 32 0.43 0.43 0.53 0.56
> > 64 0.85 0.86 1.04 1.10
> > 128 1.63 1.71 2.07 2.13
> > 256 3.48 3.35 4.02 4.22
> > 512 6.80 6.67 7.97 8.63
> > 1K 13.32 13.31 15.72 15.94
> > 2K 25.79 25.92 30.84 30.98
> > 4K 50.37 50.48 58.79 59.69
> > 8K 95.90 96.15 107.04 110.33
> > 16K 145.80 145.43 143.97 174.70
> > 32K 147.06 144.74 146.02 282.48
> > 64K 145.25 143.99 141.62 406.40
> > 128K 149.34 146.96 147.49 489.34
> > 256K 156.35 149.81 152.21 536.37
> > 512K 151.65 150.74 151.52 519.93
> >
> > [1] https://github.com/stefano-garzarella/iperf/
> >
> > Stefano Garzarella (5):
> > vsock/virtio: limit the memory used per-socket
> > vsock/virtio: reduce credit update messages
> > vsock/virtio: fix locking in virtio_transport_inc_tx_pkt()
> > vhost/vsock: split packets to send using multiple buffers
> > vsock/virtio: change the maximum packet size allowed
> >
> > drivers/vhost/vsock.c | 68 ++++++++++++-----
> > include/linux/virtio_vsock.h | 4 +-
> > net/vmw_vsock/virtio_transport.c | 1 +
> > net/vmw_vsock/virtio_transport_common.c | 99 ++++++++++++++++++++-----
> > 4 files changed, 134 insertions(+), 38 deletions(-)
> >
> > --
> > 2.20.1
^ permalink raw reply
* Re: [PATCH 03/12] block: bio_release_pages: use flags arg instead of bool
From: Jerome Glisse @ 2019-07-30 15:57 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Christoph Hellwig, john.hubbard, Andrew Morton, Alexander Viro,
Anna Schumaker, David S . Miller, Dominique Martinet,
Eric Van Hensbergen, Jason Gunthorpe, Jason Wang, Jens Axboe,
Latchesar Ionkov, Michael S . Tsirkin, Miklos Szeredi,
Trond Myklebust, Matthew Wilcox, linux-mm, LKML, ceph-devel, kvm,
linux-block, linux-cifs, linux-fsdevel, linux-nfs, linux-rdma,
netdev, samba-technical, v9fs-developer, virtualization,
John Hubbard, Minwoo Im
In-Reply-To: <20190730102557.GA1700@lst.de>
On Tue, Jul 30, 2019 at 12:25:57PM +0200, Christoph Hellwig wrote:
> On Mon, Jul 29, 2019 at 04:57:21PM -0400, Jerome Glisse wrote:
> > > All pages releases by bio_release_pages should come from
> > > get_get_user_pages, so I don't really see the point here.
> >
> > No they do not all comes from GUP for see various callers
> > of bio_check_pages_dirty() for instance iomap_dio_zero()
> >
> > I have carefully tracked down all this and i did not do
> > anyconvertion just for the fun of it :)
>
> Well, the point is _should_ not necessarily do. iomap_dio_zero adds the
> ZERO_PAGE, which we by definition don't need to refcount. So we can
> mark this bio BIO_NO_PAGE_REF safely after removing the get_page there.
>
> Note that the equivalent in the old direct I/O code, dio_refill_pages,
> will be a little more complicated as it can match user pages and the
> ZERO_PAGE in a single bio, so a per-bio flag won't handle it easily.
> Maybe we just need to use a separate bio there as well.
>
> In general with series like this we should not encode the status quo an
> pile new hacks upon the old one, but thing where we should be and fix
> up the old warts while having to wade through all that code.
Other user can also add page that are not coming from GUP but need to
have a reference see __blkdev_direct_IO() saddly bio get fill from many
different places and not always with GUP. So we can not say that all
pages here are coming from bio. I had a different version of the patchset
i think that was adding a new release dirty function for GUP versus non
GUP bio. I posted it a while ago, i will try to dig it up once i am
back.
Cheers,
Jérôme
^ permalink raw reply
* Re: [PATCH 4/4] net: dsa: mv88e6xxx: add PTP support for MV88E6250 family
From: Richard Cochran @ 2019-07-30 16:00 UTC (permalink / raw)
To: Hubert Feurstein
Cc: netdev, linux-kernel, Andrew Lunn, Vivien Didelot,
Florian Fainelli, David S. Miller, Rasmus Villemoes
In-Reply-To: <20190730100429.32479-5-h.feurstein@gmail.com>
On Tue, Jul 30, 2019 at 12:04:29PM +0200, Hubert Feurstein wrote:
> diff --git a/drivers/net/dsa/mv88e6xxx/ptp.c b/drivers/net/dsa/mv88e6xxx/ptp.c
> index 768d256f7c9f..51cdf4712517 100644
> --- a/drivers/net/dsa/mv88e6xxx/ptp.c
> +++ b/drivers/net/dsa/mv88e6xxx/ptp.c
> @@ -15,11 +15,38 @@
> #include "hwtstamp.h"
> #include "ptp.h"
>
> -/* Raw timestamps are in units of 8-ns clock periods. */
> -#define CC_SHIFT 28
> -#define CC_MULT (8 << CC_SHIFT)
> -#define CC_MULT_NUM (1 << 9)
> -#define CC_MULT_DEM 15625ULL
> +/* The adjfine API clamps ppb between [-32,768,000, 32,768,000], and
That is not true.
> + * therefore scaled_ppm between [-2,147,483,648, 2,147,483,647].
> + * Set the maximum supported ppb to a round value smaller than the maximum.
> + *
> + * Percentually speaking, this is a +/- 0.032x adjustment of the
> + * free-running counter (0.968x to 1.032x).
> + */
> +#define MV88E6XXX_MAX_ADJ_PPB 32000000
I had set an arbitrary limit of 1000 ppm. I can't really see any
point in raising the limit.
> +/* Family MV88E6250:
> + * Raw timestamps are in units of 10-ns clock periods.
> + *
> + * clkadj = scaled_ppm * 10*2^28 / (10^6 * 2^16)
> + * simplifies to
> + * clkadj = scaled_ppm * 2^7 / 5^5
> + */
> +#define MV88E6250_CC_SHIFT 28
> +#define MV88E6250_CC_MULT (10 << MV88E6250_CC_SHIFT)
> +#define MV88E6250_CC_MULT_NUM (1 << 7)
> +#define MV88E6250_CC_MULT_DEM 3125ULL
> +
> +/* Other families:
> + * Raw timestamps are in units of 8-ns clock periods.
> + *
> + * clkadj = scaled_ppm * 8*2^28 / (10^6 * 2^16)
> + * simplifies to
> + * clkadj = scaled_ppm * 2^9 / 5^6
> + */
> +#define MV88E6XXX_CC_SHIFT 28
> +#define MV88E6XXX_CC_MULT (8 << MV88E6XXX_CC_SHIFT)
> +#define MV88E6XXX_CC_MULT_NUM (1 << 9)
> +#define MV88E6XXX_CC_MULT_DEM 15625ULL
>
> #define TAI_EVENT_WORK_INTERVAL msecs_to_jiffies(100)
>
> @@ -179,24 +206,14 @@ static void mv88e6352_tai_event_work(struct work_struct *ugly)
> static int mv88e6xxx_ptp_adjfine(struct ptp_clock_info *ptp, long scaled_ppm)
> {
> struct mv88e6xxx_chip *chip = ptp_to_chip(ptp);
> - int neg_adj = 0;
> - u32 diff, mult;
> - u64 adj;
> + s64 adj;
>
> - if (scaled_ppm < 0) {
> - neg_adj = 1;
> - scaled_ppm = -scaled_ppm;
> - }
Please don't re-write this logic. It is written like that for a reason.
> - mult = CC_MULT;
> - adj = CC_MULT_NUM;
> - adj *= scaled_ppm;
> - diff = div_u64(adj, CC_MULT_DEM);
Just substitute CC_MULT* with your new chip->ptp_cc_mult*
and leave the rest alone.
Thanks,
Richard
^ permalink raw reply
* Re: [PATCH net-next v5 0/5] vsock/virtio: optimizations to increase the throughput
From: Stefano Garzarella @ 2019-07-30 16:02 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: netdev, virtualization, linux-kernel, Jason Wang, kvm,
David S. Miller, Stefan Hajnoczi
In-Reply-To: <20190730115503-mutt-send-email-mst@kernel.org>
On Tue, Jul 30, 2019 at 11:55:09AM -0400, Michael S. Tsirkin wrote:
> On Tue, Jul 30, 2019 at 11:54:53AM -0400, Michael S. Tsirkin wrote:
> > On Tue, Jul 30, 2019 at 05:43:29PM +0200, Stefano Garzarella wrote:
> > > This series tries to increase the throughput of virtio-vsock with slight
> > > changes.
> > > While I was testing the v2 of this series I discovered an huge use of memory,
> > > so I added patch 1 to mitigate this issue. I put it in this series in order
> > > to better track the performance trends.
> > >
> > > v5:
> > > - rebased all patches on net-next
> > > - added Stefan's R-b and Michael's A-b
> >
> > This doesn't solve all issues around allocation - as I mentioned I think
> > we will need to improve accounting for that,
> > and maybe add pre-allocation.
Yes, I'll work on it following your suggestions.
> > But it's a great series of steps in the right direction!
> >
Thank you very much :)
Stefano
>
>
> So
>
> Acked-by: Michael S. Tsirkin <mst@redhat.com>
>
> >
> > > v4: https://patchwork.kernel.org/cover/11047717
> > > v3: https://patchwork.kernel.org/cover/10970145
> > > v2: https://patchwork.kernel.org/cover/10938743
> > > v1: https://patchwork.kernel.org/cover/10885431
> > >
> > > Below are the benchmarks step by step. I used iperf3 [1] modified with VSOCK
> > > support. As Michael suggested in the v1, I booted host and guest with 'nosmap'.
> > >
> > > A brief description of patches:
> > > - Patches 1: limit the memory usage with an extra copy for small packets
> > > - Patches 2+3: reduce the number of credit update messages sent to the
> > > transmitter
> > > - Patches 4+5: allow the host to split packets on multiple buffers and use
> > > VIRTIO_VSOCK_MAX_PKT_BUF_SIZE as the max packet size allowed
> > >
> > > host -> guest [Gbps]
> > > pkt_size before opt p 1 p 2+3 p 4+5
> > >
> > > 32 0.032 0.030 0.048 0.051
> > > 64 0.061 0.059 0.108 0.117
> > > 128 0.122 0.112 0.227 0.234
> > > 256 0.244 0.241 0.418 0.415
> > > 512 0.459 0.466 0.847 0.865
> > > 1K 0.927 0.919 1.657 1.641
> > > 2K 1.884 1.813 3.262 3.269
> > > 4K 3.378 3.326 6.044 6.195
> > > 8K 5.637 5.676 10.141 11.287
> > > 16K 8.250 8.402 15.976 16.736
> > > 32K 13.327 13.204 19.013 20.515
> > > 64K 21.241 21.341 20.973 21.879
> > > 128K 21.851 22.354 21.816 23.203
> > > 256K 21.408 21.693 21.846 24.088
> > > 512K 21.600 21.899 21.921 24.106
> > >
> > > guest -> host [Gbps]
> > > pkt_size before opt p 1 p 2+3 p 4+5
> > >
> > > 32 0.045 0.046 0.057 0.057
> > > 64 0.089 0.091 0.103 0.104
> > > 128 0.170 0.179 0.192 0.200
> > > 256 0.364 0.351 0.361 0.379
> > > 512 0.709 0.699 0.731 0.790
> > > 1K 1.399 1.407 1.395 1.427
> > > 2K 2.670 2.684 2.745 2.835
> > > 4K 5.171 5.199 5.305 5.451
> > > 8K 8.442 8.500 10.083 9.941
> > > 16K 12.305 12.259 13.519 15.385
> > > 32K 11.418 11.150 11.988 24.680
> > > 64K 10.778 10.659 11.589 35.273
> > > 128K 10.421 10.339 10.939 40.338
> > > 256K 10.300 9.719 10.508 36.562
> > > 512K 9.833 9.808 10.612 35.979
> > >
> > > As Stefan suggested in the v1, I measured also the efficiency in this way:
> > > efficiency = Mbps / (%CPU_Host + %CPU_Guest)
> > >
> > > The '%CPU_Guest' is taken inside the VM. I know that it is not the best way,
> > > but it's provided for free from iperf3 and could be an indication.
> > >
> > > host -> guest efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> > > pkt_size before opt p 1 p 2+3 p 4+5
> > >
> > > 32 0.35 0.45 0.79 1.02
> > > 64 0.56 0.80 1.41 1.54
> > > 128 1.11 1.52 3.03 3.12
> > > 256 2.20 2.16 5.44 5.58
> > > 512 4.17 4.18 10.96 11.46
> > > 1K 8.30 8.26 20.99 20.89
> > > 2K 16.82 16.31 39.76 39.73
> > > 4K 30.89 30.79 74.07 75.73
> > > 8K 53.74 54.49 124.24 148.91
> > > 16K 80.68 83.63 200.21 232.79
> > > 32K 132.27 132.52 260.81 357.07
> > > 64K 229.82 230.40 300.19 444.18
> > > 128K 332.60 329.78 331.51 492.28
> > > 256K 331.06 337.22 339.59 511.59
> > > 512K 335.58 328.50 331.56 504.56
> > >
> > > guest -> host efficiency [Mbps / (%CPU_Host + %CPU_Guest)]
> > > pkt_size before opt p 1 p 2+3 p 4+5
> > >
> > > 32 0.43 0.43 0.53 0.56
> > > 64 0.85 0.86 1.04 1.10
> > > 128 1.63 1.71 2.07 2.13
> > > 256 3.48 3.35 4.02 4.22
> > > 512 6.80 6.67 7.97 8.63
> > > 1K 13.32 13.31 15.72 15.94
> > > 2K 25.79 25.92 30.84 30.98
> > > 4K 50.37 50.48 58.79 59.69
> > > 8K 95.90 96.15 107.04 110.33
> > > 16K 145.80 145.43 143.97 174.70
> > > 32K 147.06 144.74 146.02 282.48
> > > 64K 145.25 143.99 141.62 406.40
> > > 128K 149.34 146.96 147.49 489.34
> > > 256K 156.35 149.81 152.21 536.37
> > > 512K 151.65 150.74 151.52 519.93
> > >
> > > [1] https://github.com/stefano-garzarella/iperf/
> > >
> > > Stefano Garzarella (5):
> > > vsock/virtio: limit the memory used per-socket
> > > vsock/virtio: reduce credit update messages
> > > vsock/virtio: fix locking in virtio_transport_inc_tx_pkt()
> > > vhost/vsock: split packets to send using multiple buffers
> > > vsock/virtio: change the maximum packet size allowed
> > >
> > > drivers/vhost/vsock.c | 68 ++++++++++++-----
> > > include/linux/virtio_vsock.h | 4 +-
> > > net/vmw_vsock/virtio_transport.c | 1 +
> > > net/vmw_vsock/virtio_transport_common.c | 99 ++++++++++++++++++++-----
> > > 4 files changed, 134 insertions(+), 38 deletions(-)
> > >
> > > --
> > > 2.20.1
--
^ permalink raw reply
* Re: [net-next 08/13] net/mlx5e: Protect tc flows hashtable with rcu
From: Willem de Bruijn @ 2019-07-30 16:15 UTC (permalink / raw)
To: Saeed Mahameed
Cc: David S. Miller, netdev@vger.kernel.org, Vlad Buslov, Jianbo Liu,
Roi Dayan
In-Reply-To: <20190729234934.23595-9-saeedm@mellanox.com>
On Mon, Jul 29, 2019 at 7:50 PM Saeed Mahameed <saeedm@mellanox.com> wrote:
>
> From: Vlad Buslov <vladbu@mellanox.com>
>
> In order to remove dependency on rtnl lock, access to tc flows hashtable
> must be explicitly protected from concurrent flows removal.
>
> Extend tc flow structure with rcu to allow concurrent parallel access. Use
> rcu read lock to safely lookup flow in tc flows hash table, and take
> reference to it. Use rcu free for flow deletion to accommodate concurrent
> stats requests.
>
> Add new DELETED flow flag. Imlement new flow_flag_test_and_set() helper
> that is used to set a flag and return its previous value. Use it to
> atomically set the flag in mlx5e_delete_flower() to guarantee that flow can
> only be deleted once, even when same flow is deleted concurrently by
> multiple tasks.
>
> Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
> Reviewed-by: Jianbo Liu <jianbol@mellanox.com>
> Reviewed-by: Roi Dayan <roid@mellanox.com>
> Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
> ---
> @@ -3492,16 +3507,32 @@ int mlx5e_delete_flower(struct net_device *dev, struct mlx5e_priv *priv,
> {
> struct rhashtable *tc_ht = get_tc_ht(priv, flags);
> struct mlx5e_tc_flow *flow;
> + int err;
>
> + rcu_read_lock();
> flow = rhashtable_lookup_fast(tc_ht, &f->cookie, tc_ht_params);
> - if (!flow || !same_flow_direction(flow, flags))
> - return -EINVAL;
> + if (!flow || !same_flow_direction(flow, flags)) {
> + err = -EINVAL;
> + goto errout;
> + }
>
> + /* Only delete the flow if it doesn't have MLX5E_TC_FLOW_DELETED flag
> + * set.
> + */
> + if (flow_flag_test_and_set(flow, DELETED)) {
> + err = -EINVAL;
> + goto errout;
> + }
> rhashtable_remove_fast(tc_ht, &flow->node, tc_ht_params);
> + rcu_read_unlock();
>
> mlx5e_flow_put(priv, flow);
Dereferencing flow outside rcu readside critical section? Does a build
with lockdep not complain?
>
> return 0;
> +
> +errout:
> + rcu_read_unlock();
> + return err;
> }
>
> int mlx5e_stats_flower(struct net_device *dev, struct mlx5e_priv *priv,
> @@ -3517,8 +3548,10 @@ int mlx5e_stats_flower(struct net_device *dev, struct mlx5e_priv *priv,
> u64 bytes = 0;
> int err = 0;
>
> - flow = mlx5e_flow_get(rhashtable_lookup_fast(tc_ht, &f->cookie,
> - tc_ht_params));
> + rcu_read_lock();
> + flow = mlx5e_flow_get(rhashtable_lookup(tc_ht, &f->cookie,
> + tc_ht_params));
> + rcu_read_unlock();
> if (IS_ERR(flow))
> return PTR_ERR(flow);
Same, in code below this check?
^ permalink raw reply
* Re: [PATCH 4/4] net: dsa: mv88e6xxx: add PTP support for MV88E6250 family
From: Hubert Feurstein @ 2019-07-30 16:20 UTC (permalink / raw)
To: Richard Cochran
Cc: netdev, linux-kernel, Andrew Lunn, Vivien Didelot,
Florian Fainelli, David S. Miller, Rasmus Villemoes
In-Reply-To: <20190730160032.GA1251@localhost>
Hi Richard,
thank you for your comments.
Am Di., 30. Juli 2019 um 18:00 Uhr schrieb Richard Cochran
<richardcochran@gmail.com>:
[...]
> > -/* Raw timestamps are in units of 8-ns clock periods. */
> > -#define CC_SHIFT 28
> > -#define CC_MULT (8 << CC_SHIFT)
> > -#define CC_MULT_NUM (1 << 9)
> > -#define CC_MULT_DEM 15625ULL
> > +/* The adjfine API clamps ppb between [-32,768,000, 32,768,000], and
>
> That is not true.
>
> > + * therefore scaled_ppm between [-2,147,483,648, 2,147,483,647].
> > + * Set the maximum supported ppb to a round value smaller than the maximum.
> > + *
> > + * Percentually speaking, this is a +/- 0.032x adjustment of the
> > + * free-running counter (0.968x to 1.032x).
> > + */
> > +#define MV88E6XXX_MAX_ADJ_PPB 32000000
>
> I had set an arbitrary limit of 1000 ppm. I can't really see any
> point in raising the limit.
>
> > +/* Family MV88E6250:
> > + * Raw timestamps are in units of 10-ns clock periods.
> > + *
> > + * clkadj = scaled_ppm * 10*2^28 / (10^6 * 2^16)
> > + * simplifies to
> > + * clkadj = scaled_ppm * 2^7 / 5^5
> > + */
> > +#define MV88E6250_CC_SHIFT 28
> > +#define MV88E6250_CC_MULT (10 << MV88E6250_CC_SHIFT)
> > +#define MV88E6250_CC_MULT_NUM (1 << 7)
> > +#define MV88E6250_CC_MULT_DEM 3125ULL
> > +
> > +/* Other families:
> > + * Raw timestamps are in units of 8-ns clock periods.
> > + *
> > + * clkadj = scaled_ppm * 8*2^28 / (10^6 * 2^16)
> > + * simplifies to
> > + * clkadj = scaled_ppm * 2^9 / 5^6
> > + */
> > +#define MV88E6XXX_CC_SHIFT 28
> > +#define MV88E6XXX_CC_MULT (8 << MV88E6XXX_CC_SHIFT)
> > +#define MV88E6XXX_CC_MULT_NUM (1 << 9)
> > +#define MV88E6XXX_CC_MULT_DEM 15625ULL
> >
> > #define TAI_EVENT_WORK_INTERVAL msecs_to_jiffies(100)
> >
> > @@ -179,24 +206,14 @@ static void mv88e6352_tai_event_work(struct work_struct *ugly)
> > static int mv88e6xxx_ptp_adjfine(struct ptp_clock_info *ptp, long scaled_ppm)
> > {
> > struct mv88e6xxx_chip *chip = ptp_to_chip(ptp);
> > - int neg_adj = 0;
> > - u32 diff, mult;
> > - u64 adj;
> > + s64 adj;
> >
> > - if (scaled_ppm < 0) {
> > - neg_adj = 1;
> > - scaled_ppm = -scaled_ppm;
> > - }
>
> Please don't re-write this logic. It is written like that for a reason.
I used the sja1105_ptp.c as a reference. So it is also wrong there.
Hubert
^ permalink raw reply
* Re: [net-next 08/13] net/mlx5e: Protect tc flows hashtable with rcu
From: Willem de Bruijn @ 2019-07-30 16:37 UTC (permalink / raw)
To: Saeed Mahameed
Cc: David S. Miller, netdev@vger.kernel.org, Vlad Buslov, Jianbo Liu,
Roi Dayan
In-Reply-To: <CA+FuTSfnikCV_J2cUEeafCaui8KxrK4njRR9rqgpo+5JhBxR9g@mail.gmail.com>
On Tue, Jul 30, 2019 at 12:16 PM Willem de Bruijn
<willemdebruijn.kernel@gmail.com> wrote:
>
> On Mon, Jul 29, 2019 at 7:50 PM Saeed Mahameed <saeedm@mellanox.com> wrote:
> >
> > From: Vlad Buslov <vladbu@mellanox.com>
> >
> > In order to remove dependency on rtnl lock, access to tc flows hashtable
> > must be explicitly protected from concurrent flows removal.
> >
> > Extend tc flow structure with rcu to allow concurrent parallel access. Use
> > rcu read lock to safely lookup flow in tc flows hash table, and take
> > reference to it. Use rcu free for flow deletion to accommodate concurrent
> > stats requests.
> >
> > Add new DELETED flow flag. Imlement new flow_flag_test_and_set() helper
> > that is used to set a flag and return its previous value. Use it to
> > atomically set the flag in mlx5e_delete_flower() to guarantee that flow can
> > only be deleted once, even when same flow is deleted concurrently by
> > multiple tasks.
> >
> > Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
> > Reviewed-by: Jianbo Liu <jianbol@mellanox.com>
> > Reviewed-by: Roi Dayan <roid@mellanox.com>
> > Signed-off-by: Saeed Mahameed <saeedm@mellanox.com>
> > ---
>
> > @@ -3492,16 +3507,32 @@ int mlx5e_delete_flower(struct net_device *dev, struct mlx5e_priv *priv,
> > {
> > struct rhashtable *tc_ht = get_tc_ht(priv, flags);
> > struct mlx5e_tc_flow *flow;
> > + int err;
> >
> > + rcu_read_lock();
> > flow = rhashtable_lookup_fast(tc_ht, &f->cookie, tc_ht_params);
> > - if (!flow || !same_flow_direction(flow, flags))
> > - return -EINVAL;
> > + if (!flow || !same_flow_direction(flow, flags)) {
> > + err = -EINVAL;
> > + goto errout;
> > + }
> >
> > + /* Only delete the flow if it doesn't have MLX5E_TC_FLOW_DELETED flag
> > + * set.
> > + */
> > + if (flow_flag_test_and_set(flow, DELETED)) {
> > + err = -EINVAL;
> > + goto errout;
> > + }
> > rhashtable_remove_fast(tc_ht, &flow->node, tc_ht_params);
> > + rcu_read_unlock();
> >
> > mlx5e_flow_put(priv, flow);
>
> Dereferencing flow outside rcu readside critical section? Does a build
> with lockdep not complain?
Eh no, it won't. The surprising part to me was to use a readside
critical section when performing a write action on an RCU ptr. The
DELETED flag ensures that multiple writers will not compete to call
rhashtable_remove_fast. rcu_read_lock is a common pattern to do
rhashtable lookup + delete.
>
> >
> > return 0;
> > +
> > +errout:
> > + rcu_read_unlock();
> > + return err;
> > }
> >
> > int mlx5e_stats_flower(struct net_device *dev, struct mlx5e_priv *priv,
> > @@ -3517,8 +3548,10 @@ int mlx5e_stats_flower(struct net_device *dev, struct mlx5e_priv *priv,
> > u64 bytes = 0;
> > int err = 0;
> >
> > - flow = mlx5e_flow_get(rhashtable_lookup_fast(tc_ht, &f->cookie,
> > - tc_ht_params));
> > + rcu_read_lock();
> > + flow = mlx5e_flow_get(rhashtable_lookup(tc_ht, &f->cookie,
> > + tc_ht_params));
> > + rcu_read_unlock();
> > if (IS_ERR(flow))
> > return PTR_ERR(flow);
>
> Same, in code below this check?
Never mind, sorry. I missed that this took a reference on the ptr
returned from rhashtable_lookup.
^ permalink raw reply
* Re: [PATCH net-next v4 0/4] enetc: Add mdio bus driver for the PCIe MDIO endpoint
From: David Miller @ 2019-07-30 16:44 UTC (permalink / raw)
To: claudiu.manoil
Cc: andrew, robh+dt, leoyang.li, alexandru.marginean, netdev,
devicetree, linux-arm-kernel, linux-kernel
In-Reply-To: <1564479919-18835-1-git-send-email-claudiu.manoil@nxp.com>
From: Claudiu Manoil <claudiu.manoil@nxp.com>
Date: Tue, 30 Jul 2019 12:45:15 +0300
> First patch fixes a sparse issue and cleans up accessors to avoid
> casting to __iomem.
> Second patch just registers the PCIe endpoint device containing
> the MDIO registers as a standalone MDIO bus driver, to allow
> an alternative way to control the MDIO bus. The same code used
> by the ENETC ports (eth controllers) to manage MDIO via local
> registers applies and is reused.
>
> Bindings are provided for the new MDIO node, similarly to ENETC
> port nodes bindings.
>
> Last patch enables the ENETC port 1 and its RGMII PHY on the
> LS1028A QDS board, where the MDIO muxing configuration relies
> on the MDIO support provided in the first patch.
...
Series applied, thank you.
^ permalink raw reply
* Re: [PATCH net-next v4 0/4] enetc: Add mdio bus driver for the PCIe MDIO endpoint
From: David Miller @ 2019-07-30 16:53 UTC (permalink / raw)
To: claudiu.manoil
Cc: andrew, robh+dt, leoyang.li, alexandru.marginean, netdev,
devicetree, linux-arm-kernel, linux-kernel
In-Reply-To: <20190730.094436.855806617449032791.davem@davemloft.net>
From: David Miller <davem@davemloft.net>
Date: Tue, 30 Jul 2019 09:44:36 -0700 (PDT)
> From: Claudiu Manoil <claudiu.manoil@nxp.com>
> Date: Tue, 30 Jul 2019 12:45:15 +0300
>
>> First patch fixes a sparse issue and cleans up accessors to avoid
>> casting to __iomem.
>> Second patch just registers the PCIe endpoint device containing
>> the MDIO registers as a standalone MDIO bus driver, to allow
>> an alternative way to control the MDIO bus. The same code used
>> by the ENETC ports (eth controllers) to manage MDIO via local
>> registers applies and is reused.
>>
>> Bindings are provided for the new MDIO node, similarly to ENETC
>> port nodes bindings.
>>
>> Last patch enables the ENETC port 1 and its RGMII PHY on the
>> LS1028A QDS board, where the MDIO muxing configuration relies
>> on the MDIO support provided in the first patch.
> ...
>
> Series applied, thank you.
Actually this doesn't compile, I had to revert:
In file included from ./include/linux/phy.h:20,
from ./include/linux/of_mdio.h:11,
from drivers/net/ethernet/freescale/enetc/enetc_mdio.c:5:
drivers/net/ethernet/freescale/enetc/enetc_mdio.c:284:26: error: ‘enetc_mdio_id_table’ undeclared here (not in a function); did you mean ‘enetc_pci_mdio_id_table’?
MODULE_DEVICE_TABLE(pci, enetc_mdio_id_table);
^~~~~~~~~~~~~~~~~~~
./include/linux/module.h:230:15: note: in definition of macro ‘MODULE_DEVICE_TABLE’
extern typeof(name) __mod_##type##__##name##_device_table \
^~~~
./include/linux/module.h:230:21: error: ‘__mod_pci__enetc_mdio_id_table_device_table’ aliased to undefined symbol ‘enetc_mdio_id_table’
extern typeof(name) __mod_##type##__##name##_device_table \
^~~~~~
drivers/net/ethernet/freescale/enetc/enetc_mdio.c:284:1: note: in expansion of macro ‘MODULE_DEVICE_TABLE’
MODULE_DEVICE_TABLE(pci, enetc_mdio_id_table);
^~~~~~~~~~~~~~~~~~~
^ permalink raw reply
* Re: [PATCH] net: phy: fixed_phy: print gpio error only if gpio node is present
From: David Miller @ 2019-07-30 16:55 UTC (permalink / raw)
To: h.feurstein; +Cc: netdev, linux-kernel, andrew, f.fainelli, hkallweit1
In-Reply-To: <20190730094623.31640-1-h.feurstein@gmail.com>
From: Hubert Feurstein <h.feurstein@gmail.com>
Date: Tue, 30 Jul 2019 11:46:23 +0200
> It is perfectly ok to not have an gpio attached to the fixed-link node. So
> the driver should not throw an error message when the gpio is missing.
>
> Signed-off-by: Hubert Feurstein <h.feurstein@gmail.com>
Applied and queued up for -stable, thanks.
^ permalink raw reply
* Re: [PATCH] net: dsa: mv88e6xxx: use link-down-define instead of plain value
From: David Miller @ 2019-07-30 16:56 UTC (permalink / raw)
To: h.feurstein; +Cc: netdev, linux-kernel, andrew, vivien.didelot, f.fainelli
In-Reply-To: <20190730101142.548-1-h.feurstein@gmail.com>
From: Hubert Feurstein <h.feurstein@gmail.com>
Date: Tue, 30 Jul 2019 12:11:42 +0200
> Using the define here makes the code more expressive.
>
> Signed-off-by: Hubert Feurstein <h.feurstein@gmail.com>
Applied, thank you.
^ permalink raw reply
* Re: [PATCH v2 0/2] mmc: core: Fix Marvell WiFi reset by adding SDIO API to replug card
From: Doug Anderson @ 2019-07-30 16:59 UTC (permalink / raw)
To: Andreas Fenkart
Cc: Ulf Hansson, Kalle Valo, Adrian Hunter, Ganapathi Bhat,
linux-wireless, Brian Norris, Amitkumar Karwar,
open list:ARM/Rockchip SoC..., Wolfram Sang, Nishant Sarmukadam,
netdev, Avri Altman, linux-mmc, David Miller, Xinming Hu, LKML,
Thomas Gleixner, Kate Stewart
In-Reply-To: <CALtMJEB871Redpzx1u6G5GVEXz-kAP=vT6Wt98=X=xm4SEMeAQ@mail.gmail.com>
Hi,
On Tue, Jul 30, 2019 at 1:47 AM Andreas Fenkart <afenkart@gmail.com> wrote:
>
> > * Sometimes while I was testing I saw "Fail WiFi 1" indicating a
> > transitory failure. Usually this was an association failure, but in
> > one case I saw the device do "Firmware wakeup failed" after I
> > triggered the reset. This caused the driver to trigger a re-reset
> > of itself which eventually recovered things. This was good because
> > it was an actual test of the normal reset flow (not the one
> > triggered via sysfs).
>
> This error triggers something. I remember that when I was working on
> suspend-to-ram feature, we had problems to wake up the firmware
> reliable. I found this patch in one of my old 3.13 tree
>
> the missing bit -- ugly hack to force cmd52 before cmd53.
Thanks for the reference! At the moment I'm not terribly worried
about this particular failure case (compared to other failure modes)
because it's rare and it self-heals.
...my best guess, though, is that the problem isn't exactly the same.
The "Firmware wakeup failed" is a pretty generic error message, kind
of like "something went wrong" and not all instances of this message
will have the same root cause.
I actually dealt with a few suspend/resume issues around mwifiex
recently though. If you ever uprev, you might be interested in:
b82d6c1f8f82 mwifiex: Make resume actually do something useful again
on SDIO cards
83293386bc95 mmc: core: Prevent processing SDIO IRQs when the card is suspended
-Doug
^ permalink raw reply
* [PATCH bpf-next v4 00/11] XDP unaligned chunk placement support
From: Kevin Laatz @ 2019-07-30 8:53 UTC (permalink / raw)
To: netdev, ast, daniel, bjorn.topel, magnus.karlsson, jakub.kicinski,
jonathan.lemon, saeedm, maximmi, stephen
Cc: bruce.richardson, ciara.loftus, bpf, intel-wired-lan, Kevin Laatz
In-Reply-To: <20190724051043.14348-1-kevin.laatz@intel.com>
This patch set adds the ability to use unaligned chunks in the XDP umem.
Currently, all chunk addresses passed to the umem are masked to be chunk
size aligned (max is PAGE_SIZE). This limits where we can place chunks
within the umem as well as limiting the packet sizes that are supported.
The changes in this patch set removes these restrictions, allowing XDP to
be more flexible in where it can place a chunk within a umem. By relaxing
where the chunks can be placed, it allows us to use an arbitrary buffer
size and place that wherever we have a free address in the umem. These
changes add the ability to support arbitrary frame sizes up to 4k
(PAGE_SIZE) and make it easy to integrate with other existing frameworks
that have their own memory management systems, such as DPDK.
In DPDK, for example, there is already support for AF_XDP with zero-copy.
However, with this patch set the integration will be much more seamless.
You can find the DPDK AF_XDP driver at:
https://git.dpdk.org/dpdk/tree/drivers/net/af_xdp
Since we are now dealing with arbitrary frame sizes, we need also need to
update how we pass around addresses. Currently, the addresses can simply be
masked to 2k to get back to the original address. This becomes less trivial
when using frame sizes that are not a 'power of 2' size. This patch set
modifies the Rx/Tx descriptor format to use the upper 16-bits of the addr
field for an offset value, leaving the lower 48-bits for the address (this
leaves us with 256 Terabytes, which should be enough!). We only need to use
the upper 16-bits to store the offset when running in unaligned mode.
Rather than adding the offset (headroom etc) to the address, we will store
it in the upper 16-bits of the address field. This way, we can easily add
the offset to the address where we need it, using some bit manipulation and
addition, and we can also easily get the original address wherever we need
it (for example in i40e_zca_free) by simply masking to get the lower
48-bits of the address field.
The numbers below were recorded with the following set up:
- Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
- Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
- Driver: i40e
- Application: xdpsock with l2fwd (single interface)
- Turbo disabled in BIOS
These are solely for comparing performance with and without the patches.
The largest drop was ~1.5% (in zero-copy mode).
+-------------------------+-------------+-----------------+-------------+
| Buffer size: 2048 | SKB mode | Zero-copy | Copy |
+-------------------------+-------------+-----------------+-------------+
| Aligned (baseline) | 1.25 Mpps | 10.0 Mpps | 1.66 Mpps |
+-------------------------+-------------+-----------------+-------------+
| Aligned (with patches) | 1.25 Mpps | 9.85 Mpps | 1.66 Mpps |
+-------------------------+-------------+-----------------+-------------+
| Unaligned | 1.25 Mpps | 9.65 Mpps | 1.66 Mpps |
+-------------------------+-------------+-----------------+-------------+
This patch set has been applied against
commit 475e31f8da1b("Merge branch 'revamp-test_progs'")
Structure of the patch set:
Patch 1:
- Remove unnecessary masking and headroom addition during zero-copy Rx
buffer recycling in i40e. This change is required in order for the
buffer recycling to work in the unaligned chunk mode.
Patch 2:
- Remove unnecessary masking and headroom addition during
zero-copy Rx buffer recycling in ixgbe. This change is required in
order for the buffer recycling to work in the unaligned chunk mode.
Patch 3:
- Add flags for umem configuration to libbpf
Patch 4:
- Add infrastructure for unaligned chunks. Since we are dealing with
unaligned chunks that could potentially cross a physical page boundary,
we add checks to keep track of that information. We can later use this
information to correctly handle buffers that are placed at an address
where they cross a page boundary. This patch also modifies the
existing Rx and Tx functions to use the new descriptor format. To
handle addresses correctly, we need to mask appropriately based on
whether we are in aligned or unaligned mode.
Patch 5:
- This patch updates the i40e driver to make use of the new descriptor
format.
Patch 6:
- This patch updates the ixgbe driver to make use of the new descriptor
format.
Patch 7:
- This patch updates the mlx5e driver to make use of the new descriptor
format. These changes are required to handle the new descriptor format
and for unaligned chunks support.
Patch 8:
- Modify xdpsock application to add a command line option for
unaligned chunks
Patch 9:
- Since we can now run the application in unaligned chunk mode, we need
to make sure we recycle the buffers appropriately.
Patch 10:
- Adds hugepage support to the xdpsock application
Patch 11:
- Documentation update to include the unaligned chunk scenario. We need
to explicitly state that the incoming addresses are only masked in the
aligned chunk mode and not the unaligned chunk mode.
---
v2:
- fixed checkpatch issues
- fixed Rx buffer recycling for unaligned chunks in xdpsock
- removed unused defines
- fixed how chunk_size is calculated in xsk_diag.c
- added some performance numbers to cover letter
- modified descriptor format to make it easier to retrieve original
address
- removed patch adding off_t off to the zero copy allocator. This is no
longer needed with the new descriptor format.
v3:
- added patch for mlx5 driver changes needed for unaligned chunks
- moved offset handling to new helper function
- changed value used for the umem chunk_mask. Now using the new
descriptor format to save us doing the calculations in a number of
places meaning more of the code is left unchanged while adding
unaligned chunk support.
v4:
- reworked the next_pg_contig field in the xdp_umem_page struct. We now
use the low 12 bits of the addr for flags rather than adding an extra
field in the struct.
- modified unaligned chunks flag define
- fixed page_start calculation in __xsk_rcv_memcpy().
- move offset handling to the xdp_umem_get_* functions
- modified the len field in xdp_umem_reg struct. We now use 16 bits from
this for the flags field.
- removed next_pg_contig field from xdp_umem_page struct. Using low 12
bits of addr to store flags instead.
- fixed headroom addition to handle in the mlx5e driver
- other minor changes based on review comments
Kevin Laatz (11):
i40e: simplify Rx buffer recycle
ixgbe: simplify Rx buffer recycle
libbpf: add flags to umem config
xsk: add support to allow unaligned chunk placement
i40e: modify driver for handling offsets
ixgbe: modify driver for handling offsets
mlx5e: modify driver for handling offsets
samples/bpf: add unaligned chunks mode support to xdpsock
samples/bpf: add buffer recycling for unaligned chunks to xdpsock
samples/bpf: use hugepages in xdpsock app
doc/af_xdp: include unaligned chunk case
Documentation/networking/af_xdp.rst | 10 ++-
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 26 +++---
drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 26 +++---
.../net/ethernet/mellanox/mlx5/core/en/xdp.c | 8 +-
.../ethernet/mellanox/mlx5/core/en/xsk/rx.c | 3 +-
include/net/xdp_sock.h | 40 +++++++++-
include/uapi/linux/if_xdp.h | 14 +++-
net/xdp/xdp_umem.c | 18 +++--
net/xdp/xsk.c | 79 +++++++++++++++----
net/xdp/xsk_diag.c | 2 +-
net/xdp/xsk_queue.h | 69 ++++++++++++++--
samples/bpf/xdpsock_user.c | 59 ++++++++++----
tools/include/uapi/linux/if_xdp.h | 9 ++-
tools/lib/bpf/xsk.c | 3 +
tools/lib/bpf/xsk.h | 2 +
15 files changed, 280 insertions(+), 88 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH bpf-next v4 01/11] i40e: simplify Rx buffer recycle
From: Kevin Laatz @ 2019-07-30 8:53 UTC (permalink / raw)
To: netdev, ast, daniel, bjorn.topel, magnus.karlsson, jakub.kicinski,
jonathan.lemon, saeedm, maximmi, stephen
Cc: bruce.richardson, ciara.loftus, bpf, intel-wired-lan, Kevin Laatz
In-Reply-To: <20190730085400.10376-1-kevin.laatz@intel.com>
Currently, the dma, addr and handle are modified when we reuse Rx buffers
in zero-copy mode. However, this is not required as the inputs to the
function are copies, not the original values themselves. As we use the
copies within the function, we can use the original 'old_bi' values
directly without having to mask and add the headroom.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
---
drivers/net/ethernet/intel/i40e/i40e_xsk.c | 13 +++----------
1 file changed, 3 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
index 32bad014d76c..dfa096db2244 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c
@@ -420,8 +420,6 @@ static void i40e_reuse_rx_buffer_zc(struct i40e_ring *rx_ring,
struct i40e_rx_buffer *old_bi)
{
struct i40e_rx_buffer *new_bi = &rx_ring->rx_bi[rx_ring->next_to_alloc];
- unsigned long mask = (unsigned long)rx_ring->xsk_umem->chunk_mask;
- u64 hr = rx_ring->xsk_umem->headroom + XDP_PACKET_HEADROOM;
u16 nta = rx_ring->next_to_alloc;
/* update, and store next to alloc */
@@ -429,14 +427,9 @@ static void i40e_reuse_rx_buffer_zc(struct i40e_ring *rx_ring,
rx_ring->next_to_alloc = (nta < rx_ring->count) ? nta : 0;
/* transfer page from old buffer to new buffer */
- new_bi->dma = old_bi->dma & mask;
- new_bi->dma += hr;
-
- new_bi->addr = (void *)((unsigned long)old_bi->addr & mask);
- new_bi->addr += hr;
-
- new_bi->handle = old_bi->handle & mask;
- new_bi->handle += rx_ring->xsk_umem->headroom;
+ new_bi->dma = old_bi->dma;
+ new_bi->addr = old_bi->addr;
+ new_bi->handle = old_bi->handle;
old_bi->addr = NULL;
}
--
2.17.1
^ permalink raw reply related
* [PATCH bpf-next v4 02/11] ixgbe: simplify Rx buffer recycle
From: Kevin Laatz @ 2019-07-30 8:53 UTC (permalink / raw)
To: netdev, ast, daniel, bjorn.topel, magnus.karlsson, jakub.kicinski,
jonathan.lemon, saeedm, maximmi, stephen
Cc: bruce.richardson, ciara.loftus, bpf, intel-wired-lan, Kevin Laatz
In-Reply-To: <20190730085400.10376-1-kevin.laatz@intel.com>
Currently, the dma, addr and handle are modified when we reuse Rx buffers
in zero-copy mode. However, this is not required as the inputs to the
function are copies, not the original values themselves. As we use the
copies within the function, we can use the original 'obi' values
directly without having to mask and add the headroom.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
---
drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 13 +++----------
1 file changed, 3 insertions(+), 10 deletions(-)
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
index 6b609553329f..bc86057628c8 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c
@@ -201,8 +201,6 @@ ixgbe_rx_buffer *ixgbe_get_rx_buffer_zc(struct ixgbe_ring *rx_ring,
static void ixgbe_reuse_rx_buffer_zc(struct ixgbe_ring *rx_ring,
struct ixgbe_rx_buffer *obi)
{
- unsigned long mask = (unsigned long)rx_ring->xsk_umem->chunk_mask;
- u64 hr = rx_ring->xsk_umem->headroom + XDP_PACKET_HEADROOM;
u16 nta = rx_ring->next_to_alloc;
struct ixgbe_rx_buffer *nbi;
@@ -212,14 +210,9 @@ static void ixgbe_reuse_rx_buffer_zc(struct ixgbe_ring *rx_ring,
rx_ring->next_to_alloc = (nta < rx_ring->count) ? nta : 0;
/* transfer page from old buffer to new buffer */
- nbi->dma = obi->dma & mask;
- nbi->dma += hr;
-
- nbi->addr = (void *)((unsigned long)obi->addr & mask);
- nbi->addr += hr;
-
- nbi->handle = obi->handle & mask;
- nbi->handle += rx_ring->xsk_umem->headroom;
+ nbi->dma = obi->dma;
+ nbi->addr = obi->addr;
+ nbi->handle = obi->handle;
obi->addr = NULL;
obi->skb = NULL;
--
2.17.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox