* Re: [PATCH net] add snmp counter document
From: David Miller @ 2019-02-03 18:10 UTC (permalink / raw)
To: yupeng0921; +Cc: netdev, xiyou.wangcong, rdunlap
In-Reply-To: <20190201054559.8082-1-yupeng0921@gmail.com>
From: yupeng <yupeng0921@gmail.com>
Date: Thu, 31 Jan 2019 21:45:59 -0800
> add document for tcp retransmission, tcp fast open, syn cookies,
> challenge ack, prune and several general counters
>
> Signed-off-by: yupeng <yupeng0921@gmail.com>
Please respin this against net-next, thank you.
^ permalink raw reply
* Re: [PATCH net-next] ipv4/igmp: Don't drop IGMP pkt with zeros src addr
From: David Miller @ 2019-02-03 17:54 UTC (permalink / raw)
To: echron; +Cc: netdev, colona
In-Reply-To: <20190131230040.15232-1-echron@arista.com>
From: Edward Chron <echron@arista.com>
Date: Thu, 31 Jan 2019 15:00:40 -0800
> Don't drop IGMP packets with a source address of all zeros which are
> IGMP proxy reports. This is documented in Section 2.1.1 IGMP
> Forwarding Rules of RFC 4541 IGMP and MLD Snooping Switches
> Considerations.
>
> Signed-off-by: Edward Chron <echron@arista.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH net-next] net: phy: realtek: add generic Realtek PHY driver
From: David Miller @ 2019-02-03 17:37 UTC (permalink / raw)
To: hkallweit1; +Cc: andrew, f.fainelli, netdev
In-Reply-To: <74416e89-bcd5-6c3d-b38a-2ed9ad6a0c0b@gmail.com>
From: Heiner Kallweit <hkallweit1@gmail.com>
Date: Sun, 3 Feb 2019 16:07:33 +0100
> The integrated PHY's of later RTL8168 network chips report the generic
> PHYID 0x001cc800 (Realtek OUI, model and revision number both set to
> zero) and therefore currently the genphy driver is used.
>
> To be able to use the paged version of e.g. phy_write() we need a
> PHY driver with the read_page and write_page callbacks implemented.
> So basically make a copy of the genphy driver, just with the
> read_page and write_page callbacks being set.
>
> Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
Applied, thanks.
^ permalink raw reply
* Re: [PATCH iproute2-next] tc: add 'kind' property to 'csum' action
From: David Ahern @ 2019-02-03 17:12 UTC (permalink / raw)
To: Davide Caratti, Stephen Hemminger; +Cc: netdev
In-Reply-To: <60a8d248feaea84e6ebddaf1b8f28871a8a25ffc.1548957208.git.dcaratti@redhat.com>
On 1/31/19 10:58 AM, Davide Caratti wrote:
> unlike other TC actions already supporting JSON printout, 'csum' does not
> print the value of TCA_KIND in the 'kind' property: remove 'csum' word
> from 'csum' property, and add a separate 'kind' property containing the
> action name. The human-readable printout is preserved.
>
> Tested with:
> # ./tdc.py -c csum
>
> Cc: Andrea Claudi <aclaudi@redhat.com>
> Signed-off-by: Davide Caratti <dcaratti@redhat.com>
> ---
> tc/m_csum.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
applied to iproute2-next. Thanks
^ permalink raw reply
* Re: [PATCH iproute2-next] tc: full JSON support for 'bpf' actions
From: David Ahern @ 2019-02-03 17:11 UTC (permalink / raw)
To: Davide Caratti, David Ahern, Stephen Hemminger; +Cc: netdev
In-Reply-To: <c1bb8538298d24e302ef0e84ead3bfb97f2438dc.1548957196.git.dcaratti@redhat.com>
On 1/31/19 10:58 AM, Davide Caratti wrote:
> Add full JSON output support in the dump of 'act_bpf'.
>
> Example using eBPF:
>
> # tc actions flush action bpf
> # tc action add action bpf object bpf/action.o section 'action-ok'
> # tc -j action list action bpf | jq
> [
> {
> "total acts": 1
> },
> {
> "actions": [
> {
> "order": 0,
> "kind": "bpf",
> "bpf_name": "action.o:[action-ok]",
> "prog": {
> "id": 33,
> "tag": "a04f5eef06a7f555",
> "jited": 1
> },
> "control_action": {
> "type": "pipe"
> },
> "index": 1,
> "ref": 1,
> "bind": 0
> }
> ]
> }
> ]
>
> Example using cBPF:
>
> # tc actions flush action bpf
> # a=$(mktemp)
> # tcpdump -ddd not ether proto 0x888e >$a
> # tc action add action bpf bytecode-file $a index 42
> # rm $a
> # tc -j action list action bpf | jq
> [
> {
> "total acts": 1
> },
> {
> "actions": [
> {
> "order": 0,
> "kind": "bpf",
> "bytecode": {
> "length": 4,
> "insns": [
> {
> "code": 40,
> "jt": 0,
> "jf": 0,
> "k": 12
> },
> {
> "code": 21,
> "jt": 0,
> "jf": 1,
> "k": 34958
> },
> {
> "code": 6,
> "jt": 0,
> "jf": 0,
> "k": 0
> },
> {
> "code": 6,
> "jt": 0,
> "jf": 0,
> "k": 262144
> }
> ]
> },
> "control_action": {
> "type": "pipe"
> },
> "index": 42,
> "ref": 1,
> "bind": 0
> }
> ]
> }
> ]
>
> Tested with:
> # ./tdc.py -c bpf
>
> Cc: Andrea Claudi <aclaudi@redhat.com>
> Signed-off-by: Davide Caratti <dcaratti@redhat.com>
> ---
> include/bpf_util.h | 2 +-
> lib/bpf.c | 26 ++++++++++++++++++--------
> tc/f_bpf.c | 2 +-
> tc/m_bpf.c | 32 +++++++++++++++++---------------
> 4 files changed, 37 insertions(+), 25 deletions(-)
>
applied to iproute2-next. Thanks
^ permalink raw reply
* Re: [PATCH bpf-next 2/3] MAINTAINERS: add RISC-V BPF JIT maintainer
From: David Miller @ 2019-02-03 17:08 UTC (permalink / raw)
To: bjorn.topel; +Cc: linux-riscv, daniel, ast, netdev, palmer, hch
In-Reply-To: <20190203115132.8766-3-bjorn.topel@gmail.com>
From: bjorn.topel@gmail.com
Date: Sun, 3 Feb 2019 12:51:30 +0100
> From: Björn Töpel <bjorn.topel@gmail.com>
>
> Add Björn Töpel as RISC-V BPF JIT maintainer.
>
> Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
Acked-by: David S. Miller <davem@davemloft.net>
^ permalink raw reply
* Re: [PATCH bpf-next 3/3] bpf, doc: add RISC-V to filter.txt
From: David Miller @ 2019-02-03 17:08 UTC (permalink / raw)
To: bjorn.topel; +Cc: linux-riscv, daniel, ast, netdev, palmer, hch
In-Reply-To: <20190203115132.8766-4-bjorn.topel@gmail.com>
From: bjorn.topel@gmail.com
Date: Sun, 3 Feb 2019 12:51:31 +0100
> From: Björn Töpel <bjorn.topel@gmail.com>
>
> Update Documentation/networking/filter.txt to mention RISC-V.
>
> Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
Acked-by: David S. Miller <davem@davemloft.net>
^ permalink raw reply
* Re: [PATCH bpf-next 1/3] bpf, riscv: add BPF JIT for RV64G
From: David Miller @ 2019-02-03 17:08 UTC (permalink / raw)
To: bjorn.topel; +Cc: linux-riscv, daniel, ast, netdev, palmer, hch
In-Reply-To: <20190203115132.8766-2-bjorn.topel@gmail.com>
From: bjorn.topel@gmail.com
Date: Sun, 3 Feb 2019 12:51:29 +0100
> From: Björn Töpel <bjorn.topel@gmail.com>
>
> This commit adds BPF JIT for RV64G.
>
> The JIT is a two-pass JIT, and has a dynamic prolog/epilogue (similar
> to the MIPS64 BPF JIT) instead of static ones (e.g. x86_64).
>
> At the moment the RISC-V Linux port does not support HAVE_KPROBES,
> which means that CONFIG_BPF_EVENTS is not supported. Thus, no tests
> involving BPF_PROG_TYPE_TRACEPOINT passes.
>
> Further, the implementation does not support "far branching" (>4KiB).
>
> The implementation passes all the test_bpf.ko tests:
> test_bpf: Summary: 378 PASSED, 0 FAILED, [366/366 JIT'ed]
>
> All the tail_call tests in the selftest/bpf/test_verifier program
> passes.
>
> All tests where done on QEMU (QEMU emulator version 3.1.50
> (v3.1.0-688-g8ae951fbc106)).
>
> Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
Acked-by: David S. Miller <davem@davemloft.net>
^ permalink raw reply
* Re: [PATCH] net: fix IPv6 prefix route residue
From: David Miller @ 2019-02-03 17:04 UTC (permalink / raw)
To: liuzhiqiang26
Cc: kuznet, yoshfuji, 0xeffeff, edumazet, netdev, mingfangsen,
zhangwenhao8, wangxiaogang3, zhoukang7
In-Reply-To: <98d563a5-34f0-a504-d62f-d20ed7c770da@huawei.com>
From: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Date: Sun, 3 Feb 2019 14:10:31 +0800
> @@ -1165,7 +1165,8 @@ enum cleanup_prefix_rt_t {
> list_for_each_entry(ifa, &idev->addr_list, if_list) {
> if (ifa == ifp)
> continue;
> - if (!ipv6_prefix_equal(&ifa->addr, &ifp->addr,
> + if (ifa->prefix_len != ifp->prefix_len ||
> + !ipv6_prefix_equal(&ifa->addr, &ifp->addr,
> ifp->prefix_len))
Please fix the indentation of this conditional, it should be:
if (ifa->prefix_len != ifp->prefix_len ||
!ipv6_prefix_equal(&ifa->addr, &ifp->addr,
ifp->prefix_len))
Thank you.
^ permalink raw reply
* [PATCH net-next] net: phy: realtek: add generic Realtek PHY driver
From: Heiner Kallweit @ 2019-02-03 15:07 UTC (permalink / raw)
To: Andrew Lunn, Florian Fainelli, David Miller; +Cc: netdev@vger.kernel.org
The integrated PHY's of later RTL8168 network chips report the generic
PHYID 0x001cc800 (Realtek OUI, model and revision number both set to
zero) and therefore currently the genphy driver is used.
To be able to use the paged version of e.g. phy_write() we need a
PHY driver with the read_page and write_page callbacks implemented.
So basically make a copy of the genphy driver, just with the
read_page and write_page callbacks being set.
Signed-off-by: Heiner Kallweit <hkallweit1@gmail.com>
---
drivers/net/phy/realtek.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/drivers/net/phy/realtek.c b/drivers/net/phy/realtek.c
index 75543f937..d8c4b9ae0 100644
--- a/drivers/net/phy/realtek.c
+++ b/drivers/net/phy/realtek.c
@@ -272,6 +272,15 @@ static struct phy_driver realtek_drvs[] = {
.resume = genphy_resume,
.read_page = rtl821x_read_page,
.write_page = rtl821x_write_page,
+ }, {
+ PHY_ID_MATCH_EXACT(0x001cc800),
+ .name = "Generic Realtek PHY",
+ .features = PHY_GBIT_FEATURES,
+ .config_init = genphy_config_init,
+ .suspend = genphy_suspend,
+ .resume = genphy_resume,
+ .read_page = rtl821x_read_page,
+ .write_page = rtl821x_write_page,
}, {
PHY_ID_MATCH_EXACT(0x001cc961),
.name = "RTL8366RB Gigabit Ethernet",
--
2.20.1
^ permalink raw reply related
* [RFC/PATCH net-next 4/9] linux/dim: Rename net_dim_sample() to net_dim_create_sample()
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
In order to avoid confusion between the function and the similarly
named struct.
In preparation for removing the 'net' prefix from dim members.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 4 ++--
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 8 ++++----
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 4 ++--
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 6 ++----
include/linux/dim.h | 8 ++++----
include/linux/net_dim.h | 4 ++--
6 files changed, 16 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 0dbdef541f3c..00958c4bf740 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -1034,8 +1034,8 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
}
if (priv->dim.use_dim) {
- net_dim_sample(priv->dim.event_ctr, priv->dim.packets,
- priv->dim.bytes, &dim_sample);
+ net_dim_create_sample(priv->dim.event_ctr, priv->dim.packets,
+ priv->dim.bytes, &dim_sample);
net_dim(&priv->dim.dim, dim_sample);
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 08d0679ccf63..9bf7b51d9405 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2107,10 +2107,10 @@ static int bnxt_poll(struct napi_struct *napi, int budget)
if (bp->flags & BNXT_FLAG_DIM) {
struct net_dim_sample dim_sample;
- net_dim_sample(cpr->event_ctr,
- cpr->rx_packets,
- cpr->rx_bytes,
- &dim_sample);
+ net_dim_create_sample(cpr->event_ctr,
+ cpr->rx_packets,
+ cpr->rx_bytes,
+ &dim_sample);
net_dim(&cpr->dim, dim_sample);
}
mmiowb();
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index eee48dae8ed5..8ab2d672d2ba 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -1909,8 +1909,8 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
}
if (ring->dim.use_dim) {
- net_dim_sample(ring->dim.event_ctr, ring->dim.packets,
- ring->dim.bytes, &dim_sample);
+ net_dim_create_sample(ring->dim.event_ctr, ring->dim.packets,
+ ring->dim.bytes, &dim_sample);
net_dim(&ring->dim.dim, dim_sample);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index b4af5e19f6ac..6dd820242f4b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -53,8 +53,7 @@ static void mlx5e_handle_tx_dim(struct mlx5e_txqsq *sq)
if (unlikely(!test_bit(MLX5E_SQ_STATE_AM, &sq->state)))
return;
- net_dim_sample(sq->cq.event_ctr, stats->packets, stats->bytes,
- &dim_sample);
+ net_dim_create_sample(sq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
net_dim(&sq->dim, dim_sample);
}
@@ -66,8 +65,7 @@ static void mlx5e_handle_rx_dim(struct mlx5e_rq *rq)
if (unlikely(!test_bit(MLX5E_RQ_STATE_AM, &rq->state)))
return;
- net_dim_sample(rq->cq.event_ctr, stats->packets, stats->bytes,
- &dim_sample);
+ net_dim_create_sample(rq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
net_dim(&rq->dim, dim_sample);
}
diff --git a/include/linux/dim.h b/include/linux/dim.h
index adb20e58bf19..7809ffe470ff 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -150,10 +150,10 @@ static inline void dim_park_tired(struct net_dim *dim)
dim->tune_state = DIM_PARKING_TIRED;
}
-static inline void net_dim_sample(u16 event_ctr,
- u64 packets,
- u64 bytes,
- struct net_dim_sample *s)
+static inline void net_dim_create_sample(u16 event_ctr,
+ u64 packets,
+ u64 bytes,
+ struct net_dim_sample *s)
{
s->time = ktime_get();
s->pkt_ctr = packets;
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index 98ea0bb6f130..8cddfc93819c 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -262,8 +262,8 @@ static inline void net_dim(struct net_dim *dim,
}
/* fall through */
case DIM_START_MEASURE:
- net_dim_sample(end_sample.event_ctr, end_sample.pkt_ctr, end_sample.byte_ctr,
- &dim->start_sample);
+ net_dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
+ end_sample.byte_ctr, &dim->start_sample);
dim->state = DIM_MEASURE_IN_PROGRESS;
break;
case DIM_APPLY_NEW_PROFILE:
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 0/9] net/dim: Support for multiple implementations
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
net_dim.h lib exposes an implementation of the DIM algorithm for dynamically-tuned interrupt
moderation for networking interfaces.
We need the same behavior for any block CQ. The main motivation is two benefit from maximized
completion rate and reduced interrupt overhead that DIM may provide.
Current DIM implementation prioritizes reducing interrupt overhead over latency. Also, in
order to reduce DIM's own overhead, the algorithm might take take some time to identify it
needs to change profiles. For these reasons we got to the understanding that a slightly
modified algorithm is needed. Early tests with current implementation show it doesn't react
fast and sharply enough in order to satisfy the block CQ needs.
I would like to suggest an implementation for block DIM. The idea is to expose the new
functionality without the risk of breaking Net DIM behavior for netdev. Below are main
similarities and differences between the two implementations and general guidelines for the
suggested solution.
Performance tests over ConnectX-5 100GbE NIC show a 200% improvement on tail latency when
switching from high load traffic to low load traffic.
Common logic, main DIM procedure:
- Calculate current stats from a given sample
- Compare current stats vs. previous iteration stats
- Make a decision -> choose a new profile
Differences:
- Different parameters for moving between profiles
- Different moderation values and number of profiles
- Different sampled data
Suggested solution:
- Common logic will be declared in include/linux/dim.h and implemented in lib/dim/dim.c
- Net DIM (existing) logic will be declared in include/linux/net_dim.h and implemented in
lib/dim/net_dim.c, which will use the common logic from dim.h
- Block DIM logic will be declared in /include/linux/block_dim.h and implemented in
lib/dim/blk_dim.c.
This new implementation will expose modified versions of profiles, dim_step() and dim_decision()
Pros for this solution are:
- Zero impact on existing net_dim implementation and usage
- Relatively more code reuse (compared to two separate solutions)
- Readiness for future implementations
Tal Gilboa (6):
linux/dim: Move logic to dim.h
linux/dim: Remove "net" prefix from internal DIM members
linux/dim: Rename externally exposed macros
linux/dim: Rename net_dim_sample() to net_dim_create_sample()
linux/dim: Rename externally used net_dim members
linux/dim: Move implementation to .c files
Yamin Friedman (3):
linux/dim: Add completions count to dim_sample
linux/dim: Implement blk_dim.h
drivers/infiniband: Use blk_dim in infiniband driver
MAINTAINERS | 3 +
drivers/infiniband/core/cq.c | 75 +++-
drivers/infiniband/hw/mlx4/qp.c | 2 +-
drivers/infiniband/hw/mlx5/qp.c | 2 +-
drivers/net/ethernet/broadcom/bcmsysport.c | 20 +-
drivers/net/ethernet/broadcom/bcmsysport.h | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 13 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +-
.../net/ethernet/broadcom/bnxt/bnxt_debugfs.c | 4 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 7 +-
.../net/ethernet/broadcom/genet/bcmgenet.c | 18 +-
.../net/ethernet/broadcom/genet/bcmgenet.h | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en.h | 8 +-
.../net/ethernet/mellanox/mlx5/core/en_dim.c | 12 +-
.../ethernet/mellanox/mlx5/core/en_ethtool.c | 4 +-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 22 +-
.../net/ethernet/mellanox/mlx5/core/en_txrx.c | 10 +-
include/linux/blk_dim.h | 56 +++
include/linux/dim.h | 126 +++++++
include/linux/irq_poll.h | 7 +
include/linux/net_dim.h | 338 +-----------------
include/rdma/ib_verbs.h | 11 +-
lib/Kconfig | 7 +
lib/Makefile | 1 +
lib/dim/Makefile | 14 +
lib/dim/blk_dim.c | 114 ++++++
lib/dim/dim.c | 92 +++++
lib/dim/net_dim.c | 193 ++++++++++
lib/irq_poll.c | 13 +-
29 files changed, 778 insertions(+), 400 deletions(-)
create mode 100644 include/linux/blk_dim.h
create mode 100644 include/linux/dim.h
create mode 100644 lib/dim/Makefile
create mode 100644 lib/dim/blk_dim.c
create mode 100644 lib/dim/dim.c
create mode 100644 lib/dim/net_dim.c
--
2.19.1
^ permalink raw reply
* [RFC/PATCH net-next 2/9] linux/dim: Remove "net" prefix from internal DIM members
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
Only renaming functions and structs which aren't used by an external code.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
include/linux/dim.h | 86 ++++++++++++++++++++---------------------
include/linux/net_dim.h | 86 ++++++++++++++++++++---------------------
2 files changed, 86 insertions(+), 86 deletions(-)
diff --git a/include/linux/dim.h b/include/linux/dim.h
index c745c5d457ff..16ef59524b69 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -36,7 +36,7 @@
#include <linux/module.h>
-#define NET_DIM_NEVENTS 64
+#define DIM_NEVENTS 64
#define IS_SIGNIFICANT_DIFF(val, ref) \
(((100UL * abs((val) - (ref))) / (ref)) > 10) /* more than 10% difference */
#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1))
@@ -55,7 +55,7 @@ struct net_dim_sample {
u16 event_ctr;
};
-struct net_dim_stats {
+struct dim_stats {
int ppms; /* packets per msec */
int bpms; /* bytes per msec */
int epms; /* events per msec */
@@ -63,7 +63,7 @@ struct net_dim_stats {
struct net_dim { /* Dynamic Interrupt Moderation */
u8 state;
- struct net_dim_stats prev_stats;
+ struct dim_stats prev_stats;
struct net_dim_sample start_sample;
struct work_struct work;
u8 profile_ix;
@@ -87,67 +87,67 @@ enum {
};
enum {
- NET_DIM_PARKING_ON_TOP,
- NET_DIM_PARKING_TIRED,
- NET_DIM_GOING_RIGHT,
- NET_DIM_GOING_LEFT,
+ DIM_PARKING_ON_TOP,
+ DIM_PARKING_TIRED,
+ DIM_GOING_RIGHT,
+ DIM_GOING_LEFT,
};
enum {
- NET_DIM_STATS_WORSE,
- NET_DIM_STATS_SAME,
- NET_DIM_STATS_BETTER,
+ DIM_STATS_WORSE,
+ DIM_STATS_SAME,
+ DIM_STATS_BETTER,
};
enum {
- NET_DIM_STEPPED,
- NET_DIM_TOO_TIRED,
- NET_DIM_ON_EDGE,
+ DIM_STEPPED,
+ DIM_TOO_TIRED,
+ DIM_ON_EDGE,
};
-static inline bool net_dim_on_top(struct net_dim *net_dim)
+static inline bool dim_on_top(struct net_dim *dim)
{
- switch (net_dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
- case NET_DIM_PARKING_TIRED:
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
return true;
- case NET_DIM_GOING_RIGHT:
- return (net_dim->steps_left > 1) && (net_dim->steps_right == 1);
- default: /* NET_DIM_GOING_LEFT */
- return (net_dim->steps_right > 1) && (net_dim->steps_left == 1);
+ case DIM_GOING_RIGHT:
+ return (dim->steps_left > 1) && (dim->steps_right == 1);
+ default: /* DIM_GOING_LEFT */
+ return (dim->steps_right > 1) && (dim->steps_left == 1);
}
}
-static inline void net_dim_turn(struct net_dim *net_dim)
+static inline void dim_turn(struct net_dim *dim)
{
- switch (net_dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
- case NET_DIM_PARKING_TIRED:
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
break;
- case NET_DIM_GOING_RIGHT:
- net_dim->tune_state = NET_DIM_GOING_LEFT;
- net_dim->steps_left = 0;
+ case DIM_GOING_RIGHT:
+ dim->tune_state = DIM_GOING_LEFT;
+ dim->steps_left = 0;
break;
- case NET_DIM_GOING_LEFT:
- net_dim->tune_state = NET_DIM_GOING_RIGHT;
- net_dim->steps_right = 0;
+ case DIM_GOING_LEFT:
+ dim->tune_state = DIM_GOING_RIGHT;
+ dim->steps_right = 0;
break;
}
}
-static inline void net_dim_park_on_top(struct net_dim *net_dim)
+static inline void dim_park_on_top(struct net_dim *dim)
{
- net_dim->steps_right = 0;
- net_dim->steps_left = 0;
- net_dim->tired = 0;
- net_dim->tune_state = NET_DIM_PARKING_ON_TOP;
+ dim->steps_right = 0;
+ dim->steps_left = 0;
+ dim->tired = 0;
+ dim->tune_state = DIM_PARKING_ON_TOP;
}
-static inline void net_dim_park_tired(struct net_dim *net_dim)
+static inline void dim_park_tired(struct net_dim *dim)
{
- net_dim->steps_right = 0;
- net_dim->steps_left = 0;
- net_dim->tune_state = NET_DIM_PARKING_TIRED;
+ dim->steps_right = 0;
+ dim->steps_left = 0;
+ dim->tune_state = DIM_PARKING_TIRED;
}
static inline void net_dim_sample(u16 event_ctr,
@@ -161,9 +161,9 @@ static inline void net_dim_sample(u16 event_ctr,
s->event_ctr = event_ctr;
}
-static inline void net_dim_calc_stats(struct net_dim_sample *start,
+static inline void dim_calc_stats(struct net_dim_sample *start,
struct net_dim_sample *end,
- struct net_dim_stats *curr_stats)
+ struct dim_stats *curr_stats)
{
/* u32 holds up to 71 minutes, should be enough */
u32 delta_us = ktime_us_delta(end->time, start->time);
@@ -176,7 +176,7 @@ static inline void net_dim_calc_stats(struct net_dim_sample *start,
curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us);
curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
- curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC,
+ curr_stats->epms = DIV_ROUND_UP(DIM_NEVENTS * USEC_PER_MSEC,
delta_us);
}
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index 373cda74b167..1ce0899b5f30 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -128,67 +128,67 @@ net_dim_get_def_tx_moderation(u8 cq_period_mode)
static inline int net_dim_step(struct net_dim *dim)
{
if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2))
- return NET_DIM_TOO_TIRED;
+ return DIM_TOO_TIRED;
switch (dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
- case NET_DIM_PARKING_TIRED:
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
break;
- case NET_DIM_GOING_RIGHT:
+ case DIM_GOING_RIGHT:
if (dim->profile_ix == (NET_DIM_PARAMS_NUM_PROFILES - 1))
- return NET_DIM_ON_EDGE;
+ return DIM_ON_EDGE;
dim->profile_ix++;
dim->steps_right++;
break;
- case NET_DIM_GOING_LEFT:
+ case DIM_GOING_LEFT:
if (dim->profile_ix == 0)
- return NET_DIM_ON_EDGE;
+ return DIM_ON_EDGE;
dim->profile_ix--;
dim->steps_left++;
break;
}
dim->tired++;
- return NET_DIM_STEPPED;
+ return DIM_STEPPED;
}
static inline void net_dim_exit_parking(struct net_dim *dim)
{
- dim->tune_state = dim->profile_ix ? NET_DIM_GOING_LEFT :
- NET_DIM_GOING_RIGHT;
+ dim->tune_state = dim->profile_ix ? DIM_GOING_LEFT :
+ DIM_GOING_RIGHT;
net_dim_step(dim);
}
-static inline int net_dim_stats_compare(struct net_dim_stats *curr,
- struct net_dim_stats *prev)
+static inline int net_dim_stats_compare(struct dim_stats *curr,
+ struct dim_stats *prev)
{
if (!prev->bpms)
- return curr->bpms ? NET_DIM_STATS_BETTER :
- NET_DIM_STATS_SAME;
+ return curr->bpms ? DIM_STATS_BETTER :
+ DIM_STATS_SAME;
if (IS_SIGNIFICANT_DIFF(curr->bpms, prev->bpms))
- return (curr->bpms > prev->bpms) ? NET_DIM_STATS_BETTER :
- NET_DIM_STATS_WORSE;
+ return (curr->bpms > prev->bpms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
if (!prev->ppms)
- return curr->ppms ? NET_DIM_STATS_BETTER :
- NET_DIM_STATS_SAME;
+ return curr->ppms ? DIM_STATS_BETTER :
+ DIM_STATS_SAME;
if (IS_SIGNIFICANT_DIFF(curr->ppms, prev->ppms))
- return (curr->ppms > prev->ppms) ? NET_DIM_STATS_BETTER :
- NET_DIM_STATS_WORSE;
+ return (curr->ppms > prev->ppms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
if (!prev->epms)
- return NET_DIM_STATS_SAME;
+ return DIM_STATS_SAME;
if (IS_SIGNIFICANT_DIFF(curr->epms, prev->epms))
- return (curr->epms < prev->epms) ? NET_DIM_STATS_BETTER :
- NET_DIM_STATS_WORSE;
+ return (curr->epms < prev->epms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
- return NET_DIM_STATS_SAME;
+ return DIM_STATS_SAME;
}
-static inline bool net_dim_decision(struct net_dim_stats *curr_stats,
+static inline bool net_dim_decision(struct dim_stats *curr_stats,
struct net_dim *dim)
{
int prev_state = dim->tune_state;
@@ -197,44 +197,44 @@ static inline bool net_dim_decision(struct net_dim_stats *curr_stats,
int step_res;
switch (dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
+ case DIM_PARKING_ON_TOP:
stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
- if (stats_res != NET_DIM_STATS_SAME)
+ if (stats_res != DIM_STATS_SAME)
net_dim_exit_parking(dim);
break;
- case NET_DIM_PARKING_TIRED:
+ case DIM_PARKING_TIRED:
dim->tired--;
if (!dim->tired)
net_dim_exit_parking(dim);
break;
- case NET_DIM_GOING_RIGHT:
- case NET_DIM_GOING_LEFT:
+ case DIM_GOING_RIGHT:
+ case DIM_GOING_LEFT:
stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
- if (stats_res != NET_DIM_STATS_BETTER)
- net_dim_turn(dim);
+ if (stats_res != DIM_STATS_BETTER)
+ dim_turn(dim);
- if (net_dim_on_top(dim)) {
- net_dim_park_on_top(dim);
+ if (dim_on_top(dim)) {
+ dim_park_on_top(dim);
break;
}
step_res = net_dim_step(dim);
switch (step_res) {
- case NET_DIM_ON_EDGE:
- net_dim_park_on_top(dim);
+ case DIM_ON_EDGE:
+ dim_park_on_top(dim);
break;
- case NET_DIM_TOO_TIRED:
- net_dim_park_tired(dim);
+ case DIM_TOO_TIRED:
+ dim_park_tired(dim);
break;
}
break;
}
- if ((prev_state != NET_DIM_PARKING_ON_TOP) ||
- (dim->tune_state != NET_DIM_PARKING_ON_TOP))
+ if ((prev_state != DIM_PARKING_ON_TOP) ||
+ (dim->tune_state != DIM_PARKING_ON_TOP))
dim->prev_stats = *curr_stats;
return dim->profile_ix != prev_ix;
@@ -243,7 +243,7 @@ static inline bool net_dim_decision(struct net_dim_stats *curr_stats,
static inline void net_dim(struct net_dim *dim,
struct net_dim_sample end_sample)
{
- struct net_dim_stats curr_stats;
+ struct dim_stats curr_stats;
u16 nevents;
switch (dim->state) {
@@ -251,9 +251,9 @@ static inline void net_dim(struct net_dim *dim,
nevents = BIT_GAP(BITS_PER_TYPE(u16),
end_sample.event_ctr,
dim->start_sample.event_ctr);
- if (nevents < NET_DIM_NEVENTS)
+ if (nevents < DIM_NEVENTS)
break;
- net_dim_calc_stats(&dim->start_sample, &end_sample,
+ dim_calc_stats(&dim->start_sample, &end_sample,
&curr_stats);
if (net_dim_decision(&curr_stats, dim)) {
dim->state = NET_DIM_APPLY_NEW_PROFILE;
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 3/9] linux/dim: Rename externally exposed macros
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
Renamed macros in use by external drivers.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 4 ++--
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 2 +-
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 4 ++--
.../net/ethernet/mellanox/mlx5/core/en_dim.c | 2 +-
.../net/ethernet/mellanox/mlx5/core/en_main.c | 10 +++++-----
include/linux/dim.h | 12 ++++++------
include/linux/net_dim.h | 18 +++++++++---------
8 files changed, 27 insertions(+), 27 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index f9521d0274b7..0dbdef541f3c 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -1114,7 +1114,7 @@ static void bcm_sysport_dim_work(struct work_struct *work)
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
bcm_sysport_set_rx_coalesce(priv, cur_profile.usec, cur_profile.pkts);
- dim->state = NET_DIM_START_MEASURE;
+ dim->state = DIM_START_MEASURE;
}
/* RX and misc interrupt routine */
@@ -1466,7 +1466,7 @@ static void bcm_sysport_init_dim(struct bcm_sysport_priv *priv,
struct bcm_sysport_net_dim *dim = &priv->dim;
INIT_WORK(&dim->dim.work, cb);
- dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ dim->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE;
dim->event_ctr = 0;
dim->packets = 0;
dim->bytes = 0;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 6a512871176b..08d0679ccf63 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -7673,7 +7673,7 @@ static void bnxt_enable_napi(struct bnxt *bp)
if (bp->bnapi[i]->rx_ring) {
INIT_WORK(&cpr->dim.work, bnxt_dim_work);
- cpr->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ cpr->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE;
}
napi_enable(&bp->bnapi[i]->napi);
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
index afa97c8bb081..16a4588709d1 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
@@ -28,5 +28,5 @@ void bnxt_dim_work(struct work_struct *work)
cpr->rx_ring_coal.coal_bufs = cur_moder.pkts;
bnxt_hwrm_set_ring_coal(bnapi->bp, bnapi);
- dim->state = NET_DIM_START_MEASURE;
+ dim->state = DIM_START_MEASURE;
}
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 983245c0867c..eee48dae8ed5 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -1928,7 +1928,7 @@ static void bcmgenet_dim_work(struct work_struct *work)
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
bcmgenet_set_rx_coalesce(ring, cur_profile.usec, cur_profile.pkts);
- dim->state = NET_DIM_START_MEASURE;
+ dim->state = DIM_START_MEASURE;
}
/* Assign skb to RX DMA descriptor. */
@@ -2085,7 +2085,7 @@ static void bcmgenet_init_dim(struct bcmgenet_rx_ring *ring,
struct bcmgenet_net_dim *dim = &ring->dim;
INIT_WORK(&dim->dim.work, cb);
- dim->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ dim->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE;
dim->event_ctr = 0;
dim->packets = 0;
dim->bytes = 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
index d67adf70a97b..a80303add7c0 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
@@ -38,7 +38,7 @@ mlx5e_complete_dim_work(struct net_dim *dim, struct net_dim_cq_moder moder,
struct mlx5_core_dev *mdev, struct mlx5_core_cq *mcq)
{
mlx5_core_modify_cq_moderation(mdev, mcq, moder.usec, moder.pkts);
- dim->state = NET_DIM_START_MEASURE;
+ dim->state = DIM_START_MEASURE;
}
void mlx5e_rx_dim_work(struct work_struct *work)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 099d307e6f25..7d80f803f445 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -679,11 +679,11 @@ static int mlx5e_alloc_rq(struct mlx5e_channel *c,
switch (params->rx_cq_moderation.cq_period_mode) {
case MLX5_CQ_PERIOD_MODE_START_FROM_CQE:
- rq->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE;
+ rq->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_CQE;
break;
case MLX5_CQ_PERIOD_MODE_START_FROM_EQE:
default:
- rq->dim.mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ rq->dim.mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE;
}
rq->page_cache.head = 0;
@@ -2345,7 +2345,7 @@ static void mlx5e_build_ico_cq_param(struct mlx5e_priv *priv,
mlx5e_build_common_cq_param(priv, param);
- param->cq_period_mode = NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ param->cq_period_mode = DIM_CQ_PERIOD_MODE_START_FROM_EQE;
}
static void mlx5e_build_icosq_param(struct mlx5e_priv *priv,
@@ -4545,8 +4545,8 @@ static struct net_dim_cq_moder mlx5e_get_def_rx_moderation(u8 cq_period_mode)
static u8 mlx5_to_net_dim_cq_period_mode(u8 cq_period_mode)
{
return cq_period_mode == MLX5_CQ_PERIOD_MODE_START_FROM_CQE ?
- NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE :
- NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE;
+ DIM_CQ_PERIOD_MODE_START_FROM_CQE :
+ DIM_CQ_PERIOD_MODE_START_FROM_EQE;
}
void mlx5e_set_tx_cq_mode_params(struct mlx5e_params *params, u8 cq_period_mode)
diff --git a/include/linux/dim.h b/include/linux/dim.h
index 16ef59524b69..adb20e58bf19 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -75,15 +75,15 @@ struct net_dim { /* Dynamic Interrupt Moderation */
};
enum {
- NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0,
- NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1,
- NET_DIM_CQ_PERIOD_NUM_MODES
+ DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0,
+ DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1,
+ DIM_CQ_PERIOD_NUM_MODES
};
enum {
- NET_DIM_START_MEASURE,
- NET_DIM_MEASURE_IN_PROGRESS,
- NET_DIM_APPLY_NEW_PROFILE,
+ DIM_START_MEASURE,
+ DIM_MEASURE_IN_PROGRESS,
+ DIM_APPLY_NEW_PROFILE,
};
enum {
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index 1ce0899b5f30..98ea0bb6f130 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -78,13 +78,13 @@
}
static const struct net_dim_cq_moder
-rx_profile[NET_DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
+rx_profile[DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
NET_DIM_RX_EQE_PROFILES,
NET_DIM_RX_CQE_PROFILES,
};
static const struct net_dim_cq_moder
-tx_profile[NET_DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
+tx_profile[DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
NET_DIM_TX_EQE_PROFILES,
NET_DIM_TX_CQE_PROFILES,
};
@@ -101,7 +101,7 @@ net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
static inline struct net_dim_cq_moder
net_dim_get_def_rx_moderation(u8 cq_period_mode)
{
- u8 profile_ix = cq_period_mode == NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
return net_dim_get_rx_moderation(cq_period_mode, profile_ix);
@@ -119,7 +119,7 @@ net_dim_get_tx_moderation(u8 cq_period_mode, int ix)
static inline struct net_dim_cq_moder
net_dim_get_def_tx_moderation(u8 cq_period_mode)
{
- u8 profile_ix = cq_period_mode == NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
@@ -247,7 +247,7 @@ static inline void net_dim(struct net_dim *dim,
u16 nevents;
switch (dim->state) {
- case NET_DIM_MEASURE_IN_PROGRESS:
+ case DIM_MEASURE_IN_PROGRESS:
nevents = BIT_GAP(BITS_PER_TYPE(u16),
end_sample.event_ctr,
dim->start_sample.event_ctr);
@@ -256,17 +256,17 @@ static inline void net_dim(struct net_dim *dim,
dim_calc_stats(&dim->start_sample, &end_sample,
&curr_stats);
if (net_dim_decision(&curr_stats, dim)) {
- dim->state = NET_DIM_APPLY_NEW_PROFILE;
+ dim->state = DIM_APPLY_NEW_PROFILE;
schedule_work(&dim->work);
break;
}
/* fall through */
- case NET_DIM_START_MEASURE:
+ case DIM_START_MEASURE:
net_dim_sample(end_sample.event_ctr, end_sample.pkt_ctr, end_sample.byte_ctr,
&dim->start_sample);
- dim->state = NET_DIM_MEASURE_IN_PROGRESS;
+ dim->state = DIM_MEASURE_IN_PROGRESS;
break;
- case NET_DIM_APPLY_NEW_PROFILE:
+ case DIM_APPLY_NEW_PROFILE:
break;
}
}
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 1/9] linux/dim: Move logic to dim.h
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
In preparation for supporting more implementations of the DIM
algorithm, I'm moving what would become common logic to a common
library. Downstream DIM implementations will use the common lib
for their implementation.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
include/linux/dim.h | 183 ++++++++++++++++++++++++++++++++++++++++
include/linux/net_dim.h | 148 +-------------------------------
2 files changed, 185 insertions(+), 146 deletions(-)
create mode 100644 include/linux/dim.h
diff --git a/include/linux/dim.h b/include/linux/dim.h
new file mode 100644
index 000000000000..c745c5d457ff
--- /dev/null
+++ b/include/linux/dim.h
@@ -0,0 +1,183 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies. All rights reserved.
+ * Copyright (c) 2017-2018, Broadcom Limited. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef DIM_H
+#define DIM_H
+
+#include <linux/module.h>
+
+#define NET_DIM_NEVENTS 64
+#define IS_SIGNIFICANT_DIFF(val, ref) \
+ (((100UL * abs((val) - (ref))) / (ref)) > 10) /* more than 10% difference */
+#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1))
+
+
+struct net_dim_cq_moder {
+ u16 usec;
+ u16 pkts;
+ u8 cq_period_mode;
+};
+
+struct net_dim_sample {
+ ktime_t time;
+ u32 pkt_ctr;
+ u32 byte_ctr;
+ u16 event_ctr;
+};
+
+struct net_dim_stats {
+ int ppms; /* packets per msec */
+ int bpms; /* bytes per msec */
+ int epms; /* events per msec */
+};
+
+struct net_dim { /* Dynamic Interrupt Moderation */
+ u8 state;
+ struct net_dim_stats prev_stats;
+ struct net_dim_sample start_sample;
+ struct work_struct work;
+ u8 profile_ix;
+ u8 mode;
+ u8 tune_state;
+ u8 steps_right;
+ u8 steps_left;
+ u8 tired;
+};
+
+enum {
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0,
+ NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1,
+ NET_DIM_CQ_PERIOD_NUM_MODES
+};
+
+enum {
+ NET_DIM_START_MEASURE,
+ NET_DIM_MEASURE_IN_PROGRESS,
+ NET_DIM_APPLY_NEW_PROFILE,
+};
+
+enum {
+ NET_DIM_PARKING_ON_TOP,
+ NET_DIM_PARKING_TIRED,
+ NET_DIM_GOING_RIGHT,
+ NET_DIM_GOING_LEFT,
+};
+
+enum {
+ NET_DIM_STATS_WORSE,
+ NET_DIM_STATS_SAME,
+ NET_DIM_STATS_BETTER,
+};
+
+enum {
+ NET_DIM_STEPPED,
+ NET_DIM_TOO_TIRED,
+ NET_DIM_ON_EDGE,
+};
+
+static inline bool net_dim_on_top(struct net_dim *net_dim)
+{
+ switch (net_dim->tune_state) {
+ case NET_DIM_PARKING_ON_TOP:
+ case NET_DIM_PARKING_TIRED:
+ return true;
+ case NET_DIM_GOING_RIGHT:
+ return (net_dim->steps_left > 1) && (net_dim->steps_right == 1);
+ default: /* NET_DIM_GOING_LEFT */
+ return (net_dim->steps_right > 1) && (net_dim->steps_left == 1);
+ }
+}
+
+static inline void net_dim_turn(struct net_dim *net_dim)
+{
+ switch (net_dim->tune_state) {
+ case NET_DIM_PARKING_ON_TOP:
+ case NET_DIM_PARKING_TIRED:
+ break;
+ case NET_DIM_GOING_RIGHT:
+ net_dim->tune_state = NET_DIM_GOING_LEFT;
+ net_dim->steps_left = 0;
+ break;
+ case NET_DIM_GOING_LEFT:
+ net_dim->tune_state = NET_DIM_GOING_RIGHT;
+ net_dim->steps_right = 0;
+ break;
+ }
+}
+
+static inline void net_dim_park_on_top(struct net_dim *net_dim)
+{
+ net_dim->steps_right = 0;
+ net_dim->steps_left = 0;
+ net_dim->tired = 0;
+ net_dim->tune_state = NET_DIM_PARKING_ON_TOP;
+}
+
+static inline void net_dim_park_tired(struct net_dim *net_dim)
+{
+ net_dim->steps_right = 0;
+ net_dim->steps_left = 0;
+ net_dim->tune_state = NET_DIM_PARKING_TIRED;
+}
+
+static inline void net_dim_sample(u16 event_ctr,
+ u64 packets,
+ u64 bytes,
+ struct net_dim_sample *s)
+{
+ s->time = ktime_get();
+ s->pkt_ctr = packets;
+ s->byte_ctr = bytes;
+ s->event_ctr = event_ctr;
+}
+
+static inline void net_dim_calc_stats(struct net_dim_sample *start,
+ struct net_dim_sample *end,
+ struct net_dim_stats *curr_stats)
+{
+ /* u32 holds up to 71 minutes, should be enough */
+ u32 delta_us = ktime_us_delta(end->time, start->time);
+ u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr);
+ u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr,
+ start->byte_ctr);
+
+ if (!delta_us)
+ return;
+
+ curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us);
+ curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
+ curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC,
+ delta_us);
+}
+
+#endif /* DIM_H */
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index fd458389f7d1..373cda74b167 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -35,73 +35,10 @@
#define NET_DIM_H
#include <linux/module.h>
-
-struct net_dim_cq_moder {
- u16 usec;
- u16 pkts;
- u8 cq_period_mode;
-};
-
-struct net_dim_sample {
- ktime_t time;
- u32 pkt_ctr;
- u32 byte_ctr;
- u16 event_ctr;
-};
-
-struct net_dim_stats {
- int ppms; /* packets per msec */
- int bpms; /* bytes per msec */
- int epms; /* events per msec */
-};
-
-struct net_dim { /* Adaptive Moderation */
- u8 state;
- struct net_dim_stats prev_stats;
- struct net_dim_sample start_sample;
- struct work_struct work;
- u8 profile_ix;
- u8 mode;
- u8 tune_state;
- u8 steps_right;
- u8 steps_left;
- u8 tired;
-};
-
-enum {
- NET_DIM_CQ_PERIOD_MODE_START_FROM_EQE = 0x0,
- NET_DIM_CQ_PERIOD_MODE_START_FROM_CQE = 0x1,
- NET_DIM_CQ_PERIOD_NUM_MODES
-};
-
-/* Adaptive moderation logic */
-enum {
- NET_DIM_START_MEASURE,
- NET_DIM_MEASURE_IN_PROGRESS,
- NET_DIM_APPLY_NEW_PROFILE,
-};
-
-enum {
- NET_DIM_PARKING_ON_TOP,
- NET_DIM_PARKING_TIRED,
- NET_DIM_GOING_RIGHT,
- NET_DIM_GOING_LEFT,
-};
-
-enum {
- NET_DIM_STATS_WORSE,
- NET_DIM_STATS_SAME,
- NET_DIM_STATS_BETTER,
-};
-
-enum {
- NET_DIM_STEPPED,
- NET_DIM_TOO_TIRED,
- NET_DIM_ON_EDGE,
-};
+#include <linux/dim.h>
#define NET_DIM_PARAMS_NUM_PROFILES 5
-/* Adaptive moderation profiles */
+/* Netdev dynamic interrupt moderation profiles */
#define NET_DIM_DEFAULT_RX_CQ_MODERATION_PKTS_FROM_EQE 256
#define NET_DIM_DEFAULT_TX_CQ_MODERATION_PKTS_FROM_EQE 128
#define NET_DIM_DEF_PROFILE_CQE 1
@@ -188,36 +125,6 @@ net_dim_get_def_tx_moderation(u8 cq_period_mode)
return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
}
-static inline bool net_dim_on_top(struct net_dim *dim)
-{
- switch (dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
- case NET_DIM_PARKING_TIRED:
- return true;
- case NET_DIM_GOING_RIGHT:
- return (dim->steps_left > 1) && (dim->steps_right == 1);
- default: /* NET_DIM_GOING_LEFT */
- return (dim->steps_right > 1) && (dim->steps_left == 1);
- }
-}
-
-static inline void net_dim_turn(struct net_dim *dim)
-{
- switch (dim->tune_state) {
- case NET_DIM_PARKING_ON_TOP:
- case NET_DIM_PARKING_TIRED:
- break;
- case NET_DIM_GOING_RIGHT:
- dim->tune_state = NET_DIM_GOING_LEFT;
- dim->steps_left = 0;
- break;
- case NET_DIM_GOING_LEFT:
- dim->tune_state = NET_DIM_GOING_RIGHT;
- dim->steps_right = 0;
- break;
- }
-}
-
static inline int net_dim_step(struct net_dim *dim)
{
if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2))
@@ -245,21 +152,6 @@ static inline int net_dim_step(struct net_dim *dim)
return NET_DIM_STEPPED;
}
-static inline void net_dim_park_on_top(struct net_dim *dim)
-{
- dim->steps_right = 0;
- dim->steps_left = 0;
- dim->tired = 0;
- dim->tune_state = NET_DIM_PARKING_ON_TOP;
-}
-
-static inline void net_dim_park_tired(struct net_dim *dim)
-{
- dim->steps_right = 0;
- dim->steps_left = 0;
- dim->tune_state = NET_DIM_PARKING_TIRED;
-}
-
static inline void net_dim_exit_parking(struct net_dim *dim)
{
dim->tune_state = dim->profile_ix ? NET_DIM_GOING_LEFT :
@@ -267,9 +159,6 @@ static inline void net_dim_exit_parking(struct net_dim *dim)
net_dim_step(dim);
}
-#define IS_SIGNIFICANT_DIFF(val, ref) \
- (((100UL * abs((val) - (ref))) / (ref)) > 10) /* more than 10% difference */
-
static inline int net_dim_stats_compare(struct net_dim_stats *curr,
struct net_dim_stats *prev)
{
@@ -351,39 +240,6 @@ static inline bool net_dim_decision(struct net_dim_stats *curr_stats,
return dim->profile_ix != prev_ix;
}
-static inline void net_dim_sample(u16 event_ctr,
- u64 packets,
- u64 bytes,
- struct net_dim_sample *s)
-{
- s->time = ktime_get();
- s->pkt_ctr = packets;
- s->byte_ctr = bytes;
- s->event_ctr = event_ctr;
-}
-
-#define NET_DIM_NEVENTS 64
-#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1))
-
-static inline void net_dim_calc_stats(struct net_dim_sample *start,
- struct net_dim_sample *end,
- struct net_dim_stats *curr_stats)
-{
- /* u32 holds up to 71 minutes, should be enough */
- u32 delta_us = ktime_us_delta(end->time, start->time);
- u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr);
- u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr,
- start->byte_ctr);
-
- if (!delta_us)
- return;
-
- curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us);
- curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
- curr_stats->epms = DIV_ROUND_UP(NET_DIM_NEVENTS * USEC_PER_MSEC,
- delta_us);
-}
-
static inline void net_dim(struct net_dim *dim,
struct net_dim_sample end_sample)
{
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 5/9] linux/dim: Rename externally used net_dim members
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
Removed 'net' prefix from functions and structs used by external drivers.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 16 +++++-----
drivers/net/ethernet/broadcom/bcmsysport.h | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 10 +++----
drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +-
.../net/ethernet/broadcom/bnxt/bnxt_debugfs.c | 4 +--
drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 5 ++--
.../net/ethernet/broadcom/genet/bcmgenet.c | 14 ++++-----
.../net/ethernet/broadcom/genet/bcmgenet.h | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en.h | 8 ++---
.../net/ethernet/mellanox/mlx5/core/en_dim.c | 10 +++----
.../ethernet/mellanox/mlx5/core/en_ethtool.c | 4 +--
.../net/ethernet/mellanox/mlx5/core/en_main.c | 12 ++++----
.../net/ethernet/mellanox/mlx5/core/en_txrx.c | 8 ++---
include/linux/dim.h | 28 ++++++++---------
include/linux/net_dim.h | 30 +++++++++----------
15 files changed, 77 insertions(+), 78 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 00958c4bf740..840b3bf1ae3e 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -627,7 +627,7 @@ static int bcm_sysport_set_coalesce(struct net_device *dev,
struct ethtool_coalesce *ec)
{
struct bcm_sysport_priv *priv = netdev_priv(dev);
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
u32 usecs, pkts;
unsigned int i;
@@ -1010,7 +1010,7 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
{
struct bcm_sysport_priv *priv =
container_of(napi, struct bcm_sysport_priv, napi);
- struct net_dim_sample dim_sample;
+ struct dim_sample dim_sample;
unsigned int work_done = 0;
work_done = bcm_sysport_desc_rx(priv, budget);
@@ -1034,8 +1034,8 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
}
if (priv->dim.use_dim) {
- net_dim_create_sample(priv->dim.event_ctr, priv->dim.packets,
- priv->dim.bytes, &dim_sample);
+ dim_create_sample(priv->dim.event_ctr, priv->dim.packets,
+ priv->dim.bytes, &dim_sample);
net_dim(&priv->dim.dim, dim_sample);
}
@@ -1105,13 +1105,13 @@ static void bcm_sysport_resume_from_wol(struct bcm_sysport_priv *priv)
static void bcm_sysport_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct dim *dim = container_of(work, struct dim, work);
struct bcm_sysport_net_dim *ndim =
container_of(dim, struct bcm_sysport_net_dim, dim);
struct bcm_sysport_priv *priv =
container_of(ndim, struct bcm_sysport_priv, dim);
- struct net_dim_cq_moder cur_profile =
- net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
+ struct dim_cq_moder cur_profile = net_dim_get_rx_moderation(dim->mode,
+ dim->profile_ix);
bcm_sysport_set_rx_coalesce(priv, cur_profile.usec, cur_profile.pkts);
dim->state = DIM_START_MEASURE;
@@ -1475,7 +1475,7 @@ static void bcm_sysport_init_dim(struct bcm_sysport_priv *priv,
static void bcm_sysport_init_rx_coalesce(struct bcm_sysport_priv *priv)
{
struct bcm_sysport_net_dim *dim = &priv->dim;
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
u32 usecs, pkts;
usecs = priv->rx_coalesce_usecs;
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.h b/drivers/net/ethernet/broadcom/bcmsysport.h
index 0887e6356649..db9143eeb817 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.h
+++ b/drivers/net/ethernet/broadcom/bcmsysport.h
@@ -710,7 +710,7 @@ struct bcm_sysport_net_dim {
u16 event_ctr;
unsigned long packets;
unsigned long bytes;
- struct net_dim dim;
+ struct dim dim;
};
/* Software view of the TX ring */
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 9bf7b51d9405..131ab07aad83 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2105,12 +2105,12 @@ static int bnxt_poll(struct napi_struct *napi, int budget)
}
}
if (bp->flags & BNXT_FLAG_DIM) {
- struct net_dim_sample dim_sample;
+ struct dim_sample dim_sample;
- net_dim_create_sample(cpr->event_ctr,
- cpr->rx_packets,
- cpr->rx_bytes,
- &dim_sample);
+ dim_create_sample(cpr->event_ctr,
+ cpr->rx_packets,
+ cpr->rx_bytes,
+ &dim_sample);
net_dim(&cpr->dim, dim_sample);
}
mmiowb();
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
index 5c886a700bcc..211f68d3e14b 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h
@@ -810,7 +810,7 @@ struct bnxt_cp_ring_info {
u64 rx_bytes;
u64 event_ctr;
- struct net_dim dim;
+ struct dim dim;
union {
struct tx_cmp *cp_desc_ring[MAX_CP_PAGES];
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.c
index 94e208e9789f..3d1d53fbb135 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.c
@@ -21,7 +21,7 @@ static ssize_t debugfs_dim_read(struct file *filep,
char __user *buffer,
size_t count, loff_t *ppos)
{
- struct net_dim *dim = filep->private_data;
+ struct dim *dim = filep->private_data;
int len;
char *buf;
@@ -61,7 +61,7 @@ static const struct file_operations debugfs_dim_fops = {
.read = debugfs_dim_read,
};
-static struct dentry *debugfs_dim_ring_init(struct net_dim *dim, int ring_idx,
+static struct dentry *debugfs_dim_ring_init(struct dim *dim, int ring_idx,
struct dentry *dd)
{
static char qname[16];
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
index 16a4588709d1..11605f9fa61e 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c
@@ -13,15 +13,14 @@
void bnxt_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim,
- work);
+ struct dim *dim = container_of(work, struct dim, work);
struct bnxt_cp_ring_info *cpr = container_of(dim,
struct bnxt_cp_ring_info,
dim);
struct bnxt_napi *bnapi = container_of(cpr,
struct bnxt_napi,
cp_ring);
- struct net_dim_cq_moder cur_moder =
+ struct dim_cq_moder cur_moder =
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
cpr->rx_ring_coal.coal_ticks = cur_moder.usec;
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 8ab2d672d2ba..68d96e333c6d 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -643,7 +643,7 @@ static void bcmgenet_set_rx_coalesce(struct bcmgenet_rx_ring *ring,
static void bcmgenet_set_ring_rx_coalesce(struct bcmgenet_rx_ring *ring,
struct ethtool_coalesce *ec)
{
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
u32 usecs, pkts;
ring->rx_coalesce_usecs = ec->rx_coalesce_usecs;
@@ -1898,7 +1898,7 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
{
struct bcmgenet_rx_ring *ring = container_of(napi,
struct bcmgenet_rx_ring, napi);
- struct net_dim_sample dim_sample;
+ struct dim_sample dim_sample;
unsigned int work_done;
work_done = bcmgenet_desc_rx(ring, budget);
@@ -1909,8 +1909,8 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
}
if (ring->dim.use_dim) {
- net_dim_create_sample(ring->dim.event_ctr, ring->dim.packets,
- ring->dim.bytes, &dim_sample);
+ dim_create_sample(ring->dim.event_ctr, ring->dim.packets,
+ ring->dim.bytes, &dim_sample);
net_dim(&ring->dim.dim, dim_sample);
}
@@ -1919,12 +1919,12 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
static void bcmgenet_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct dim *dim = container_of(work, struct dim, work);
struct bcmgenet_net_dim *ndim =
container_of(dim, struct bcmgenet_net_dim, dim);
struct bcmgenet_rx_ring *ring =
container_of(ndim, struct bcmgenet_rx_ring, dim);
- struct net_dim_cq_moder cur_profile =
+ struct dim_cq_moder cur_profile =
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
bcmgenet_set_rx_coalesce(ring, cur_profile.usec, cur_profile.pkts);
@@ -2094,7 +2094,7 @@ static void bcmgenet_init_dim(struct bcmgenet_rx_ring *ring,
static void bcmgenet_init_rx_coalesce(struct bcmgenet_rx_ring *ring)
{
struct bcmgenet_net_dim *dim = &ring->dim;
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
u32 usecs, pkts;
usecs = ring->rx_coalesce_usecs;
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.h b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
index 14b49612aa86..6e418d9c3706 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.h
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.h
@@ -581,7 +581,7 @@ struct bcmgenet_net_dim {
u16 event_ctr;
unsigned long packets;
unsigned long bytes;
- struct net_dim dim;
+ struct dim dim;
};
struct bcmgenet_rx_ring {
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h
index 6dd74ef69389..94826b48aead 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en.h
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h
@@ -238,8 +238,8 @@ struct mlx5e_params {
u16 num_channels;
u8 num_tc;
bool rx_cqe_compress_def;
- struct net_dim_cq_moder rx_cq_moderation;
- struct net_dim_cq_moder tx_cq_moderation;
+ struct dim_cq_moder rx_cq_moderation;
+ struct dim_cq_moder tx_cq_moderation;
bool lro_en;
u32 lro_wqe_sz;
u8 tx_min_inline_mode;
@@ -356,7 +356,7 @@ struct mlx5e_txqsq {
/* dirtied @completion */
u16 cc;
u32 dma_fifo_cc;
- struct net_dim dim; /* Adaptive Moderation */
+ struct dim dim; /* Adaptive Moderation */
/* dirtied @xmit */
u16 pc ____cacheline_aligned_in_smp;
@@ -593,7 +593,7 @@ struct mlx5e_rq {
int ix;
unsigned int hw_mtu;
- struct net_dim dim; /* Dynamic Interrupt Moderation */
+ struct dim dim; /* Dynamic Interrupt Moderation */
/* XDP */
struct bpf_prog *xdp_prog;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
index a80303add7c0..ba3c1be9f2d3 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_dim.c
@@ -34,7 +34,7 @@
#include "en.h"
static void
-mlx5e_complete_dim_work(struct net_dim *dim, struct net_dim_cq_moder moder,
+mlx5e_complete_dim_work(struct dim *dim, struct dim_cq_moder moder,
struct mlx5_core_dev *mdev, struct mlx5_core_cq *mcq)
{
mlx5_core_modify_cq_moderation(mdev, mcq, moder.usec, moder.pkts);
@@ -43,9 +43,9 @@ mlx5e_complete_dim_work(struct net_dim *dim, struct net_dim_cq_moder moder,
void mlx5e_rx_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct dim *dim = container_of(work, struct dim, work);
struct mlx5e_rq *rq = container_of(dim, struct mlx5e_rq, dim);
- struct net_dim_cq_moder cur_moder =
+ struct dim_cq_moder cur_moder =
net_dim_get_rx_moderation(dim->mode, dim->profile_ix);
mlx5e_complete_dim_work(dim, cur_moder, rq->mdev, &rq->cq.mcq);
@@ -53,9 +53,9 @@ void mlx5e_rx_dim_work(struct work_struct *work)
void mlx5e_tx_dim_work(struct work_struct *work)
{
- struct net_dim *dim = container_of(work, struct net_dim, work);
+ struct dim *dim = container_of(work, struct dim, work);
struct mlx5e_txqsq *sq = container_of(dim, struct mlx5e_txqsq, dim);
- struct net_dim_cq_moder cur_moder =
+ struct dim_cq_moder cur_moder =
net_dim_get_tx_moderation(dim->mode, dim->profile_ix);
mlx5e_complete_dim_work(dim, cur_moder, sq->cq.mdev, &sq->cq.mcq);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
index 3bbccead2f63..5c693556cd1e 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c
@@ -399,7 +399,7 @@ static int mlx5e_set_channels(struct net_device *dev,
int mlx5e_ethtool_get_coalesce(struct mlx5e_priv *priv,
struct ethtool_coalesce *coal)
{
- struct net_dim_cq_moder *rx_moder, *tx_moder;
+ struct dim_cq_moder *rx_moder, *tx_moder;
if (!MLX5_CAP_GEN(priv->mdev, cq_moderation))
return -EOPNOTSUPP;
@@ -454,7 +454,7 @@ mlx5e_set_priv_channels_coalesce(struct mlx5e_priv *priv, struct ethtool_coalesc
int mlx5e_ethtool_set_coalesce(struct mlx5e_priv *priv,
struct ethtool_coalesce *coal)
{
- struct net_dim_cq_moder *rx_moder, *tx_moder;
+ struct dim_cq_moder *rx_moder, *tx_moder;
struct mlx5_core_dev *mdev = priv->mdev;
struct mlx5e_channels new_channels = {};
int err = 0;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 7d80f803f445..6ed12a77cdb4 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -1773,7 +1773,7 @@ static void mlx5e_destroy_cq(struct mlx5e_cq *cq)
}
static int mlx5e_open_cq(struct mlx5e_channel *c,
- struct net_dim_cq_moder moder,
+ struct dim_cq_moder moder,
struct mlx5e_cq_param *param,
struct mlx5e_cq *cq)
{
@@ -1978,7 +1978,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
struct mlx5e_channel **cp)
{
int cpu = cpumask_first(mlx5_comp_irq_get_affinity_mask(priv->mdev, ix));
- struct net_dim_cq_moder icocq_moder = {0, 0};
+ struct dim_cq_moder icocq_moder = {0, 0};
struct net_device *netdev = priv->netdev;
struct mlx5e_channel *c;
unsigned int irq;
@@ -4516,9 +4516,9 @@ static bool slow_pci_heuristic(struct mlx5_core_dev *mdev)
link_speed > MLX5E_SLOW_PCI_RATIO * pci_bw;
}
-static struct net_dim_cq_moder mlx5e_get_def_tx_moderation(u8 cq_period_mode)
+static struct dim_cq_moder mlx5e_get_def_tx_moderation(u8 cq_period_mode)
{
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
moder.cq_period_mode = cq_period_mode;
moder.pkts = MLX5E_PARAMS_DEFAULT_TX_CQ_MODERATION_PKTS;
@@ -4529,9 +4529,9 @@ static struct net_dim_cq_moder mlx5e_get_def_tx_moderation(u8 cq_period_mode)
return moder;
}
-static struct net_dim_cq_moder mlx5e_get_def_rx_moderation(u8 cq_period_mode)
+static struct dim_cq_moder mlx5e_get_def_rx_moderation(u8 cq_period_mode)
{
- struct net_dim_cq_moder moder;
+ struct dim_cq_moder moder;
moder.cq_period_mode = cq_period_mode;
moder.pkts = MLX5E_PARAMS_DEFAULT_RX_CQ_MODERATION_PKTS;
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index 6dd820242f4b..432474754d77 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -48,24 +48,24 @@ static inline bool mlx5e_channel_no_affinity_change(struct mlx5e_channel *c)
static void mlx5e_handle_tx_dim(struct mlx5e_txqsq *sq)
{
struct mlx5e_sq_stats *stats = sq->stats;
- struct net_dim_sample dim_sample;
+ struct dim_sample dim_sample;
if (unlikely(!test_bit(MLX5E_SQ_STATE_AM, &sq->state)))
return;
- net_dim_create_sample(sq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
+ dim_create_sample(sq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
net_dim(&sq->dim, dim_sample);
}
static void mlx5e_handle_rx_dim(struct mlx5e_rq *rq)
{
struct mlx5e_rq_stats *stats = rq->stats;
- struct net_dim_sample dim_sample;
+ struct dim_sample dim_sample;
if (unlikely(!test_bit(MLX5E_RQ_STATE_AM, &rq->state)))
return;
- net_dim_create_sample(rq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
+ dim_create_sample(rq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
net_dim(&rq->dim, dim_sample);
}
diff --git a/include/linux/dim.h b/include/linux/dim.h
index 7809ffe470ff..8d9f8279ebee 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -42,13 +42,13 @@
#define BIT_GAP(bits, end, start) ((((end) - (start)) + BIT_ULL(bits)) & (BIT_ULL(bits) - 1))
-struct net_dim_cq_moder {
+struct dim_cq_moder {
u16 usec;
u16 pkts;
u8 cq_period_mode;
};
-struct net_dim_sample {
+struct dim_sample {
ktime_t time;
u32 pkt_ctr;
u32 byte_ctr;
@@ -61,10 +61,10 @@ struct dim_stats {
int epms; /* events per msec */
};
-struct net_dim { /* Dynamic Interrupt Moderation */
+struct dim { /* Dynamic Interrupt Moderation */
u8 state;
struct dim_stats prev_stats;
- struct net_dim_sample start_sample;
+ struct dim_sample start_sample;
struct work_struct work;
u8 profile_ix;
u8 mode;
@@ -105,7 +105,7 @@ enum {
DIM_ON_EDGE,
};
-static inline bool dim_on_top(struct net_dim *dim)
+static inline bool dim_on_top(struct dim *dim)
{
switch (dim->tune_state) {
case DIM_PARKING_ON_TOP:
@@ -118,7 +118,7 @@ static inline bool dim_on_top(struct net_dim *dim)
}
}
-static inline void dim_turn(struct net_dim *dim)
+static inline void dim_turn(struct dim *dim)
{
switch (dim->tune_state) {
case DIM_PARKING_ON_TOP:
@@ -135,7 +135,7 @@ static inline void dim_turn(struct net_dim *dim)
}
}
-static inline void dim_park_on_top(struct net_dim *dim)
+static inline void dim_park_on_top(struct dim *dim)
{
dim->steps_right = 0;
dim->steps_left = 0;
@@ -143,17 +143,17 @@ static inline void dim_park_on_top(struct net_dim *dim)
dim->tune_state = DIM_PARKING_ON_TOP;
}
-static inline void dim_park_tired(struct net_dim *dim)
+static inline void dim_park_tired(struct dim *dim)
{
dim->steps_right = 0;
dim->steps_left = 0;
dim->tune_state = DIM_PARKING_TIRED;
}
-static inline void net_dim_create_sample(u16 event_ctr,
- u64 packets,
- u64 bytes,
- struct net_dim_sample *s)
+static inline void dim_create_sample(u16 event_ctr,
+ u64 packets,
+ u64 bytes,
+ struct dim_sample *s)
{
s->time = ktime_get();
s->pkt_ctr = packets;
@@ -161,8 +161,8 @@ static inline void net_dim_create_sample(u16 event_ctr,
s->event_ctr = event_ctr;
}
-static inline void dim_calc_stats(struct net_dim_sample *start,
- struct net_dim_sample *end,
+static inline void dim_calc_stats(struct dim_sample *start,
+ struct dim_sample *end,
struct dim_stats *curr_stats)
{
/* u32 holds up to 71 minutes, should be enough */
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index 8cddfc93819c..e9363c372b68 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -77,28 +77,28 @@
{64, 32} \
}
-static const struct net_dim_cq_moder
+static const struct dim_cq_moder
rx_profile[DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
NET_DIM_RX_EQE_PROFILES,
NET_DIM_RX_CQE_PROFILES,
};
-static const struct net_dim_cq_moder
+static const struct dim_cq_moder
tx_profile[DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
NET_DIM_TX_EQE_PROFILES,
NET_DIM_TX_CQE_PROFILES,
};
-static inline struct net_dim_cq_moder
+static inline struct dim_cq_moder
net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
{
- struct net_dim_cq_moder cq_moder = rx_profile[cq_period_mode][ix];
+ struct dim_cq_moder cq_moder = rx_profile[cq_period_mode][ix];
cq_moder.cq_period_mode = cq_period_mode;
return cq_moder;
}
-static inline struct net_dim_cq_moder
+static inline struct dim_cq_moder
net_dim_get_def_rx_moderation(u8 cq_period_mode)
{
u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
@@ -107,16 +107,16 @@ net_dim_get_def_rx_moderation(u8 cq_period_mode)
return net_dim_get_rx_moderation(cq_period_mode, profile_ix);
}
-static inline struct net_dim_cq_moder
+static inline struct dim_cq_moder
net_dim_get_tx_moderation(u8 cq_period_mode, int ix)
{
- struct net_dim_cq_moder cq_moder = tx_profile[cq_period_mode][ix];
+ struct dim_cq_moder cq_moder = tx_profile[cq_period_mode][ix];
cq_moder.cq_period_mode = cq_period_mode;
return cq_moder;
}
-static inline struct net_dim_cq_moder
+static inline struct dim_cq_moder
net_dim_get_def_tx_moderation(u8 cq_period_mode)
{
u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
@@ -125,7 +125,7 @@ net_dim_get_def_tx_moderation(u8 cq_period_mode)
return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
}
-static inline int net_dim_step(struct net_dim *dim)
+static inline int net_dim_step(struct dim *dim)
{
if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2))
return DIM_TOO_TIRED;
@@ -152,7 +152,7 @@ static inline int net_dim_step(struct net_dim *dim)
return DIM_STEPPED;
}
-static inline void net_dim_exit_parking(struct net_dim *dim)
+static inline void net_dim_exit_parking(struct dim *dim)
{
dim->tune_state = dim->profile_ix ? DIM_GOING_LEFT :
DIM_GOING_RIGHT;
@@ -189,7 +189,7 @@ static inline int net_dim_stats_compare(struct dim_stats *curr,
}
static inline bool net_dim_decision(struct dim_stats *curr_stats,
- struct net_dim *dim)
+ struct dim *dim)
{
int prev_state = dim->tune_state;
int prev_ix = dim->profile_ix;
@@ -240,8 +240,8 @@ static inline bool net_dim_decision(struct dim_stats *curr_stats,
return dim->profile_ix != prev_ix;
}
-static inline void net_dim(struct net_dim *dim,
- struct net_dim_sample end_sample)
+static inline void net_dim(struct dim *dim,
+ struct dim_sample end_sample)
{
struct dim_stats curr_stats;
u16 nevents;
@@ -262,8 +262,8 @@ static inline void net_dim(struct net_dim *dim,
}
/* fall through */
case DIM_START_MEASURE:
- net_dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
- end_sample.byte_ctr, &dim->start_sample);
+ dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
+ end_sample.byte_ctr, &dim->start_sample);
dim->state = DIM_MEASURE_IN_PROGRESS;
break;
case DIM_APPLY_NEW_PROFILE:
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 8/9] linux/dim: Implement blk_dim.h
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
From: Yamin Friedman <yaminf@mellanox.com>
blk_dim implements a different algorithm than net_dim that is optimized for nvmf
storage applications.
The algorithm optimizes for number of completions and ratio between completions
and events.
It also has a feature for fast reduction of moderation level when the traffic
changes in such a way as to no longer require high moderation in order to avoid
long latencies.
blk_dim.h will be called from the ib_core module.
Signed-off-by: Yamin Friedman <yaminf@mellanox.com>
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
MAINTAINERS | 1 +
include/linux/blk_dim.h | 56 ++++++++++++++++++++
lib/dim/Makefile | 7 ++-
lib/dim/blk_dim.c | 114 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 177 insertions(+), 1 deletion(-)
create mode 100644 include/linux/blk_dim.h
create mode 100644 lib/dim/blk_dim.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 6ae949be8b83..2860a3316be5 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5337,6 +5337,7 @@ DYNAMIC INTERRUPT MODERATION
M: Tal Gilboa <talgi@mellanox.com>
S: Maintained
F: include/linux/net_dim.h
+F: include/linux/blk_dim.h
F: include/linux/dim.h
F: lib/dim/
diff --git a/include/linux/blk_dim.h b/include/linux/blk_dim.h
new file mode 100644
index 000000000000..a044f62ec8fe
--- /dev/null
+++ b/include/linux/blk_dim.h
@@ -0,0 +1,56 @@
+/*
+ * Copyright (c) 2016, Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef BLK_DIM_H
+#define BLK_DIM_H
+
+#include <linux/module.h>
+#include <linux/dim.h>
+
+#define BLK_DIM_PARAMS_NUM_PROFILES 8
+#define BLK_DIM_START_PROFILE 0
+
+static const struct dim_cq_moder
+blk_dim_prof[BLK_DIM_PARAMS_NUM_PROFILES] = {
+ {1, 0, 1, 0},
+ {2, 0, 2, 0},
+ {4, 0, 4, 0},
+ {16, 0, 4, 0},
+ {32, 0, 4, 0},
+ {32, 0, 16, 0},
+ {256, 0, 16, 0},
+ {256, 0, 32, 0},
+};
+
+void blk_dim(struct dim *dim, struct dim_sample end_sample);
+
+#endif /* BLK_DIM_H */
diff --git a/lib/dim/Makefile b/lib/dim/Makefile
index 160afe288df0..2b3c57318dbb 100644
--- a/lib/dim/Makefile
+++ b/lib/dim/Makefile
@@ -2,8 +2,13 @@
# DIM Dynamic Interrupt Moderation library
#
-obj-$(CONFIG_DIMLIB) = net_dim.o
+obj-$(CONFIG_DIMLIB) += net_dim.o
+obj-$(CONFIG_DIMLIB) += blk_dim.o
net_dim-y = \
dim.o \
net_dim.o
+
+blk_dim-y = \
+ dim.o \
+ blk_dim.o
diff --git a/lib/dim/blk_dim.c b/lib/dim/blk_dim.c
new file mode 100644
index 000000000000..49107c169b56
--- /dev/null
+++ b/lib/dim/blk_dim.c
@@ -0,0 +1,114 @@
+#include <linux/blk_dim.h>
+
+static inline int blk_dim_step(struct dim *dim)
+{
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
+ break;
+ case DIM_GOING_RIGHT:
+ if (dim->profile_ix == (BLK_DIM_PARAMS_NUM_PROFILES - 1))
+ return DIM_ON_EDGE;
+ dim->profile_ix++;
+ dim->steps_right++;
+ break;
+ case DIM_GOING_LEFT:
+ if (dim->profile_ix == 0)
+ return DIM_ON_EDGE;
+ dim->profile_ix--;
+ dim->steps_left++;
+ break;
+ }
+
+ return DIM_STEPPED;
+}
+
+static inline int blk_dim_stats_compare(struct dim_stats *curr, struct dim_stats *prev)
+{
+ /* first stat */
+ if (!prev->cpms)
+ return DIM_STATS_SAME;
+
+ if (IS_SIGNIFICANT_DIFF(curr->cpms, prev->cpms))
+ return (curr->cpms > prev->cpms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
+
+ if (IS_SIGNIFICANT_DIFF(curr->cpe_ratio, prev->cpe_ratio))
+ return (curr->cpe_ratio > prev->cpe_ratio) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
+
+ return DIM_STATS_SAME;
+}
+
+static inline bool blk_dim_decision(struct dim_stats *curr_stats, struct dim *dim)
+{
+ int prev_ix = dim->profile_ix;
+ int stats_res;
+ int step_res;
+
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ break;
+ case DIM_PARKING_TIRED:
+ break;
+
+ case DIM_GOING_RIGHT:
+ case DIM_GOING_LEFT:
+ stats_res = blk_dim_stats_compare(curr_stats, &dim->prev_stats);
+
+ switch (stats_res) {
+ case DIM_STATS_SAME:
+ if (curr_stats->cpe_ratio <= 50*prev_ix)
+ dim->profile_ix = 0;
+ break;
+ case DIM_STATS_WORSE:
+ dim_turn(dim);
+ default:
+ case DIM_STATS_BETTER:
+ /* fall through */
+ step_res = blk_dim_step(dim);
+ if (step_res == DIM_ON_EDGE)
+ dim_turn(dim);
+ break;
+ }
+ break;
+ }
+
+ dim->prev_stats = *curr_stats;
+
+ return dim->profile_ix != prev_ix;
+}
+
+void blk_dim(struct dim *dim, struct dim_sample end_sample)
+{
+ struct dim_stats curr_stats;
+ u16 nevents;
+
+ switch (dim->state) {
+ case DIM_MEASURE_IN_PROGRESS:
+ nevents = end_sample.event_ctr - dim->start_sample.event_ctr;
+ if (nevents < DIM_NEVENTS) {
+ dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
+ end_sample.byte_ctr, end_sample.comp_ctr, &dim->measuring_sample);
+ break;
+ }
+ dim_calc_stats(&dim->start_sample, &end_sample,
+ &curr_stats);
+ if (blk_dim_decision(&curr_stats, dim)) {
+ dim->state = DIM_APPLY_NEW_PROFILE;
+ schedule_work(&dim->work);
+ break;
+ }
+ /* fall through */
+ case DIM_START_MEASURE:
+ dim->state = DIM_MEASURE_IN_PROGRESS;
+ dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr, end_sample.byte_ctr,
+ end_sample.comp_ctr, &dim->start_sample);
+ dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr, end_sample.byte_ctr,
+ end_sample.comp_ctr, &dim->measuring_sample);
+ break;
+ case DIM_APPLY_NEW_PROFILE:
+ break;
+ }
+}
+EXPORT_SYMBOL(blk_dim);
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 7/9] linux/dim: Add completions count to dim_sample
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
From: Yamin Friedman <yaminf@mellanox.com>
Added a measurement of completions per/msec to allow for completion based dim
algorithms.
Signed-off-by: Yamin Friedman <yaminf@mellanox.com>
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/net/ethernet/broadcom/bcmsysport.c | 2 +-
drivers/net/ethernet/broadcom/bnxt/bnxt.c | 1 +
drivers/net/ethernet/broadcom/genet/bcmgenet.c | 2 +-
drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 4 ++--
include/linux/dim.h | 7 ++++++-
lib/dim/dim.c | 11 ++++++++++-
lib/dim/net_dim.c | 2 +-
7 files changed, 22 insertions(+), 7 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/bcmsysport.c b/drivers/net/ethernet/broadcom/bcmsysport.c
index 840b3bf1ae3e..df38c8fd373f 100644
--- a/drivers/net/ethernet/broadcom/bcmsysport.c
+++ b/drivers/net/ethernet/broadcom/bcmsysport.c
@@ -1035,7 +1035,7 @@ static int bcm_sysport_poll(struct napi_struct *napi, int budget)
if (priv->dim.use_dim) {
dim_create_sample(priv->dim.event_ctr, priv->dim.packets,
- priv->dim.bytes, &dim_sample);
+ priv->dim.bytes, 0, &dim_sample);
net_dim(&priv->dim.dim, dim_sample);
}
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
index 131ab07aad83..516703ac0009 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c
@@ -2110,6 +2110,7 @@ static int bnxt_poll(struct napi_struct *napi, int budget)
dim_create_sample(cpr->event_ctr,
cpr->rx_packets,
cpr->rx_bytes,
+ 0,
&dim_sample);
net_dim(&cpr->dim, dim_sample);
}
diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.c b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
index 68d96e333c6d..aca82ef12d28 100644
--- a/drivers/net/ethernet/broadcom/genet/bcmgenet.c
+++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.c
@@ -1910,7 +1910,7 @@ static int bcmgenet_rx_poll(struct napi_struct *napi, int budget)
if (ring->dim.use_dim) {
dim_create_sample(ring->dim.event_ctr, ring->dim.packets,
- ring->dim.bytes, &dim_sample);
+ ring->dim.bytes, 0, &dim_sample);
net_dim(&ring->dim.dim, dim_sample);
}
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
index 432474754d77..76fc57762083 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c
@@ -53,7 +53,7 @@ static void mlx5e_handle_tx_dim(struct mlx5e_txqsq *sq)
if (unlikely(!test_bit(MLX5E_SQ_STATE_AM, &sq->state)))
return;
- dim_create_sample(sq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
+ dim_create_sample(sq->cq.event_ctr, stats->packets, stats->bytes, 0, &dim_sample);
net_dim(&sq->dim, dim_sample);
}
@@ -65,7 +65,7 @@ static void mlx5e_handle_rx_dim(struct mlx5e_rq *rq)
if (unlikely(!test_bit(MLX5E_RQ_STATE_AM, &rq->state)))
return;
- dim_create_sample(rq->cq.event_ctr, stats->packets, stats->bytes, &dim_sample);
+ dim_create_sample(rq->cq.event_ctr, stats->packets, stats->bytes, 0, &dim_sample);
net_dim(&rq->dim, dim_sample);
}
diff --git a/include/linux/dim.h b/include/linux/dim.h
index 88a74cd83d95..39b621dc8e3e 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -45,6 +45,7 @@
struct dim_cq_moder {
u16 usec;
u16 pkts;
+ u16 comps;
u8 cq_period_mode;
};
@@ -53,18 +54,22 @@ struct dim_sample {
u32 pkt_ctr;
u32 byte_ctr;
u16 event_ctr;
+ u32 comp_ctr;
};
struct dim_stats {
int ppms; /* packets per msec */
int bpms; /* bytes per msec */
int epms; /* events per msec */
+ int cpms; /* completions per msec */
+ int cpe_ratio; /* ratio of completions to events */
};
struct dim { /* Dynamic Interrupt Moderation */
u8 state;
struct dim_stats prev_stats;
struct dim_sample start_sample;
+ struct dim_sample measuring_sample;
struct work_struct work;
u8 profile_ix;
u8 mode;
@@ -113,7 +118,7 @@ void dim_park_on_top(struct dim *dim);
void dim_park_tired(struct dim *dim);
-void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, struct dim_sample *s);
+void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, u64 comps, struct dim_sample *s);
void dim_calc_stats(struct dim_sample *start, struct dim_sample *end,
struct dim_stats *curr_stats);
diff --git a/lib/dim/dim.c b/lib/dim/dim.c
index 93e1ddd701b0..b7283f1cb000 100644
--- a/lib/dim/dim.c
+++ b/lib/dim/dim.c
@@ -54,12 +54,13 @@ void dim_park_tired(struct dim *dim)
}
EXPORT_SYMBOL(dim_park_tired);
-void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, struct dim_sample *s)
+void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, u64 comps, struct dim_sample *s)
{
s->time = ktime_get();
s->pkt_ctr = packets;
s->byte_ctr = bytes;
s->event_ctr = event_ctr;
+ s->comp_ctr = comps;
}
EXPORT_SYMBOL(dim_create_sample);
@@ -71,6 +72,8 @@ void dim_calc_stats(struct dim_sample *start, struct dim_sample *end,
u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr);
u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr,
start->byte_ctr);
+ u32 ncomps = BIT_GAP(BITS_PER_TYPE(u32), end->comp_ctr,
+ start->comp_ctr);
if (!delta_us)
return;
@@ -79,5 +82,11 @@ void dim_calc_stats(struct dim_sample *start, struct dim_sample *end,
curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
curr_stats->epms = DIV_ROUND_UP(DIM_NEVENTS * USEC_PER_MSEC,
delta_us);
+ curr_stats->cpms = DIV_ROUND_UP(ncomps * USEC_PER_MSEC, delta_us);
+ if (curr_stats->epms != 0)
+ curr_stats->cpe_ratio = (curr_stats->cpms * 100) / curr_stats->epms;
+ else
+ curr_stats->cpe_ratio = 0;
+
}
EXPORT_SYMBOL(dim_calc_stats);
diff --git a/lib/dim/net_dim.c b/lib/dim/net_dim.c
index cf95cd20cf02..10605b77bbc5 100644
--- a/lib/dim/net_dim.c
+++ b/lib/dim/net_dim.c
@@ -183,7 +183,7 @@ void net_dim(struct dim *dim, struct dim_sample end_sample)
/* fall through */
case DIM_START_MEASURE:
dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
- end_sample.byte_ctr, &dim->start_sample);
+ end_sample.byte_ctr, 0, &dim->start_sample);
dim->state = DIM_MEASURE_IN_PROGRESS;
break;
case DIM_APPLY_NEW_PROFILE:
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 6/9] linux/dim: Move implementation to .c files
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
Moved all logic from dim.h and net_dim.h to dim.c and net_dim.c.
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
MAINTAINERS | 2 +
include/linux/dim.h | 86 +++---------------
include/linux/net_dim.h | 182 ++-----------------------------------
lib/Kconfig | 7 ++
lib/Makefile | 1 +
lib/dim/Makefile | 9 ++
lib/dim/dim.c | 83 +++++++++++++++++
lib/dim/net_dim.c | 193 ++++++++++++++++++++++++++++++++++++++++
8 files changed, 312 insertions(+), 251 deletions(-)
create mode 100644 lib/dim/Makefile
create mode 100644 lib/dim/dim.c
create mode 100644 lib/dim/net_dim.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 019a2bcfbd09..6ae949be8b83 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5337,6 +5337,8 @@ DYNAMIC INTERRUPT MODERATION
M: Tal Gilboa <talgi@mellanox.com>
S: Maintained
F: include/linux/net_dim.h
+F: include/linux/dim.h
+F: lib/dim/
DZ DECSTATION DZ11 SERIAL DRIVER
M: "Maciej W. Rozycki" <macro@linux-mips.org>
diff --git a/include/linux/dim.h b/include/linux/dim.h
index 8d9f8279ebee..88a74cd83d95 100644
--- a/include/linux/dim.h
+++ b/include/linux/dim.h
@@ -105,79 +105,17 @@ enum {
DIM_ON_EDGE,
};
-static inline bool dim_on_top(struct dim *dim)
-{
- switch (dim->tune_state) {
- case DIM_PARKING_ON_TOP:
- case DIM_PARKING_TIRED:
- return true;
- case DIM_GOING_RIGHT:
- return (dim->steps_left > 1) && (dim->steps_right == 1);
- default: /* DIM_GOING_LEFT */
- return (dim->steps_right > 1) && (dim->steps_left == 1);
- }
-}
-
-static inline void dim_turn(struct dim *dim)
-{
- switch (dim->tune_state) {
- case DIM_PARKING_ON_TOP:
- case DIM_PARKING_TIRED:
- break;
- case DIM_GOING_RIGHT:
- dim->tune_state = DIM_GOING_LEFT;
- dim->steps_left = 0;
- break;
- case DIM_GOING_LEFT:
- dim->tune_state = DIM_GOING_RIGHT;
- dim->steps_right = 0;
- break;
- }
-}
-
-static inline void dim_park_on_top(struct dim *dim)
-{
- dim->steps_right = 0;
- dim->steps_left = 0;
- dim->tired = 0;
- dim->tune_state = DIM_PARKING_ON_TOP;
-}
-
-static inline void dim_park_tired(struct dim *dim)
-{
- dim->steps_right = 0;
- dim->steps_left = 0;
- dim->tune_state = DIM_PARKING_TIRED;
-}
-
-static inline void dim_create_sample(u16 event_ctr,
- u64 packets,
- u64 bytes,
- struct dim_sample *s)
-{
- s->time = ktime_get();
- s->pkt_ctr = packets;
- s->byte_ctr = bytes;
- s->event_ctr = event_ctr;
-}
-
-static inline void dim_calc_stats(struct dim_sample *start,
- struct dim_sample *end,
- struct dim_stats *curr_stats)
-{
- /* u32 holds up to 71 minutes, should be enough */
- u32 delta_us = ktime_us_delta(end->time, start->time);
- u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr);
- u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr,
- start->byte_ctr);
-
- if (!delta_us)
- return;
-
- curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us);
- curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
- curr_stats->epms = DIV_ROUND_UP(DIM_NEVENTS * USEC_PER_MSEC,
- delta_us);
-}
+bool dim_on_top(struct dim *dim);
+
+void dim_turn(struct dim *dim);
+
+void dim_park_on_top(struct dim *dim);
+
+void dim_park_tired(struct dim *dim);
+
+void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, struct dim_sample *s);
+
+void dim_calc_stats(struct dim_sample *start, struct dim_sample *end,
+ struct dim_stats *curr_stats);
#endif /* DIM_H */
diff --git a/include/linux/net_dim.h b/include/linux/net_dim.h
index e9363c372b68..debf548cfb33 100644
--- a/include/linux/net_dim.h
+++ b/include/linux/net_dim.h
@@ -89,186 +89,14 @@ tx_profile[DIM_CQ_PERIOD_NUM_MODES][NET_DIM_PARAMS_NUM_PROFILES] = {
NET_DIM_TX_CQE_PROFILES,
};
-static inline struct dim_cq_moder
-net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
-{
- struct dim_cq_moder cq_moder = rx_profile[cq_period_mode][ix];
+struct dim_cq_moder net_dim_get_rx_moderation(u8 cq_period_mode, int ix);
- cq_moder.cq_period_mode = cq_period_mode;
- return cq_moder;
-}
-
-static inline struct dim_cq_moder
-net_dim_get_def_rx_moderation(u8 cq_period_mode)
-{
- u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
- NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
-
- return net_dim_get_rx_moderation(cq_period_mode, profile_ix);
-}
-
-static inline struct dim_cq_moder
-net_dim_get_tx_moderation(u8 cq_period_mode, int ix)
-{
- struct dim_cq_moder cq_moder = tx_profile[cq_period_mode][ix];
-
- cq_moder.cq_period_mode = cq_period_mode;
- return cq_moder;
-}
-
-static inline struct dim_cq_moder
-net_dim_get_def_tx_moderation(u8 cq_period_mode)
-{
- u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
- NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
-
- return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
-}
-
-static inline int net_dim_step(struct dim *dim)
-{
- if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2))
- return DIM_TOO_TIRED;
-
- switch (dim->tune_state) {
- case DIM_PARKING_ON_TOP:
- case DIM_PARKING_TIRED:
- break;
- case DIM_GOING_RIGHT:
- if (dim->profile_ix == (NET_DIM_PARAMS_NUM_PROFILES - 1))
- return DIM_ON_EDGE;
- dim->profile_ix++;
- dim->steps_right++;
- break;
- case DIM_GOING_LEFT:
- if (dim->profile_ix == 0)
- return DIM_ON_EDGE;
- dim->profile_ix--;
- dim->steps_left++;
- break;
- }
-
- dim->tired++;
- return DIM_STEPPED;
-}
-
-static inline void net_dim_exit_parking(struct dim *dim)
-{
- dim->tune_state = dim->profile_ix ? DIM_GOING_LEFT :
- DIM_GOING_RIGHT;
- net_dim_step(dim);
-}
-
-static inline int net_dim_stats_compare(struct dim_stats *curr,
- struct dim_stats *prev)
-{
- if (!prev->bpms)
- return curr->bpms ? DIM_STATS_BETTER :
- DIM_STATS_SAME;
+struct dim_cq_moder net_dim_get_def_rx_moderation(u8 cq_period_mode);
- if (IS_SIGNIFICANT_DIFF(curr->bpms, prev->bpms))
- return (curr->bpms > prev->bpms) ? DIM_STATS_BETTER :
- DIM_STATS_WORSE;
+struct dim_cq_moder net_dim_get_tx_moderation(u8 cq_period_mode, int ix);
- if (!prev->ppms)
- return curr->ppms ? DIM_STATS_BETTER :
- DIM_STATS_SAME;
+struct dim_cq_moder net_dim_get_def_tx_moderation(u8 cq_period_mode);
- if (IS_SIGNIFICANT_DIFF(curr->ppms, prev->ppms))
- return (curr->ppms > prev->ppms) ? DIM_STATS_BETTER :
- DIM_STATS_WORSE;
-
- if (!prev->epms)
- return DIM_STATS_SAME;
-
- if (IS_SIGNIFICANT_DIFF(curr->epms, prev->epms))
- return (curr->epms < prev->epms) ? DIM_STATS_BETTER :
- DIM_STATS_WORSE;
-
- return DIM_STATS_SAME;
-}
-
-static inline bool net_dim_decision(struct dim_stats *curr_stats,
- struct dim *dim)
-{
- int prev_state = dim->tune_state;
- int prev_ix = dim->profile_ix;
- int stats_res;
- int step_res;
-
- switch (dim->tune_state) {
- case DIM_PARKING_ON_TOP:
- stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
- if (stats_res != DIM_STATS_SAME)
- net_dim_exit_parking(dim);
- break;
-
- case DIM_PARKING_TIRED:
- dim->tired--;
- if (!dim->tired)
- net_dim_exit_parking(dim);
- break;
-
- case DIM_GOING_RIGHT:
- case DIM_GOING_LEFT:
- stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
- if (stats_res != DIM_STATS_BETTER)
- dim_turn(dim);
-
- if (dim_on_top(dim)) {
- dim_park_on_top(dim);
- break;
- }
-
- step_res = net_dim_step(dim);
- switch (step_res) {
- case DIM_ON_EDGE:
- dim_park_on_top(dim);
- break;
- case DIM_TOO_TIRED:
- dim_park_tired(dim);
- break;
- }
-
- break;
- }
-
- if ((prev_state != DIM_PARKING_ON_TOP) ||
- (dim->tune_state != DIM_PARKING_ON_TOP))
- dim->prev_stats = *curr_stats;
-
- return dim->profile_ix != prev_ix;
-}
-
-static inline void net_dim(struct dim *dim,
- struct dim_sample end_sample)
-{
- struct dim_stats curr_stats;
- u16 nevents;
-
- switch (dim->state) {
- case DIM_MEASURE_IN_PROGRESS:
- nevents = BIT_GAP(BITS_PER_TYPE(u16),
- end_sample.event_ctr,
- dim->start_sample.event_ctr);
- if (nevents < DIM_NEVENTS)
- break;
- dim_calc_stats(&dim->start_sample, &end_sample,
- &curr_stats);
- if (net_dim_decision(&curr_stats, dim)) {
- dim->state = DIM_APPLY_NEW_PROFILE;
- schedule_work(&dim->work);
- break;
- }
- /* fall through */
- case DIM_START_MEASURE:
- dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
- end_sample.byte_ctr, &dim->start_sample);
- dim->state = DIM_MEASURE_IN_PROGRESS;
- break;
- case DIM_APPLY_NEW_PROFILE:
- break;
- }
-}
+void net_dim(struct dim *dim, struct dim_sample end_sample);
#endif /* NET_DIM_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9e56539bd11..153465c65624 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -551,6 +551,13 @@ config SIGNATURE
Digital signature verification. Currently only RSA is supported.
Implementation is done using GnuPG MPI library
+config DIMLIB
+ bool "DIM library"
+ default y
+ help
+ Dynamic Interrupt Moderation library.
+ It is used to implement DIM logic.
+
#
# libfdt files, only selected if needed.
#
diff --git a/lib/Makefile b/lib/Makefile
index e1b59da71418..0f2155eb9a85 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -185,6 +185,7 @@ obj-$(CONFIG_GLOB) += glob.o
obj-$(CONFIG_GLOB_SELFTEST) += globtest.o
obj-$(CONFIG_MPILIB) += mpi/
+obj-$(CONFIG_DIMLIB) += dim/
obj-$(CONFIG_SIGNATURE) += digsig.o
lib-$(CONFIG_CLZ_TAB) += clz_tab.o
diff --git a/lib/dim/Makefile b/lib/dim/Makefile
new file mode 100644
index 000000000000..160afe288df0
--- /dev/null
+++ b/lib/dim/Makefile
@@ -0,0 +1,9 @@
+#
+# DIM Dynamic Interrupt Moderation library
+#
+
+obj-$(CONFIG_DIMLIB) = net_dim.o
+
+net_dim-y = \
+ dim.o \
+ net_dim.o
diff --git a/lib/dim/dim.c b/lib/dim/dim.c
new file mode 100644
index 000000000000..93e1ddd701b0
--- /dev/null
+++ b/lib/dim/dim.c
@@ -0,0 +1,83 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright (c) 2018, Mellanox Technologies inc. All rights reserved.
+ */
+
+#include <linux/dim.h>
+
+bool dim_on_top(struct dim *dim)
+{
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
+ return true;
+ case DIM_GOING_RIGHT:
+ return (dim->steps_left > 1) && (dim->steps_right == 1);
+ default: /* DIM_GOING_LEFT */
+ return (dim->steps_right > 1) && (dim->steps_left == 1);
+ }
+}
+EXPORT_SYMBOL(dim_on_top);
+
+void dim_turn(struct dim *dim)
+{
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
+ break;
+ case DIM_GOING_RIGHT:
+ dim->tune_state = DIM_GOING_LEFT;
+ dim->steps_left = 0;
+ break;
+ case DIM_GOING_LEFT:
+ dim->tune_state = DIM_GOING_RIGHT;
+ dim->steps_right = 0;
+ break;
+ }
+}
+EXPORT_SYMBOL(dim_turn);
+
+void dim_park_on_top(struct dim *dim)
+{
+ dim->steps_right = 0;
+ dim->steps_left = 0;
+ dim->tired = 0;
+ dim->tune_state = DIM_PARKING_ON_TOP;
+}
+EXPORT_SYMBOL(dim_park_on_top);
+
+void dim_park_tired(struct dim *dim)
+{
+ dim->steps_right = 0;
+ dim->steps_left = 0;
+ dim->tune_state = DIM_PARKING_TIRED;
+}
+EXPORT_SYMBOL(dim_park_tired);
+
+void dim_create_sample(u16 event_ctr, u64 packets, u64 bytes, struct dim_sample *s)
+{
+ s->time = ktime_get();
+ s->pkt_ctr = packets;
+ s->byte_ctr = bytes;
+ s->event_ctr = event_ctr;
+}
+EXPORT_SYMBOL(dim_create_sample);
+
+void dim_calc_stats(struct dim_sample *start, struct dim_sample *end,
+ struct dim_stats *curr_stats)
+{
+ /* u32 holds up to 71 minutes, should be enough */
+ u32 delta_us = ktime_us_delta(end->time, start->time);
+ u32 npkts = BIT_GAP(BITS_PER_TYPE(u32), end->pkt_ctr, start->pkt_ctr);
+ u32 nbytes = BIT_GAP(BITS_PER_TYPE(u32), end->byte_ctr,
+ start->byte_ctr);
+
+ if (!delta_us)
+ return;
+
+ curr_stats->ppms = DIV_ROUND_UP(npkts * USEC_PER_MSEC, delta_us);
+ curr_stats->bpms = DIV_ROUND_UP(nbytes * USEC_PER_MSEC, delta_us);
+ curr_stats->epms = DIV_ROUND_UP(DIM_NEVENTS * USEC_PER_MSEC,
+ delta_us);
+}
+EXPORT_SYMBOL(dim_calc_stats);
diff --git a/lib/dim/net_dim.c b/lib/dim/net_dim.c
new file mode 100644
index 000000000000..cf95cd20cf02
--- /dev/null
+++ b/lib/dim/net_dim.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
+/*
+ * Copyright (c) 2018, Mellanox Technologies inc. All rights reserved.
+ */
+
+#include <linux/net_dim.h>
+#include <linux/dim.h>
+
+struct dim_cq_moder
+net_dim_get_rx_moderation(u8 cq_period_mode, int ix)
+{
+ struct dim_cq_moder cq_moder = rx_profile[cq_period_mode][ix];
+
+ cq_moder.cq_period_mode = cq_period_mode;
+ return cq_moder;
+}
+EXPORT_SYMBOL(net_dim_get_rx_moderation);
+
+struct dim_cq_moder
+net_dim_get_def_rx_moderation(u8 cq_period_mode)
+{
+ u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
+
+ return net_dim_get_rx_moderation(cq_period_mode, profile_ix);
+}
+EXPORT_SYMBOL(net_dim_get_def_rx_moderation);
+
+struct dim_cq_moder
+net_dim_get_tx_moderation(u8 cq_period_mode, int ix)
+{
+ struct dim_cq_moder cq_moder = tx_profile[cq_period_mode][ix];
+
+ cq_moder.cq_period_mode = cq_period_mode;
+ return cq_moder;
+}
+EXPORT_SYMBOL(net_dim_get_tx_moderation);
+
+struct dim_cq_moder
+net_dim_get_def_tx_moderation(u8 cq_period_mode)
+{
+ u8 profile_ix = cq_period_mode == DIM_CQ_PERIOD_MODE_START_FROM_CQE ?
+ NET_DIM_DEF_PROFILE_CQE : NET_DIM_DEF_PROFILE_EQE;
+
+ return net_dim_get_tx_moderation(cq_period_mode, profile_ix);
+}
+EXPORT_SYMBOL(net_dim_get_def_tx_moderation);
+
+static inline int net_dim_step(struct dim *dim)
+{
+ if (dim->tired == (NET_DIM_PARAMS_NUM_PROFILES * 2))
+ return DIM_TOO_TIRED;
+
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ case DIM_PARKING_TIRED:
+ break;
+ case DIM_GOING_RIGHT:
+ if (dim->profile_ix == (NET_DIM_PARAMS_NUM_PROFILES - 1))
+ return DIM_ON_EDGE;
+ dim->profile_ix++;
+ dim->steps_right++;
+ break;
+ case DIM_GOING_LEFT:
+ if (dim->profile_ix == 0)
+ return DIM_ON_EDGE;
+ dim->profile_ix--;
+ dim->steps_left++;
+ break;
+ }
+
+ dim->tired++;
+ return DIM_STEPPED;
+}
+
+static inline void net_dim_exit_parking(struct dim *dim)
+{
+ dim->tune_state = dim->profile_ix ? DIM_GOING_LEFT :
+ DIM_GOING_RIGHT;
+ net_dim_step(dim);
+}
+
+static inline int net_dim_stats_compare(struct dim_stats *curr,
+ struct dim_stats *prev)
+{
+ if (!prev->bpms)
+ return curr->bpms ? DIM_STATS_BETTER :
+ DIM_STATS_SAME;
+
+ if (IS_SIGNIFICANT_DIFF(curr->bpms, prev->bpms))
+ return (curr->bpms > prev->bpms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
+
+ if (!prev->ppms)
+ return curr->ppms ? DIM_STATS_BETTER :
+ DIM_STATS_SAME;
+
+ if (IS_SIGNIFICANT_DIFF(curr->ppms, prev->ppms))
+ return (curr->ppms > prev->ppms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
+
+ if (!prev->epms)
+ return DIM_STATS_SAME;
+
+ if (IS_SIGNIFICANT_DIFF(curr->epms, prev->epms))
+ return (curr->epms < prev->epms) ? DIM_STATS_BETTER :
+ DIM_STATS_WORSE;
+
+ return DIM_STATS_SAME;
+}
+
+static inline bool net_dim_decision(struct dim_stats *curr_stats,
+ struct dim *dim)
+{
+ int prev_state = dim->tune_state;
+ int prev_ix = dim->profile_ix;
+ int stats_res;
+ int step_res;
+
+ switch (dim->tune_state) {
+ case DIM_PARKING_ON_TOP:
+ stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
+ if (stats_res != DIM_STATS_SAME)
+ net_dim_exit_parking(dim);
+ break;
+
+ case DIM_PARKING_TIRED:
+ dim->tired--;
+ if (!dim->tired)
+ net_dim_exit_parking(dim);
+ break;
+
+ case DIM_GOING_RIGHT:
+ case DIM_GOING_LEFT:
+ stats_res = net_dim_stats_compare(curr_stats, &dim->prev_stats);
+ if (stats_res != DIM_STATS_BETTER)
+ dim_turn(dim);
+
+ if (dim_on_top(dim)) {
+ dim_park_on_top(dim);
+ break;
+ }
+
+ step_res = net_dim_step(dim);
+ switch (step_res) {
+ case DIM_ON_EDGE:
+ dim_park_on_top(dim);
+ break;
+ case DIM_TOO_TIRED:
+ dim_park_tired(dim);
+ break;
+ }
+
+ break;
+ }
+
+ if ((prev_state != DIM_PARKING_ON_TOP) ||
+ (dim->tune_state != DIM_PARKING_ON_TOP))
+ dim->prev_stats = *curr_stats;
+
+ return dim->profile_ix != prev_ix;
+}
+
+void net_dim(struct dim *dim, struct dim_sample end_sample)
+{
+ struct dim_stats curr_stats;
+ u16 nevents;
+
+ switch (dim->state) {
+ case DIM_MEASURE_IN_PROGRESS:
+ nevents = BIT_GAP(BITS_PER_TYPE(u16),
+ end_sample.event_ctr,
+ dim->start_sample.event_ctr);
+ if (nevents < DIM_NEVENTS)
+ break;
+ dim_calc_stats(&dim->start_sample, &end_sample,
+ &curr_stats);
+ if (net_dim_decision(&curr_stats, dim)) {
+ dim->state = DIM_APPLY_NEW_PROFILE;
+ schedule_work(&dim->work);
+ break;
+ }
+ /* fall through */
+ case DIM_START_MEASURE:
+ dim_create_sample(end_sample.event_ctr, end_sample.pkt_ctr,
+ end_sample.byte_ctr, &dim->start_sample);
+ dim->state = DIM_MEASURE_IN_PROGRESS;
+ break;
+ case DIM_APPLY_NEW_PROFILE:
+ break;
+ }
+}
+EXPORT_SYMBOL(net_dim);
--
2.19.1
^ permalink raw reply related
* [RFC/PATCH net-next 9/9] drivers/infiniband: Use blk_dim in infiniband driver
From: Tal Gilboa @ 2019-02-03 13:50 UTC (permalink / raw)
To: David Miller
Cc: netdev, Tariq Toukan, Tal Gilboa, Saeed Mahameed, Idan Burstein,
Yamin Friedman, Max Gurtovoy, Florian Fainelli, Andy Gospodarek
In-Reply-To: <20190203135040.28871-1-talgi@mellanox.com>
From: Yamin Friedman <yaminf@mellanox.com>
Added the interface in the infiniband driver that applies the blk_dim adaptive
moderation.
Performance improvment (ConnectX-5 100GbE, x86) running FIO benchmark over NVMf
between two equal end-hosts across a Mellanox switch:
Running long tests that switch between periods of high bandwidth high latency
and low bandwidth low latency, using the blk_dim algorithm there is a much
shorter wait before the moderation is reduced and thus tail latency is reduced.
There is a 200% reduction on tail latency when switching from high bandwidth to
low bandwidth traffic without degredation of other flow parameters.
The blk_dim algorithm was designed to measure the effectiveness of moderation
on the flow in a general way and thus should be appropriate for all RDMA storage
protocols.
Signed-off-by: Yamin Friedman <yaminf@mellanox.com>
Signed-off-by: Tal Gilboa <talgi@mellanox.com>
---
drivers/infiniband/core/cq.c | 75 ++++++++++++++++++++++++++++++---
drivers/infiniband/hw/mlx4/qp.c | 2 +-
drivers/infiniband/hw/mlx5/qp.c | 2 +-
include/linux/irq_poll.h | 7 +++
include/rdma/ib_verbs.h | 11 ++++-
lib/irq_poll.c | 13 +++++-
6 files changed, 100 insertions(+), 10 deletions(-)
diff --git a/drivers/infiniband/core/cq.c b/drivers/infiniband/core/cq.c
index d61e5e1427c2..065b54978dae 100644
--- a/drivers/infiniband/core/cq.c
+++ b/drivers/infiniband/core/cq.c
@@ -14,6 +14,7 @@
#include <linux/err.h>
#include <linux/slab.h>
#include <rdma/ib_verbs.h>
+#include <linux/blk_dim.h>
/* # of WCs to poll for with a single call to ib_poll_cq */
#define IB_POLL_BATCH 16
@@ -26,6 +27,51 @@
#define IB_POLL_FLAGS \
(IB_CQ_NEXT_COMP | IB_CQ_REPORT_MISSED_EVENTS)
+static bool use_am = true;
+module_param(use_am, bool, 0444);
+MODULE_PARM_DESC(use_am, "Use cq adaptive moderation");
+
+static int ib_cq_dim_modify_cq(struct ib_cq *cq, unsigned short level)
+{
+ u16 usec = blk_dim_prof[level].usec;
+ u16 comps = blk_dim_prof[level].comps;
+
+ return cq->device->modify_cq(cq, comps, usec);
+}
+
+static void update_cq_moderation(struct dim *dim, struct ib_cq *cq)
+{
+ dim->state = DIM_START_MEASURE;
+
+ ib_cq_dim_modify_cq(cq, dim->profile_ix);
+}
+
+static void ib_cq_blk_dim_workqueue_work(struct work_struct *w)
+{
+ struct dim *dim = container_of(w, struct dim, work);
+ struct ib_cq *cq = container_of(dim, struct ib_cq, workqueue_poll.dim);
+
+ update_cq_moderation(dim, cq);
+}
+
+static void ib_cq_blk_dim_irqpoll_work(struct work_struct *w)
+{
+ struct dim *dim = container_of(w, struct dim, work);
+ struct irq_poll *iop = container_of(dim, struct irq_poll, dim);
+ struct ib_cq *cq = container_of(iop, struct ib_cq, iop);
+
+ update_cq_moderation(dim, cq);
+}
+
+void blk_dim_init(struct dim *dim, work_func_t func)
+{
+ memset(dim, 0, sizeof(*dim));
+ dim->state = DIM_START_MEASURE;
+ dim->tune_state = DIM_GOING_RIGHT;
+ dim->profile_ix = BLK_DIM_START_PROFILE;
+ INIT_WORK(&dim->work, func);
+}
+
static int __ib_process_cq(struct ib_cq *cq, int budget, struct ib_wc *wcs,
int batch)
{
@@ -105,19 +151,28 @@ static void ib_cq_completion_softirq(struct ib_cq *cq, void *private)
static void ib_cq_poll_work(struct work_struct *work)
{
- struct ib_cq *cq = container_of(work, struct ib_cq, work);
+ struct ib_cq *cq = container_of(work, struct ib_cq, workqueue_poll.work);
int completed;
+ struct dim_sample e_sample;
+ struct dim_sample *m_sample = &cq->workqueue_poll.dim.measuring_sample;
completed = __ib_process_cq(cq, IB_POLL_BUDGET_WORKQUEUE, cq->wc,
IB_POLL_BATCH);
+
+ if (cq->workqueue_poll.dim_used)
+ dim_create_sample(m_sample->event_ctr + 1, m_sample->pkt_ctr, m_sample->byte_ctr,
+ m_sample->comp_ctr + completed, &e_sample);
+
if (completed >= IB_POLL_BUDGET_WORKQUEUE ||
ib_req_notify_cq(cq, IB_POLL_FLAGS) > 0)
- queue_work(cq->comp_wq, &cq->work);
+ queue_work(cq->comp_wq, &cq->workqueue_poll.work);
+ else if (cq->workqueue_poll.dim_used)
+ blk_dim(&cq->workqueue_poll.dim, e_sample);
}
static void ib_cq_completion_workqueue(struct ib_cq *cq, void *private)
{
- queue_work(cq->comp_wq, &cq->work);
+ queue_work(cq->comp_wq, &cq->workqueue_poll.work);
}
/**
@@ -172,12 +227,20 @@ struct ib_cq *__ib_alloc_cq(struct ib_device *dev, void *private,
cq->comp_handler = ib_cq_completion_softirq;
irq_poll_init(&cq->iop, IB_POLL_BUDGET_IRQ, ib_poll_handler);
+ if (cq->device->modify_cq && use_am) {
+ blk_dim_init(&cq->iop.dim, ib_cq_blk_dim_irqpoll_work);
+ cq->iop.dim_used = true;
+ }
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
break;
case IB_POLL_WORKQUEUE:
case IB_POLL_UNBOUND_WORKQUEUE:
cq->comp_handler = ib_cq_completion_workqueue;
- INIT_WORK(&cq->work, ib_cq_poll_work);
+ INIT_WORK(&cq->workqueue_poll.work, ib_cq_poll_work);
+ if (cq->device->modify_cq && use_am) {
+ blk_dim_init(&cq->workqueue_poll.dim, ib_cq_blk_dim_workqueue_work);
+ cq->workqueue_poll.dim_used = true;
+ }
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
cq->comp_wq = (cq->poll_ctx == IB_POLL_WORKQUEUE) ?
ib_comp_wq : ib_comp_unbound_wq;
@@ -217,7 +280,9 @@ void ib_free_cq(struct ib_cq *cq)
break;
case IB_POLL_WORKQUEUE:
case IB_POLL_UNBOUND_WORKQUEUE:
- cancel_work_sync(&cq->work);
+ cancel_work_sync(&cq->workqueue_poll.work);
+ if (cq->workqueue_poll.dim_used)
+ flush_work(&cq->iop.dim.work);
break;
default:
WARN_ON_ONCE(1);
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 971e9a9ebdaf..f3e5dbe4689a 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -4365,7 +4365,7 @@ static void handle_drain_completion(struct ib_cq *cq,
irq_poll_enable(&cq->iop);
break;
case IB_POLL_WORKQUEUE:
- cancel_work_sync(&cq->work);
+ cancel_work_sync(&cq->workqueue_poll.work);
break;
default:
WARN_ON_ONCE(1);
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index dd2ae640bc84..4b65147010cc 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -6102,7 +6102,7 @@ static void handle_drain_completion(struct ib_cq *cq,
irq_poll_enable(&cq->iop);
break;
case IB_POLL_WORKQUEUE:
- cancel_work_sync(&cq->work);
+ cancel_work_sync(&cq->workqueue_poll.work);
break;
default:
WARN_ON_ONCE(1);
diff --git a/include/linux/irq_poll.h b/include/linux/irq_poll.h
index 16aaeccb65cb..ede1a390159b 100644
--- a/include/linux/irq_poll.h
+++ b/include/linux/irq_poll.h
@@ -2,14 +2,21 @@
#ifndef IRQ_POLL_H
#define IRQ_POLL_H
+#include <linux/blk_dim.h>
+
struct irq_poll;
typedef int (irq_poll_fn)(struct irq_poll *, int);
+typedef int (irq_poll_dim_fn)(struct irq_poll *);
struct irq_poll {
struct list_head list;
unsigned long state;
int weight;
irq_poll_fn *poll;
+
+ bool dim_used;
+ struct dim dim;
+ irq_poll_dim_fn *dimfn;
};
enum {
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index a3ceed3a040a..d8060c3cee06 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -1584,6 +1584,13 @@ enum ib_poll_context {
IB_POLL_UNBOUND_WORKQUEUE, /* poll from unbound workqueue */
};
+struct ib_cq_workqueue_poll {
+ struct dim dim;
+ struct work_struct work;
+ bool dim_used;
+};
+
+
struct ib_cq {
struct ib_device *device;
struct ib_uobject *uobject;
@@ -1595,8 +1602,8 @@ struct ib_cq {
enum ib_poll_context poll_ctx;
struct ib_wc *wc;
union {
- struct irq_poll iop;
- struct work_struct work;
+ struct irq_poll iop;
+ struct ib_cq_workqueue_poll workqueue_poll;
};
struct workqueue_struct *comp_wq;
/*
diff --git a/lib/irq_poll.c b/lib/irq_poll.c
index 86a709954f5a..2b5e41f0e583 100644
--- a/lib/irq_poll.c
+++ b/lib/irq_poll.c
@@ -53,6 +53,8 @@ static void __irq_poll_complete(struct irq_poll *iop)
list_del(&iop->list);
smp_mb__before_atomic();
clear_bit_unlock(IRQ_POLL_F_SCHED, &iop->state);
+ if (iop->dim_used)
+ blk_dim(&iop->dim, iop->dim.measuring_sample);
}
/**
@@ -86,6 +88,7 @@ static void __latent_entropy irq_poll_softirq(struct softirq_action *h)
while (!list_empty(list)) {
struct irq_poll *iop;
int work, weight;
+ struct dim_sample *m_sample;
/*
* If softirq window is exhausted then punt.
@@ -104,10 +107,16 @@ static void __latent_entropy irq_poll_softirq(struct softirq_action *h)
*/
iop = list_entry(list->next, struct irq_poll, list);
+ m_sample = &iop->dim.measuring_sample;
weight = iop->weight;
work = 0;
- if (test_bit(IRQ_POLL_F_SCHED, &iop->state))
+ if (test_bit(IRQ_POLL_F_SCHED, &iop->state)) {
work = iop->poll(iop, weight);
+ if (iop->dim_used)
+ dim_create_sample(m_sample->event_ctr + 1, m_sample->pkt_ctr,
+ m_sample->byte_ctr, m_sample->comp_ctr + work,
+ &iop->dim.measuring_sample);
+ }
budget -= work;
@@ -144,6 +153,8 @@ static void __latent_entropy irq_poll_softirq(struct softirq_action *h)
**/
void irq_poll_disable(struct irq_poll *iop)
{
+ if (iop->dim_used)
+ flush_work(&iop->dim.work);
set_bit(IRQ_POLL_F_DISABLE, &iop->state);
while (test_and_set_bit(IRQ_POLL_F_SCHED, &iop->state))
msleep(1);
--
2.19.1
^ permalink raw reply related
* [PATCH bpf-next 3/3] bpf, doc: add RISC-V to filter.txt
From: bjorn.topel @ 2019-02-03 11:51 UTC (permalink / raw)
To: linux-riscv, daniel, ast, netdev; +Cc: Björn Töpel, palmer, hch
In-Reply-To: <20190203115132.8766-1-bjorn.topel@gmail.com>
From: Björn Töpel <bjorn.topel@gmail.com>
Update Documentation/networking/filter.txt to mention RISC-V.
Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
---
Documentation/networking/filter.txt | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt
index 01603bc2eff1..b5e060edfc38 100644
--- a/Documentation/networking/filter.txt
+++ b/Documentation/networking/filter.txt
@@ -464,10 +464,11 @@ breakpoints: 0 1
JIT compiler
------------
-The Linux kernel has a built-in BPF JIT compiler for x86_64, SPARC, PowerPC,
-ARM, ARM64, MIPS and s390 and can be enabled through CONFIG_BPF_JIT. The JIT
-compiler is transparently invoked for each attached filter from user space
-or for internal kernel users if it has been previously enabled by root:
+The Linux kernel has a built-in BPF JIT compiler for x86_64, SPARC,
+PowerPC, ARM, ARM64, MIPS, RISC-V and s390 and can be enabled through
+CONFIG_BPF_JIT. The JIT compiler is transparently invoked for each
+attached filter from user space or for internal kernel users if it has
+been previously enabled by root:
echo 1 > /proc/sys/net/core/bpf_jit_enable
@@ -603,9 +604,10 @@ got from bpf_prog_create(), and 'ctx' the given context (e.g.
skb pointer). All constraints and restrictions from bpf_check_classic() apply
before a conversion to the new layout is being done behind the scenes!
-Currently, the classic BPF format is being used for JITing on most 32-bit
-architectures, whereas x86-64, aarch64, s390x, powerpc64, sparc64, arm32 perform
-JIT compilation from eBPF instruction set.
+Currently, the classic BPF format is being used for JITing on most
+32-bit architectures, whereas x86-64, aarch64, s390x, powerpc64,
+sparc64, arm32, riscv (RV64G) perform JIT compilation from eBPF
+instruction set.
Some core changes of the new internal format:
--
2.19.1
^ permalink raw reply related
* [PATCH bpf-next 2/3] MAINTAINERS: add RISC-V BPF JIT maintainer
From: bjorn.topel @ 2019-02-03 11:51 UTC (permalink / raw)
To: linux-riscv, daniel, ast, netdev; +Cc: Björn Töpel, palmer, hch
In-Reply-To: <20190203115132.8766-1-bjorn.topel@gmail.com>
From: Björn Töpel <bjorn.topel@gmail.com>
Add Björn Töpel as RISC-V BPF JIT maintainer.
Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
---
MAINTAINERS | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 019a2bcfbd09..b4491132b9ce 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -2907,6 +2907,12 @@ L: netdev@vger.kernel.org
S: Maintained
F: arch/powerpc/net/
+BPF JIT for RISC-V (RV64G)
+M: Björn Töpel <bjorn.topel@gmail.com>
+L: netdev@vger.kernel.org
+S: Maintained
+F: arch/riscv/net/
+
BPF JIT for S390
M: Martin Schwidefsky <schwidefsky@de.ibm.com>
M: Heiko Carstens <heiko.carstens@de.ibm.com>
--
2.19.1
^ permalink raw reply related
* [PATCH bpf-next 1/3] bpf, riscv: add BPF JIT for RV64G
From: bjorn.topel @ 2019-02-03 11:51 UTC (permalink / raw)
To: linux-riscv, daniel, ast, netdev; +Cc: Björn Töpel, palmer, hch
In-Reply-To: <20190203115132.8766-1-bjorn.topel@gmail.com>
From: Björn Töpel <bjorn.topel@gmail.com>
This commit adds BPF JIT for RV64G.
The JIT is a two-pass JIT, and has a dynamic prolog/epilogue (similar
to the MIPS64 BPF JIT) instead of static ones (e.g. x86_64).
At the moment the RISC-V Linux port does not support HAVE_KPROBES,
which means that CONFIG_BPF_EVENTS is not supported. Thus, no tests
involving BPF_PROG_TYPE_TRACEPOINT passes.
Further, the implementation does not support "far branching" (>4KiB).
The implementation passes all the test_bpf.ko tests:
test_bpf: Summary: 378 PASSED, 0 FAILED, [366/366 JIT'ed]
All the tail_call tests in the selftest/bpf/test_verifier program
passes.
All tests where done on QEMU (QEMU emulator version 3.1.50
(v3.1.0-688-g8ae951fbc106)).
Signed-off-by: Björn Töpel <bjorn.topel@gmail.com>
---
arch/riscv/Kconfig | 1 +
arch/riscv/Makefile | 2 +-
arch/riscv/net/Makefile | 1 +
arch/riscv/net/bpf_jit_comp.c | 1502 +++++++++++++++++++++++++++++++++
4 files changed, 1505 insertions(+), 1 deletion(-)
create mode 100644 arch/riscv/net/Makefile
create mode 100644 arch/riscv/net/bpf_jit_comp.c
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index feeeaa60697c..e64c657060bb 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -49,6 +49,7 @@ config RISCV
select RISCV_TIMER
select GENERIC_IRQ_MULTI_HANDLER
select ARCH_HAS_PTE_SPECIAL
+ select HAVE_EBPF_JIT if 64BIT
config MMU
def_bool y
diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
index 4b594f2e4f7e..c6342e638ef7 100644
--- a/arch/riscv/Makefile
+++ b/arch/riscv/Makefile
@@ -77,7 +77,7 @@ KBUILD_IMAGE := $(boot)/Image.gz
head-y := arch/riscv/kernel/head.o
-core-y += arch/riscv/kernel/ arch/riscv/mm/
+core-y += arch/riscv/kernel/ arch/riscv/mm/ arch/riscv/net/
libs-y += arch/riscv/lib/
diff --git a/arch/riscv/net/Makefile b/arch/riscv/net/Makefile
new file mode 100644
index 000000000000..a132220cc582
--- /dev/null
+++ b/arch/riscv/net/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
diff --git a/arch/riscv/net/bpf_jit_comp.c b/arch/riscv/net/bpf_jit_comp.c
new file mode 100644
index 000000000000..83b1e2f89074
--- /dev/null
+++ b/arch/riscv/net/bpf_jit_comp.c
@@ -0,0 +1,1502 @@
+// SPDX-License-Identifier: GPL-2.0
+/* BPF JIT compiler for RV64G
+ *
+ * Copyright(c) 2019 Björn Töpel <bjorn.topel@gmail.com>
+ *
+ */
+
+#include <linux/bpf.h>
+#include <linux/filter.h>
+#include <asm/cacheflush.h>
+
+enum {
+ RV_REG_ZERO = 0, /* The constant value 0 */
+ RV_REG_RA = 1, /* Return address */
+ RV_REG_SP = 2, /* Stack pointer */
+ RV_REG_GP = 3, /* Global pointer */
+ RV_REG_TP = 4, /* Thread pointer */
+ RV_REG_T0 = 5, /* Temporaries */
+ RV_REG_T1 = 6,
+ RV_REG_T2 = 7,
+ RV_REG_FP = 8,
+ RV_REG_S1 = 9, /* Saved registers */
+ RV_REG_A0 = 10, /* Function argument/return values */
+ RV_REG_A1 = 11, /* Function arguments */
+ RV_REG_A2 = 12,
+ RV_REG_A3 = 13,
+ RV_REG_A4 = 14,
+ RV_REG_A5 = 15,
+ RV_REG_A6 = 16,
+ RV_REG_A7 = 17,
+ RV_REG_S2 = 18, /* Saved registers */
+ RV_REG_S3 = 19,
+ RV_REG_S4 = 20,
+ RV_REG_S5 = 21,
+ RV_REG_S6 = 22,
+ RV_REG_S7 = 23,
+ RV_REG_S8 = 24,
+ RV_REG_S9 = 25,
+ RV_REG_S10 = 26,
+ RV_REG_S11 = 27,
+ RV_REG_T3 = 28, /* Temporaries */
+ RV_REG_T4 = 29,
+ RV_REG_T5 = 30,
+ RV_REG_T6 = 31,
+};
+
+#define RV_REG_TCC RV_REG_A6
+#define RV_REG_TCC_SAVED RV_REG_S6 /* Store A6 in S6 if program do calls */
+
+static const int regmap[] = {
+ [BPF_REG_0] = RV_REG_A5,
+ [BPF_REG_1] = RV_REG_A0,
+ [BPF_REG_2] = RV_REG_A1,
+ [BPF_REG_3] = RV_REG_A2,
+ [BPF_REG_4] = RV_REG_A3,
+ [BPF_REG_5] = RV_REG_A4,
+ [BPF_REG_6] = RV_REG_S1,
+ [BPF_REG_7] = RV_REG_S2,
+ [BPF_REG_8] = RV_REG_S3,
+ [BPF_REG_9] = RV_REG_S4,
+ [BPF_REG_FP] = RV_REG_S5,
+ [BPF_REG_AX] = RV_REG_T0,
+};
+
+enum {
+ RV_CTX_F_SEEN_TAIL_CALL = 0,
+ RV_CTX_F_SEEN_CALL = RV_REG_RA,
+ RV_CTX_F_SEEN_S1 = RV_REG_S1,
+ RV_CTX_F_SEEN_S2 = RV_REG_S2,
+ RV_CTX_F_SEEN_S3 = RV_REG_S3,
+ RV_CTX_F_SEEN_S4 = RV_REG_S4,
+ RV_CTX_F_SEEN_S5 = RV_REG_S5,
+ RV_CTX_F_SEEN_S6 = RV_REG_S6,
+};
+
+struct rv_jit_context {
+ struct bpf_prog *prog;
+ u32 *insns; /* RV insns */
+ int ninsns;
+ int epilogue_offset;
+ int *offset; /* BPF to RV */
+ unsigned long flags;
+ int stack_size;
+};
+
+struct rv_jit_data {
+ struct bpf_binary_header *header;
+ u8 *image;
+ struct rv_jit_context ctx;
+};
+
+static u8 bpf_to_rv_reg(int bpf_reg, struct rv_jit_context *ctx)
+{
+ u8 reg = regmap[bpf_reg];
+
+ switch (reg) {
+ case RV_CTX_F_SEEN_S1:
+ case RV_CTX_F_SEEN_S2:
+ case RV_CTX_F_SEEN_S3:
+ case RV_CTX_F_SEEN_S4:
+ case RV_CTX_F_SEEN_S5:
+ case RV_CTX_F_SEEN_S6:
+ __set_bit(reg, &ctx->flags);
+ }
+ return reg;
+};
+
+static bool seen_reg(int reg, struct rv_jit_context *ctx)
+{
+ switch (reg) {
+ case RV_CTX_F_SEEN_CALL:
+ case RV_CTX_F_SEEN_S1:
+ case RV_CTX_F_SEEN_S2:
+ case RV_CTX_F_SEEN_S3:
+ case RV_CTX_F_SEEN_S4:
+ case RV_CTX_F_SEEN_S5:
+ case RV_CTX_F_SEEN_S6:
+ return test_bit(reg, &ctx->flags);
+ }
+ return false;
+}
+
+static void mark_call(struct rv_jit_context *ctx)
+{
+ __set_bit(RV_CTX_F_SEEN_CALL, &ctx->flags);
+}
+
+static bool seen_call(struct rv_jit_context *ctx)
+{
+ return seen_reg(RV_REG_RA, ctx);
+}
+
+static void mark_tail_call(struct rv_jit_context *ctx)
+{
+ __set_bit(RV_CTX_F_SEEN_TAIL_CALL, &ctx->flags);
+}
+
+static bool seen_tail_call(struct rv_jit_context *ctx)
+{
+ return test_bit(RV_CTX_F_SEEN_TAIL_CALL, &ctx->flags);
+}
+
+static u8 rv_tail_call_reg(struct rv_jit_context *ctx)
+{
+ mark_tail_call(ctx);
+
+ if (seen_call(ctx)) {
+ __set_bit(RV_CTX_F_SEEN_S6, &ctx->flags);
+ return RV_REG_S6;
+ }
+ return RV_REG_A6;
+}
+
+static void emit(const u32 insn, struct rv_jit_context *ctx)
+{
+ if (ctx->insns)
+ ctx->insns[ctx->ninsns] = insn;
+
+ ctx->ninsns++;
+}
+
+static u32 rv_r_insn(u8 funct7, u8 rs2, u8 rs1, u8 funct3, u8 rd, u8 opcode)
+{
+ return (funct7 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
+ (rd << 7) | opcode;
+}
+
+static u32 rv_i_insn(u16 imm11_0, u8 rs1, u8 funct3, u8 rd, u8 opcode)
+{
+ return (imm11_0 << 20) | (rs1 << 15) | (funct3 << 12) | (rd << 7) |
+ opcode;
+}
+
+static u32 rv_s_insn(u16 imm11_0, u8 rs2, u8 rs1, u8 funct3, u8 opcode)
+{
+ u8 imm11_5 = imm11_0 >> 5, imm4_0 = imm11_0 & 0x1f;
+
+ return (imm11_5 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
+ (imm4_0 << 7) | opcode;
+}
+
+static u32 rv_sb_insn(u16 imm12_1, u8 rs2, u8 rs1, u8 funct3, u8 opcode)
+{
+ u8 imm12 = ((imm12_1 & 0x800) >> 5) | ((imm12_1 & 0x3f0) >> 4);
+ u8 imm4_1 = ((imm12_1 & 0xf) << 1) | ((imm12_1 & 0x400) >> 10);
+
+ return (imm12 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
+ (imm4_1 << 7) | opcode;
+}
+
+static u32 rv_u_insn(u32 imm31_12, u8 rd, u8 opcode)
+{
+ return (imm31_12 << 12) | (rd << 7) | opcode;
+}
+
+static u32 rv_uj_insn(u32 imm20_1, u8 rd, u8 opcode)
+{
+ u32 imm;
+
+ imm = (imm20_1 & 0x80000) | ((imm20_1 & 0x3ff) << 9) |
+ ((imm20_1 & 0x400) >> 2) | ((imm20_1 & 0x7f800) >> 11);
+
+ return (imm << 12) | (rd << 7) | opcode;
+}
+
+static u32 rv_amo_insn(u8 funct5, u8 aq, u8 rl, u8 rs2, u8 rs1,
+ u8 funct3, u8 rd, u8 opcode)
+{
+ u8 funct7 = (funct5 << 2) | (aq << 1) | rl;
+
+ return rv_r_insn(funct7, rs2, rs1, funct3, rd, opcode);
+}
+
+static u32 rv_addiw(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 0, rd, 0x1b);
+}
+
+static u32 rv_addi(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 0, rd, 0x13);
+}
+
+static u32 rv_addw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 0, rd, 0x3b);
+}
+
+static u32 rv_add(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 0, rd, 0x33);
+}
+
+static u32 rv_subw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0x20, rs2, rs1, 0, rd, 0x3b);
+}
+
+static u32 rv_sub(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0x20, rs2, rs1, 0, rd, 0x33);
+}
+
+static u32 rv_and(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 7, rd, 0x33);
+}
+
+static u32 rv_or(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 6, rd, 0x33);
+}
+
+static u32 rv_xor(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 4, rd, 0x33);
+}
+
+static u32 rv_mulw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 0, rd, 0x3b);
+}
+
+static u32 rv_mul(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 0, rd, 0x33);
+}
+
+static u32 rv_divuw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 5, rd, 0x3b);
+}
+
+static u32 rv_divu(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 5, rd, 0x33);
+}
+
+static u32 rv_remuw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 7, rd, 0x3b);
+}
+
+static u32 rv_remu(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(1, rs2, rs1, 7, rd, 0x33);
+}
+
+static u32 rv_sllw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 1, rd, 0x3b);
+}
+
+static u32 rv_sll(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 1, rd, 0x33);
+}
+
+static u32 rv_srlw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 5, rd, 0x3b);
+}
+
+static u32 rv_srl(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0, rs2, rs1, 5, rd, 0x33);
+}
+
+static u32 rv_sraw(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0x20, rs2, rs1, 5, rd, 0x3b);
+}
+
+static u32 rv_sra(u8 rd, u8 rs1, u8 rs2)
+{
+ return rv_r_insn(0x20, rs2, rs1, 5, rd, 0x33);
+}
+
+static u32 rv_lui(u8 rd, u32 imm31_12)
+{
+ return rv_u_insn(imm31_12, rd, 0x37);
+}
+
+static u32 rv_slli(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 1, rd, 0x13);
+}
+
+static u32 rv_andi(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 7, rd, 0x13);
+}
+
+static u32 rv_ori(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 6, rd, 0x13);
+}
+
+static u32 rv_xori(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 4, rd, 0x13);
+}
+
+static u32 rv_slliw(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 1, rd, 0x1b);
+}
+
+static u32 rv_srliw(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 5, rd, 0x1b);
+}
+
+static u32 rv_srli(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 5, rd, 0x13);
+}
+
+static u32 rv_sraiw(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(0x400 | imm11_0, rs1, 5, rd, 0x1b);
+}
+
+static u32 rv_srai(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(0x400 | imm11_0, rs1, 5, rd, 0x13);
+}
+
+static u32 rv_jal(u8 rd, u32 imm20_1)
+{
+ return rv_uj_insn(imm20_1, rd, 0x6f);
+}
+
+static u32 rv_jalr(u8 rd, u8 rs1, u16 imm11_0)
+{
+ return rv_i_insn(imm11_0, rs1, 0, rd, 0x67);
+}
+
+static u32 rv_beq(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 0, 0x63);
+}
+
+static u32 rv_bltu(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 6, 0x63);
+}
+
+static u32 rv_bgeu(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 7, 0x63);
+}
+
+static u32 rv_bne(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 1, 0x63);
+}
+
+static u32 rv_blt(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 4, 0x63);
+}
+
+static u32 rv_bge(u8 rs1, u8 rs2, u16 imm12_1)
+{
+ return rv_sb_insn(imm12_1, rs2, rs1, 5, 0x63);
+}
+
+static u32 rv_sb(u8 rs1, u16 imm11_0, u8 rs2)
+{
+ return rv_s_insn(imm11_0, rs2, rs1, 0, 0x23);
+}
+
+static u32 rv_sh(u8 rs1, u16 imm11_0, u8 rs2)
+{
+ return rv_s_insn(imm11_0, rs2, rs1, 1, 0x23);
+}
+
+static u32 rv_sw(u8 rs1, u16 imm11_0, u8 rs2)
+{
+ return rv_s_insn(imm11_0, rs2, rs1, 2, 0x23);
+}
+
+static u32 rv_sd(u8 rs1, u16 imm11_0, u8 rs2)
+{
+ return rv_s_insn(imm11_0, rs2, rs1, 3, 0x23);
+}
+
+static u32 rv_lbu(u8 rd, u16 imm11_0, u8 rs1)
+{
+ return rv_i_insn(imm11_0, rs1, 4, rd, 0x03);
+}
+
+static u32 rv_lhu(u8 rd, u16 imm11_0, u8 rs1)
+{
+ return rv_i_insn(imm11_0, rs1, 5, rd, 0x03);
+}
+
+static u32 rv_lwu(u8 rd, u16 imm11_0, u8 rs1)
+{
+ return rv_i_insn(imm11_0, rs1, 6, rd, 0x03);
+}
+
+static u32 rv_ld(u8 rd, u16 imm11_0, u8 rs1)
+{
+ return rv_i_insn(imm11_0, rs1, 3, rd, 0x03);
+}
+
+static u32 rv_amoadd_w(u8 rd, u8 rs2, u8 rs1, u8 aq, u8 rl)
+{
+ return rv_amo_insn(0, aq, rl, rs2, rs1, 2, rd, 0x2f);
+}
+
+static u32 rv_amoadd_d(u8 rd, u8 rs2, u8 rs1, u8 aq, u8 rl)
+{
+ return rv_amo_insn(0, aq, rl, rs2, rs1, 3, rd, 0x2f);
+}
+
+static bool is_12b_int(s64 val)
+{
+ return -(1 << 11) <= val && val < (1 << 11);
+}
+
+static bool is_13b_int(s64 val)
+{
+ return -(1 << 12) <= val && val < (1 << 12);
+}
+
+static bool is_21b_int(s64 val)
+{
+ return -(1L << 20) <= val && val < (1L << 20);
+}
+
+static bool is_32b_int(s64 val)
+{
+ return -(1L << 31) <= val && val < (1L << 31);
+}
+
+static int is_12b_check(int off, int insn)
+{
+ if (!is_12b_int(off)) {
+ pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
+ insn, (int)off);
+ return -1;
+ }
+ return 0;
+}
+
+static int is_13b_check(int off, int insn)
+{
+ if (!is_13b_int(off)) {
+ pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
+ insn, (int)off);
+ return -1;
+ }
+ return 0;
+}
+
+static int is_21b_check(int off, int insn)
+{
+ if (!is_21b_int(off)) {
+ pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
+ insn, (int)off);
+ return -1;
+ }
+ return 0;
+}
+
+static void emit_imm(u8 rd, s64 val, struct rv_jit_context *ctx)
+{
+ /* Note that the immediate from the add is sign-extended,
+ * which means that we need to compensate this by adding 2^12,
+ * when the 12th bit is set. A simpler way of doing this, and
+ * getting rid of the check, is to just add 2**11 before the
+ * shift. The "Loading a 32-Bit constant" example from the
+ * "Computer Organization and Design, RISC-V edition" book by
+ * Patterson/Hennessy highlights this fact.
+ *
+ * This also means that we need to process LSB to MSB.
+ */
+ s64 upper = (val + (1 << 11)) >> 12, lower = val & 0xfff;
+ int shift;
+
+ if (is_32b_int(val)) {
+ if (upper)
+ emit(rv_lui(rd, upper), ctx);
+
+ if (!upper) {
+ emit(rv_addi(rd, RV_REG_ZERO, lower), ctx);
+ return;
+ }
+
+ emit(rv_addiw(rd, rd, lower), ctx);
+ return;
+ }
+
+ shift = __ffs(upper);
+ upper >>= shift;
+ shift += 12;
+
+ emit_imm(rd, upper, ctx);
+
+ emit(rv_slli(rd, rd, shift), ctx);
+ if (lower)
+ emit(rv_addi(rd, rd, lower), ctx);
+}
+
+static int rv_offset(int bpf_to, int bpf_from, struct rv_jit_context *ctx)
+{
+ int from = ctx->offset[bpf_from] - 1, to = ctx->offset[bpf_to];
+
+ return (to - from) << 2;
+}
+
+static int epilogue_offset(struct rv_jit_context *ctx)
+{
+ int to = ctx->epilogue_offset, from = ctx->ninsns;
+
+ return (to - from) << 2;
+}
+
+static void __build_epilogue(u8 reg, struct rv_jit_context *ctx)
+{
+ int stack_adjust = ctx->stack_size, store_offset = stack_adjust - 8;
+
+ if (seen_reg(RV_REG_RA, ctx)) {
+ emit(rv_ld(RV_REG_RA, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ emit(rv_ld(RV_REG_FP, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ if (seen_reg(RV_REG_S1, ctx)) {
+ emit(rv_ld(RV_REG_S1, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S2, ctx)) {
+ emit(rv_ld(RV_REG_S2, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S3, ctx)) {
+ emit(rv_ld(RV_REG_S3, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S4, ctx)) {
+ emit(rv_ld(RV_REG_S4, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S5, ctx)) {
+ emit(rv_ld(RV_REG_S5, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S6, ctx)) {
+ emit(rv_ld(RV_REG_S6, store_offset, RV_REG_SP), ctx);
+ store_offset -= 8;
+ }
+
+ emit(rv_addi(RV_REG_SP, RV_REG_SP, stack_adjust), ctx);
+ /* Set return value. */
+ emit(rv_addi(RV_REG_A0, RV_REG_A5, 0), ctx);
+ emit(rv_jalr(RV_REG_ZERO, reg, 0), ctx);
+}
+
+static void emit_zero_extend_32(u8 reg, struct rv_jit_context *ctx)
+{
+ emit(rv_slli(reg, reg, 32), ctx);
+ emit(rv_srli(reg, reg, 32), ctx);
+}
+
+static int emit_bpf_tail_call(int insn, struct rv_jit_context *ctx)
+{
+ int tc_ninsn, off, start_insn = ctx->ninsns;
+ u8 tcc = rv_tail_call_reg(ctx);
+
+ /* a0: &ctx
+ * a1: &array
+ * a2: index
+ *
+ * if (index >= array->map.max_entries)
+ * goto out;
+ */
+ tc_ninsn = insn ? ctx->offset[insn] - ctx->offset[insn - 1] :
+ ctx->offset[0];
+ emit_zero_extend_32(RV_REG_A2, ctx);
+
+ off = offsetof(struct bpf_array, map.max_entries);
+ if (is_12b_check(off, insn))
+ return -1;
+ emit(rv_lwu(RV_REG_T1, off, RV_REG_A1), ctx);
+ off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
+ if (is_13b_check(off, insn))
+ return -1;
+ emit(rv_bgeu(RV_REG_A2, RV_REG_T1, off >> 1), ctx);
+
+ /* if (--TCC < 0)
+ * goto out;
+ */
+ emit(rv_addi(RV_REG_T1, tcc, -1), ctx);
+ off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
+ if (is_13b_check(off, insn))
+ return -1;
+ emit(rv_blt(RV_REG_T1, RV_REG_ZERO, off >> 1), ctx);
+
+ /* prog = array->ptrs[index];
+ * if (!prog)
+ * goto out;
+ */
+ emit(rv_slli(RV_REG_T2, RV_REG_A2, 3), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_A1), ctx);
+ off = offsetof(struct bpf_array, ptrs);
+ if (is_12b_check(off, insn))
+ return -1;
+ emit(rv_ld(RV_REG_T2, off, RV_REG_T2), ctx);
+ off = (tc_ninsn - (ctx->ninsns - start_insn)) << 2;
+ if (is_13b_check(off, insn))
+ return -1;
+ emit(rv_beq(RV_REG_T2, RV_REG_ZERO, off >> 1), ctx);
+
+ /* goto *(prog->bpf_func + 4); */
+ off = offsetof(struct bpf_prog, bpf_func);
+ if (is_12b_check(off, insn))
+ return -1;
+ emit(rv_ld(RV_REG_T3, off, RV_REG_T2), ctx);
+ emit(rv_addi(RV_REG_T3, RV_REG_T3, 4), ctx);
+ emit(rv_addi(RV_REG_TCC, RV_REG_T1, 0), ctx);
+ __build_epilogue(RV_REG_T3, ctx);
+ return 0;
+}
+
+static void init_regs(u8 *rd, u8 *rs, const struct bpf_insn *insn,
+ struct rv_jit_context *ctx)
+{
+ u8 code = insn->code;
+
+ switch (code) {
+ case BPF_JMP | BPF_JA:
+ case BPF_JMP | BPF_CALL:
+ case BPF_JMP | BPF_EXIT:
+ case BPF_JMP | BPF_TAIL_CALL:
+ break;
+ default:
+ *rd = bpf_to_rv_reg(insn->dst_reg, ctx);
+ }
+
+ if (code & (BPF_ALU | BPF_X) || code & (BPF_ALU64 | BPF_X) ||
+ code & (BPF_JMP | BPF_X) || code & BPF_LDX || code & BPF_STX)
+ *rs = bpf_to_rv_reg(insn->src_reg, ctx);
+}
+
+static int rv_offset_check(int *rvoff, s16 off, int insn,
+ struct rv_jit_context *ctx)
+{
+ *rvoff = rv_offset(insn + off, insn, ctx);
+ return is_13b_check(*rvoff, insn);
+}
+
+static int emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
+ bool extra_pass)
+{
+ bool is64 = BPF_CLASS(insn->code) == BPF_ALU64;
+ int rvoff, i = insn - ctx->prog->insnsi;
+ u8 rd = -1, rs = -1, code = insn->code;
+ s16 off = insn->off;
+ s32 imm = insn->imm;
+
+ init_regs(&rd, &rs, insn, ctx);
+
+ switch (code) {
+ /* dst = src */
+ case BPF_ALU | BPF_MOV | BPF_X:
+ case BPF_ALU64 | BPF_MOV | BPF_X:
+ emit(is64 ? rv_addi(rd, rs, 0) : rv_addiw(rd, rs, 0), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+
+ /* dst = dst OP src */
+ case BPF_ALU | BPF_ADD | BPF_X:
+ case BPF_ALU64 | BPF_ADD | BPF_X:
+ emit(is64 ? rv_add(rd, rd, rs) : rv_addw(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_SUB | BPF_X:
+ case BPF_ALU64 | BPF_SUB | BPF_X:
+ emit(is64 ? rv_sub(rd, rd, rs) : rv_subw(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_AND | BPF_X:
+ case BPF_ALU64 | BPF_AND | BPF_X:
+ emit(rv_and(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_OR | BPF_X:
+ case BPF_ALU64 | BPF_OR | BPF_X:
+ emit(rv_or(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_XOR | BPF_X:
+ case BPF_ALU64 | BPF_XOR | BPF_X:
+ emit(rv_xor(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_MUL | BPF_X:
+ case BPF_ALU64 | BPF_MUL | BPF_X:
+ emit(is64 ? rv_mul(rd, rd, rs) : rv_mulw(rd, rd, rs), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_DIV | BPF_X:
+ case BPF_ALU64 | BPF_DIV | BPF_X:
+ emit(is64 ? rv_divu(rd, rd, rs) : rv_divuw(rd, rd, rs), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_MOD | BPF_X:
+ case BPF_ALU64 | BPF_MOD | BPF_X:
+ emit(is64 ? rv_remu(rd, rd, rs) : rv_remuw(rd, rd, rs), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_LSH | BPF_X:
+ case BPF_ALU64 | BPF_LSH | BPF_X:
+ emit(is64 ? rv_sll(rd, rd, rs) : rv_sllw(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_RSH | BPF_X:
+ case BPF_ALU64 | BPF_RSH | BPF_X:
+ emit(is64 ? rv_srl(rd, rd, rs) : rv_srlw(rd, rd, rs), ctx);
+ break;
+ case BPF_ALU | BPF_ARSH | BPF_X:
+ case BPF_ALU64 | BPF_ARSH | BPF_X:
+ emit(is64 ? rv_sra(rd, rd, rs) : rv_sraw(rd, rd, rs), ctx);
+ break;
+
+ /* dst = -dst */
+ case BPF_ALU | BPF_NEG:
+ case BPF_ALU64 | BPF_NEG:
+ emit(is64 ? rv_sub(rd, RV_REG_ZERO, rd) :
+ rv_subw(rd, RV_REG_ZERO, rd), ctx);
+ break;
+
+ /* dst = BSWAP##imm(dst) */
+ case BPF_ALU | BPF_END | BPF_FROM_LE:
+ {
+ int shift = 64 - imm;
+
+ emit(rv_slli(rd, rd, shift), ctx);
+ emit(rv_srli(rd, rd, shift), ctx);
+ break;
+ }
+ case BPF_ALU | BPF_END | BPF_FROM_BE:
+ emit(rv_addi(RV_REG_T2, RV_REG_ZERO, 0), ctx);
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+ if (imm == 16)
+ goto out_be;
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+ if (imm == 32)
+ goto out_be;
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+ emit(rv_slli(RV_REG_T2, RV_REG_T2, 8), ctx);
+ emit(rv_srli(rd, rd, 8), ctx);
+out_be:
+ emit(rv_andi(RV_REG_T1, rd, 0xff), ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, RV_REG_T1), ctx);
+
+ emit(rv_addi(rd, RV_REG_T2, 0), ctx);
+ break;
+
+ /* dst = imm */
+ case BPF_ALU | BPF_MOV | BPF_K:
+ case BPF_ALU64 | BPF_MOV | BPF_K:
+ emit_imm(rd, imm, ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+
+ /* dst = dst OP imm */
+ case BPF_ALU | BPF_ADD | BPF_K:
+ case BPF_ALU64 | BPF_ADD | BPF_K:
+ if (is_12b_int(imm)) {
+ emit(is64 ? rv_addi(rd, rd, imm) :
+ rv_addiw(rd, rd, imm), ctx);
+ } else {
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(is64 ? rv_add(rd, rd, RV_REG_T1) :
+ rv_addw(rd, rd, RV_REG_T1), ctx);
+ }
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_SUB | BPF_K:
+ case BPF_ALU64 | BPF_SUB | BPF_K:
+ if (is_12b_int(-imm)) {
+ emit(is64 ? rv_addi(rd, rd, -imm) :
+ rv_addiw(rd, rd, -imm), ctx);
+ } else {
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(is64 ? rv_sub(rd, rd, RV_REG_T1) :
+ rv_subw(rd, rd, RV_REG_T1), ctx);
+ }
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_AND | BPF_K:
+ case BPF_ALU64 | BPF_AND | BPF_K:
+ if (is_12b_int(imm)) {
+ emit(rv_andi(rd, rd, imm), ctx);
+ } else {
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_and(rd, rd, RV_REG_T1), ctx);
+ }
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_OR | BPF_K:
+ case BPF_ALU64 | BPF_OR | BPF_K:
+ if (is_12b_int(imm)) {
+ emit(rv_ori(rd, rd, imm), ctx);
+ } else {
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_or(rd, rd, RV_REG_T1), ctx);
+ }
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_XOR | BPF_K:
+ case BPF_ALU64 | BPF_XOR | BPF_K:
+ if (is_12b_int(imm)) {
+ emit(rv_xori(rd, rd, imm), ctx);
+ } else {
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_xor(rd, rd, RV_REG_T1), ctx);
+ }
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_MUL | BPF_K:
+ case BPF_ALU64 | BPF_MUL | BPF_K:
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(is64 ? rv_mul(rd, rd, RV_REG_T1) :
+ rv_mulw(rd, rd, RV_REG_T1), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_DIV | BPF_K:
+ case BPF_ALU64 | BPF_DIV | BPF_K:
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(is64 ? rv_divu(rd, rd, RV_REG_T1) :
+ rv_divuw(rd, rd, RV_REG_T1), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_MOD | BPF_K:
+ case BPF_ALU64 | BPF_MOD | BPF_K:
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(is64 ? rv_remu(rd, rd, RV_REG_T1) :
+ rv_remuw(rd, rd, RV_REG_T1), ctx);
+ if (!is64)
+ emit_zero_extend_32(rd, ctx);
+ break;
+ case BPF_ALU | BPF_LSH | BPF_K:
+ case BPF_ALU64 | BPF_LSH | BPF_K:
+ emit(is64 ? rv_slli(rd, rd, imm) : rv_slliw(rd, rd, imm), ctx);
+ break;
+ case BPF_ALU | BPF_RSH | BPF_K:
+ case BPF_ALU64 | BPF_RSH | BPF_K:
+ emit(is64 ? rv_srli(rd, rd, imm) : rv_srliw(rd, rd, imm), ctx);
+ break;
+ case BPF_ALU | BPF_ARSH | BPF_K:
+ case BPF_ALU64 | BPF_ARSH | BPF_K:
+ emit(is64 ? rv_srai(rd, rd, imm) : rv_sraiw(rd, rd, imm), ctx);
+ break;
+
+ /* JUMP off */
+ case BPF_JMP | BPF_JA:
+ rvoff = rv_offset(i + off, i, ctx);
+ if (!is_21b_int(rvoff)) {
+ pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
+ i, rvoff);
+ return -1;
+ }
+
+ emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx);
+ break;
+
+ /* IF (dst COND src) JUMP off */
+ case BPF_JMP | BPF_JEQ | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_beq(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JGT | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bltu(rs, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JLT | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bltu(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JGE | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bgeu(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JLE | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bgeu(rs, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JNE | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bne(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSGT | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_blt(rs, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSLT | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_blt(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSGE | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bge(rd, rs, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSLE | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_bge(rs, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSET | BPF_X:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit(rv_and(RV_REG_T1, rd, rs), ctx);
+ emit(rv_bne(RV_REG_T1, RV_REG_ZERO, rvoff >> 1), ctx);
+ break;
+
+ /* IF (dst COND imm) JUMP off */
+ case BPF_JMP | BPF_JEQ | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_beq(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JGT | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bltu(RV_REG_T1, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JLT | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bltu(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JGE | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bgeu(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JLE | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bgeu(RV_REG_T1, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JNE | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bne(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSGT | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_blt(RV_REG_T1, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSLT | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_blt(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSGE | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bge(rd, RV_REG_T1, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSLE | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T1, imm, ctx);
+ emit(rv_bge(RV_REG_T1, rd, rvoff >> 1), ctx);
+ break;
+ case BPF_JMP | BPF_JSET | BPF_K:
+ if (rv_offset_check(&rvoff, off, i, ctx))
+ return -1;
+ emit_imm(RV_REG_T2, imm, ctx);
+ emit(rv_and(RV_REG_T1, rd, RV_REG_T2), ctx);
+ emit(rv_bne(RV_REG_T1, RV_REG_ZERO, rvoff >> 1), ctx);
+ break;
+
+ /* function call */
+ case BPF_JMP | BPF_CALL:
+ {
+ bool fixed;
+ int i, ret;
+ u64 addr;
+
+ mark_call(ctx);
+ ret = bpf_jit_get_func_addr(ctx->prog, insn, extra_pass, &addr,
+ &fixed);
+ if (ret < 0)
+ return ret;
+ if (fixed) {
+ emit_imm(RV_REG_T1, addr, ctx);
+ } else {
+ i = ctx->ninsns;
+ emit_imm(RV_REG_T1, addr, ctx);
+ for (i = ctx->ninsns - i; i < 8; i++) {
+ /* nop */
+ emit(rv_addi(RV_REG_ZERO, RV_REG_ZERO, 0),
+ ctx);
+ }
+ }
+ emit(rv_jalr(RV_REG_RA, RV_REG_T1, 0), ctx);
+ rd = bpf_to_rv_reg(BPF_REG_0, ctx);
+ emit(rv_addi(rd, RV_REG_A0, 0), ctx);
+ break;
+ }
+ /* tail call */
+ case BPF_JMP | BPF_TAIL_CALL:
+ if (emit_bpf_tail_call(i, ctx))
+ return -1;
+ break;
+
+ /* function return */
+ case BPF_JMP | BPF_EXIT:
+ if (i == ctx->prog->len - 1)
+ break;
+
+ rvoff = epilogue_offset(ctx);
+ if (is_21b_check(rvoff, i))
+ return -1;
+ emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx);
+ break;
+
+ /* dst = imm64 */
+ case BPF_LD | BPF_IMM | BPF_DW:
+ {
+ struct bpf_insn insn1 = insn[1];
+ u64 imm64;
+
+ imm64 = (u64)insn1.imm << 32 | (u32)imm;
+ emit_imm(rd, imm64, ctx);
+ return 1;
+ }
+
+ /* LDX: dst = *(size *)(src + off) */
+ case BPF_LDX | BPF_MEM | BPF_B:
+ if (is_12b_int(off)) {
+ emit(rv_lbu(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rs), ctx);
+ emit(rv_lbu(rd, 0, RV_REG_T1), ctx);
+ break;
+ case BPF_LDX | BPF_MEM | BPF_H:
+ if (is_12b_int(off)) {
+ emit(rv_lhu(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rs), ctx);
+ emit(rv_lhu(rd, 0, RV_REG_T1), ctx);
+ break;
+ case BPF_LDX | BPF_MEM | BPF_W:
+ if (is_12b_int(off)) {
+ emit(rv_lwu(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rs), ctx);
+ emit(rv_lwu(rd, 0, RV_REG_T1), ctx);
+ break;
+ case BPF_LDX | BPF_MEM | BPF_DW:
+ if (is_12b_int(off)) {
+ emit(rv_ld(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rs), ctx);
+ emit(rv_ld(rd, 0, RV_REG_T1), ctx);
+ break;
+
+ /* ST: *(size *)(dst + off) = imm */
+ case BPF_ST | BPF_MEM | BPF_B:
+ emit_imm(RV_REG_T1, imm, ctx);
+ if (is_12b_int(off)) {
+ emit(rv_sb(rd, off, RV_REG_T1), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T2, off, ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, rd), ctx);
+ emit(rv_sb(RV_REG_T2, 0, RV_REG_T1), ctx);
+ break;
+
+ case BPF_ST | BPF_MEM | BPF_H:
+ emit_imm(RV_REG_T1, imm, ctx);
+ if (is_12b_int(off)) {
+ emit(rv_sh(rd, off, RV_REG_T1), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T2, off, ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, rd), ctx);
+ emit(rv_sh(RV_REG_T2, 0, RV_REG_T1), ctx);
+ break;
+ case BPF_ST | BPF_MEM | BPF_W:
+ emit_imm(RV_REG_T1, imm, ctx);
+ if (is_12b_int(off)) {
+ emit(rv_sw(rd, off, RV_REG_T1), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T2, off, ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, rd), ctx);
+ emit(rv_sw(RV_REG_T2, 0, RV_REG_T1), ctx);
+ break;
+ case BPF_ST | BPF_MEM | BPF_DW:
+ emit_imm(RV_REG_T1, imm, ctx);
+ if (is_12b_int(off)) {
+ emit(rv_sd(rd, off, RV_REG_T1), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T2, off, ctx);
+ emit(rv_add(RV_REG_T2, RV_REG_T2, rd), ctx);
+ emit(rv_sd(RV_REG_T2, 0, RV_REG_T1), ctx);
+ break;
+
+ /* STX: *(size *)(dst + off) = src */
+ case BPF_STX | BPF_MEM | BPF_B:
+ if (is_12b_int(off)) {
+ emit(rv_sb(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rd), ctx);
+ emit(rv_sb(RV_REG_T1, 0, rs), ctx);
+ break;
+ case BPF_STX | BPF_MEM | BPF_H:
+ if (is_12b_int(off)) {
+ emit(rv_sh(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rd), ctx);
+ emit(rv_sh(RV_REG_T1, 0, rs), ctx);
+ break;
+ case BPF_STX | BPF_MEM | BPF_W:
+ if (is_12b_int(off)) {
+ emit(rv_sw(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rd), ctx);
+ emit(rv_sw(RV_REG_T1, 0, rs), ctx);
+ break;
+ case BPF_STX | BPF_MEM | BPF_DW:
+ if (is_12b_int(off)) {
+ emit(rv_sd(rd, off, rs), ctx);
+ break;
+ }
+
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rd), ctx);
+ emit(rv_sd(RV_REG_T1, 0, rs), ctx);
+ break;
+ /* STX XADD: lock *(u32 *)(dst + off) += src */
+ case BPF_STX | BPF_XADD | BPF_W:
+ /* STX XADD: lock *(u64 *)(dst + off) += src */
+ case BPF_STX | BPF_XADD | BPF_DW:
+ if (off) {
+ if (is_12b_int(off)) {
+ emit(rv_addi(RV_REG_T1, rd, off), ctx);
+ } else {
+ emit_imm(RV_REG_T1, off, ctx);
+ emit(rv_add(RV_REG_T1, RV_REG_T1, rd), ctx);
+ }
+
+ rd = RV_REG_T1;
+ }
+
+ emit(BPF_SIZE(code) == BPF_W ?
+ rv_amoadd_w(RV_REG_ZERO, rs, rd, 0, 0) :
+ rv_amoadd_d(RV_REG_ZERO, rs, rd, 0, 0), ctx);
+ break;
+ default:
+ pr_err("bpf-jit: unknown opcode %02x\n", code);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void build_prologue(struct rv_jit_context *ctx)
+{
+ int stack_adjust = 0, store_offset, bpf_stack_adjust;
+
+ if (seen_reg(RV_REG_RA, ctx))
+ stack_adjust += 8;
+ stack_adjust += 8; /* RV_REG_FP */
+ if (seen_reg(RV_REG_S1, ctx))
+ stack_adjust += 8;
+ if (seen_reg(RV_REG_S2, ctx))
+ stack_adjust += 8;
+ if (seen_reg(RV_REG_S3, ctx))
+ stack_adjust += 8;
+ if (seen_reg(RV_REG_S4, ctx))
+ stack_adjust += 8;
+ if (seen_reg(RV_REG_S5, ctx))
+ stack_adjust += 8;
+ if (seen_reg(RV_REG_S6, ctx))
+ stack_adjust += 8;
+
+ stack_adjust = round_up(stack_adjust, 16);
+ bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, 16);
+ stack_adjust += bpf_stack_adjust;
+
+ store_offset = stack_adjust - 8;
+
+ /* First instruction is always setting the tail-call-counter
+ * (TCC) register. This instruction is skipped for tail calls.
+ */
+ emit(rv_addi(RV_REG_TCC, RV_REG_ZERO, MAX_TAIL_CALL_CNT), ctx);
+
+ emit(rv_addi(RV_REG_SP, RV_REG_SP, -stack_adjust), ctx);
+
+ if (seen_reg(RV_REG_RA, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_RA), ctx);
+ store_offset -= 8;
+ }
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_FP), ctx);
+ store_offset -= 8;
+ if (seen_reg(RV_REG_S1, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S1), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S2, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S2), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S3, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S3), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S4, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S4), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S5, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S5), ctx);
+ store_offset -= 8;
+ }
+ if (seen_reg(RV_REG_S6, ctx)) {
+ emit(rv_sd(RV_REG_SP, store_offset, RV_REG_S6), ctx);
+ store_offset -= 8;
+ }
+
+ emit(rv_addi(RV_REG_FP, RV_REG_SP, stack_adjust), ctx);
+
+ if (bpf_stack_adjust)
+ emit(rv_addi(RV_REG_S5, RV_REG_SP, bpf_stack_adjust), ctx);
+
+ /* Program contains calls and tail calls, so RV_REG_TCC need
+ * to be saved across calls.
+ */
+ if (seen_tail_call(ctx) && seen_call(ctx))
+ emit(rv_addi(RV_REG_TCC_SAVED, RV_REG_TCC, 0), ctx);
+
+ ctx->stack_size = stack_adjust;
+}
+
+static void build_epilogue(struct rv_jit_context *ctx)
+{
+ __build_epilogue(RV_REG_RA, ctx);
+}
+
+static int build_body(struct rv_jit_context *ctx, bool extra_pass)
+{
+ const struct bpf_prog *prog = ctx->prog;
+ int i;
+
+ for (i = 0; i < prog->len; i++) {
+ const struct bpf_insn *insn = &prog->insnsi[i];
+ int ret;
+
+ ret = emit_insn(insn, ctx, extra_pass);
+ if (ret > 0) {
+ i++;
+ if (ctx->insns == NULL)
+ ctx->offset[i] = ctx->ninsns;
+ continue;
+ }
+ if (ctx->insns == NULL)
+ ctx->offset[i] = ctx->ninsns;
+ if (ret)
+ return ret;
+ }
+ return 0;
+}
+
+static void bpf_fill_ill_insns(void *area, unsigned int size)
+{
+ memset(area, 0, size);
+}
+
+static void bpf_flush_icache(void *start, void *end)
+{
+ flush_icache_range((unsigned long)start, (unsigned long)end);
+}
+
+struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
+{
+ bool tmp_blinded = false, extra_pass = false;
+ struct bpf_prog *tmp, *orig_prog = prog;
+ struct rv_jit_data *jit_data;
+ struct rv_jit_context *ctx;
+ unsigned int image_size;
+
+ if (!prog->jit_requested)
+ return orig_prog;
+
+ tmp = bpf_jit_blind_constants(prog);
+ if (IS_ERR(tmp))
+ return orig_prog;
+ if (tmp != prog) {
+ tmp_blinded = true;
+ prog = tmp;
+ }
+
+ jit_data = prog->aux->jit_data;
+ if (!jit_data) {
+ jit_data = kzalloc(sizeof(*jit_data), GFP_KERNEL);
+ if (!jit_data) {
+ prog = orig_prog;
+ goto out;
+ }
+ prog->aux->jit_data = jit_data;
+ }
+
+ ctx = &jit_data->ctx;
+
+ if (ctx->offset) {
+ extra_pass = true;
+ image_size = sizeof(u32) * ctx->ninsns;
+ goto skip_init_ctx;
+ }
+
+ ctx->prog = prog;
+ ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
+ if (!ctx->offset) {
+ prog = orig_prog;
+ goto out_offset;
+ }
+
+ /* First pass generates the ctx->offset, but does not emit an image. */
+ if (build_body(ctx, extra_pass)) {
+ prog = orig_prog;
+ goto out_offset;
+ }
+ build_prologue(ctx);
+ ctx->epilogue_offset = ctx->ninsns;
+ build_epilogue(ctx);
+
+ /* Allocate image, now that we know the size. */
+ image_size = sizeof(u32) * ctx->ninsns;
+ jit_data->header = bpf_jit_binary_alloc(image_size, &jit_data->image,
+ sizeof(u32),
+ bpf_fill_ill_insns);
+ if (!jit_data->header) {
+ prog = orig_prog;
+ goto out_offset;
+ }
+
+ /* Second, real pass, that acutally emits the image. */
+ ctx->insns = (u32 *)jit_data->image;
+skip_init_ctx:
+ ctx->ninsns = 0;
+
+ build_prologue(ctx);
+ if (build_body(ctx, extra_pass)) {
+ bpf_jit_binary_free(jit_data->header);
+ prog = orig_prog;
+ goto out_offset;
+ }
+ build_epilogue(ctx);
+
+ if (bpf_jit_enable > 1)
+ bpf_jit_dump(prog->len, image_size, 2, ctx->insns);
+
+ prog->bpf_func = (void *)ctx->insns;
+ prog->jited = 1;
+ prog->jited_len = image_size;
+
+ bpf_flush_icache(jit_data->header, (u8 *)ctx->insns + ctx->ninsns);
+
+ if (!prog->is_func || extra_pass) {
+out_offset:
+ kfree(ctx->offset);
+ kfree(jit_data);
+ prog->aux->jit_data = NULL;
+ }
+out:
+ if (tmp_blinded)
+ bpf_jit_prog_release_other(prog, prog == orig_prog ?
+ tmp : orig_prog);
+ return prog;
+}
--
2.19.1
^ permalink raw reply related
* [PATCH bpf-next 0/3] Add RISC-V (RV64G) BPF JIT
From: bjorn.topel @ 2019-02-03 11:51 UTC (permalink / raw)
To: linux-riscv, daniel, ast, netdev; +Cc: Björn Töpel, palmer, hch
From: Björn Töpel <bjorn.topel@gmail.com>
Hi!
This series adds an RV64G BPF JIT to the kernel. I've sent out a RFC
for a couple of weeks ago, and think this code is
feature-complete/stable enough (famous last words) for a proper patch.
Unfortunately, kprobes isn't supported by RISC-V yet (Patrick Stählin
sent out an RFC last year), which means that all tests involving
BPF_PROG_TYPE_TRACEPOINT, BPF_PROG_TYPE_PERF_EVENT,
BPF_PROG_TYPE_KPROBE and BPF_PROG_TYPE_RAW_TRACEPOINT will fail.
Further, the implementation does not support "far branching" (>4KiB).
The implementation passes all the test_bpf.ko tests: test_bpf:
Summary: 378 PASSED, 0 FAILED, [366/366 JIT'ed]
All the tail_call tests in the selftest/bpf/test_verifier program
passes.
All tests where done on QEMU emulator version 3.1.50
(v3.1.0-688-g8ae951fbc106). I'll test it on real hardware, when I get
access to it.
I'm routing this patch via bpf-next/netdev mailing list (after a
conversation with Palmer at FOSDEM), mainly because the other JITs
went that path.
Thanks for all the comments!
Cheers,
Björn
RFCv1 -> v1:
* Cleaned up the Kconfig and net/Makefile. (Christoph)
* Removed the entry-stub and squashed the build/config changes to be
part of the JIT implementation. (Christoph)
* Simplified the register tracking code. (Daniel)
* Removed unused macros. (Daniel)
* Added myself as maintainer and updated documentation. (Daniel)
* Removed HAVE_EFFICIENT_UNALIGNED_ACCESS. (Christoph, Palmer)
* Added tail-calls and cleaned up the code.
Björn Töpel (3):
bpf, riscv: add BPF JIT for RV64G
MAINTAINERS: add RISC-V BPF JIT maintainer
bpf, doc: add RISC-V to filter.txt
Documentation/networking/filter.txt | 16 +-
MAINTAINERS | 6 +
arch/riscv/Kconfig | 1 +
arch/riscv/Makefile | 2 +-
arch/riscv/net/Makefile | 1 +
arch/riscv/net/bpf_jit_comp.c | 1502 +++++++++++++++++++++++++++
6 files changed, 1520 insertions(+), 8 deletions(-)
create mode 100644 arch/riscv/net/Makefile
create mode 100644 arch/riscv/net/bpf_jit_comp.c
--
2.19.1
^ permalink raw reply
* Re: [PATCH iproute2-next] devlink: add info subcommand
From: Jiri Pirko @ 2019-02-03 11:04 UTC (permalink / raw)
To: Jakub Kicinski; +Cc: dsahern, stephen, netdev, oss-drivers
In-Reply-To: <20190202140608.27691bbc@cakuba.hsd1.ca.comcast.net>
Sat, Feb 02, 2019 at 11:06:08PM CET, jakub.kicinski@netronome.com wrote:
>On Sat, 2 Feb 2019 16:59:38 +0100, Jiri Pirko wrote:
>> Sat, Feb 02, 2019 at 01:03:38AM CET, jakub.kicinski@netronome.com wrote:
>> >Add support for reading the device serial number and versions
>> >from the kernel.
>> >
>> >RFCv2:
>> > - make info subcommand of dev.
>>
>> Please add some examples of inputs and outputs.
>
>Sorry, yes, will do.
>
>> >Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
>> >---
>> > devlink/devlink.c | 207 +++++++++++++++++++++++++++++++++++++++++
>> > man/man8/devlink-dev.8 | 36 +++++++
>> > 2 files changed, 243 insertions(+)
>> >
>> >diff --git a/devlink/devlink.c b/devlink/devlink.c
>> >index 3651e90c1159..3ab046ace8f8 100644
>> >--- a/devlink/devlink.c
>> >+++ b/devlink/devlink.c
>> >@@ -199,6 +199,7 @@ static void ifname_map_free(struct ifname_map *ifname_map)
>> > #define DL_OPT_REGION_SNAPSHOT_ID BIT(22)
>> > #define DL_OPT_REGION_ADDRESS BIT(23)
>> > #define DL_OPT_REGION_LENGTH BIT(24)
>> >+#define DL_OPT_VERSIONS_TYPE BIT(25)
>>
>> Why "type"? Confusing.
>
>Right, I think this dates back to day 1 when the whole thing was called
>versions not info. How about DL_OPT_INFO_VERSION_TYPE?
>
>> >
>> > struct dl_opts {
>> > uint32_t present; /* flags of present items */
>
>> >@@ -943,6 +952,21 @@ static int param_cmode_get(const char *cmodestr,
>> > return 0;
>> > }
>> >
>> >+static int versions_type_get(const char *typestr, int *p_attr)
>> >+{
>> >+ if (strcmp(typestr, "fixed") == 0) {
>> >+ *p_attr = DEVLINK_ATTR_INFO_VERSION_FIXED;
>> >+ } else if (strcmp(typestr, "stored") == 0) {
>> >+ *p_attr = DEVLINK_ATTR_INFO_VERSION_STORED;
>> >+ } else if (strcmp(typestr, "running") == 0) {
>> >+ *p_attr = DEVLINK_ATTR_INFO_VERSION_RUNNING;
>> >+ } else {
>> >+ pr_err("Unknown versions type \"%s\"\n", typestr);
>> >+ return -EINVAL;
>> >+ }
>> >+ return 0;
>> >+}
>> >+
>> > static int dl_argv_parse(struct dl *dl, uint32_t o_required,
>> > uint32_t o_optional)
>> > {
>> >@@ -1178,6 +1202,19 @@ static int dl_argv_parse(struct dl *dl, uint32_t o_required,
>> > if (err)
>> > return err;
>> > o_found |= DL_OPT_REGION_LENGTH;
>> >+ } else if (dl_argv_match(dl, "versions") &&
>> >+ (o_all & DL_OPT_VERSIONS_TYPE)) {
>> >+ const char *versionstr;
>> >+
>> >+ dl_arg_inc(dl);
>> >+ err = dl_argv_str(dl, &versionstr);
>> >+ if (err)
>> >+ return err;
>> >+ err = versions_type_get(versionstr,
>> >+ &opts->versions_attr);
>> >+ if (err)
>> >+ return err;
>> >+ o_found |= DL_OPT_VERSIONS_TYPE;
>> > } else {
>> > pr_err("Unknown option \"%s\"\n", dl_argv(dl));
>> > return -EINVAL;
>> >@@ -1443,6 +1480,9 @@ static void cmd_dev_help(void)
>> > pr_err(" devlink dev param set DEV name PARAMETER value VALUE cmode { permanent | driverinit | runtime }\n");
>> > pr_err(" devlink dev param show [DEV name PARAMETER]\n");
>> > pr_err(" devlink dev reload DEV\n");
>> >+ pr_err(" devlink dev info [ DEV [ { versions [ VTYPE ] } ] ]\n");
>> >+ pr_err("\n");
>> >+ pr_err(" VTYPE := { fixed | running | stored }\n");
>>
>> So you would like to filter according to the version type? Why?
>
>if devlink dev info bus/train versions stored != devlink ... running
>then reboot is needed. That one of main uses for reporting running vs
>stored in sections, it's nice to be able to just compare the outputs.
I don't know. In the "info", there are more things than versions. Also,
I believe in near future there are going to be even more things there.
Providing a filtering option for one item and not for other seems
incorrect. I think that devlink should not filter in this case. Let it
dump the whole output and let the user take care of it by other tools.
Each tool should do one thing.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox