* WARNING: ODEBUG bug in netdev_freemem (2)
From: syzbot @ 2019-06-24 8:53 UTC (permalink / raw)
To: alexander.h.duyck, amritha.nambiar, andriy.shevchenko, davem,
dmitry.torokhov, f.fainelli, gregkh, idosch, linux-kernel, netdev,
syzkaller-bugs, tglx, tyhicks, wanghai26, yuehaibing
Hello,
syzbot found the following crash on:
HEAD commit: fd6b99fa Merge branch 'akpm' (patches from Andrew)
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=144de256a00000
kernel config: https://syzkaller.appspot.com/x/.config?x=fa9f7e1b6a8bb586
dashboard link: https://syzkaller.appspot.com/bug?extid=c4521ac872a4ccc3afec
compiler: gcc (GCC) 9.0.0 20181231 (experimental)
Unfortunately, I don't have any reproducer for this crash yet.
IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+c4521ac872a4ccc3afec@syzkaller.appspotmail.com
device hsr_slave_0 left promiscuous mode
team0 (unregistering): Port device team_slave_1 removed
team0 (unregistering): Port device team_slave_0 removed
bond0 (unregistering): Releasing backup interface bond_slave_1
bond0 (unregistering): Releasing backup interface bond_slave_0
bond0 (unregistering): Released all slaves
------------[ cut here ]------------
ODEBUG: free active (active state 0) object type: timer_list hint:
delayed_work_timer_fn+0x0/0x90 arch/x86/include/asm/paravirt.h:767
WARNING: CPU: 1 PID: 25149 at lib/debugobjects.c:325
debug_print_object+0x168/0x250 lib/debugobjects.c:325
Kernel panic - not syncing: panic_on_warn set ...
CPU: 1 PID: 25149 Comm: kworker/u4:1 Not tainted 5.2.0-rc4+ #31
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Workqueue: netns cleanup_net
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x172/0x1f0 lib/dump_stack.c:113
panic+0x2cb/0x744 kernel/panic.c:219
__warn.cold+0x20/0x4d kernel/panic.c:576
report_bug+0x263/0x2b0 lib/bug.c:186
fixup_bug arch/x86/kernel/traps.c:179 [inline]
fixup_bug arch/x86/kernel/traps.c:174 [inline]
do_error_trap+0x11b/0x200 arch/x86/kernel/traps.c:272
do_invalid_op+0x37/0x50 arch/x86/kernel/traps.c:291
invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:986
RIP: 0010:debug_print_object+0x168/0x250 lib/debugobjects.c:325
Code: dd e0 c9 a4 87 48 89 fa 48 c1 ea 03 80 3c 02 00 0f 85 b5 00 00 00 48
8b 14 dd e0 c9 a4 87 48 c7 c7 80 bf a4 87 e8 16 75 0d fe <0f> 0b 83 05 4b
46 4b 06 01 48 83 c4 20 5b 41 5c 41 5d 41 5e 5d c3
RSP: 0018:ffff888058c07838 EFLAGS: 00010086
RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
RDX: 0000000000000000 RSI: ffffffff815ac956 RDI: ffffed100b180ef9
RBP: ffff888058c07878 R08: ffff88805692a340 R09: ffffed1015d240f1
R10: ffffed1015d240f0 R11: ffff8880ae920787 R12: 0000000000000001
R13: ffffffff88bad1a0 R14: ffffffff816039d0 R15: ffff88805f992e60
__debug_check_no_obj_freed lib/debugobjects.c:785 [inline]
debug_check_no_obj_freed+0x29f/0x464 lib/debugobjects.c:817
kfree+0xbd/0x220 mm/slab.c:3754
kvfree+0x61/0x70 mm/util.c:460
netdev_freemem+0x4c/0x60 net/core/dev.c:9070
netdev_release+0x86/0xb0 net/core/net-sysfs.c:1635
device_release+0x7a/0x210 drivers/base/core.c:1064
kobject_cleanup lib/kobject.c:691 [inline]
kobject_release lib/kobject.c:720 [inline]
kref_put include/linux/kref.h:65 [inline]
kobject_put.cold+0x289/0x2e6 lib/kobject.c:737
netdev_run_todo+0x53b/0x7c0 net/core/dev.c:8975
rtnl_unlock+0xe/0x10 net/core/rtnetlink.c:112
default_device_exit_batch+0x358/0x410 net/core/dev.c:9756
ops_exit_list.isra.0+0xfc/0x150 net/core/net_namespace.c:157
cleanup_net+0x3fb/0x960 net/core/net_namespace.c:553
process_one_work+0x989/0x1790 kernel/workqueue.c:2269
worker_thread+0x98/0xe40 kernel/workqueue.c:2415
kthread+0x354/0x420 kernel/kthread.c:255
ret_from_fork+0x24/0x30 arch/x86/entry/entry_64.S:352
======================================================
---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
^ permalink raw reply
* [PATCH 3/3] module: Properly propagate MODULE_STATE_COMING failure
From: Peter Zijlstra @ 2019-06-24 9:18 UTC (permalink / raw)
To: Jessica Yu, linux-kernel, jpoimboe, jikos, mbenes, pmladek, ast,
daniel, akpm, peterz
Cc: Martin KaFai Lau, Song Liu, Yonghong Song, netdev, bpf
In-Reply-To: <20190624091843.859714294@infradead.org>
Now that notifiers got unbroken; use the proper interface to handle
notifier errors and propagate them.
There were already MODULE_STATE_COMING notifiers that failed; notably:
- jump_label_module_notifier()
- tracepoint_module_notify()
- bpf_event_notify()
By propagating this error, we fix those users.
Cc: Jessica Yu <jeyu@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: linux-kernel@vger.kernel.org
Cc: netdev@vger.kernel.org
Cc: bpf@vger.kernel.org
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/module.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -3643,9 +3643,10 @@ static int prepare_coming_module(struct
if (err)
return err;
- blocking_notifier_call_chain(&module_notify_list,
- MODULE_STATE_COMING, mod);
- return 0;
+ err = blocking_notifier_call_chain_error(&module_notify_list,
+ MODULE_STATE_COMING, MODULE_STATE_GOING, mod);
+
+ return notifier_to_errno(err);
}
static int unknown_module_param_cb(char *param, char *val, const char *modname,
^ permalink raw reply
* [PATCH 2/3] module: Fix up module_notifier return values.
From: Peter Zijlstra @ 2019-06-24 9:18 UTC (permalink / raw)
To: Jessica Yu, linux-kernel, jpoimboe, jikos, mbenes, pmladek, ast,
daniel, akpm, peterz
Cc: Robert Richter, Steven Rostedt, Ingo Molnar, Martin KaFai Lau,
Song Liu, Yonghong Song, Mathieu Desnoyers, Paul E. McKenney,
Joel Fernandes (Google), Ard Biesheuvel, Thomas Gleixner,
oprofile-list, netdev, bpf
In-Reply-To: <20190624091843.859714294@infradead.org>
While auditing all module notifiers I noticed a whole bunch of fail
wrt the return value. Notifiers have a 'special' return semantics.
Cc: Robert Richter <rric@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: "Paul E. McKenney" <paulmck@linux.ibm.com>
Cc: "Joel Fernandes (Google)" <joel@joelfernandes.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: oprofile-list@lists.sf.net
Cc: linux-kernel@vger.kernel.org
Cc: netdev@vger.kernel.org
Cc: bpf@vger.kernel.org
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
drivers/oprofile/buffer_sync.c | 4 ++--
kernel/module.c | 9 +++++----
kernel/trace/bpf_trace.c | 8 ++++++--
kernel/trace/trace.c | 2 +-
kernel/trace/trace_events.c | 2 +-
kernel/trace/trace_printk.c | 4 ++--
kernel/tracepoint.c | 2 +-
7 files changed, 18 insertions(+), 13 deletions(-)
--- a/drivers/oprofile/buffer_sync.c
+++ b/drivers/oprofile/buffer_sync.c
@@ -116,7 +116,7 @@ module_load_notify(struct notifier_block
{
#ifdef CONFIG_MODULES
if (val != MODULE_STATE_COMING)
- return 0;
+ return NOTIFY_DONE;
/* FIXME: should we process all CPU buffers ? */
mutex_lock(&buffer_mutex);
@@ -124,7 +124,7 @@ module_load_notify(struct notifier_block
add_event_entry(MODULE_LOADED_CODE);
mutex_unlock(&buffer_mutex);
#endif
- return 0;
+ return NOTIFY_OK;
}
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -1302,10 +1302,11 @@ static int bpf_event_notify(struct notif
{
struct bpf_trace_module *btm, *tmp;
struct module *mod = module;
+ int ret = 0;
if (mod->num_bpf_raw_events == 0 ||
(op != MODULE_STATE_COMING && op != MODULE_STATE_GOING))
- return 0;
+ goto out;
mutex_lock(&bpf_module_mutex);
@@ -1315,6 +1316,8 @@ static int bpf_event_notify(struct notif
if (btm) {
btm->module = module;
list_add(&btm->list, &bpf_trace_modules);
+ } else {
+ ret = -ENOMEM;
}
break;
case MODULE_STATE_GOING:
@@ -1330,7 +1333,8 @@ static int bpf_event_notify(struct notif
mutex_unlock(&bpf_module_mutex);
- return 0;
+out:
+ return notifier_from_errno(ret);
}
static struct notifier_block bpf_module_nb = {
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -8685,7 +8685,7 @@ static int trace_module_notify(struct no
break;
}
- return 0;
+ return NOTIFY_OK;
}
static struct notifier_block trace_module_nb = {
--- a/kernel/trace/trace_events.c
+++ b/kernel/trace/trace_events.c
@@ -2450,7 +2450,7 @@ static int trace_module_notify(struct no
mutex_unlock(&trace_types_lock);
mutex_unlock(&event_mutex);
- return 0;
+ return NOTIFY_OK;
}
static struct notifier_block trace_module_nb = {
--- a/kernel/trace/trace_printk.c
+++ b/kernel/trace/trace_printk.c
@@ -95,7 +95,7 @@ static int module_trace_bprintk_format_n
if (val == MODULE_STATE_COMING)
hold_module_trace_bprintk_format(start, end);
}
- return 0;
+ return NOTIFY_OK;
}
/*
@@ -173,7 +173,7 @@ __init static int
module_trace_bprintk_format_notify(struct notifier_block *self,
unsigned long val, void *data)
{
- return 0;
+ return NOTIFY_OK;
}
static inline const char **
find_next_mod_format(int start_index, void *v, const char **fmt, loff_t *pos)
--- a/kernel/tracepoint.c
+++ b/kernel/tracepoint.c
@@ -538,7 +538,7 @@ static int tracepoint_module_notify(stru
case MODULE_STATE_UNFORMED:
break;
}
- return ret;
+ return notifier_from_errno(ret);
}
static struct notifier_block tracepoint_module_nb = {
^ permalink raw reply
* Re: WARNING: ODEBUG bug in netdev_freemem (2)
From: Thomas Gleixner @ 2019-06-24 9:33 UTC (permalink / raw)
To: syzbot
Cc: alexander.h.duyck, amritha.nambiar, andriy.shevchenko, davem,
dmitry.torokhov, f.fainelli, gregkh, idosch, linux-kernel, netdev,
syzkaller-bugs, tyhicks, wanghai26, yuehaibing
In-Reply-To: <000000000000d6a8ba058c0df076@google.com>
On Mon, 24 Jun 2019, syzbot wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit: fd6b99fa Merge branch 'akpm' (patches from Andrew)
> git tree: upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=144de256a00000
> kernel config: https://syzkaller.appspot.com/x/.config?x=fa9f7e1b6a8bb586
> dashboard link: https://syzkaller.appspot.com/bug?extid=c4521ac872a4ccc3afec
> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+c4521ac872a4ccc3afec@syzkaller.appspotmail.com
>
> device hsr_slave_0 left promiscuous mode
> team0 (unregistering): Port device team_slave_1 removed
> team0 (unregistering): Port device team_slave_0 removed
> bond0 (unregistering): Releasing backup interface bond_slave_1
> bond0 (unregistering): Releasing backup interface bond_slave_0
> bond0 (unregistering): Released all slaves
> ------------[ cut here ]------------
> ODEBUG: free active (active state 0) object type: timer_list hint:
> delayed_work_timer_fn+0x0/0x90 arch/x86/include/asm/paravirt.h:767
One of the cleaned up devices has left an active timer which belongs to a
delayed work. That's all I can decode out of that splat. :(
Thanks,
tglx
^ permalink raw reply
* Re: [PATCH v4 2/5] net: macb: add support for sgmii MAC-PHY interface
From: Russell King - ARM Linux admin @ 2019-06-24 9:35 UTC (permalink / raw)
To: Parshuram Raju Thombare
Cc: andrew@lunn.ch, nicolas.ferre@microchip.com, davem@davemloft.net,
f.fainelli@gmail.com, netdev@vger.kernel.org,
hkallweit1@gmail.com, linux-kernel@vger.kernel.org,
Rafal Ciepiela, Anil Joy Varughese, Piotr Sroka
In-Reply-To: <CO2PR07MB246931C79F736F39D0523D3BC1E00@CO2PR07MB2469.namprd07.prod.outlook.com>
On Mon, Jun 24, 2019 at 06:35:44AM +0000, Parshuram Raju Thombare wrote:
>
> >> + if (change_interface) {
> >> + if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII) {
> >> + gem_writel(bp, NCFGR, ~GEM_BIT(SGMIIEN) &
> >> + ~GEM_BIT(PCSSEL) &
> >> + gem_readl(bp, NCFGR));
> >> + gem_writel(bp, NCR, ~GEM_BIT(TWO_PT_FIVE_GIG) &
> >> + gem_readl(bp, NCR));
> >> + gem_writel(bp, PCS_CTRL, gem_readl(bp, PCS_CTRL) |
> >> + GEM_BIT(PCS_CTRL_RST));
> >> + }
> >I still don't think this makes much sense, splitting the interface
> >configuration between here and below.
> Do you mean splitting mac_config in two *_configure functions ?
> This was done as per Andrew's suggestion to make code mode readable
> and easy to manage by splitting MAC configuration for different interfaces.
No, I mean here you disable SGMII if we're switching away from SGMII
mode.... (note, this means there is more to come for this sentence)
>
> >> + bp->phy_interface = state->interface;
> >> + }
> >> +
> >> if (!phylink_autoneg_inband(mode) &&
> >> (bp->speed != state->speed ||
> >> - bp->duplex != state->duplex)) {
> >> + bp->duplex != state->duplex ||
> >> + change_interface)) {
> >> u32 reg;
> >>
> >> reg = macb_readl(bp, NCFGR);
> >> reg &= ~(MACB_BIT(SPD) | MACB_BIT(FD));
> >> if (macb_is_gem(bp))
> >> reg &= ~GEM_BIT(GBE);
> >> + macb_or_gem_writel(bp, NCFGR, reg);
> >> +
> >> + if (bp->phy_interface == PHY_INTERFACE_MODE_SGMII)
> >> + gem_writel(bp, NCFGR, GEM_BIT(SGMIIEN) |
> >> + GEM_BIT(PCSSEL) |
> >> + gem_readl(bp, NCFGR));
> >This will only be executed when we are not using inband mode, which
> >basically means it's not possible to switch to SGMII in-band mode.
> SGMII is used in default PHY mode. And above code is to program MAC to
> select PCS and SGMII interface.
... and here you enable it for SGMII mode, but only for non-inband
modes.
For inband modes, you do not have any code that enables SGMII mode.
Since the only inband mode you support is SGMII, this is not very
good behaviour.
Why not:
if (change_interface) {
u32 ncfgr;
bp->phy_interface = state->interface;
// We don't support 2.5G modes
gem_writel(bp, NCR, ~GEM_BIT(TWO_PT_FIVE_GIG) &
gem_readl(bp, NCR));
ncfgr = gem_readl(bp, NCFGR);
if (state->interface == PHY_INTERFACE_MODE_SGMII) {
// Enable SGMII mode and PCS
gem_writel(bp, NCFGR, ncfgr | GEM_BIT(SGMIIEN) |
GEM_BIT(PCSSEL));
} else {
// Disable SGMII mode and PCS
gem_writel(bp, NCFGR, ncfgr & ~(GEM_BIT(SGMIIEN) |
GEM_BIT(PCSSEL)));
// Reset PCS
gem_writel(bp, PCS_CTRL, gem_readl(bp, PCS_CTRL) |
GEM_BIT(PCS_CTRL_RST));
}
}
if (!phylink_autoneg_inband(mode) &&
(bp->speed != state->speed || bp->duplex != state->duplex)) {
?
>
> >> +
> >> + if (!interface_supported) {
> >> + netdev_err(dev, "Phy mode %s not supported",
> >> + phy_modes(phy_mode));
> >> + goto err_out_free_netdev;
> >> + }
> >> +
> >> bp->phy_interface = phy_mode;
> >> + } else {
> >> + bp->phy_interface = phy_mode;
> >> + }
> >If bp->phy_interface is PHY_INTERFACE_MODE_SGMII here, and mac_config()
> >is called with state->interface = PHY_INTERFACE_MODE_SGMII, then
> >mac_config() won't configure the MAC for the interface type - is that
> >intentional?
>
> In mac_config configure MAC for non in-band mode, there is also check for speed, duplex
> changes. bp->speed and bp->duplex are initialized to SPEED_UNKNOWN and DUPLEX_UNKNOWN
> values so it is expected that for non in band mode state contains valid speed and duplex mode
> which are different from *_UNKNOWN values.
Sorry, this reply doesn't answer my question. I'm not asking about
bp->speed and bp->duplex. I'm asking:
1) why you are initialising bp->phy_interface here
2) you to consider the impact that has on the mac_config() implementation
you are proposing
because I think it's buggy.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
^ permalink raw reply
* Re: [PATCH bpf-next] libbpf: add xsk_ring_prod__free() function
From: Eelco Chaudron @ 2019-06-24 9:37 UTC (permalink / raw)
To: Andrii Nakryiko, Karlsson, Magnus
Cc: Networking, Alexei Starovoitov, Daniel Borkmann, Martin Lau,
Song Liu, Yonghong Song
In-Reply-To: <CAEf4BzZsmH+4A0dADeXYUDqeEK9N_-PVqzHW_=vPytjEX1hqTA@mail.gmail.com>
On 21 Jun 2019, at 21:13, Andrii Nakryiko wrote:
> On Fri, Jun 21, 2019 at 8:26 AM Eelco Chaudron <echaudro@redhat.com>
> wrote:
>>
>> When an AF_XDP application received X packets, it does not mean X
>> frames can be stuffed into the producer ring. To make it easier for
>> AF_XDP applications this API allows them to check how many frames can
>> be added into the ring.
>>
>> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
>> ---
>> tools/lib/bpf/xsk.h | 6 ++++++
>> 1 file changed, 6 insertions(+)
>>
>> diff --git a/tools/lib/bpf/xsk.h b/tools/lib/bpf/xsk.h
>> index 82ea71a0f3ec..86f3d485e957 100644
>> --- a/tools/lib/bpf/xsk.h
>> +++ b/tools/lib/bpf/xsk.h
>> @@ -95,6 +95,12 @@ static inline __u32 xsk_prod_nb_free(struct
>> xsk_ring_prod *r, __u32 nb)
>> return r->cached_cons - r->cached_prod;
>> }
>>
>> +static inline __u32 xsk_ring_prod__free(struct xsk_ring_prod *r)
>
> This is a very bad name choice. __free is used for functions that free
> memory and resources. One function below I see avail is used in the
> name, why not xsk_ring_prog__avail?
Must agree that free sound like you are freeing entries… However, I
just kept the naming already in the API/file (see above,
xsk_prod_nb_free()).
Reading the code there is a difference as xx_avail() means available
filled entries, where xx_free() means available free entries.
So I could rename it to xsk_ring_prod__nb_free() maybe?
Forgot to include Magnus in the email, so copied him in, for some
comments.
>> +{
>> + r->cached_cons = *r->consumer + r->size;
>> + return r->cached_cons - r->cached_prod;
>> +}
>> +
>> static inline __u32 xsk_cons_nb_avail(struct xsk_ring_cons *r, __u32
>> nb)
>> {
>> __u32 entries = r->cached_prod - r->cached_cons;
>> --
>> 2.20.1
>>
^ permalink raw reply
* [PATCH net] net/packet: fix memory leak in packet_set_ring()
From: Eric Dumazet @ 2019-06-24 9:38 UTC (permalink / raw)
To: David S . Miller
Cc: netdev, Eric Dumazet, Eric Dumazet, Sowmini Varadhan, syzbot
syzbot found we can leak memory in packet_set_ring(), if user application
provides buggy parameters.
Fixes: 7f953ab2ba46 ("af_packet: TX_RING support for TPACKET_V3")
Signed-off-by: Eric Dumazet <edumazet@google.com>
Cc: Sowmini Varadhan <sowmini.varadhan@oracle.com>
Reported-by: syzbot <syzkaller@googlegroups.com>
---
net/packet/af_packet.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index a29d66da7394bb48c4012b29ff2ad3d256cf7bff..0b4cf94f0233f86c196e767dc0179c38330ecca7 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -4314,7 +4314,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
req3->tp_sizeof_priv ||
req3->tp_feature_req_word) {
err = -EINVAL;
- goto out;
+ goto out_free_pg_vec;
}
}
break;
@@ -4378,6 +4378,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u,
prb_shutdown_retire_blk_timer(po, rb_queue);
}
+out_free_pg_vec:
if (pg_vec)
free_pg_vec(pg_vec, order, req->tp_block_nr);
out:
--
2.22.0.410.gd8fdbe21b5-goog
^ permalink raw reply related
* Re: [PATCH 1/2] net: macb: Fix compilation on systems without COMMON_CLK
From: Nicolas.Ferre @ 2019-06-24 9:40 UTC (permalink / raw)
To: palmer, davem; +Cc: netdev, linux-kernel
In-Reply-To: <20190624061603.1704-2-palmer@sifive.com>
On 24/06/2019 at 08:16, Palmer Dabbelt wrote:
> External E-Mail
>
>
> The patch to add support for the FU540-C000 added a dependency on
> COMMON_CLK, but didn't express that via Kconfig. This fixes the build
> failure by adding CONFIG_MACB_FU540, which depends on COMMON_CLK and
> conditionally enables the FU540-C000 support.
Let's try to limit the use of #ifdef's throughout the code. We are
using them in this driver but only for the hot paths and things that
have an impact on performance. I don't think it's the case here: so
please find another option => NACK.
> I've built this with a powerpc allyesconfig (which pointed out the bug)
> and on RISC-V, manually checking to ensure the code was built. I
> haven't even booted the resulting kernels.
>
> Fixes: c218ad559020 ("macb: Add support for SiFive FU540-C000")
> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
> ---
> drivers/net/ethernet/cadence/Kconfig | 11 +++++++++++
> drivers/net/ethernet/cadence/macb_main.c | 12 ++++++++++++
> 2 files changed, 23 insertions(+)
>
> diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
> index 1766697c9c5a..74ee2bfd2369 100644
> --- a/drivers/net/ethernet/cadence/Kconfig
> +++ b/drivers/net/ethernet/cadence/Kconfig
> @@ -40,6 +40,17 @@ config MACB_USE_HWSTAMP
> ---help---
> Enable IEEE 1588 Precision Time Protocol (PTP) support for MACB.
>
> +config MACB_FU540
> + bool "Enable support for the SiFive FU540 clock controller"
> + depends on MACB && COMMON_CLK
> + default y
> + ---help---
> + Enable support for the MACB/GEM clock controller on the SiFive
> + FU540-C000. This device is necessary for switching between 10/100
> + and gigabit modes on the FU540-C000 SoC, without which it is only
> + possible to bring up the Ethernet link in whatever mode the
> + bootloader probed.
> +
> config MACB_PCI
> tristate "Cadence PCI MACB/GEM support"
> depends on MACB && PCI && COMMON_CLK
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index c545c5b435d8..a903dfdd4183 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -41,6 +41,7 @@
> #include <linux/pm_runtime.h>
> #include "macb.h"
>
> +#ifdef CONFIG_MACB_FU540
> /* This structure is only used for MACB on SiFive FU540 devices */
> struct sifive_fu540_macb_mgmt {
> void __iomem *reg;
> @@ -49,6 +50,7 @@ struct sifive_fu540_macb_mgmt {
> };
>
> static struct sifive_fu540_macb_mgmt *mgmt;
> +#endif
>
> #define MACB_RX_BUFFER_SIZE 128
> #define RX_BUFFER_MULTIPLE 64 /* bytes */
> @@ -3956,6 +3958,7 @@ static int at91ether_init(struct platform_device *pdev)
> return 0;
> }
>
> +#ifdef CONFIG_MACB_FU540
> static unsigned long fu540_macb_tx_recalc_rate(struct clk_hw *hw,
> unsigned long parent_rate)
> {
> @@ -4056,7 +4059,9 @@ static int fu540_c000_init(struct platform_device *pdev)
>
> return macb_init(pdev);
> }
> +#endif
>
> +#ifdef CONFIG_MACB_FU540
> static const struct macb_config fu540_c000_config = {
> .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO |
> MACB_CAPS_GEM_HAS_PTP,
> @@ -4065,6 +4070,7 @@ static const struct macb_config fu540_c000_config = {
> .init = fu540_c000_init,
> .jumbo_max_len = 10240,
> };
> +#endif
>
> static const struct macb_config at91sam9260_config = {
> .caps = MACB_CAPS_USRIO_HAS_CLKEN | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
> @@ -4155,7 +4161,9 @@ static const struct of_device_id macb_dt_ids[] = {
> { .compatible = "cdns,emac", .data = &emac_config },
> { .compatible = "cdns,zynqmp-gem", .data = &zynqmp_config},
> { .compatible = "cdns,zynq-gem", .data = &zynq_config },
> +#ifdef CONFIG_MACB_FU540
> { .compatible = "sifive,fu540-macb", .data = &fu540_c000_config },
> +#endif
> { /* sentinel */ }
> };
> MODULE_DEVICE_TABLE(of, macb_dt_ids);
> @@ -4363,7 +4371,9 @@ static int macb_probe(struct platform_device *pdev)
>
> err_disable_clocks:
> clk_disable_unprepare(tx_clk);
> +#ifdef CONFIG_MACB_FU540
> clk_unregister(tx_clk);
> +#endif
> clk_disable_unprepare(hclk);
> clk_disable_unprepare(pclk);
> clk_disable_unprepare(rx_clk);
> @@ -4398,7 +4408,9 @@ static int macb_remove(struct platform_device *pdev)
> pm_runtime_dont_use_autosuspend(&pdev->dev);
> if (!pm_runtime_suspended(&pdev->dev)) {
> clk_disable_unprepare(bp->tx_clk);
> +#ifdef CONFIG_MACB_FU540
> clk_unregister(bp->tx_clk);
> +#endif
> clk_disable_unprepare(bp->hclk);
> clk_disable_unprepare(bp->pclk);
> clk_disable_unprepare(bp->rx_clk);
>
--
Nicolas Ferre
^ permalink raw reply
* Re: [PATCH v4 3/5] net: macb: add support for c45 PHY
From: Russell King - ARM Linux admin @ 2019-06-24 9:42 UTC (permalink / raw)
To: Parshuram Raju Thombare
Cc: andrew@lunn.ch, nicolas.ferre@microchip.com, davem@davemloft.net,
f.fainelli@gmail.com, netdev@vger.kernel.org,
hkallweit1@gmail.com, linux-kernel@vger.kernel.org,
Rafal Ciepiela, Anil Joy Varughese, Piotr Sroka
In-Reply-To: <CO2PR07MB24695E11E3931BE2E5664054C1E00@CO2PR07MB2469.namprd07.prod.outlook.com>
On Mon, Jun 24, 2019 at 06:47:48AM +0000, Parshuram Raju Thombare wrote:
> >Which Clause 45 PHY are you using?
>
> I am using emulated PHY in our CSP environment.
Concentrated Solar Power? Chartered Society of Physiotherapy? Center
for Space Physics?
Sorry, I don't know what a "CSP environment" is in this context, neither
it seems does google. TLAs in general tend to be bad when it comes to
communication.
However, it seems from that comment that you're not talking about real
hardware. Is there no real hardware out there supporting 10G mode with
these proposed driver changes yet?
> This is using 10G generic PHY driver, with PHY having compatible = "ethernet-phy-ieee802.3-c45"
The generic 10G PHY driver is really dumb and basic - it only supports
a very basic 10G mode.
>
> Hi Andrew,
> Can I add your "Reviewed-by" tag for this patch. You added it to this patch in last series.
>
> Regards,
> Parshuram Thombare
>
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
^ permalink raw reply
* Re: [PATCH 2/2] net: macb: Kconfig: Rename Atmel to Cadence
From: Nicolas.Ferre @ 2019-06-24 9:49 UTC (permalink / raw)
To: palmer, davem; +Cc: netdev, linux-kernel
In-Reply-To: <20190624061603.1704-3-palmer@sifive.com>
On 24/06/2019 at 08:16, Palmer Dabbelt wrote:
> External E-Mail
>
>
> When touching the Kconfig for this driver I noticed that both the
> Kconfig help text and a comment referred to this being an Atmel driver.
> As far as I know, this is a Cadence driver. The fix is just
Indeed: was written and then maintained by Atmel (now Microchip) for
years... So I would say that more than a "Cadence driver" it's a driver
that applies to a Cadence peripheral.
I won't hold the patch just for this as the patch makes perfect sense,
but would love that it's been highlighted...
> s/Atmel/Cadence/, but I did go and re-wrap the Kconfig help text as that
> change caused it to go over 80 characters.
>
> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
> ---
> drivers/net/ethernet/cadence/Kconfig | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
> index 74ee2bfd2369..29b6132b418e 100644
> --- a/drivers/net/ethernet/cadence/Kconfig
> +++ b/drivers/net/ethernet/cadence/Kconfig
> @@ -1,6 +1,6 @@
> # SPDX-License-Identifier: GPL-2.0-only
> #
> -# Atmel device configuration
> +# Cadence device configuration
> #
>
> config NET_VENDOR_CADENCE
> @@ -13,8 +13,8 @@ config NET_VENDOR_CADENCE
> If unsure, say Y.
>
> Note that the answer to this question doesn't directly affect the
> - kernel: saying N will just cause the configurator to skip all
> - the remaining Atmel network card questions. If you say Y, you will be
> + kernel: saying N will just cause the configurator to skip all the
> + remaining Cadence network card questions. If you say Y, you will be
> asked for your specific card in the following questions.
>
> if NET_VENDOR_CADENCE
>
--
Nicolas Ferre
^ permalink raw reply
* Re: [PATCH 2/2] net: macb: Kconfig: Rename Atmel to Cadence
From: Palmer Dabbelt @ 2019-06-24 9:57 UTC (permalink / raw)
To: Nicolas.Ferre; +Cc: davem, netdev, linux-kernel
In-Reply-To: <0c714db9-a3c1-e89b-8889-e9cdb2ac6c52@microchip.com>
On Mon, 24 Jun 2019 02:49:16 PDT (-0700), Nicolas.Ferre@microchip.com wrote:
> On 24/06/2019 at 08:16, Palmer Dabbelt wrote:
>> External E-Mail
>>
>>
>> When touching the Kconfig for this driver I noticed that both the
>> Kconfig help text and a comment referred to this being an Atmel driver.
>> As far as I know, this is a Cadence driver. The fix is just
>
> Indeed: was written and then maintained by Atmel (now Microchip) for
> years... So I would say that more than a "Cadence driver" it's a driver
> that applies to a Cadence peripheral.
>
> I won't hold the patch just for this as the patch makes perfect sense,
> but would love that it's been highlighted...
OK, I don't mind changing it. Does this look OK? I have to submit a v2 anyway
for the first patch.
Author: Palmer Dabbelt <palmer@sifive.com>
Date: Sun Jun 23 23:04:14 2019 -0700
net: macb: Kconfig: Rename Atmel to Cadence
The help text makes it look like NET_VENDOR_CADENCE enables support for
Atmel devices, when in reality it's a driver written by Atmel that
supports Cadence devices. This may confuse users that have this device
on a non-Atmel SoC.
The fix is just s/Atmel/Cadence/, but I did go and re-wrap the Kconfig
help text as that change caused it to go over 80 characters.
Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
index 74ee2bfd2369..29b6132b418e 100644
--- a/drivers/net/ethernet/cadence/Kconfig
+++ b/drivers/net/ethernet/cadence/Kconfig
@@ -1,6 +1,6 @@
# SPDX-License-Identifier: GPL-2.0-only
#
-# Atmel device configuration
+# Cadence device configuration
#
config NET_VENDOR_CADENCE
@@ -13,8 +13,8 @@ config NET_VENDOR_CADENCE
If unsure, say Y.
Note that the answer to this question doesn't directly affect the
- kernel: saying N will just cause the configurator to skip all
- the remaining Atmel network card questions. If you say Y, you will be
+ kernel: saying N will just cause the configurator to skip all the
+ remaining Cadence network card questions. If you say Y, you will be
asked for your specific card in the following questions.
if NET_VENDOR_CADENCE
>
>> s/Atmel/Cadence/, but I did go and re-wrap the Kconfig help text as that
>> change caused it to go over 80 characters.
>>
>> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
>> ---
>> drivers/net/ethernet/cadence/Kconfig | 6 +++---
>> 1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
>> index 74ee2bfd2369..29b6132b418e 100644
>> --- a/drivers/net/ethernet/cadence/Kconfig
>> +++ b/drivers/net/ethernet/cadence/Kconfig
>> @@ -1,6 +1,6 @@
>> # SPDX-License-Identifier: GPL-2.0-only
>> #
>> -# Atmel device configuration
>> +# Cadence device configuration
>> #
>>
>> config NET_VENDOR_CADENCE
>> @@ -13,8 +13,8 @@ config NET_VENDOR_CADENCE
>> If unsure, say Y.
>>
>> Note that the answer to this question doesn't directly affect the
>> - kernel: saying N will just cause the configurator to skip all
>> - the remaining Atmel network card questions. If you say Y, you will be
>> + kernel: saying N will just cause the configurator to skip all the
>> + remaining Cadence network card questions. If you say Y, you will be
>> asked for your specific card in the following questions.
>>
>> if NET_VENDOR_CADENCE
>>
>
>
> --
> Nicolas Ferre
^ permalink raw reply related
* Re: [PATCH 1/2] net: macb: Fix compilation on systems without COMMON_CLK
From: Palmer Dabbelt @ 2019-06-24 9:57 UTC (permalink / raw)
To: Nicolas.Ferre; +Cc: davem, netdev, linux-kernel
In-Reply-To: <c440e194-dc93-5a3e-7608-710afade9774@microchip.com>
On Mon, 24 Jun 2019 02:40:21 PDT (-0700), Nicolas.Ferre@microchip.com wrote:
> On 24/06/2019 at 08:16, Palmer Dabbelt wrote:
>> External E-Mail
>>
>>
>> The patch to add support for the FU540-C000 added a dependency on
>> COMMON_CLK, but didn't express that via Kconfig. This fixes the build
>> failure by adding CONFIG_MACB_FU540, which depends on COMMON_CLK and
>> conditionally enables the FU540-C000 support.
>
> Let's try to limit the use of #ifdef's throughout the code. We are
> using them in this driver but only for the hot paths and things that
> have an impact on performance. I don't think it's the case here: so
> please find another option => NACK.
OK. Would you accept adding a Kconfig dependency of the generic MACB driver on
COMMON_CLK, as suggested in the cover letter?
>
>> I've built this with a powerpc allyesconfig (which pointed out the bug)
>> and on RISC-V, manually checking to ensure the code was built. I
>> haven't even booted the resulting kernels.
>>
>> Fixes: c218ad559020 ("macb: Add support for SiFive FU540-C000")
>> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
>> ---
>> drivers/net/ethernet/cadence/Kconfig | 11 +++++++++++
>> drivers/net/ethernet/cadence/macb_main.c | 12 ++++++++++++
>> 2 files changed, 23 insertions(+)
>>
>> diff --git a/drivers/net/ethernet/cadence/Kconfig b/drivers/net/ethernet/cadence/Kconfig
>> index 1766697c9c5a..74ee2bfd2369 100644
>> --- a/drivers/net/ethernet/cadence/Kconfig
>> +++ b/drivers/net/ethernet/cadence/Kconfig
>> @@ -40,6 +40,17 @@ config MACB_USE_HWSTAMP
>> ---help---
>> Enable IEEE 1588 Precision Time Protocol (PTP) support for MACB.
>>
>> +config MACB_FU540
>> + bool "Enable support for the SiFive FU540 clock controller"
>> + depends on MACB && COMMON_CLK
>> + default y
>> + ---help---
>> + Enable support for the MACB/GEM clock controller on the SiFive
>> + FU540-C000. This device is necessary for switching between 10/100
>> + and gigabit modes on the FU540-C000 SoC, without which it is only
>> + possible to bring up the Ethernet link in whatever mode the
>> + bootloader probed.
>> +
>> config MACB_PCI
>> tristate "Cadence PCI MACB/GEM support"
>> depends on MACB && PCI && COMMON_CLK
>> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
>> index c545c5b435d8..a903dfdd4183 100644
>> --- a/drivers/net/ethernet/cadence/macb_main.c
>> +++ b/drivers/net/ethernet/cadence/macb_main.c
>> @@ -41,6 +41,7 @@
>> #include <linux/pm_runtime.h>
>> #include "macb.h"
>>
>> +#ifdef CONFIG_MACB_FU540
>> /* This structure is only used for MACB on SiFive FU540 devices */
>> struct sifive_fu540_macb_mgmt {
>> void __iomem *reg;
>> @@ -49,6 +50,7 @@ struct sifive_fu540_macb_mgmt {
>> };
>>
>> static struct sifive_fu540_macb_mgmt *mgmt;
>> +#endif
>>
>> #define MACB_RX_BUFFER_SIZE 128
>> #define RX_BUFFER_MULTIPLE 64 /* bytes */
>> @@ -3956,6 +3958,7 @@ static int at91ether_init(struct platform_device *pdev)
>> return 0;
>> }
>>
>> +#ifdef CONFIG_MACB_FU540
>> static unsigned long fu540_macb_tx_recalc_rate(struct clk_hw *hw,
>> unsigned long parent_rate)
>> {
>> @@ -4056,7 +4059,9 @@ static int fu540_c000_init(struct platform_device *pdev)
>>
>> return macb_init(pdev);
>> }
>> +#endif
>>
>> +#ifdef CONFIG_MACB_FU540
>> static const struct macb_config fu540_c000_config = {
>> .caps = MACB_CAPS_GIGABIT_MODE_AVAILABLE | MACB_CAPS_JUMBO |
>> MACB_CAPS_GEM_HAS_PTP,
>> @@ -4065,6 +4070,7 @@ static const struct macb_config fu540_c000_config = {
>> .init = fu540_c000_init,
>> .jumbo_max_len = 10240,
>> };
>> +#endif
>>
>> static const struct macb_config at91sam9260_config = {
>> .caps = MACB_CAPS_USRIO_HAS_CLKEN | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII,
>> @@ -4155,7 +4161,9 @@ static const struct of_device_id macb_dt_ids[] = {
>> { .compatible = "cdns,emac", .data = &emac_config },
>> { .compatible = "cdns,zynqmp-gem", .data = &zynqmp_config},
>> { .compatible = "cdns,zynq-gem", .data = &zynq_config },
>> +#ifdef CONFIG_MACB_FU540
>> { .compatible = "sifive,fu540-macb", .data = &fu540_c000_config },
>> +#endif
>> { /* sentinel */ }
>> };
>> MODULE_DEVICE_TABLE(of, macb_dt_ids);
>> @@ -4363,7 +4371,9 @@ static int macb_probe(struct platform_device *pdev)
>>
>> err_disable_clocks:
>> clk_disable_unprepare(tx_clk);
>> +#ifdef CONFIG_MACB_FU540
>> clk_unregister(tx_clk);
>> +#endif
>> clk_disable_unprepare(hclk);
>> clk_disable_unprepare(pclk);
>> clk_disable_unprepare(rx_clk);
>> @@ -4398,7 +4408,9 @@ static int macb_remove(struct platform_device *pdev)
>> pm_runtime_dont_use_autosuspend(&pdev->dev);
>> if (!pm_runtime_suspended(&pdev->dev)) {
>> clk_disable_unprepare(bp->tx_clk);
>> +#ifdef CONFIG_MACB_FU540
>> clk_unregister(bp->tx_clk);
>> +#endif
>> clk_disable_unprepare(bp->hclk);
>> clk_disable_unprepare(bp->pclk);
>> clk_disable_unprepare(bp->rx_clk);
>>
>
>
> --
> Nicolas Ferre
^ permalink raw reply
* Re: [PATCH] xsk: sample kernel code is now in libbpf
From: Björn Töpel @ 2019-06-24 10:01 UTC (permalink / raw)
To: Eric Leblond; +Cc: Netdev, bpf, Karlsson, Magnus
In-Reply-To: <20190621201310.12791-1-eric@regit.org>
On Fri, 21 Jun 2019 at 22:55, Eric Leblond <eric@regit.org> wrote:
>
> Fix documentation that mention xdpsock_kern.c which has been
> replaced by code embedded in libbpf.
>
> Signed-off-by: Eric Leblond <eric@regit.org>
Thanks Eric!
Acked-by: Björn Töpel <bjorn.topel@intel.com>
> ---
> Documentation/networking/af_xdp.rst | 16 +++++++++++++++-
> 1 file changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/Documentation/networking/af_xdp.rst b/Documentation/networking/af_xdp.rst
> index e14d7d40fc75..83dddc20f5d6 100644
> --- a/Documentation/networking/af_xdp.rst
> +++ b/Documentation/networking/af_xdp.rst
> @@ -220,7 +220,21 @@ Usage
> In order to use AF_XDP sockets there are two parts needed. The
> user-space application and the XDP program. For a complete setup and
> usage example, please refer to the sample application. The user-space
> -side is xdpsock_user.c and the XDP side xdpsock_kern.c.
> +side is xdpsock_user.c and the XDP side is part of libbpf.
> +
> +The XDP code sample included in tools/lib/bpf/xsk.c is the following::
> +
> + SEC("xdp_sock") int xdp_sock_prog(struct xdp_md *ctx)
> + {
> + int index = ctx->rx_queue_index;
> +
> + // A set entry here means that the correspnding queue_id
> + // has an active AF_XDP socket bound to it.
> + if (bpf_map_lookup_elem(&xsks_map, &index))
> + return bpf_redirect_map(&xsks_map, index, 0);
> +
> + return XDP_PASS;
> + }
>
> Naive ring dequeue and enqueue could look like this::
>
> --
> 2.20.1
>
^ permalink raw reply
* Re: [RFC PATCH bpf-next] RV32G eBPF JIT
From: Björn Töpel @ 2019-06-24 10:11 UTC (permalink / raw)
To: Luke Nelson
Cc: Luke Nelson, Xi Wang, Palmer Dabbelt, Albert Ou,
Alexei Starovoitov, Daniel Borkmann, Martin KaFai Lau, Song Liu,
Yonghong Song, linux-riscv, LKML, Netdev, bpf
In-Reply-To: <20190621225938.27030-1-lukenels@cs.washington.edu>
On Sat, 22 Jun 2019 at 00:59, Luke Nelson <lukenels@cs.washington.edu> wrote:
>
> From: Luke Nelson <luke.r.nels@gmail.com>
>
Yay! Thanks for working on this! Nice work!
> This is an eBPF JIT for RV32G, adapted from the JIT for RV64G.
> Any feedback would be greatly appreciated.
>
> It passes 359 out of 378 tests in test_bpf.ko. The failing tests are
> features that are not supported right now:
> - ALU64 DIV/MOD:
> These require loops to emulate on 32-bit hardware,
> and are not supported on other 32-bit JITs like
> ARM32.
> - BPF_XADD | BPF_DW:
> RV32G does not have atomic instructions for operating
> on double words. This is similar to ARM32.
> - Tail calls:
> I'm working on adding support for these now, but couldn't
> find any test cases that use them. What's the best way
> of testing tail call code?
Have a look at the selftests (tools/testing/selftests/). The bpf
section has a lot a tests involving tail calls, e.g. test_progs and
test_verifier.
> - Far branches
> These are not supported in RV64G either.
This would be really nice to have, now that the size of BPF programs
are getting larger.
I'll try to a review/test run when I have some time over!
Björn
>
> There are two main changes required for this to work compared to the
> RV64 JIT.
>
> First, eBPF registers are 64-bit, while RV32G registers are 32-bit.
> I take an approach similar to ARM32: most BPF registers map directly to
> 2 RISC-V registers, while some reside in stack scratch space and must
> be saved / restored when used.
>
> Second, many 64-bit ALU operations do not trivially map to 32-bit
> operations. Operations that move bits between high and low words, such
> as ADD, LSH, MUL, and others must emulate the 64-bit behavior in terms
> of 32-bit instructions.
>
> Signed-off-by: Luke Nelson <luke.r.nels@gmail.com>
> Cc: Xi Wang <xi.wang@gmail.com>
> ---
> arch/riscv/Kconfig | 2 +-
> arch/riscv/net/Makefile | 7 +-
> arch/riscv/net/bpf_jit_comp32.c | 1460 +++++++++++++++++++++++++++++++
> 3 files changed, 1467 insertions(+), 2 deletions(-)
> create mode 100644 arch/riscv/net/bpf_jit_comp32.c
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 0c4b12205632..153ff9ee6e6c 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -49,7 +49,7 @@ config RISCV
> select GENERIC_IRQ_MULTI_HANDLER
> select ARCH_HAS_PTE_SPECIAL
> select ARCH_HAS_MMIOWB
> - select HAVE_EBPF_JIT if 64BIT
> + select HAVE_EBPF_JIT
>
> config MMU
> def_bool y
> diff --git a/arch/riscv/net/Makefile b/arch/riscv/net/Makefile
> index ec5b14763316..5511fb1dd40e 100644
> --- a/arch/riscv/net/Makefile
> +++ b/arch/riscv/net/Makefile
> @@ -1,2 +1,7 @@
> # SPDX-License-Identifier: GPL-2.0-only
> -obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
> +
> +ifeq ($(CONFIG_ARCH_RV64I),y)
> + obj-$(CONFIG_BPF_JIT) += bpf_jit_comp.o
> +else
> + obj-$(CONFIG_BPF_JIT) += bpf_jit_comp32.o
> +endif
> diff --git a/arch/riscv/net/bpf_jit_comp32.c b/arch/riscv/net/bpf_jit_comp32.c
> new file mode 100644
> index 000000000000..8790fd931aed
> --- /dev/null
> +++ b/arch/riscv/net/bpf_jit_comp32.c
> @@ -0,0 +1,1460 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* BPF JIT compiler for RV32G
> + *
> + * Copyright(c) 2019 Luke Nelson <luke.r.nels@gmail.com>
> + * This code is based on the code and ideas from
> + * Björn Töpel <bjorn.topel@gmail.com>,
> + * who wrote the rv64g BPF JIT, and
> + * Shubham Bansal <illusionist.neo@gmail.com> and
> + * Mircea Gherzan <mgherzan@gmail.com>,
> + * who wrote the 32-bit ARM BPF JIT.
> + */
> +
> +#include <linux/bpf.h>
> +#include <linux/filter.h>
> +#include <asm/cacheflush.h>
> +
> +enum {
> + RV_REG_ZERO = 0, /* The constant value 0 */
> + RV_REG_RA = 1, /* Return address */
> + RV_REG_SP = 2, /* Stack pointer */
> + RV_REG_GP = 3, /* Global pointer */
> + RV_REG_TP = 4, /* Thread pointer */
> + RV_REG_T0 = 5, /* Temporaries */
> + RV_REG_T1 = 6,
> + RV_REG_T2 = 7,
> + RV_REG_FP = 8,
> + RV_REG_S1 = 9, /* Saved registers */
> + RV_REG_A0 = 10, /* Function argument/return values */
> + RV_REG_A1 = 11, /* Function arguments */
> + RV_REG_A2 = 12,
> + RV_REG_A3 = 13,
> + RV_REG_A4 = 14,
> + RV_REG_A5 = 15,
> + RV_REG_A6 = 16,
> + RV_REG_A7 = 17,
> + RV_REG_S2 = 18, /* Saved registers */
> + RV_REG_S3 = 19,
> + RV_REG_S4 = 20,
> + RV_REG_S5 = 21,
> + RV_REG_S6 = 22,
> + RV_REG_S7 = 23,
> + RV_REG_S8 = 24,
> + RV_REG_S9 = 25,
> + RV_REG_S10 = 26,
> + RV_REG_S11 = 27,
> + RV_REG_T3 = 28, /* Temporaries */
> + RV_REG_T4 = 29,
> + RV_REG_T5 = 30,
> + RV_REG_T6 = 31,
> +};
> +
> +enum {
> + /* Stack layout - these are offsets from (top of stack - 4) */
> + BPF_R6_HI,
> + BPF_R6_LO,
> + BPF_R7_HI,
> + BPF_R7_LO,
> + BPF_R8_HI,
> + BPF_R8_LO,
> + BPF_R9_HI,
> + BPF_R9_LO,
> + BPF_TC_HI,
> + BPF_TC_LO,
> + BPF_AX_HI,
> + BPF_AX_LO,
> + /* Stack space for BPF_REG_6, BPF_REG_7, BPF_REG_8, BPF_REG_9,
> + * BPF_REG_AX and tail call counts.
> + */
> + BPF_JIT_SCRATCH_REGS,
> +};
> +
> +#define STACK_OFFSET(k) (-4 - ((k) * 4))
> +
> +#define TMP_REG_1 (MAX_BPF_JIT_REG + 0) /* TEMP Register 1 */
> +#define TMP_REG_2 (MAX_BPF_JIT_REG + 1) /* TEMP Register 2 */
> +#define TCALL_CNT (MAX_BPF_JIT_REG + 2) /* Tail Call Count */
> +
> +static const s8 bpf2rv32[][2] = {
> + /* Return value from in-kernel function, and exit value from eBPF */
> + [BPF_REG_0] = {RV_REG_S2, RV_REG_S1},
> + /* Arguments from eBPF program to in-kernel function */
> + [BPF_REG_1] = {RV_REG_A1, RV_REG_A0},
> + [BPF_REG_2] = {RV_REG_A3, RV_REG_A2},
> + [BPF_REG_3] = {RV_REG_A5, RV_REG_A4},
> + [BPF_REG_4] = {RV_REG_A7, RV_REG_A6},
> + [BPF_REG_5] = {RV_REG_S4, RV_REG_S3},
> + /* Callee saved registers that in-kernel function will preserve */
> + /* Stored on stack scratch space */
> + [BPF_REG_6] = {STACK_OFFSET(BPF_R6_HI), STACK_OFFSET(BPF_R6_LO)},
> + [BPF_REG_7] = {STACK_OFFSET(BPF_R7_HI), STACK_OFFSET(BPF_R7_LO)},
> + [BPF_REG_8] = {STACK_OFFSET(BPF_R8_HI), STACK_OFFSET(BPF_R8_LO)},
> + [BPF_REG_9] = {STACK_OFFSET(BPF_R9_HI), STACK_OFFSET(BPF_R9_LO)},
> + /* Read only Frame Pointer to access Stack */
> + [BPF_REG_FP] = {RV_REG_S6, RV_REG_S5},
> + /* Temporary Register for internal BPF JIT, can be used
> + * for constant blindings and others. Save T0 and T1
> + * for use internal to one instruction.
> + */
> + [TMP_REG_1] = {RV_REG_T3, RV_REG_T2},
> + [TMP_REG_2] = {RV_REG_T5, RV_REG_T4},
> + /* Tail call count. Stored on stack scratch space. */
> + [TCALL_CNT] = {STACK_OFFSET(BPF_TC_HI), STACK_OFFSET(BPF_TC_LO)},
> + /* Temporary register for blinding constants.
> + * Stored on stack scratch space.
> + */
> + [BPF_REG_AX] = {STACK_OFFSET(BPF_AX_HI), STACK_OFFSET(BPF_AX_LO)},
> +};
> +
> +struct rv_jit_context {
> + struct bpf_prog *prog;
> + u32 *insns; /* RV insns */
> + int ninsns;
> + int epilogue_offset;
> + int *offset; /* BPF to RV */
> + unsigned long flags;
> + int stack_size;
> +};
> +
> +struct rv_jit_data {
> + struct bpf_binary_header *header;
> + u8 *image;
> + struct rv_jit_context ctx;
> +};
> +
> +static void emit(const u32 insn, struct rv_jit_context *ctx)
> +{
> + if (ctx->insns)
> + ctx->insns[ctx->ninsns] = insn;
> +
> + ctx->ninsns++;
> +}
> +
> +static u32 rv_r_insn(u8 funct7, u8 rs2, u8 rs1, u8 funct3, u8 rd, u8 opcode)
> +{
> + return (funct7 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
> + (rd << 7) | opcode;
> +}
> +
> +static u32 rv_i_insn(u16 imm11_0, u8 rs1, u8 funct3, u8 rd, u8 opcode)
> +{
> + return (imm11_0 << 20) | (rs1 << 15) | (funct3 << 12) | (rd << 7) |
> + opcode;
> +}
> +
> +static u32 rv_s_insn(u16 imm11_0, u8 rs2, u8 rs1, u8 funct3, u8 opcode)
> +{
> + u8 imm11_5 = imm11_0 >> 5, imm4_0 = imm11_0 & 0x1f;
> +
> + return (imm11_5 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
> + (imm4_0 << 7) | opcode;
> +}
> +
> +static u32 rv_sb_insn(u16 imm12_1, u8 rs2, u8 rs1, u8 funct3, u8 opcode)
> +{
> + u8 imm12 = ((imm12_1 & 0x800) >> 5) | ((imm12_1 & 0x3f0) >> 4);
> + u8 imm4_1 = ((imm12_1 & 0xf) << 1) | ((imm12_1 & 0x400) >> 10);
> +
> + return (imm12 << 25) | (rs2 << 20) | (rs1 << 15) | (funct3 << 12) |
> + (imm4_1 << 7) | opcode;
> +}
> +
> +static u32 rv_u_insn(u32 imm31_12, u8 rd, u8 opcode)
> +{
> + return (imm31_12 << 12) | (rd << 7) | opcode;
> +}
> +
> +static u32 rv_uj_insn(u32 imm20_1, u8 rd, u8 opcode)
> +{
> + u32 imm;
> +
> + imm = (imm20_1 & 0x80000) | ((imm20_1 & 0x3ff) << 9) |
> + ((imm20_1 & 0x400) >> 2) | ((imm20_1 & 0x7f800) >> 11);
> +
> + return (imm << 12) | (rd << 7) | opcode;
> +}
> +
> +static u32 rv_amo_insn(u8 funct5, u8 aq, u8 rl, u8 rs2, u8 rs1,
> + u8 funct3, u8 rd, u8 opcode)
> +{
> + u8 funct7 = (funct5 << 2) | (aq << 1) | rl;
> +
> + return rv_r_insn(funct7, rs2, rs1, funct3, rd, opcode);
> +}
> +
> +static u32 rv_addi(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(imm11_0, rs1, 0, rd, 0x13);
> +}
> +
> +static u32 rv_add(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 0, rd, 0x33);
> +}
> +
> +static u32 rv_sub(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0x20, rs2, rs1, 0, rd, 0x33);
> +}
> +
> +static u32 rv_and(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 7, rd, 0x33);
> +}
> +
> +static u32 rv_or(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 6, rd, 0x33);
> +}
> +
> +static u32 rv_xor(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 4, rd, 0x33);
> +}
> +
> +static u32 rv_mul(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(1, rs2, rs1, 0, rd, 0x33);
> +}
> +
> +static u32 rv_mulhu(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(1, rs2, rs1, 3, rd, 0x33);
> +}
> +
> +static u32 rv_divu(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(1, rs2, rs1, 5, rd, 0x33);
> +}
> +
> +static u32 rv_remu(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(1, rs2, rs1, 7, rd, 0x33);
> +}
> +
> +static u32 rv_sll(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 1, rd, 0x33);
> +}
> +
> +static u32 rv_srl(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 5, rd, 0x33);
> +}
> +
> +static u32 rv_sra(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0x20, rs2, rs1, 5, rd, 0x33);
> +}
> +
> +static u32 rv_sltu(u8 rd, u8 rs1, u8 rs2)
> +{
> + return rv_r_insn(0, rs2, rs1, 3, rd, 0x33);
> +}
> +
> +static u32 rv_lui(u8 rd, u32 imm31_12)
> +{
> + return rv_u_insn(imm31_12, rd, 0x37);
> +}
> +
> +static u32 rv_slli(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(imm11_0, rs1, 1, rd, 0x13);
> +}
> +
> +static u32 rv_srli(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(imm11_0, rs1, 5, rd, 0x13);
> +}
> +
> +static u32 rv_srai(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(0x400 | imm11_0, rs1, 5, rd, 0x13);
> +}
> +
> +static u32 rv_andi(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(imm11_0, rs1, 7, rd, 0x13);
> +}
> +
> +static u32 rv_jal(u8 rd, u32 imm20_1)
> +{
> + return rv_uj_insn(imm20_1, rd, 0x6f);
> +}
> +
> +static u32 rv_jalr(u8 rd, u8 rs1, u16 imm11_0)
> +{
> + return rv_i_insn(imm11_0, rs1, 0, rd, 0x67);
> +}
> +
> +static u32 rv_beq(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 0, 0x63);
> +}
> +
> +static u32 rv_bltu(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 6, 0x63);
> +}
> +
> +static u32 rv_bgeu(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 7, 0x63);
> +}
> +
> +static u32 rv_bne(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 1, 0x63);
> +}
> +
> +static u32 rv_blt(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 4, 0x63);
> +}
> +
> +static u32 rv_bge(u8 rs1, u8 rs2, u16 imm12_1)
> +{
> + return rv_sb_insn(imm12_1, rs2, rs1, 5, 0x63);
> +}
> +
> +static u32 rv_sb(u8 rs1, u16 imm11_0, u8 rs2)
> +{
> + return rv_s_insn(imm11_0, rs2, rs1, 0, 0x23);
> +}
> +
> +static u32 rv_sh(u8 rs1, u16 imm11_0, u8 rs2)
> +{
> + return rv_s_insn(imm11_0, rs2, rs1, 1, 0x23);
> +}
> +
> +static u32 rv_sw(u8 rs1, u16 imm11_0, u8 rs2)
> +{
> + return rv_s_insn(imm11_0, rs2, rs1, 2, 0x23);
> +}
> +
> +static u32 rv_lbu(u8 rd, u16 imm11_0, u8 rs1)
> +{
> + return rv_i_insn(imm11_0, rs1, 4, rd, 0x03);
> +}
> +
> +static u32 rv_lhu(u8 rd, u16 imm11_0, u8 rs1)
> +{
> + return rv_i_insn(imm11_0, rs1, 5, rd, 0x03);
> +}
> +
> +static u32 rv_lw(u8 rd, u16 imm11_0, u8 rs1)
> +{
> + return rv_i_insn(imm11_0, rs1, 2, rd, 0x03);
> +}
> +
> +static u32 rv_amoadd_w(u8 rd, u8 rs2, u8 rs1, u8 aq, u8 rl)
> +{
> + return rv_amo_insn(0, aq, rl, rs2, rs1, 2, rd, 0x2f);
> +}
> +
> +static bool is_13b_int(s32 val)
> +{
> + return -(1 << 12) <= val && val < (1 << 12);
> +}
> +
> +static bool is_21b_int(s32 val)
> +{
> + return -(1L << 20) <= val && val < (1L << 20);
> +}
> +
> +static int is_13b_check(int off, int insn)
> +{
> + if (!is_13b_int(off)) {
> + pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
> + insn, off);
> + return -1;
> + }
> + return 0;
> +}
> +
> +static int is_21b_check(int off, int insn)
> +{
> + if (!is_21b_int(off)) {
> + pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
> + insn, off);
> + return -1;
> + }
> + return 0;
> +}
> +
> +static void emit_imm(const s8 rd, s32 imm, struct rv_jit_context *ctx)
> +{
> + u32 upper = (imm + (1 << 11)) >> 12;
> + u32 lower = imm & 0xfff;
> +
> + if (upper) {
> + emit(rv_lui(rd, upper), ctx);
> + emit(rv_addi(rd, rd, lower), ctx);
> + } else {
> + emit(rv_addi(rd, RV_REG_ZERO, lower), ctx);
> + }
> +}
> +
> +static void emit_imm32(const s8 *rd, s32 imm, struct rv_jit_context *ctx)
> +{
> + /* Emit immediate into lower bits */
> + emit_imm(rd[1], imm, ctx);
> +
> + /* Sign-extend into upper bits */
> + emit(rv_addi(rd[0], rd[1], 0), ctx);
> + emit(rv_srai(rd[0], rd[0], 31), ctx);
> +}
> +
> +static void emit_imm64(const s8 *rd, s32 imm_hi, s32 imm_lo,
> + struct rv_jit_context *ctx)
> +{
> + emit_imm(rd[1], imm_lo, ctx);
> + emit_imm(rd[0], imm_hi, ctx);
> +}
> +
> +static int rv_offset(int bpf_to, int bpf_from, struct rv_jit_context *ctx)
> +{
> + int from = ctx->offset[bpf_from] - 1, to = ctx->offset[bpf_to];
> +
> + return (to - from) << 2;
> +}
> +
> +static int epilogue_offset(struct rv_jit_context *ctx)
> +{
> + int to = ctx->epilogue_offset, from = ctx->ninsns;
> +
> + return (to - from) << 2;
> +}
> +
> +static void build_epilogue(struct rv_jit_context *ctx)
> +{
> + int stack_adjust = ctx->stack_size, store_offset = stack_adjust - 8;
> + const s8 *r0 = bpf2rv32[BPF_REG_0];
> +
> + store_offset -= 4 * BPF_JIT_SCRATCH_REGS;
> +
> + /* Set return value. */
> + emit(rv_addi(RV_REG_A0, r0[1], 0), ctx);
> + emit(rv_addi(RV_REG_A1, r0[0], 0), ctx);
> +
> + /* Restore callee-saved registers. */
> + emit(rv_lw(RV_REG_RA, store_offset - 0, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_FP, store_offset - 4, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S1, store_offset - 8, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S2, store_offset - 12, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S3, store_offset - 16, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S4, store_offset - 20, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S5, store_offset - 24, RV_REG_SP), ctx);
> + emit(rv_lw(RV_REG_S6, store_offset - 28, RV_REG_SP), ctx);
> +
> + emit(rv_addi(RV_REG_SP, RV_REG_SP, stack_adjust), ctx);
> + emit(rv_jalr(RV_REG_ZERO, RV_REG_RA, 0), ctx);
> +}
> +
> +static int rv_offset_check(int *rvoff, s16 off, int insn,
> + struct rv_jit_context *ctx)
> +{
> + *rvoff = rv_offset(insn + off, insn, ctx);
> + return is_13b_check(*rvoff, insn);
> +}
> +
> +static bool is_stacked(s8 reg)
> +{
> + return reg < 0;
> +}
> +
> +static const s8 *rv32_bpf_get_reg64(const s8 *reg, const s8 *tmp,
> + struct rv_jit_context *ctx)
> +{
> + if (is_stacked(reg[0])) {
> + emit(rv_lw(tmp[0], reg[0], RV_REG_FP), ctx);
> + emit(rv_lw(tmp[1], reg[1], RV_REG_FP), ctx);
> + reg = tmp;
> + }
> + return reg;
> +}
> +
> +static void rv32_bpf_put_reg64(const s8 *reg, const s8 *src,
> + struct rv_jit_context *ctx)
> +{
> + if (is_stacked(reg[0])) {
> + emit(rv_sw(RV_REG_FP, reg[0], src[0]), ctx);
> + emit(rv_sw(RV_REG_FP, reg[1], src[1]), ctx);
> + }
> +}
> +
> +static const s8 *rv32_bpf_get_reg32(const s8 *reg, const s8 *tmp,
> + struct rv_jit_context *ctx)
> +{
> + if (is_stacked(reg[1])) {
> + emit(rv_lw(tmp[1], reg[1], RV_REG_FP), ctx);
> + reg = tmp;
> + }
> + return reg;
> +}
> +
> +static void rv32_bpf_put_reg32(const s8 *reg, const s8 *src,
> + struct rv_jit_context *ctx)
> +{
> + if (is_stacked(reg[1])) {
> + emit(rv_sw(RV_REG_FP, reg[1], src[1]), ctx);
> + emit(rv_sw(RV_REG_FP, reg[0], RV_REG_ZERO), ctx);
> + } else {
> + emit(rv_addi(reg[0], RV_REG_ZERO, 0), ctx);
> + }
> +}
> +
> +static void emit_rv32_alu_i64(const s8 dst[], s32 imm,
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> +
> + switch (op) {
> + case BPF_MOV:
> + emit_imm32(rd, imm, ctx);
> + break;
> + }
> +
> + rv32_bpf_put_reg64(dst, rd, ctx);
> +}
> +
> +static void emit_rv32_alu_i32(const s8 dst[], s32 imm,
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *rd = rv32_bpf_get_reg32(dst, tmp1, ctx);
> +
> + switch (op) {
> + case BPF_MOV:
> + emit_imm32(rd, imm, ctx);
> + break;
> + }
> +
> + rv32_bpf_put_reg32(dst, rd, ctx);
> +}
> +
> +static void emit_rv32_alu_r64(const s8 dst[], const s8 src[],
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> + const s8 *rs = rv32_bpf_get_reg64(src, tmp2, ctx);
> +
> + switch (op) {
> + case BPF_MOV:
> + emit(rv_addi(rd[0], rs[0], 0), ctx);
> + emit(rv_addi(rd[1], rs[1], 0), ctx);
> + break;
> + case BPF_ADD:
> + emit(rv_addi(RV_REG_T0, rd[1], 0), ctx);
> + emit(rv_add(rd[1], rd[1], rs[1]), ctx);
> + emit(rv_sltu(RV_REG_T0, rd[1], RV_REG_T0), ctx);
> + emit(rv_add(rd[0], rd[0], rs[0]), ctx);
> + emit(rv_add(rd[0], rd[0], RV_REG_T0), ctx);
> + break;
> + case BPF_SUB:
> + emit(rv_addi(RV_REG_T0, rd[1], 0), ctx);
> + emit(rv_sub(rd[1], rd[1], rs[1]), ctx);
> + emit(rv_sltu(RV_REG_T0, RV_REG_T0, rd[1]), ctx);
> + emit(rv_sub(rd[0], rd[0], rs[0]), ctx);
> + emit(rv_sub(rd[0], rd[0], RV_REG_T0), ctx);
> + break;
> + case BPF_AND:
> + emit(rv_and(rd[0], rd[0], rs[0]), ctx);
> + emit(rv_and(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_OR:
> + emit(rv_or(rd[0], rd[0], rs[0]), ctx);
> + emit(rv_or(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_XOR:
> + emit(rv_xor(rd[0], rd[0], rs[0]), ctx);
> + emit(rv_xor(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_MUL:
> + emit(rv_mul(RV_REG_T0, rs[0], rd[1]), ctx);
> + emit(rv_mul(rd[0], rd[0], rs[1]), ctx);
> + emit(rv_mulhu(RV_REG_T1, rd[1], rs[1]), ctx);
> + emit(rv_add(rd[0], rd[0], RV_REG_T0), ctx);
> + emit(rv_mul(rd[1], rd[1], rs[1]), ctx);
> + emit(rv_add(rd[0], rd[0], RV_REG_T1), ctx);
> + break;
> + case BPF_LSH:
> + emit(rv_addi(RV_REG_T0, rs[1], -32), ctx);
> + emit(rv_blt(RV_REG_T0, RV_REG_ZERO, 16 >> 1), ctx);
> +
> + emit(rv_sll(rd[0], rd[1], RV_REG_T0), ctx);
> + emit(rv_addi(rd[1], RV_REG_ZERO, 0), ctx);
> + emit(rv_jal(RV_REG_ZERO, 32 >> 1), ctx);
> +
> + emit(rv_addi(RV_REG_T1, RV_REG_ZERO, 31), ctx);
> + emit(rv_srli(RV_REG_T0, rd[1], 1), ctx);
> + emit(rv_sub(RV_REG_T1, RV_REG_T1, rs[1]), ctx);
> + emit(rv_srl(RV_REG_T0, RV_REG_T0, RV_REG_T1), ctx);
> + emit(rv_sll(rd[0], rd[0], rs[1]), ctx);
> + emit(rv_or(rd[0], RV_REG_T0, rd[0]), ctx);
> + emit(rv_sll(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_RSH:
> + emit(rv_addi(RV_REG_T0, rs[1], -32), ctx);
> + emit(rv_blt(RV_REG_T0, RV_REG_ZERO, 16 >> 1), ctx);
> +
> + emit(rv_srl(rd[1], rd[0], RV_REG_T0), ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + emit(rv_jal(RV_REG_ZERO, 32 >> 1), ctx);
> +
> + emit(rv_addi(RV_REG_T1, RV_REG_ZERO, 31), ctx);
> + emit(rv_slli(RV_REG_T0, rd[0], 1), ctx);
> + emit(rv_sub(RV_REG_T1, RV_REG_T1, rs[1]), ctx);
> + emit(rv_sll(RV_REG_T0, RV_REG_T0, RV_REG_T1), ctx);
> + emit(rv_srl(rd[1], rd[1], rs[1]), ctx);
> + emit(rv_or(rd[1], RV_REG_T0, rd[1]), ctx);
> + emit(rv_srl(rd[0], rd[0], rs[1]), ctx);
> + break;
> + case BPF_ARSH:
> + emit(rv_addi(RV_REG_T0, rs[1], -32), ctx);
> + emit(rv_blt(RV_REG_T0, RV_REG_ZERO, 16 >> 1), ctx);
> +
> + emit(rv_sra(rd[1], rd[0], RV_REG_T0), ctx);
> + emit(rv_srai(rd[0], rd[0], 0x1f), ctx);
> + emit(rv_jal(RV_REG_ZERO, 32 >> 1), ctx);
> +
> + emit(rv_addi(RV_REG_T1, RV_REG_ZERO, 31), ctx);
> + emit(rv_slli(RV_REG_T0, rd[0], 1), ctx);
> + emit(rv_sub(RV_REG_T1, RV_REG_T1, rs[1]), ctx);
> + emit(rv_sll(RV_REG_T0, RV_REG_T0, RV_REG_T1), ctx);
> + emit(rv_srl(rd[1], rd[1], rs[1]), ctx);
> + emit(rv_or(rd[1], RV_REG_T0, rd[1]), ctx);
> + emit(rv_sra(rd[0], rd[0], rs[1]), ctx);
> + break;
> + case BPF_NEG:
> + emit(rv_sub(rd[1], RV_REG_ZERO, rd[1]), ctx);
> + emit(rv_sltu(RV_REG_T0, RV_REG_ZERO, rd[1]), ctx);
> + emit(rv_sub(rd[0], RV_REG_ZERO, rd[0]), ctx);
> + emit(rv_sub(rd[0], rd[0], RV_REG_T0), ctx);
> + break;
> + }
> +
> + rv32_bpf_put_reg64(dst, rd, ctx);
> +}
> +
> +static void emit_rv32_alu_r32(const s8 dst[], const s8 src[],
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rd = rv32_bpf_get_reg32(dst, tmp1, ctx);
> + const s8 *rs = rv32_bpf_get_reg32(src, tmp2, ctx);
> +
> + switch (op) {
> + case BPF_MOV:
> + emit(rv_addi(rd[1], rs[1], 0), ctx);
> + break;
> + case BPF_ADD:
> + emit(rv_add(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_SUB:
> + emit(rv_sub(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_AND:
> + emit(rv_and(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_OR:
> + emit(rv_or(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_XOR:
> + emit(rv_xor(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_MUL:
> + emit(rv_mul(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_DIV:
> + emit(rv_divu(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_MOD:
> + emit(rv_remu(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_LSH:
> + emit(rv_sll(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_RSH:
> + emit(rv_srl(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_ARSH:
> + emit(rv_sra(rd[1], rd[1], rs[1]), ctx);
> + break;
> + case BPF_NEG:
> + emit(rv_sub(rd[1], RV_REG_ZERO, rd[1]), ctx);
> + break;
> + }
> +
> + rv32_bpf_put_reg32(dst, rd, ctx);
> +}
> +
> +static int emit_rv32_jump_r64(const s8 src1[], const s8 src2[],
> + s16 off, int insn,
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + int rvoff;
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rs1 = rv32_bpf_get_reg64(src1, tmp1, ctx);
> + const s8 *rs2 = rv32_bpf_get_reg64(src2, tmp2, ctx);
> +
> + if (rv_offset_check(&rvoff, off, insn, ctx))
> + return -1;
> +
> + switch (op) {
> + case BPF_JEQ:
> + emit(rv_bne(rs1[0], rs2[0], 8 >> 1), ctx);
> + emit(rv_beq(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JNE:
> + emit(rv_bne(rs1[0], rs2[0], (rvoff + 4) >> 1), ctx);
> + emit(rv_bne(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JLE:
> + emit(rv_bltu(rs1[0], rs2[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bltu(rs2[0], rs1[0], 8 >> 1), ctx);
> + emit(rv_bgeu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JLT:
> + emit(rv_bltu(rs1[0], rs2[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bltu(rs2[0], rs1[0], 8 >> 1), ctx);
> + emit(rv_bltu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JGE:
> + emit(rv_bltu(rs2[0], rs1[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bltu(rs1[0], rs2[0], 8 >> 1), ctx);
> + emit(rv_bgeu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JGT:
> + emit(rv_bltu(rs2[0], rs1[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bltu(rs1[0], rs2[0], 8 >> 1), ctx);
> + emit(rv_bltu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSLE:
> + emit(rv_blt(rs1[0], rs2[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bne(rs2[0], rs1[0], 8 >> 1), ctx);
> + emit(rv_bgeu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSLT:
> + emit(rv_blt(rs1[0], rs2[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bne(rs2[0], rs1[0], 8 >> 1), ctx);
> + emit(rv_bltu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSGE:
> + emit(rv_blt(rs2[0], rs1[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bne(rs1[0], rs2[0], 8 >> 1), ctx);
> + emit(rv_bgeu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSGT:
> + emit(rv_blt(rs2[0], rs1[0], (rvoff + 8) >> 1), ctx);
> + emit(rv_bne(rs1[0], rs2[0], 8 >> 1), ctx);
> + emit(rv_bltu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSET:
> + emit(rv_and(RV_REG_T0, rs1[0], rs2[0]), ctx);
> + emit(rv_bne(RV_REG_T0, RV_REG_ZERO, (rvoff + 8) >> 1), ctx);
> + emit(rv_and(RV_REG_T0, rs1[1], rs2[1]), ctx);
> + emit(rv_bne(RV_REG_T0, RV_REG_ZERO, rvoff >> 1), ctx);
> + break;
> + }
> +
> + return 0;
> +}
> +
> +static int emit_rv32_jump_r32(const s8 src1[], const s8 src2[],
> + s16 off, int insn,
> + struct rv_jit_context *ctx,
> + const u8 op)
> +{
> + int rvoff;
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rs1 = rv32_bpf_get_reg32(src1, tmp1, ctx);
> + const s8 *rs2 = rv32_bpf_get_reg32(src2, tmp2, ctx);
> +
> + if (rv_offset_check(&rvoff, off, insn, ctx))
> + return -1;
> +
> + switch (op) {
> + case BPF_JEQ:
> + emit(rv_beq(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JNE:
> + emit(rv_bne(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JLE:
> + emit(rv_bgeu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JLT:
> + emit(rv_bltu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JGE:
> + emit(rv_bgeu(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JGT:
> + emit(rv_bltu(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSLE:
> + emit(rv_bge(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSLT:
> + emit(rv_blt(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSGE:
> + emit(rv_bge(rs1[1], rs2[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSGT:
> + emit(rv_blt(rs2[1], rs1[1], rvoff >> 1), ctx);
> + break;
> + case BPF_JSET:
> + emit(rv_and(RV_REG_T0, rs1[1], rs2[1]), ctx);
> + emit(rv_bne(RV_REG_T0, RV_REG_ZERO, rvoff >> 1), ctx);
> + break;
> + }
> +
> + return 0;
> +}
> +
> +static int emit_rv32_load_r64(const s8 dst[], const s8 src[],
> + s16 off,
> + struct rv_jit_context *ctx,
> + const u8 size)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> + const s8 *rs = rv32_bpf_get_reg64(src, tmp2, ctx);
> +
> + emit_imm(RV_REG_T0, off, ctx);
> + emit(rv_add(RV_REG_T0, RV_REG_T0, rs[1]), ctx);
> +
> + switch (size) {
> + case BPF_B:
> + emit(rv_lbu(rd[1], 0, RV_REG_T0), ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case BPF_H:
> + emit(rv_lhu(rd[1], 0, RV_REG_T0), ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case BPF_W:
> + emit(rv_lw(rd[1], 0, RV_REG_T0), ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case BPF_DW:
> + emit(rv_lw(rd[1], 0, RV_REG_T0), ctx);
> + emit(rv_lw(rd[0], 4, RV_REG_T0), ctx);
> + break;
> + }
> +
> + rv32_bpf_put_reg64(dst, rd, ctx);
> + return 0;
> +}
> +
> +static int emit_rv32_store_r64(const s8 dst[], const s8 src[],
> + s16 off,
> + struct rv_jit_context *ctx,
> + const u8 size, const u8 mode)
> +{
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> + const s8 *rs = rv32_bpf_get_reg64(src, tmp2, ctx);
> +
> + if (mode == BPF_XADD && size != BPF_W)
> + return -1;
> +
> + emit_imm(RV_REG_T0, off, ctx);
> + emit(rv_add(RV_REG_T0, RV_REG_T0, rd[1]), ctx);
> +
> + switch (size) {
> + case BPF_B:
> + emit(rv_sb(RV_REG_T0, 0, rs[1]), ctx);
> + break;
> + case BPF_H:
> + emit(rv_sh(RV_REG_T0, 0, rs[1]), ctx);
> + break;
> + case BPF_W:
> + switch (mode) {
> + case BPF_MEM:
> + emit(rv_sw(RV_REG_T0, 0, rs[1]), ctx);
> + break;
> + case BPF_XADD:
> + emit(rv_amoadd_w(RV_REG_ZERO, rs[1], RV_REG_T0, 0, 0), ctx);
> + break;
> + }
> + break;
> + case BPF_DW:
> + emit(rv_sw(RV_REG_T0, 0, rs[1]), ctx);
> + emit(rv_sw(RV_REG_T0, 4, rs[0]), ctx);
> + break;
> + }
> +
> + return 0;
> +}
> +
> +static void emit_rv32_rev16(const s8 rd, struct rv_jit_context *ctx)
> +{
> + emit(rv_addi(RV_REG_T1, RV_REG_ZERO, 0), ctx);
> +
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> + emit(rv_slli(RV_REG_T1, RV_REG_T1, 8), ctx);
> + emit(rv_srli(rd, rd, 8), ctx);
> +
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> +
> + emit(rv_addi(rd, RV_REG_T1, 0), ctx);
> +}
> +
> +static void emit_rv32_rev32(const s8 rd, struct rv_jit_context *ctx)
> +{
> + emit(rv_addi(RV_REG_T1, RV_REG_ZERO, 0), ctx);
> +
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> + emit(rv_slli(RV_REG_T1, RV_REG_T1, 8), ctx);
> + emit(rv_srli(rd, rd, 8), ctx);
> +
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> + emit(rv_slli(RV_REG_T1, RV_REG_T1, 8), ctx);
> + emit(rv_srli(rd, rd, 8), ctx);
> +
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> + emit(rv_slli(RV_REG_T1, RV_REG_T1, 8), ctx);
> + emit(rv_srli(rd, rd, 8), ctx);
> + emit(rv_andi(RV_REG_T0, rd, 0xff), ctx);
> + emit(rv_add(RV_REG_T1, RV_REG_T1, RV_REG_T0), ctx);
> +
> + emit(rv_addi(rd, RV_REG_T1, 0), ctx);
> +}
> +
> +static int emit_insn(const struct bpf_insn *insn,
> + struct rv_jit_context *ctx,
> + bool extra_pass)
> +{
> + int rvoff, i = insn - ctx->prog->insnsi;
> + u8 code = insn->code;
> + s16 off = insn->off;
> + s32 imm = insn->imm;
> +
> + const s8 *dst = bpf2rv32[insn->dst_reg];
> + const s8 *src = bpf2rv32[insn->src_reg];
> + const s8 *tmp1 = bpf2rv32[TMP_REG_1];
> + const s8 *tmp2 = bpf2rv32[TMP_REG_2];
> +
> + switch (code) {
> + case BPF_ALU64 | BPF_MOV | BPF_X:
> +
> + case BPF_ALU64 | BPF_ADD | BPF_X:
> + case BPF_ALU64 | BPF_ADD | BPF_K:
> +
> + case BPF_ALU64 | BPF_SUB | BPF_X:
> + case BPF_ALU64 | BPF_SUB | BPF_K:
> +
> + case BPF_ALU64 | BPF_AND | BPF_X:
> + case BPF_ALU64 | BPF_AND | BPF_K:
> +
> + case BPF_ALU64 | BPF_OR | BPF_X:
> + case BPF_ALU64 | BPF_OR | BPF_K:
> +
> + case BPF_ALU64 | BPF_XOR | BPF_X:
> + case BPF_ALU64 | BPF_XOR | BPF_K:
> +
> + case BPF_ALU64 | BPF_MUL | BPF_X:
> + case BPF_ALU64 | BPF_MUL | BPF_K:
> +
> + case BPF_ALU64 | BPF_LSH | BPF_X:
> + case BPF_ALU64 | BPF_LSH | BPF_K:
> +
> + case BPF_ALU64 | BPF_RSH | BPF_X:
> + case BPF_ALU64 | BPF_RSH | BPF_K:
> +
> + case BPF_ALU64 | BPF_ARSH | BPF_X:
> + case BPF_ALU64 | BPF_ARSH | BPF_K:
> + if (BPF_SRC(code) == BPF_K) {
> + emit_imm32(tmp2, imm, ctx);
> + src = tmp2;
> + }
> + emit_rv32_alu_r64(dst, src, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU64 | BPF_NEG:
> + emit_rv32_alu_r64(dst, tmp2, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU64 | BPF_DIV | BPF_X:
> + case BPF_ALU64 | BPF_DIV | BPF_K:
> + case BPF_ALU64 | BPF_MOD | BPF_X:
> + case BPF_ALU64 | BPF_MOD | BPF_K:
> + goto notsupported;
> +
> + case BPF_ALU64 | BPF_MOV | BPF_K:
> + emit_rv32_alu_i64(dst, imm, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU | BPF_MOV | BPF_X:
> +
> + case BPF_ALU | BPF_ADD | BPF_X:
> + case BPF_ALU | BPF_ADD | BPF_K:
> +
> + case BPF_ALU | BPF_SUB | BPF_X:
> + case BPF_ALU | BPF_SUB | BPF_K:
> +
> + case BPF_ALU | BPF_AND | BPF_X:
> + case BPF_ALU | BPF_AND | BPF_K:
> +
> + case BPF_ALU | BPF_OR | BPF_X:
> + case BPF_ALU | BPF_OR | BPF_K:
> +
> + case BPF_ALU | BPF_XOR | BPF_X:
> + case BPF_ALU | BPF_XOR | BPF_K:
> +
> + case BPF_ALU | BPF_MUL | BPF_X:
> + case BPF_ALU | BPF_MUL | BPF_K:
> +
> + case BPF_ALU | BPF_DIV | BPF_X:
> + case BPF_ALU | BPF_DIV | BPF_K:
> +
> + case BPF_ALU | BPF_MOD | BPF_X:
> + case BPF_ALU | BPF_MOD | BPF_K:
> +
> + case BPF_ALU | BPF_LSH | BPF_X:
> + case BPF_ALU | BPF_LSH | BPF_K:
> +
> + case BPF_ALU | BPF_RSH | BPF_X:
> + case BPF_ALU | BPF_RSH | BPF_K:
> +
> + case BPF_ALU | BPF_ARSH | BPF_X:
> + case BPF_ALU | BPF_ARSH | BPF_K:
> + if (BPF_SRC(code) == BPF_K) {
> + emit_imm32(tmp2, imm, ctx);
> + src = tmp2;
> + }
> + emit_rv32_alu_r32(dst, src, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU | BPF_MOV | BPF_K:
> + emit_rv32_alu_i32(dst, imm, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU | BPF_NEG:
> + /* src is ignored---choose a register known not to be stacked */
> + emit_rv32_alu_r32(dst, tmp2, ctx, BPF_OP(code));
> + break;
> +
> + case BPF_ALU | BPF_END | BPF_FROM_LE:
> + {
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> +
> + switch (imm) {
> + case 16:
> + emit(rv_slli(rd[1], rd[1], 16), ctx);
> + emit(rv_srli(rd[1], rd[1], 16), ctx);
> + /* Fallthrough to clear high bits. */
> + case 32:
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case 64:
> + /* Do nothing. */
> + break;
> + default:
> + pr_err("bpf-jit: BPF_END imm %d invalid\n", imm);
> + return -1;
> + }
> +
> + rv32_bpf_put_reg64(dst, rd, ctx);
> + break;
> + }
> +
> + case BPF_ALU | BPF_END | BPF_FROM_BE:
> + {
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> +
> + switch (imm) {
> + case 16:
> + emit_rv32_rev16(rd[1], ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case 32:
> + emit_rv32_rev32(rd[1], ctx);
> + emit(rv_addi(rd[0], RV_REG_ZERO, 0), ctx);
> + break;
> + case 64:
> + /* Swap upper and lower halves. */
> + emit(rv_addi(RV_REG_T0, rd[1], 0), ctx);
> + emit(rv_addi(rd[1], rd[0], 0), ctx);
> + emit(rv_addi(rd[0], RV_REG_T0, 0), ctx);
> +
> + /* Swap each half. */
> + emit_rv32_rev32(rd[1], ctx);
> + emit_rv32_rev32(rd[0], ctx);
> + break;
> + default:
> + pr_err("bpf-jit: BPF_END imm %d invalid\n", imm);
> + return -1;
> + }
> +
> + rv32_bpf_put_reg64(dst, rd, ctx);
> + break;
> + }
> +
> + case BPF_JMP | BPF_JA:
> + rvoff = rv_offset(i + off, i, ctx);
> + if (!is_21b_int(rvoff)) {
> + pr_err("bpf-jit: insn=%d offset=%d not supported yet!\n",
> + i, rvoff);
> + return -1;
> + }
> + emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx);
> + break;
> +
> + case BPF_JMP | BPF_CALL:
> + {
> + bool fixed;
> + int ret;
> + u64 addr;
> + const s8 *r0 = bpf2rv32[BPF_REG_0];
> + const s8 *r5 = bpf2rv32[BPF_REG_5];
> +
> + ret = bpf_jit_get_func_addr(ctx->prog, insn, extra_pass, &addr,
> + &fixed);
> + if (ret < 0)
> + return ret;
> + if (fixed) {
> + emit_imm(RV_REG_T0, (u32) addr, ctx);
> + } else {
> + pr_err("bpf-jit: pseudocall call not supported yet!\n");
> + return -1;
> + }
> +
> + /* R1-R4 already in correct reigsters---need to push R5 to stack */
> + emit(rv_addi(RV_REG_SP, RV_REG_SP, -8), ctx);
> + emit(rv_sw(RV_REG_SP, 0, r5[1]), ctx);
> + emit(rv_sw(RV_REG_SP, 4, r5[0]), ctx);
> +
> + emit(rv_jalr(RV_REG_RA, RV_REG_T0, 0), ctx);
> +
> + /* Set return value */
> + emit(rv_addi(r0[1], RV_REG_A0, 0), ctx);
> + emit(rv_addi(r0[0], RV_REG_A1, 0), ctx);
> + emit(rv_addi(RV_REG_SP, RV_REG_SP, 8), ctx);
> + break;
> + }
> +
> + case BPF_JMP | BPF_JEQ | BPF_X:
> + case BPF_JMP | BPF_JEQ | BPF_K:
> + case BPF_JMP32 | BPF_JEQ | BPF_X:
> + case BPF_JMP32 | BPF_JEQ | BPF_K:
> +
> + case BPF_JMP | BPF_JNE | BPF_X:
> + case BPF_JMP | BPF_JNE | BPF_K:
> + case BPF_JMP32 | BPF_JNE | BPF_X:
> + case BPF_JMP32 | BPF_JNE | BPF_K:
> +
> + case BPF_JMP | BPF_JLE | BPF_X:
> + case BPF_JMP | BPF_JLE | BPF_K:
> + case BPF_JMP32 | BPF_JLE | BPF_X:
> + case BPF_JMP32 | BPF_JLE | BPF_K:
> +
> + case BPF_JMP | BPF_JLT | BPF_X:
> + case BPF_JMP | BPF_JLT | BPF_K:
> + case BPF_JMP32 | BPF_JLT | BPF_X:
> + case BPF_JMP32 | BPF_JLT | BPF_K:
> +
> + case BPF_JMP | BPF_JGE | BPF_X:
> + case BPF_JMP | BPF_JGE | BPF_K:
> + case BPF_JMP32 | BPF_JGE | BPF_X:
> + case BPF_JMP32 | BPF_JGE | BPF_K:
> +
> + case BPF_JMP | BPF_JGT | BPF_X:
> + case BPF_JMP | BPF_JGT | BPF_K:
> + case BPF_JMP32 | BPF_JGT | BPF_X:
> + case BPF_JMP32 | BPF_JGT | BPF_K:
> +
> + case BPF_JMP | BPF_JSLE | BPF_X:
> + case BPF_JMP | BPF_JSLE | BPF_K:
> + case BPF_JMP32 | BPF_JSLE | BPF_X:
> + case BPF_JMP32 | BPF_JSLE | BPF_K:
> +
> + case BPF_JMP | BPF_JSLT | BPF_X:
> + case BPF_JMP | BPF_JSLT | BPF_K:
> + case BPF_JMP32 | BPF_JSLT | BPF_X:
> + case BPF_JMP32 | BPF_JSLT | BPF_K:
> +
> + case BPF_JMP | BPF_JSGE | BPF_X:
> + case BPF_JMP | BPF_JSGE | BPF_K:
> + case BPF_JMP32 | BPF_JSGE | BPF_X:
> + case BPF_JMP32 | BPF_JSGE | BPF_K:
> +
> + case BPF_JMP | BPF_JSGT | BPF_X:
> + case BPF_JMP | BPF_JSGT | BPF_K:
> + case BPF_JMP32 | BPF_JSGT | BPF_X:
> + case BPF_JMP32 | BPF_JSGT | BPF_K:
> +
> + case BPF_JMP | BPF_JSET | BPF_X:
> + case BPF_JMP | BPF_JSET | BPF_K:
> + case BPF_JMP32 | BPF_JSET | BPF_X:
> + case BPF_JMP32 | BPF_JSET | BPF_K:
> + if (BPF_SRC(code) == BPF_K) {
> + emit_imm32(tmp2, imm, ctx);
> + src = tmp2;
> + }
> + switch (BPF_CLASS(code)) {
> + case BPF_JMP:
> + if (emit_rv32_jump_r64(dst, src, off, i, ctx, BPF_OP(code)))
> + return -1;
> + break;
> + case BPF_JMP32:
> + if (emit_rv32_jump_r32(dst, src, off, i, ctx, BPF_OP(code)))
> + return -1;
> + break;
> + }
> + break;
> +
> + case BPF_JMP | BPF_EXIT:
> + if (i == ctx->prog->len - 1)
> + break;
> +
> + rvoff = epilogue_offset(ctx);
> + if (is_21b_check(rvoff, i))
> + return -1;
> + emit(rv_jal(RV_REG_ZERO, rvoff >> 1), ctx);
> + break;
> +
> + case BPF_LD | BPF_IMM | BPF_DW:
> + {
> + struct bpf_insn insn1 = insn[1];
> + s32 imm_lo = imm;
> + s32 imm_hi = insn1.imm;
> + const s8 *rd = rv32_bpf_get_reg64(dst, tmp1, ctx);
> +
> + emit_imm64(rd, imm_hi, imm_lo, ctx);
> + rv32_bpf_put_reg64(dst, rd, ctx);
> + return 1;
> + }
> +
> + case BPF_LDX | BPF_MEM | BPF_B:
> + case BPF_LDX | BPF_MEM | BPF_H:
> + case BPF_LDX | BPF_MEM | BPF_W:
> + case BPF_LDX | BPF_MEM | BPF_DW:
> + if (emit_rv32_load_r64(dst, src, off, ctx, BPF_SIZE(code)))
> + return -1;
> + break;
> +
> +
> + case BPF_ST | BPF_MEM | BPF_B:
> + case BPF_STX | BPF_MEM | BPF_B:
> + case BPF_ST | BPF_MEM | BPF_H:
> + case BPF_STX | BPF_MEM | BPF_H:
> + case BPF_ST | BPF_MEM | BPF_W:
> + case BPF_STX | BPF_MEM | BPF_W:
> + case BPF_ST | BPF_MEM | BPF_DW:
> + case BPF_STX | BPF_MEM | BPF_DW:
> +
> + case BPF_STX | BPF_XADD | BPF_W:
> + if (BPF_CLASS(code) == BPF_ST) {
> + emit_imm32(tmp2, imm, ctx);
> + src = tmp2;
> + }
> +
> + if (emit_rv32_store_r64(dst, src, off, ctx, BPF_SIZE(code), BPF_MODE(code)))
> + return -1;
> + break;
> +
> + case BPF_STX | BPF_XADD | BPF_DW:
> + goto notsupported;
> +
> +notsupported:
> + pr_info_once("*** NOT SUPPORTED: opcode %02x ***\n", code);
> + return -EFAULT;
> +
> + default:
> + pr_err("bpf-jit: unknown opcode %02x\n", code);
> + return -EINVAL;
> + }
> +
> + return 0;
> +}
> +
> +static void build_prologue(struct rv_jit_context *ctx)
> +{
> + int stack_adjust = 32, store_offset, bpf_stack_adjust;
> +
> + stack_adjust = round_up(stack_adjust, 16);
> + bpf_stack_adjust = round_up(ctx->prog->aux->stack_depth, 16);
> + stack_adjust += bpf_stack_adjust;
> +
> + store_offset = stack_adjust - 8;
> +
> + stack_adjust += 4 * BPF_JIT_SCRATCH_REGS;
> +
> + emit(rv_addi(RV_REG_SP, RV_REG_SP, -stack_adjust), ctx);
> +
> + /* Save callee-save registers */
> + emit(rv_sw(RV_REG_SP, store_offset - 0, RV_REG_RA), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 4, RV_REG_FP), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 8, RV_REG_S1), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 12, RV_REG_S2), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 16, RV_REG_S3), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 20, RV_REG_S4), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 24, RV_REG_S5), ctx);
> + emit(rv_sw(RV_REG_SP, store_offset - 28, RV_REG_S6), ctx);
> +
> + emit(rv_addi(RV_REG_FP, RV_REG_SP, stack_adjust), ctx);
> +
> + /* Set up BPF stack pointer */
> + emit(rv_addi(bpf2rv32[BPF_REG_FP][1], RV_REG_SP, bpf_stack_adjust), ctx);
> + emit(rv_addi(bpf2rv32[BPF_REG_FP][0], RV_REG_ZERO, 0), ctx);
> +
> + /* Set up context pointer */
> + emit(rv_addi(bpf2rv32[BPF_REG_1][1], RV_REG_A0, 0), ctx);
> + emit(rv_addi(bpf2rv32[BPF_REG_1][0], RV_REG_ZERO, 0), ctx);
> +
> + ctx->stack_size = stack_adjust;
> +}
> +
> +static int build_body(struct rv_jit_context *ctx, bool extra_pass)
> +{
> + const struct bpf_prog *prog = ctx->prog;
> + int i;
> +
> + for (i = 0; i < prog->len; i++) {
> + const struct bpf_insn *insn = &prog->insnsi[i];
> + int ret;
> +
> + ret = emit_insn(insn, ctx, extra_pass);
> + if (ret > 0) {
> + i++;
> + if (ctx->insns == NULL)
> + ctx->offset[i] = ctx->ninsns;
> + continue;
> + }
> + if (ctx->insns == NULL)
> + ctx->offset[i] = ctx->ninsns;
> + if (ret)
> + return ret;
> + }
> + return 0;
> +}
> +
> +static void bpf_fill_ill_insns(void *area, unsigned int size)
> +{
> + memset(area, 0, size);
> +}
> +
> +static void bpf_flush_icache(void *start, void *end)
> +{
> + flush_icache_range((unsigned long)start, (unsigned long)end);
> +}
> +
> +struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
> +{
> + bool tmp_blinded = false, extra_pass = false;
> + struct bpf_prog *tmp, *orig_prog = prog;
> + struct rv_jit_data *jit_data;
> + struct rv_jit_context *ctx;
> + unsigned int image_size;
> +
> + if (!prog->jit_requested)
> + return orig_prog;
> +
> + tmp = bpf_jit_blind_constants(prog);
> + if (IS_ERR(tmp))
> + return orig_prog;
> + if (tmp != prog) {
> + tmp_blinded = true;
> + prog = tmp;
> + }
> +
> + jit_data = prog->aux->jit_data;
> + if (!jit_data) {
> + jit_data = kzalloc(sizeof(*jit_data), GFP_KERNEL);
> + if (!jit_data) {
> + prog = orig_prog;
> + goto out;
> + }
> + prog->aux->jit_data = jit_data;
> + }
> +
> + ctx = &jit_data->ctx;
> +
> + if (ctx->offset) {
> + extra_pass = true;
> + image_size = sizeof(u32) * ctx->ninsns;
> + goto skip_init_ctx;
> + }
> +
> + ctx->prog = prog;
> + ctx->offset = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
> + if (!ctx->offset) {
> + prog = orig_prog;
> + goto out_offset;
> + }
> +
> + /* First pass generates the ctx->offset, but does not emit an image. */
> + if (build_body(ctx, extra_pass)) {
> + prog = orig_prog;
> + goto out_offset;
> + }
> + build_prologue(ctx);
> + ctx->epilogue_offset = ctx->ninsns;
> + build_epilogue(ctx);
> +
> + /* Allocate image, now that we know the size. */
> + image_size = sizeof(u32) * ctx->ninsns;
> + jit_data->header = bpf_jit_binary_alloc(image_size, &jit_data->image,
> + sizeof(u32),
> + bpf_fill_ill_insns);
> + if (!jit_data->header) {
> + prog = orig_prog;
> + goto out_offset;
> + }
> +
> + /* Second, real pass, that acutally emits the image. */
> + ctx->insns = (u32 *)jit_data->image;
> +skip_init_ctx:
> + ctx->ninsns = 0;
> +
> + build_prologue(ctx);
> + if (build_body(ctx, extra_pass)) {
> + bpf_jit_binary_free(jit_data->header);
> + prog = orig_prog;
> + goto out_offset;
> + }
> + build_epilogue(ctx);
> +
> + if (bpf_jit_enable > 1)
> + bpf_jit_dump(prog->len, image_size, 2, ctx->insns);
> +
> + prog->bpf_func = (void *)ctx->insns;
> + prog->jited = 1;
> + prog->jited_len = image_size;
> +
> + bpf_flush_icache(jit_data->header, ctx->insns + ctx->ninsns);
> +
> + if (!prog->is_func || extra_pass) {
> +out_offset:
> + kfree(ctx->offset);
> + kfree(jit_data);
> + prog->aux->jit_data = NULL;
> + }
> +out:
> + if (tmp_blinded)
> + bpf_jit_prog_release_other(prog, prog == orig_prog ?
> + tmp : orig_prog);
> + return prog;
> +}
> --
> 2.20.1
>
^ permalink raw reply
* RE: [PATCH v4 2/5] net: macb: add support for sgmii MAC-PHY interface
From: Parshuram Raju Thombare @ 2019-06-24 10:14 UTC (permalink / raw)
To: Russell King - ARM Linux admin
Cc: andrew@lunn.ch, nicolas.ferre@microchip.com, davem@davemloft.net,
f.fainelli@gmail.com, netdev@vger.kernel.org,
hkallweit1@gmail.com, linux-kernel@vger.kernel.org,
Rafal Ciepiela, Anil Joy Varughese, Piotr Sroka
In-Reply-To: <20190624093533.4vhvjmqqrucq2ixf@shell.armlinux.org.uk>
>> >I still don't think this makes much sense, splitting the interface
>> > configuration between here and below.
>> Do you mean splitting mac_config in two *_configure functions ?
>> This was done as per Andrew's suggestion to make code mode readable
>> and easy to manage by splitting MAC configuration for different interfaces.
>No, I mean here you disable SGMII if we're switching away from SGMII
>mode.... (note, this means there is more to come for this sentence)
Sorry, I misunderstood your original question. I think disabling old interface
and enabling new one can be done in single place. I will do this change.
>> >This will only be executed when we are not using inband mode, which
>> >basically means it's not possible to switch to SGMII in-band mode.
>> SGMII is used in default PHY mode. And above code is to program MAC to
>> select PCS and SGMII interface.
>... and here you enable it for SGMII mode, but only for non-inband
>modes.
>
>Why not:
> if (change_interface) {
> if (state->interface == PHY_INTERFACE_MODE_SGMII) {
> // Enable SGMII mode and PCS
> gem_writel(bp, NCFGR, ncfgr | GEM_BIT(SGMIIEN) |
> GEM_BIT(PCSSEL));
> } else {
> // Disable SGMII mode and PCS
> gem_writel(bp, NCFGR, ncfgr & ~(GEM_BIT(SGMIIEN)
> GEM_BIT(PCSSEL)));
> // Reset PCS
> gem_writel(bp, PCS_CTRL, gem_readl(bp, PCS_CTRL)
> GEM_BIT(PCS_CTRL_RST));
> }
> }
> if (!phylink_autoneg_inband(mode) &&
> (bp->speed != state->speed || bp->duplex != state->duplex)) {
>?
Ok
>> >> +
>> >> + if (!interface_supported) {
>> >> + netdev_err(dev, "Phy mode %s not supported",
>> >> + phy_modes(phy_mode));
>> >> + goto err_out_free_netdev;
>> >> + }
>> >> +
>> >> bp->phy_interface = phy_mode;
>> >> + } else {
>> >> + bp->phy_interface = phy_mode;
>> >> + }
>> >If bp->phy_interface is PHY_INTERFACE_MODE_SGMII here, and
>> > mac_config()
>> >is called with state->interface = PHY_INTERFACE_MODE_SGMII, then
>> >mac_config() won't configure the MAC for the interface type - is that
>> >intentional?
>> In mac_config configure MAC for non in-band mode, there is also check for
>> speed, duplex
>> changes. bp->speed and bp->duplex are initialized to SPEED_UNKNOWN
>> and DUPLEX_UNKNOWN
>> values so it is expected that for non in band mode state contains valid speed
>> and duplex mode
>> which are different from *_UNKNOWN values.
>Sorry, this reply doesn't answer my question. I'm not asking about
>bp->speed and bp->duplex. I'm asking:
>1) why you are initialising bp->phy_interface here
>2) you to consider the impact that has on the mac_config() implementation
> you are proposing
> because I think it's buggy.
bp->phy_interface is to store phy mode value from device tree. This is used later
to know what phy interface user has selected for PHY-MAC. Same is used
to configure MAC correctly and based on your suggestion code is
added to handle PHY dynamically changing phy interface, in which
case bp->phy_interface is also updated. Though it may not be what user want,
if phy interface is totally decided by PHY and is anyway going to be different from what user
has selected in DT, initializing it here doesn't make sense.
But in case of PHY not changing phy_interface dynamically bp->phy_interface need to be
initialized with value from DT.
Regards,
Parshuram Thombare
^ permalink raw reply
* Re: [PATCH bpf-next] libbpf: add xsk_ring_prod__free() function
From: Magnus Karlsson @ 2019-06-24 10:15 UTC (permalink / raw)
To: Eelco Chaudron
Cc: Andrii Nakryiko, Karlsson, Magnus, Networking, Alexei Starovoitov,
Daniel Borkmann, Martin Lau, Song Liu, Yonghong Song
In-Reply-To: <1C59E98E-7F4B-4FCA-AB95-68D3819C489C@redhat.com>
On Mon, Jun 24, 2019 at 11:53 AM Eelco Chaudron <echaudro@redhat.com> wrote:
>
>
>
> On 21 Jun 2019, at 21:13, Andrii Nakryiko wrote:
>
> > On Fri, Jun 21, 2019 at 8:26 AM Eelco Chaudron <echaudro@redhat.com>
> > wrote:
> >>
> >> When an AF_XDP application received X packets, it does not mean X
> >> frames can be stuffed into the producer ring. To make it easier for
> >> AF_XDP applications this API allows them to check how many frames can
> >> be added into the ring.
> >>
> >> Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
> >> ---
> >> tools/lib/bpf/xsk.h | 6 ++++++
> >> 1 file changed, 6 insertions(+)
> >>
> >> diff --git a/tools/lib/bpf/xsk.h b/tools/lib/bpf/xsk.h
> >> index 82ea71a0f3ec..86f3d485e957 100644
> >> --- a/tools/lib/bpf/xsk.h
> >> +++ b/tools/lib/bpf/xsk.h
> >> @@ -95,6 +95,12 @@ static inline __u32 xsk_prod_nb_free(struct
> >> xsk_ring_prod *r, __u32 nb)
> >> return r->cached_cons - r->cached_prod;
> >> }
> >>
> >> +static inline __u32 xsk_ring_prod__free(struct xsk_ring_prod *r)
> >
> > This is a very bad name choice. __free is used for functions that free
> > memory and resources. One function below I see avail is used in the
> > name, why not xsk_ring_prog__avail?
>
> Must agree that free sound like you are freeing entries… However, I
> just kept the naming already in the API/file (see above,
> xsk_prod_nb_free()).
> Reading the code there is a difference as xx_avail() means available
> filled entries, where xx_free() means available free entries.
>
> So I could rename it to xsk_ring_prod__nb_free() maybe?
xsk_ring_prod__nb_free() is fine with me. In truth, Andrii's
suggestion is fine too since the number of available entries from the
producer point of view is the number of free entries I can put stuff
in.
Your function is expensive though since it always touches global
state. I think it would be better to expose the xsk_prod_nb_free()
function as is, but with this new name. Then users can say how many
entries they want maximum and avoid touching global state when not
needed. You would also have to change all the functions that use
xsk_prod_nb_free, so it uses you new function. What do you think?
/Magnus
> Forgot to include Magnus in the email, so copied him in, for some
> comments.
>
> >> +{
> >> + r->cached_cons = *r->consumer + r->size;
> >> + return r->cached_cons - r->cached_prod;
> >> +}
> >> +
> >> static inline __u32 xsk_cons_nb_avail(struct xsk_ring_cons *r, __u32
> >> nb)
> >> {
> >> __u32 entries = r->cached_prod - r->cached_cons;
> >> --
> >> 2.20.1
> >>
^ permalink raw reply
* RE: [PATCH v4 3/5] net: macb: add support for c45 PHY
From: Parshuram Raju Thombare @ 2019-06-24 10:19 UTC (permalink / raw)
To: Russell King - ARM Linux admin
Cc: andrew@lunn.ch, nicolas.ferre@microchip.com, davem@davemloft.net,
f.fainelli@gmail.com, netdev@vger.kernel.org,
hkallweit1@gmail.com, linux-kernel@vger.kernel.org,
Rafal Ciepiela, Anil Joy Varughese, Piotr Sroka
In-Reply-To: <20190624094233.3xick3snqbcm55gu@shell.armlinux.org.uk>
>However, it seems from that comment that you're not talking about real
>hardware. Is there no real hardware out there supporting 10G mode with
>these proposed driver changes yet?
I think there are some 10GBaseT PHY out there, but I don't have any test
setup with those. This patch is tested on emulation test setup.
Regards,
Parshuram Thombare
^ permalink raw reply
* Re: [PATCH v4 2/5] net: macb: add support for sgmii MAC-PHY interface
From: Russell King - ARM Linux admin @ 2019-06-24 10:22 UTC (permalink / raw)
To: Parshuram Raju Thombare
Cc: andrew@lunn.ch, nicolas.ferre@microchip.com, davem@davemloft.net,
f.fainelli@gmail.com, netdev@vger.kernel.org,
hkallweit1@gmail.com, linux-kernel@vger.kernel.org,
Rafal Ciepiela, Anil Joy Varughese, Piotr Sroka
In-Reply-To: <CO2PR07MB24699250A3773DE76B6D2E9EC1E00@CO2PR07MB2469.namprd07.prod.outlook.com>
On Mon, Jun 24, 2019 at 10:14:41AM +0000, Parshuram Raju Thombare wrote:
> >> >I still don't think this makes much sense, splitting the interface
> >> > configuration between here and below.
> >> Do you mean splitting mac_config in two *_configure functions ?
> >> This was done as per Andrew's suggestion to make code mode readable
> >> and easy to manage by splitting MAC configuration for different interfaces.
> >No, I mean here you disable SGMII if we're switching away from SGMII
> >mode.... (note, this means there is more to come for this sentence)
> Sorry, I misunderstood your original question. I think disabling old interface
> and enabling new one can be done in single place. I will do this change.
>
> >> >This will only be executed when we are not using inband mode, which
> >> >basically means it's not possible to switch to SGMII in-band mode.
> >> SGMII is used in default PHY mode. And above code is to program MAC to
> >> select PCS and SGMII interface.
> >... and here you enable it for SGMII mode, but only for non-inband
> >modes.
> >
> >Why not:
> > if (change_interface) {
> > if (state->interface == PHY_INTERFACE_MODE_SGMII) {
> > // Enable SGMII mode and PCS
> > gem_writel(bp, NCFGR, ncfgr | GEM_BIT(SGMIIEN) |
> > GEM_BIT(PCSSEL));
> > } else {
> > // Disable SGMII mode and PCS
> > gem_writel(bp, NCFGR, ncfgr & ~(GEM_BIT(SGMIIEN)
> > GEM_BIT(PCSSEL)));
> > // Reset PCS
> > gem_writel(bp, PCS_CTRL, gem_readl(bp, PCS_CTRL)
> > GEM_BIT(PCS_CTRL_RST));
> > }
> > }
> > if (!phylink_autoneg_inband(mode) &&
> > (bp->speed != state->speed || bp->duplex != state->duplex)) {
> >?
> Ok
>
> >> >> +
> >> >> + if (!interface_supported) {
> >> >> + netdev_err(dev, "Phy mode %s not supported",
> >> >> + phy_modes(phy_mode));
> >> >> + goto err_out_free_netdev;
> >> >> + }
> >> >> +
> >> >> bp->phy_interface = phy_mode;
> >> >> + } else {
> >> >> + bp->phy_interface = phy_mode;
> >> >> + }
> >> >If bp->phy_interface is PHY_INTERFACE_MODE_SGMII here, and
> >> > mac_config()
> >> >is called with state->interface = PHY_INTERFACE_MODE_SGMII, then
> >> >mac_config() won't configure the MAC for the interface type - is that
> >> >intentional?
> >> In mac_config configure MAC for non in-band mode, there is also check for
> >> speed, duplex
> >> changes. bp->speed and bp->duplex are initialized to SPEED_UNKNOWN
> >> and DUPLEX_UNKNOWN
> >> values so it is expected that for non in band mode state contains valid speed
> >> and duplex mode
> >> which are different from *_UNKNOWN values.
>
> >Sorry, this reply doesn't answer my question. I'm not asking about
> >bp->speed and bp->duplex. I'm asking:
> >1) why you are initialising bp->phy_interface here
> >2) you to consider the impact that has on the mac_config() implementation
> > you are proposing
> > because I think it's buggy.
> bp->phy_interface is to store phy mode value from device tree. This is used later
> to know what phy interface user has selected for PHY-MAC. Same is used
> to configure MAC correctly and based on your suggestion code is
> added to handle PHY dynamically changing phy interface, in which
> case bp->phy_interface is also updated. Though it may not be what user want,
> if phy interface is totally decided by PHY and is anyway going to be different from what user
> has selected in DT, initializing it here doesn't make sense.
> But in case of PHY not changing phy_interface dynamically bp->phy_interface need to be
> initialized with value from DT.
When phylink_start() is called, you will receive a mac_config() call to
configure the MAC for the initial operating settings, which will include
the current PHY interface mode. This will initially be whatever
interface mode was passed in to phylink_create().
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
^ permalink raw reply
* [PATCH v4 rdma-next 0/3] RDMA/qedr: Use the doorbell overflow recovery mechanism for RDMA
From: Michal Kalderon @ 2019-06-24 10:28 UTC (permalink / raw)
To: michal.kalderon, ariel.elior, jgg, dledford; +Cc: linux-rdma, davem, netdev
This patch series used the doorbell overflow recovery mechanism
introduced in
commit 36907cd5cd72 ("qed: Add doorbell overflow recovery mechanism")
for rdma ( RoCE and iWARP )
rdma-core pull request #493
Changes from V3:
- Remove casts from void to u8. Pointer arithmetic can be done on void
- rebase to tip of rdma-next
Changes from V2:
- Don't use long-lived kmap. Instead use user-trigger mmap for the
doorbell recovery entries.
- Modify dpi_addr to be denoted with __iomem and avoid redundant
casts
Changes from V1:
- call kmap to map virtual address into kernel space
- modify db_rec_delete to be void
- remove some cpu_to_le16 that were added to previous patch which are
correct but not related to the overflow recovery mechanism. Will be
submitted as part of a different patch
Michal Kalderon (3):
qed*: Change dpi_addr to be denoted with __iomem
RDMA/qedr: Add doorbell overflow recovery support
RDMA/qedr: Add iWARP doorbell recovery support
drivers/infiniband/hw/qedr/main.c | 2 +-
drivers/infiniband/hw/qedr/qedr.h | 27 +-
drivers/infiniband/hw/qedr/verbs.c | 386 ++++++++++++++++++++++++-----
drivers/net/ethernet/qlogic/qed/qed_rdma.c | 5 +-
include/linux/qed/qed_rdma_if.h | 2 +-
include/uapi/rdma/qedr-abi.h | 25 ++
6 files changed, 376 insertions(+), 71 deletions(-)
--
2.14.5
^ permalink raw reply
* [PATCH v4 rdma-next 1/3] qed*: Change dpi_addr to be denoted with __iomem
From: Michal Kalderon @ 2019-06-24 10:28 UTC (permalink / raw)
To: michal.kalderon, ariel.elior, jgg, dledford; +Cc: linux-rdma, davem, netdev
In-Reply-To: <20190624102809.8793-1-michal.kalderon@marvell.com>
Several casts were required around dpi_addr parameter in qed_rdma_if.h
This is an address on the doorbell bar and should therefore be marked
with __iomem.
Reported-by: Jason Gunthorpe <jgg@mellanox.com>
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
drivers/infiniband/hw/qedr/main.c | 2 +-
drivers/infiniband/hw/qedr/qedr.h | 2 +-
drivers/net/ethernet/qlogic/qed/qed_rdma.c | 5 ++---
include/linux/qed/qed_rdma_if.h | 2 +-
4 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/qedr/main.c b/drivers/infiniband/hw/qedr/main.c
index a0a7ba0a5af4..3db4b6ba5ad6 100644
--- a/drivers/infiniband/hw/qedr/main.c
+++ b/drivers/infiniband/hw/qedr/main.c
@@ -815,7 +815,7 @@ static int qedr_init_hw(struct qedr_dev *dev)
if (rc)
goto out;
- dev->db_addr = (void __iomem *)(uintptr_t)out_params.dpi_addr;
+ dev->db_addr = out_params.dpi_addr;
dev->db_phys_addr = out_params.dpi_phys_addr;
dev->db_size = out_params.dpi_size;
dev->dpi = out_params.dpi;
diff --git a/drivers/infiniband/hw/qedr/qedr.h b/drivers/infiniband/hw/qedr/qedr.h
index 6175d1e98717..8df56aba9d2c 100644
--- a/drivers/infiniband/hw/qedr/qedr.h
+++ b/drivers/infiniband/hw/qedr/qedr.h
@@ -227,7 +227,7 @@ struct qedr_ucontext {
struct ib_ucontext ibucontext;
struct qedr_dev *dev;
struct qedr_pd *pd;
- u64 dpi_addr;
+ void __iomem *dpi_addr;
u64 dpi_phys_addr;
u32 dpi_size;
u16 dpi;
diff --git a/drivers/net/ethernet/qlogic/qed/qed_rdma.c b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
index 7873d6dfd91f..fb3fe60a1a68 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_rdma.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_rdma.c
@@ -799,9 +799,8 @@ static int qed_rdma_add_user(void *rdma_cxt,
/* Calculate the corresponding DPI address */
dpi_start_offset = p_hwfn->dpi_start_offset;
- out_params->dpi_addr = (u64)((u8 __iomem *)p_hwfn->doorbells +
- dpi_start_offset +
- ((out_params->dpi) * p_hwfn->dpi_size));
+ out_params->dpi_addr = p_hwfn->doorbells + dpi_start_offset +
+ out_params->dpi * p_hwfn->dpi_size;
out_params->dpi_phys_addr = p_hwfn->cdev->db_phys_addr +
dpi_start_offset +
diff --git a/include/linux/qed/qed_rdma_if.h b/include/linux/qed/qed_rdma_if.h
index d15f8e4815e3..834166809a6c 100644
--- a/include/linux/qed/qed_rdma_if.h
+++ b/include/linux/qed/qed_rdma_if.h
@@ -225,7 +225,7 @@ struct qed_rdma_start_in_params {
struct qed_rdma_add_user_out_params {
u16 dpi;
- u64 dpi_addr;
+ void __iomem *dpi_addr;
u64 dpi_phys_addr;
u32 dpi_size;
u16 wid_count;
--
2.14.5
^ permalink raw reply related
* [PATCH v4 rdma-next 2/3] RDMA/qedr: Add doorbell overflow recovery support
From: Michal Kalderon @ 2019-06-24 10:28 UTC (permalink / raw)
To: michal.kalderon, ariel.elior, jgg, dledford; +Cc: linux-rdma, davem, netdev
In-Reply-To: <20190624102809.8793-1-michal.kalderon@marvell.com>
Use the doorbell recovery mechanism to register rdma related doorbells
that will be restored in case there is a doorbell overflow attention.
Signed-off-by: Ariel Elior <ariel.elior@marvell.com>
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
drivers/infiniband/hw/qedr/qedr.h | 13 +-
drivers/infiniband/hw/qedr/verbs.c | 351 ++++++++++++++++++++++++++++++-------
include/uapi/rdma/qedr-abi.h | 25 +++
3 files changed, 329 insertions(+), 60 deletions(-)
diff --git a/drivers/infiniband/hw/qedr/qedr.h b/drivers/infiniband/hw/qedr/qedr.h
index 8df56aba9d2c..006712ac1c88 100644
--- a/drivers/infiniband/hw/qedr/qedr.h
+++ b/drivers/infiniband/hw/qedr/qedr.h
@@ -231,7 +231,7 @@ struct qedr_ucontext {
u64 dpi_phys_addr;
u32 dpi_size;
u16 dpi;
-
+ bool db_rec;
struct list_head mm_head;
/* Lock to protect mm list */
@@ -263,6 +263,11 @@ struct qedr_userq {
struct qedr_pbl *pbl_tbl;
u64 buf_addr;
size_t buf_len;
+
+ /* doorbell recovery */
+ void __iomem *db_addr;
+ struct qedr_user_db_rec *db_rec_data;
+ u64 db_rec_phys;
};
struct qedr_cq {
@@ -298,11 +303,17 @@ struct qedr_pd {
struct qedr_ucontext *uctx;
};
+enum qedr_mm_type {
+ QEDR_MM_TYPE_DB_BAR,
+ QEDR_MM_TYPE_DB_REC
+};
+
struct qedr_mm {
struct {
u64 phy_addr;
unsigned long len;
} key;
+ enum qedr_mm_type type;
struct list_head entry;
};
diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index 3fc7a4e901c3..cbf4cccd174d 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -257,7 +257,7 @@ int qedr_modify_port(struct ib_device *ibdev, u8 port, int mask,
}
static int qedr_add_mmap(struct qedr_ucontext *uctx, u64 phy_addr,
- unsigned long len)
+ unsigned long len, enum qedr_mm_type type)
{
struct qedr_mm *mm;
@@ -265,6 +265,7 @@ static int qedr_add_mmap(struct qedr_ucontext *uctx, u64 phy_addr,
if (!mm)
return -ENOMEM;
+ mm->type = type;
mm->key.phy_addr = phy_addr;
/* This function might be called with a length which is not a multiple
* of PAGE_SIZE, while the mapping is PAGE_SIZE grained and the kernel
@@ -281,24 +282,26 @@ static int qedr_add_mmap(struct qedr_ucontext *uctx, u64 phy_addr,
mutex_unlock(&uctx->mm_list_lock);
DP_DEBUG(uctx->dev, QEDR_MSG_MISC,
- "added (addr=0x%llx,len=0x%lx) for ctx=%p\n",
+ "added (addr=0x%llx,len=0x%lx,type=%s) for ctx=%p\n",
(unsigned long long)mm->key.phy_addr,
- (unsigned long)mm->key.len, uctx);
+ (unsigned long)mm->key.len,
+ (type == QEDR_MM_TYPE_DB_REC) ? "DB_REC" : "DB_BAR",
+ uctx);
return 0;
}
-static bool qedr_search_mmap(struct qedr_ucontext *uctx, u64 phy_addr,
- unsigned long len)
+static struct qedr_mm *qedr_remove_mmap(struct qedr_ucontext *uctx,
+ u64 phy_addr, unsigned long len)
{
+ struct qedr_mm *mm, *tmp;
bool found = false;
- struct qedr_mm *mm;
mutex_lock(&uctx->mm_list_lock);
- list_for_each_entry(mm, &uctx->mm_head, entry) {
+ list_for_each_entry_safe(mm, tmp, &uctx->mm_head, entry) {
if (len != mm->key.len || phy_addr != mm->key.phy_addr)
continue;
-
+ list_del_init(&mm->entry);
found = true;
break;
}
@@ -307,7 +310,10 @@ static bool qedr_search_mmap(struct qedr_ucontext *uctx, u64 phy_addr,
"searched for (addr=0x%llx,len=0x%lx) for ctx=%p, result=%d\n",
mm->key.phy_addr, mm->key.len, uctx, found);
- return found;
+ if (found)
+ return mm;
+
+ return NULL;
}
int qedr_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata)
@@ -316,12 +322,24 @@ int qedr_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata)
int rc;
struct qedr_ucontext *ctx = get_qedr_ucontext(uctx);
struct qedr_alloc_ucontext_resp uresp = {};
+ struct qedr_alloc_ucontext_req ureq = {};
struct qedr_dev *dev = get_qedr_dev(ibdev);
struct qed_rdma_add_user_out_params oparams;
if (!udata)
return -EFAULT;
+ if (udata->inlen) {
+ rc = ib_copy_from_udata(&ureq, udata,
+ min(sizeof(ureq), udata->inlen));
+ if (rc) {
+ DP_ERR(dev, "Problem copying data from user space\n");
+ return -EFAULT;
+ }
+
+ ctx->db_rec = !!(ureq.context_flags & QEDR_ALLOC_UCTX_DB_REC);
+ }
+
rc = dev->ops->rdma_add_user(dev->rdma_ctx, &oparams);
if (rc) {
DP_ERR(dev,
@@ -356,7 +374,8 @@ int qedr_alloc_ucontext(struct ib_ucontext *uctx, struct ib_udata *udata)
ctx->dev = dev;
- rc = qedr_add_mmap(ctx, ctx->dpi_phys_addr, ctx->dpi_size);
+ rc = qedr_add_mmap(ctx, ctx->dpi_phys_addr, ctx->dpi_size,
+ QEDR_MM_TYPE_DB_BAR);
if (rc)
return rc;
@@ -383,6 +402,43 @@ void qedr_dealloc_ucontext(struct ib_ucontext *ibctx)
}
}
+/* Map the doorbell bar */
+int qedr_mmap_db_bar(struct qedr_dev *dev, struct qedr_ucontext *ucontext,
+ struct vm_area_struct *vma, unsigned long dpi_start)
+{
+ unsigned long phys_addr = vma->vm_pgoff << PAGE_SHIFT;
+ unsigned long len = (vma->vm_end - vma->vm_start);
+
+ if (phys_addr < dpi_start ||
+ ((phys_addr + len) > (dpi_start + ucontext->dpi_size))) {
+ DP_ERR(dev,
+ "failed mmap, pages are outside of dpi; page address=0x%pK, dpi_start=0x%pK, dpi_size=0x%x\n",
+ (void *)phys_addr, (void *)dpi_start,
+ ucontext->dpi_size);
+ return -EINVAL;
+ }
+
+ if (vma->vm_flags & VM_READ) {
+ DP_ERR(dev, "failed mmap, cannot map doorbell bar for read\n");
+ return -EINVAL;
+ }
+
+ vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+
+ return io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, len,
+ vma->vm_page_prot);
+}
+
+/* Map the kernel doorbell recovery memory entry */
+int qedr_mmap_db_rec(struct vm_area_struct *vma)
+{
+ unsigned long len = vma->vm_end - vma->vm_start;
+
+ return remap_pfn_range(vma, vma->vm_start,
+ vma->vm_pgoff,
+ len, vma->vm_page_prot);
+}
+
int qedr_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
{
struct qedr_ucontext *ucontext = get_qedr_ucontext(context);
@@ -390,6 +446,8 @@ int qedr_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
unsigned long phys_addr = vma->vm_pgoff << PAGE_SHIFT;
unsigned long len = (vma->vm_end - vma->vm_start);
unsigned long dpi_start;
+ struct qedr_mm *mm;
+ int rc;
dpi_start = dev->db_phys_addr + (ucontext->dpi * ucontext->dpi_size);
@@ -405,29 +463,28 @@ int qedr_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
return -EINVAL;
}
- if (!qedr_search_mmap(ucontext, phys_addr, len)) {
- DP_ERR(dev, "failed mmap, vm_pgoff=0x%lx is not authorized\n",
+ mm = qedr_remove_mmap(ucontext, phys_addr, len);
+ if (!mm) {
+ DP_ERR(dev, "failed to remove mmap, vm_pgoff=0x%lx\n",
vma->vm_pgoff);
return -EINVAL;
}
- if (phys_addr < dpi_start ||
- ((phys_addr + len) > (dpi_start + ucontext->dpi_size))) {
- DP_ERR(dev,
- "failed mmap, pages are outside of dpi; page address=0x%pK, dpi_start=0x%pK, dpi_size=0x%x\n",
- (void *)phys_addr, (void *)dpi_start,
- ucontext->dpi_size);
- return -EINVAL;
+ switch (mm->type) {
+ case QEDR_MM_TYPE_DB_BAR:
+ rc = qedr_mmap_db_bar(dev, ucontext, vma, dpi_start);
+ break;
+ case QEDR_MM_TYPE_DB_REC:
+ rc = qedr_mmap_db_rec(vma);
+ break;
+ default:
+ rc = -EINVAL;
+ break;
}
- if (vma->vm_flags & VM_READ) {
- DP_ERR(dev, "failed mmap, cannot map doorbell bar for read\n");
- return -EINVAL;
- }
+ kfree(mm);
- vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
- return io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, len,
- vma->vm_page_prot);
+ return rc;
}
int qedr_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
@@ -657,16 +714,48 @@ static void qedr_populate_pbls(struct qedr_dev *dev, struct ib_umem *umem,
}
}
+static int qedr_db_recovery_add(struct qedr_dev *dev,
+ void __iomem *db_addr,
+ void *db_data,
+ enum qed_db_rec_width db_width,
+ enum qed_db_rec_space db_space)
+{
+ if (!db_data) {
+ DP_DEBUG(dev, QEDR_MSG_INIT, "avoiding db rec since old lib\n");
+ return 0;
+ }
+
+ return dev->ops->common->db_recovery_add(dev->cdev, db_addr, db_data,
+ db_width, db_space);
+}
+
+static void qedr_db_recovery_del(struct qedr_dev *dev,
+ void __iomem *db_addr,
+ void *db_data)
+{
+ if (!db_data) {
+ DP_DEBUG(dev, QEDR_MSG_INIT, "avoiding db rec since old lib\n");
+ return;
+ }
+
+ /* Ignore return code as there is not much we can do about it. Error
+ * log will be printed inside.
+ */
+ dev->ops->common->db_recovery_del(dev->cdev, db_addr, db_data);
+}
+
static int qedr_copy_cq_uresp(struct qedr_dev *dev,
- struct qedr_cq *cq, struct ib_udata *udata)
+ struct qedr_cq *cq, struct ib_udata *udata,
+ u32 db_offset)
{
struct qedr_create_cq_uresp uresp;
int rc;
memset(&uresp, 0, sizeof(uresp));
- uresp.db_offset = DB_ADDR_SHIFT(DQ_PWM_OFFSET_UCM_RDMA_CQ_CONS_32BIT);
+ uresp.db_offset = db_offset;
uresp.icid = cq->icid;
+ uresp.db_rec_addr = cq->q.db_rec_phys;
rc = qedr_ib_copy_to_udata(udata, &uresp, sizeof(uresp));
if (rc)
@@ -694,10 +783,36 @@ static inline int qedr_align_cq_entries(int entries)
return aligned_size / QEDR_CQE_SIZE;
}
+static int qedr_init_user_db_rec(struct ib_udata *udata,
+ struct qedr_dev *dev, struct qedr_userq *q,
+ bool requires_db_rec)
+{
+ struct qedr_ucontext *uctx =
+ rdma_udata_to_drv_context(udata, struct qedr_ucontext,
+ ibucontext);
+
+ /* Aborting for non doorbell userqueue (SRQ) or non-supporting lib */
+ if (requires_db_rec == 0 || !uctx->db_rec)
+ return 0;
+
+ /* Allocate a page for doorbell recovery, add to mmap ) */
+ q->db_rec_data = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!q->db_rec_data) {
+ DP_ERR(dev,
+ "get_free_page failed\n");
+ return -ENOMEM;
+ }
+
+ q->db_rec_phys = virt_to_phys(q->db_rec_data);
+ return qedr_add_mmap(uctx, q->db_rec_phys, PAGE_SIZE,
+ QEDR_MM_TYPE_DB_REC);
+}
+
static inline int qedr_init_user_queue(struct ib_udata *udata,
struct qedr_dev *dev,
struct qedr_userq *q, u64 buf_addr,
- size_t buf_len, int access, int dmasync,
+ size_t buf_len, bool requires_db_rec,
+ int access, int dmasync,
int alloc_and_init)
{
u32 fw_pages;
@@ -735,7 +850,8 @@ static inline int qedr_init_user_queue(struct ib_udata *udata,
}
}
- return 0;
+ /* mmap the user address used to store doorbell data for recovery */
+ return qedr_init_user_db_rec(udata, dev, q, requires_db_rec);
err0:
ib_umem_release(q->umem);
@@ -821,6 +937,7 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
int entries = attr->cqe;
struct qedr_cq *cq = get_qedr_cq(ibcq);
int chain_entries;
+ u32 db_offset;
int page_cnt;
u64 pbl_ptr;
u16 icid;
@@ -840,8 +957,12 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
chain_entries = qedr_align_cq_entries(entries);
chain_entries = min_t(int, chain_entries, QEDR_MAX_CQES);
+ /* calc db offset. user will add DPI base, kernel will add db addr */
+ db_offset = DB_ADDR_SHIFT(DQ_PWM_OFFSET_UCM_RDMA_CQ_CONS_32BIT);
+
if (udata) {
- if (ib_copy_from_udata(&ureq, udata, sizeof(ureq))) {
+ if (ib_copy_from_udata(&ureq, udata, min(sizeof(ureq),
+ udata->inlen))) {
DP_ERR(dev,
"create cq: problem copying data from user space\n");
goto err0;
@@ -856,8 +977,9 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
cq->cq_type = QEDR_CQ_TYPE_USER;
rc = qedr_init_user_queue(udata, dev, &cq->q, ureq.addr,
- ureq.len, IB_ACCESS_LOCAL_WRITE, 1,
- 1);
+ ureq.len, true,
+ IB_ACCESS_LOCAL_WRITE,
+ 1, 1);
if (rc)
goto err0;
@@ -865,6 +987,7 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
page_cnt = cq->q.pbl_info.num_pbes;
cq->ibcq.cqe = chain_entries;
+ cq->q.db_addr = ctx->dpi_addr + db_offset;
} else {
cq->cq_type = QEDR_CQ_TYPE_KERNEL;
@@ -895,14 +1018,21 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
spin_lock_init(&cq->cq_lock);
if (udata) {
- rc = qedr_copy_cq_uresp(dev, cq, udata);
+ rc = qedr_copy_cq_uresp(dev, cq, udata, db_offset);
if (rc)
goto err3;
+
+ rc = qedr_db_recovery_add(dev, cq->q.db_addr,
+ &cq->q.db_rec_data->db_data,
+ DB_REC_WIDTH_64B,
+ DB_REC_USER);
+ if (rc)
+ goto err3;
+
} else {
/* Generate doorbell address. */
- cq->db_addr = dev->db_addr +
- DB_ADDR_SHIFT(DQ_PWM_OFFSET_UCM_RDMA_CQ_CONS_32BIT);
cq->db.data.icid = cq->icid;
+ cq->db_addr = dev->db_addr + db_offset;
cq->db.data.params = DB_AGG_CMD_SET <<
RDMA_PWM_VAL32_DATA_AGG_CMD_SHIFT;
@@ -912,6 +1042,11 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
cq->latest_cqe = NULL;
consume_cqe(cq);
cq->cq_cons = qed_chain_get_cons_idx_u32(&cq->pbl);
+
+ rc = qedr_db_recovery_add(dev, cq->db_addr, &cq->db.data,
+ DB_REC_WIDTH_64B, DB_REC_KERNEL);
+ if (rc)
+ goto err3;
}
DP_DEBUG(dev, QEDR_MSG_CQ,
@@ -930,8 +1065,16 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
else
dev->ops->common->chain_free(dev->cdev, &cq->pbl);
err1:
- if (udata)
+ if (udata) {
ib_umem_release(cq->q.umem);
+ if (cq->q.db_rec_data) {
+ qedr_db_recovery_del(dev, cq->q.db_addr,
+ &cq->q.db_rec_data->db_data);
+ free_page((unsigned long)cq->q.db_rec_data);
+ }
+ } else {
+ qedr_db_recovery_del(dev, cq->db_addr, &cq->db.data);
+ }
err0:
return -EINVAL;
}
@@ -962,8 +1105,10 @@ void qedr_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
cq->destroyed = 1;
/* GSIs CQs are handled by driver, so they don't exist in the FW */
- if (cq->cq_type == QEDR_CQ_TYPE_GSI)
+ if (cq->cq_type == QEDR_CQ_TYPE_GSI) {
+ qedr_db_recovery_del(dev, cq->db_addr, &cq->db.data);
return;
+ }
iparams.icid = cq->icid;
dev->ops->rdma_destroy_cq(dev->rdma_ctx, &iparams, &oparams);
@@ -972,6 +1117,14 @@ void qedr_destroy_cq(struct ib_cq *ibcq, struct ib_udata *udata)
if (udata) {
qedr_free_pbl(dev, &cq->q.pbl_info, cq->q.pbl_tbl);
ib_umem_release(cq->q.umem);
+
+ if (cq->q.db_rec_data) {
+ qedr_db_recovery_del(dev, cq->q.db_addr,
+ &cq->q.db_rec_data->db_data);
+ free_page((unsigned long)cq->q.db_rec_data);
+ }
+ } else {
+ qedr_db_recovery_del(dev, cq->db_addr, &cq->db.data);
}
/* We don't want the IRQ handler to handle a non-existing CQ so we
@@ -1150,6 +1303,7 @@ static void qedr_copy_rq_uresp(struct qedr_dev *dev,
}
uresp->rq_icid = qp->icid;
+ uresp->rq_db_rec_addr = qp->urq.db_rec_phys;
}
static void qedr_copy_sq_uresp(struct qedr_dev *dev,
@@ -1163,22 +1317,24 @@ static void qedr_copy_sq_uresp(struct qedr_dev *dev,
uresp->sq_icid = qp->icid;
else
uresp->sq_icid = qp->icid + 1;
+
+ uresp->sq_db_rec_addr = qp->usq.db_rec_phys;
}
static int qedr_copy_qp_uresp(struct qedr_dev *dev,
- struct qedr_qp *qp, struct ib_udata *udata)
+ struct qedr_qp *qp, struct ib_udata *udata,
+ struct qedr_create_qp_uresp *uresp)
{
- struct qedr_create_qp_uresp uresp;
int rc;
- memset(&uresp, 0, sizeof(uresp));
- qedr_copy_sq_uresp(dev, &uresp, qp);
- qedr_copy_rq_uresp(dev, &uresp, qp);
+ memset(uresp, 0, sizeof(*uresp));
+ qedr_copy_sq_uresp(dev, uresp, qp);
+ qedr_copy_rq_uresp(dev, uresp, qp);
- uresp.atomic_supported = dev->atomic_cap != IB_ATOMIC_NONE;
- uresp.qp_id = qp->qp_id;
+ uresp->atomic_supported = dev->atomic_cap != IB_ATOMIC_NONE;
+ uresp->qp_id = qp->qp_id;
- rc = qedr_ib_copy_to_udata(udata, &uresp, sizeof(uresp));
+ rc = qedr_ib_copy_to_udata(udata, uresp, sizeof(*uresp));
if (rc)
DP_ERR(dev,
"create qp: failed a copy to user space with qp icid=0x%x.\n",
@@ -1222,16 +1378,35 @@ static void qedr_set_common_qp_params(struct qedr_dev *dev,
qp->sq.max_sges, qp->sq_cq->icid);
}
-static void qedr_set_roce_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
+static int qedr_set_roce_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
{
+ int rc;
+
qp->sq.db = dev->db_addr +
DB_ADDR_SHIFT(DQ_PWM_OFFSET_XCM_RDMA_SQ_PROD);
qp->sq.db_data.data.icid = qp->icid + 1;
+ rc = qedr_db_recovery_add(dev, qp->sq.db,
+ &qp->sq.db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_KERNEL);
+ if (rc)
+ return rc;
+
if (!qp->srq) {
qp->rq.db = dev->db_addr +
DB_ADDR_SHIFT(DQ_PWM_OFFSET_TCM_ROCE_RQ_PROD);
qp->rq.db_data.data.icid = qp->icid;
+
+ rc = qedr_db_recovery_add(dev, qp->rq.db,
+ &qp->rq.db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_KERNEL);
+ if (rc)
+ qedr_db_recovery_del(dev, qp->sq.db,
+ &qp->sq.db_data);
}
+
+ return rc;
}
static int qedr_check_srq_params(struct qedr_dev *dev,
@@ -1285,7 +1460,7 @@ static int qedr_init_srq_user_params(struct ib_udata *udata,
int rc;
rc = qedr_init_user_queue(udata, srq->dev, &srq->usrq, ureq->srq_addr,
- ureq->srq_len, access, dmasync, 1);
+ ureq->srq_len, false, access, dmasync, 1);
if (rc)
return rc;
@@ -1381,7 +1556,8 @@ int qedr_create_srq(struct ib_srq *ibsrq, struct ib_srq_init_attr *init_attr,
hw_srq->max_sges = init_attr->attr.max_sge;
if (udata) {
- if (ib_copy_from_udata(&ureq, udata, sizeof(ureq))) {
+ if (ib_copy_from_udata(&ureq, udata, min(sizeof(ureq),
+ udata->inlen))) {
DP_ERR(dev,
"create srq: problem copying data from user space\n");
goto err0;
@@ -1579,6 +1755,18 @@ static void qedr_cleanup_user(struct qedr_dev *dev, struct qedr_qp *qp)
if (qp->urq.umem)
ib_umem_release(qp->urq.umem);
qp->urq.umem = NULL;
+
+ if (qp->usq.db_rec_data) {
+ qedr_db_recovery_del(dev, qp->usq.db_addr,
+ &qp->usq.db_rec_data->db_data);
+ free_page((unsigned long)qp->usq.db_rec_data);
+ }
+
+ if (qp->urq.db_rec_data) {
+ qedr_db_recovery_del(dev, qp->urq.db_addr,
+ &qp->urq.db_rec_data->db_data);
+ free_page((unsigned long)qp->urq.db_rec_data);
+ }
}
static int qedr_create_user_qp(struct qedr_dev *dev,
@@ -1590,12 +1778,14 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
struct qed_rdma_create_qp_in_params in_params;
struct qed_rdma_create_qp_out_params out_params;
struct qedr_pd *pd = get_qedr_pd(ibpd);
+ struct qedr_create_qp_uresp uresp;
+ struct qedr_ucontext *ctx = NULL;
struct qedr_create_qp_ureq ureq;
int alloc_and_init = rdma_protocol_roce(&dev->ibdev, 1);
int rc = -EINVAL;
memset(&ureq, 0, sizeof(ureq));
- rc = ib_copy_from_udata(&ureq, udata, sizeof(ureq));
+ rc = ib_copy_from_udata(&ureq, udata, min(sizeof(ureq), udata->inlen));
if (rc) {
DP_ERR(dev, "Problem copying data from user space\n");
return rc;
@@ -1603,14 +1793,16 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
/* SQ - read access only (0), dma sync not required (0) */
rc = qedr_init_user_queue(udata, dev, &qp->usq, ureq.sq_addr,
- ureq.sq_len, 0, 0, alloc_and_init);
+ ureq.sq_len, true, 0, 0,
+ alloc_and_init);
if (rc)
return rc;
if (!qp->srq) {
/* RQ - read access only (0), dma sync not required (0) */
rc = qedr_init_user_queue(udata, dev, &qp->urq, ureq.rq_addr,
- ureq.rq_len, 0, 0, alloc_and_init);
+ ureq.rq_len, true,
+ 0, 0, alloc_and_init);
if (rc)
return rc;
}
@@ -1640,13 +1832,31 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
qp->qp_id = out_params.qp_id;
qp->icid = out_params.icid;
- rc = qedr_copy_qp_uresp(dev, qp, udata);
+ rc = qedr_copy_qp_uresp(dev, qp, udata, &uresp);
if (rc)
goto err;
+ /* db offset was calculated in copy_qp_uresp, now set in the user q */
+ ctx = pd->uctx;
+ qp->usq.db_addr = ctx->dpi_addr + uresp.sq_db_offset;
+ qp->urq.db_addr = ctx->dpi_addr + uresp.rq_db_offset;
+
+ rc = qedr_db_recovery_add(dev, qp->usq.db_addr,
+ &qp->usq.db_rec_data->db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_USER);
+ if (rc)
+ goto err;
+
+ rc = qedr_db_recovery_add(dev, qp->urq.db_addr,
+ &qp->urq.db_rec_data->db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_USER);
+ if (rc)
+ goto err;
qedr_qp_user_print(dev, qp);
- return 0;
+ return rc;
err:
rc = dev->ops->rdma_destroy_qp(dev->rdma_ctx, qp->qed_qp);
if (rc)
@@ -1657,12 +1867,21 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
return rc;
}
-static void qedr_set_iwarp_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
+static int qedr_set_iwarp_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
{
+ int rc;
+
qp->sq.db = dev->db_addr +
DB_ADDR_SHIFT(DQ_PWM_OFFSET_XCM_RDMA_SQ_PROD);
qp->sq.db_data.data.icid = qp->icid;
+ rc = qedr_db_recovery_add(dev, qp->sq.db,
+ &qp->sq.db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_KERNEL);
+ if (rc)
+ return rc;
+
qp->rq.db = dev->db_addr +
DB_ADDR_SHIFT(DQ_PWM_OFFSET_TCM_IWARP_RQ_PROD);
qp->rq.db_data.data.icid = qp->icid;
@@ -1670,6 +1889,13 @@ static void qedr_set_iwarp_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
DB_ADDR_SHIFT(DQ_PWM_OFFSET_TCM_FLAGS);
qp->rq.iwarp_db2_data.data.icid = qp->icid;
qp->rq.iwarp_db2_data.data.value = DQ_TCM_IWARP_POST_RQ_CF_CMD;
+
+ rc = qedr_db_recovery_add(dev, qp->rq.db,
+ &qp->rq.db_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_KERNEL);
+
+ return rc;
}
static int
@@ -1717,8 +1943,7 @@ qedr_roce_create_kernel_qp(struct qedr_dev *dev,
qp->qp_id = out_params.qp_id;
qp->icid = out_params.icid;
- qedr_set_roce_db_info(dev, qp);
- return rc;
+ return qedr_set_roce_db_info(dev, qp);
}
static int
@@ -1776,8 +2001,7 @@ qedr_iwarp_create_kernel_qp(struct qedr_dev *dev,
qp->qp_id = out_params.qp_id;
qp->icid = out_params.icid;
- qedr_set_iwarp_db_info(dev, qp);
- return rc;
+ return qedr_set_iwarp_db_info(dev, qp);
err:
dev->ops->rdma_destroy_qp(dev->rdma_ctx, qp->qed_qp);
@@ -1792,6 +2016,15 @@ static void qedr_cleanup_kernel(struct qedr_dev *dev, struct qedr_qp *qp)
dev->ops->common->chain_free(dev->cdev, &qp->rq.pbl);
kfree(qp->rqe_wr_id);
+
+ /* GSI qp is not registered to db mechanism so no need to delete */
+ if (qp->qp_type == IB_QPT_GSI)
+ return;
+
+ qedr_db_recovery_del(dev, qp->sq.db, &qp->sq.db_data);
+
+ if (!qp->srq)
+ qedr_db_recovery_del(dev, qp->rq.db, &qp->rq.db_data);
}
static int qedr_create_kernel_qp(struct qedr_dev *dev,
diff --git a/include/uapi/rdma/qedr-abi.h b/include/uapi/rdma/qedr-abi.h
index 7a10b3a325fa..aa06dfa1a031 100644
--- a/include/uapi/rdma/qedr-abi.h
+++ b/include/uapi/rdma/qedr-abi.h
@@ -38,6 +38,15 @@
#define QEDR_ABI_VERSION (8)
/* user kernel communication data structures. */
+enum qedr_alloc_ucontext_flags {
+ QEDR_ALLOC_UCTX_RESERVED = 1 << 0,
+ QEDR_ALLOC_UCTX_DB_REC = 1 << 1
+};
+
+struct qedr_alloc_ucontext_req {
+ __u32 context_flags;
+ __u32 reserved;
+};
struct qedr_alloc_ucontext_resp {
__aligned_u64 db_pa;
@@ -74,6 +83,7 @@ struct qedr_create_cq_uresp {
__u32 db_offset;
__u16 icid;
__u16 reserved;
+ __u64 db_rec_addr;
};
struct qedr_create_qp_ureq {
@@ -109,6 +119,13 @@ struct qedr_create_qp_uresp {
__u32 rq_db2_offset;
__u32 reserved;
+
+ /* address of SQ doorbell recovery user entry */
+ __u64 sq_db_rec_addr;
+
+ /* address of RQ doorbell recovery user entry */
+ __u64 rq_db_rec_addr;
+
};
struct qedr_create_srq_ureq {
@@ -128,4 +145,12 @@ struct qedr_create_srq_uresp {
__u32 reserved1;
};
+/* doorbell recovery entry allocated and populated by userspace doorbelling
+ * entities and mapped to kernel. Kernel uses this to register doorbell
+ * information with doorbell drop recovery mechanism.
+ */
+struct qedr_user_db_rec {
+ __aligned_u64 db_data; /* doorbell data */
+};
+
#endif /* __QEDR_USER_H__ */
--
2.14.5
^ permalink raw reply related
* [PATCH v4 rdma-next 3/3] RDMA/qedr: Add iWARP doorbell recovery support
From: Michal Kalderon @ 2019-06-24 10:28 UTC (permalink / raw)
To: michal.kalderon, ariel.elior, jgg, dledford; +Cc: linux-rdma, davem, netdev
In-Reply-To: <20190624102809.8793-1-michal.kalderon@marvell.com>
This patch adds the iWARP specific doorbells to the doorbell
recovery mechanism
Signed-off-by: Ariel Elior <ariel.elior@marvell.com>
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
drivers/infiniband/hw/qedr/qedr.h | 12 +++++++-----
drivers/infiniband/hw/qedr/verbs.c | 37 ++++++++++++++++++++++++++++++++++++-
2 files changed, 43 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/qedr/qedr.h b/drivers/infiniband/hw/qedr/qedr.h
index 006712ac1c88..6c5524d6b04e 100644
--- a/drivers/infiniband/hw/qedr/qedr.h
+++ b/drivers/infiniband/hw/qedr/qedr.h
@@ -238,6 +238,11 @@ struct qedr_ucontext {
struct mutex mm_list_lock;
};
+union db_prod32 {
+ struct rdma_pwm_val16_data data;
+ u32 raw;
+};
+
union db_prod64 {
struct rdma_pwm_val32_data data;
u64 raw;
@@ -268,6 +273,8 @@ struct qedr_userq {
void __iomem *db_addr;
struct qedr_user_db_rec *db_rec_data;
u64 db_rec_phys;
+ void __iomem *db_rec_db2_addr;
+ union db_prod32 db_rec_db2_data;
};
struct qedr_cq {
@@ -317,11 +324,6 @@ struct qedr_mm {
struct list_head entry;
};
-union db_prod32 {
- struct rdma_pwm_val16_data data;
- u32 raw;
-};
-
struct qedr_qp_hwq_info {
/* WQE Elements */
struct qed_chain pbl;
diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index cbf4cccd174d..c15196ce9662 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -1767,6 +1767,10 @@ static void qedr_cleanup_user(struct qedr_dev *dev, struct qedr_qp *qp)
&qp->urq.db_rec_data->db_data);
free_page((unsigned long)qp->urq.db_rec_data);
}
+
+ if (rdma_protocol_iwarp(&dev->ibdev, 1))
+ qedr_db_recovery_del(dev, qp->urq.db_rec_db2_addr,
+ &qp->urq.db_rec_db2_data);
}
static int qedr_create_user_qp(struct qedr_dev *dev,
@@ -1841,6 +1845,17 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
qp->usq.db_addr = ctx->dpi_addr + uresp.sq_db_offset;
qp->urq.db_addr = ctx->dpi_addr + uresp.rq_db_offset;
+ if (rdma_protocol_iwarp(&dev->ibdev, 1)) {
+ qp->urq.db_rec_db2_addr = ctx->dpi_addr + uresp.rq_db2_offset;
+
+ /* calculate the db_rec_db2 data since it is constant so no
+ * need to reflect from user
+ */
+ qp->urq.db_rec_db2_data.data.icid = cpu_to_le16(qp->icid);
+ qp->urq.db_rec_db2_data.data.value =
+ cpu_to_le16(DQ_TCM_IWARP_POST_RQ_CF_CMD);
+ }
+
rc = qedr_db_recovery_add(dev, qp->usq.db_addr,
&qp->usq.db_rec_data->db_data,
DB_REC_WIDTH_32B,
@@ -1854,6 +1869,15 @@ static int qedr_create_user_qp(struct qedr_dev *dev,
DB_REC_USER);
if (rc)
goto err;
+
+ if (rdma_protocol_iwarp(&dev->ibdev, 1)) {
+ rc = qedr_db_recovery_add(dev, qp->urq.db_rec_db2_addr,
+ &qp->urq.db_rec_db2_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_USER);
+ if (rc)
+ goto err;
+ }
qedr_qp_user_print(dev, qp);
return rc;
@@ -1894,7 +1918,13 @@ static int qedr_set_iwarp_db_info(struct qedr_dev *dev, struct qedr_qp *qp)
&qp->rq.db_data,
DB_REC_WIDTH_32B,
DB_REC_KERNEL);
+ if (rc)
+ return rc;
+ rc = qedr_db_recovery_add(dev, qp->rq.iwarp_db2,
+ &qp->rq.iwarp_db2_data,
+ DB_REC_WIDTH_32B,
+ DB_REC_KERNEL);
return rc;
}
@@ -2023,8 +2053,13 @@ static void qedr_cleanup_kernel(struct qedr_dev *dev, struct qedr_qp *qp)
qedr_db_recovery_del(dev, qp->sq.db, &qp->sq.db_data);
- if (!qp->srq)
+ if (!qp->srq) {
qedr_db_recovery_del(dev, qp->rq.db, &qp->rq.db_data);
+
+ if (rdma_protocol_iwarp(&dev->ibdev, 1))
+ qedr_db_recovery_del(dev, qp->rq.iwarp_db2,
+ &qp->rq.iwarp_db2_data);
+ }
}
static int qedr_create_kernel_qp(struct qedr_dev *dev,
--
2.14.5
^ permalink raw reply related
* [PATCH net-next v2 0/3] mlxsw: Thermal and hwmon extensions
From: Ido Schimmel @ 2019-06-24 10:32 UTC (permalink / raw)
To: netdev; +Cc: davem, jiri, vadimp, andrew, mlxsw, Ido Schimmel
From: Ido Schimmel <idosch@mellanox.com>
This patchset from Vadim includes various enhancements to thermal and
hwmon code in mlxsw.
Patch #1 adds a thermal zone for each inter-connect device (gearbox).
These devices are present in SN3800 systems and code to expose their
temperature via hwmon was added in commit 2e265a8b6c09 ("mlxsw: core:
Extend hwmon interface with inter-connect temperature attributes").
Currently, there are multiple thermal zones in mlxsw and only a few
cooling devices. Patch #2 detects the hottest thermal zone and the
cooling devices are switched to follow its trends. RFC was sent last
month [1].
Patch #3 allows to read and report negative temperature of the sensors
mlxsw exposes via hwmon and thermal subsystems.
v2 (Andrew Lunn):
* In patch #3, replace '%u' with '%d' in mlxsw_hwmon_module_temp_show()
[1] https://patchwork.ozlabs.org/patch/1107161/
Vadim Pasternak (3):
mlxsw: core: Extend thermal core with per inter-connect device thermal
zones
mlxsw: core: Add the hottest thermal zone detection
mlxsw: core: Add support for negative temperature readout
.../net/ethernet/mellanox/mlxsw/core_hwmon.c | 14 +-
.../ethernet/mellanox/mlxsw/core_thermal.c | 208 +++++++++++++++++-
drivers/net/ethernet/mellanox/mlxsw/reg.h | 12 +-
3 files changed, 209 insertions(+), 25 deletions(-)
--
2.20.1
^ permalink raw reply
* [PATCH net-next v2 1/3] mlxsw: core: Extend thermal core with per inter-connect device thermal zones
From: Ido Schimmel @ 2019-06-24 10:32 UTC (permalink / raw)
To: netdev; +Cc: davem, jiri, vadimp, andrew, mlxsw, Ido Schimmel
In-Reply-To: <20190624103203.22090-1-idosch@idosch.org>
From: Vadim Pasternak <vadimp@mellanox.com>
Add a dedicated thermal zone for each inter-connect device. The
current temperature is obtained from inter-connect temperature sensor
and the default trip points are set to the same values as default ASIC
trip points. These settings could be changed from the user space.
A cooling device (fan) is bound to all inter-connect devices.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/core_thermal.c | 137 +++++++++++++++++-
1 file changed, 136 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
index cfab0e330a47..88f43ad2cc4f 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
@@ -98,7 +98,7 @@ struct mlxsw_thermal_module {
struct thermal_zone_device *tzdev;
struct mlxsw_thermal_trip trips[MLXSW_THERMAL_NUM_TRIPS];
enum thermal_device_mode mode;
- int module;
+ int module; /* Module or gearbox number */
};
struct mlxsw_thermal {
@@ -111,6 +111,8 @@ struct mlxsw_thermal {
struct mlxsw_thermal_trip trips[MLXSW_THERMAL_NUM_TRIPS];
enum thermal_device_mode mode;
struct mlxsw_thermal_module *tz_module_arr;
+ struct mlxsw_thermal_module *tz_gearbox_arr;
+ u8 tz_gearbox_num;
};
static inline u8 mlxsw_state_to_duty(int state)
@@ -554,6 +556,46 @@ static struct thermal_zone_device_ops mlxsw_thermal_module_ops = {
.set_trip_hyst = mlxsw_thermal_module_trip_hyst_set,
};
+static int mlxsw_thermal_gearbox_temp_get(struct thermal_zone_device *tzdev,
+ int *p_temp)
+{
+ struct mlxsw_thermal_module *tz = tzdev->devdata;
+ struct mlxsw_thermal *thermal = tz->parent;
+ char mtmp_pl[MLXSW_REG_MTMP_LEN];
+ unsigned int temp;
+ u16 index;
+ int err;
+
+ index = MLXSW_REG_MTMP_GBOX_INDEX_MIN + tz->module;
+ mlxsw_reg_mtmp_pack(mtmp_pl, index, false, false);
+
+ err = mlxsw_reg_query(thermal->core, MLXSW_REG(mtmp), mtmp_pl);
+ if (err)
+ return err;
+
+ mlxsw_reg_mtmp_unpack(mtmp_pl, &temp, NULL, NULL);
+
+ *p_temp = (int) temp;
+ return 0;
+}
+
+static struct thermal_zone_device_ops mlxsw_thermal_gearbox_ops = {
+ .bind = mlxsw_thermal_module_bind,
+ .unbind = mlxsw_thermal_module_unbind,
+ .get_mode = mlxsw_thermal_module_mode_get,
+ .set_mode = mlxsw_thermal_module_mode_set,
+ .get_temp = mlxsw_thermal_gearbox_temp_get,
+ .get_trip_type = mlxsw_thermal_module_trip_type_get,
+ .get_trip_temp = mlxsw_thermal_module_trip_temp_get,
+ .set_trip_temp = mlxsw_thermal_module_trip_temp_set,
+ .get_trip_hyst = mlxsw_thermal_module_trip_hyst_get,
+ .set_trip_hyst = mlxsw_thermal_module_trip_hyst_set,
+};
+
+static struct thermal_zone_params mlxsw_thermal_gearbox_params = {
+ .governor_name = "user_space",
+};
+
static int mlxsw_thermal_get_max_state(struct thermal_cooling_device *cdev,
unsigned long *p_state)
{
@@ -779,6 +821,92 @@ mlxsw_thermal_modules_fini(struct mlxsw_thermal *thermal)
kfree(thermal->tz_module_arr);
}
+static int
+mlxsw_thermal_gearbox_tz_init(struct mlxsw_thermal_module *gearbox_tz)
+{
+ char tz_name[MLXSW_THERMAL_ZONE_MAX_NAME];
+
+ snprintf(tz_name, sizeof(tz_name), "mlxsw-gearbox%d",
+ gearbox_tz->module + 1);
+ gearbox_tz->tzdev = thermal_zone_device_register(tz_name,
+ MLXSW_THERMAL_NUM_TRIPS,
+ MLXSW_THERMAL_TRIP_MASK,
+ gearbox_tz,
+ &mlxsw_thermal_gearbox_ops,
+ &mlxsw_thermal_gearbox_params,
+ 0, 0);
+ if (IS_ERR(gearbox_tz->tzdev))
+ return PTR_ERR(gearbox_tz->tzdev);
+
+ return 0;
+}
+
+static void
+mlxsw_thermal_gearbox_tz_fini(struct mlxsw_thermal_module *gearbox_tz)
+{
+ thermal_zone_device_unregister(gearbox_tz->tzdev);
+}
+
+static int
+mlxsw_thermal_gearboxes_init(struct device *dev, struct mlxsw_core *core,
+ struct mlxsw_thermal *thermal)
+{
+ struct mlxsw_thermal_module *gearbox_tz;
+ char mgpir_pl[MLXSW_REG_MGPIR_LEN];
+ int i;
+ int err;
+
+ if (!mlxsw_core_res_query_enabled(core))
+ return 0;
+
+ mlxsw_reg_mgpir_pack(mgpir_pl);
+ err = mlxsw_reg_query(core, MLXSW_REG(mgpir), mgpir_pl);
+ if (err)
+ return err;
+
+ mlxsw_reg_mgpir_unpack(mgpir_pl, &thermal->tz_gearbox_num, NULL, NULL);
+ if (!thermal->tz_gearbox_num)
+ return 0;
+
+ thermal->tz_gearbox_arr = kcalloc(thermal->tz_gearbox_num,
+ sizeof(*thermal->tz_gearbox_arr),
+ GFP_KERNEL);
+ if (!thermal->tz_gearbox_arr)
+ return -ENOMEM;
+
+ for (i = 0; i < thermal->tz_gearbox_num; i++) {
+ gearbox_tz = &thermal->tz_gearbox_arr[i];
+ memcpy(gearbox_tz->trips, default_thermal_trips,
+ sizeof(thermal->trips));
+ gearbox_tz->module = i;
+ gearbox_tz->parent = thermal;
+ err = mlxsw_thermal_gearbox_tz_init(gearbox_tz);
+ if (err)
+ goto err_unreg_tz_gearbox;
+ }
+
+ return 0;
+
+err_unreg_tz_gearbox:
+ for (i--; i >= 0; i--)
+ mlxsw_thermal_gearbox_tz_fini(&thermal->tz_gearbox_arr[i]);
+ kfree(thermal->tz_gearbox_arr);
+ return err;
+}
+
+static void
+mlxsw_thermal_gearboxes_fini(struct mlxsw_thermal *thermal)
+{
+ int i;
+
+ if (!mlxsw_core_res_query_enabled(thermal->core))
+ return;
+
+ for (i = thermal->tz_gearbox_num - 1; i >= 0; i--)
+ mlxsw_thermal_gearbox_tz_fini(&thermal->tz_gearbox_arr[i]);
+ kfree(thermal->tz_gearbox_arr);
+}
+
int mlxsw_thermal_init(struct mlxsw_core *core,
const struct mlxsw_bus_info *bus_info,
struct mlxsw_thermal **p_thermal)
@@ -869,10 +997,16 @@ int mlxsw_thermal_init(struct mlxsw_core *core,
if (err)
goto err_unreg_tzdev;
+ err = mlxsw_thermal_gearboxes_init(dev, core, thermal);
+ if (err)
+ goto err_unreg_modules_tzdev;
+
thermal->mode = THERMAL_DEVICE_ENABLED;
*p_thermal = thermal;
return 0;
+err_unreg_modules_tzdev:
+ mlxsw_thermal_modules_fini(thermal);
err_unreg_tzdev:
if (thermal->tzdev) {
thermal_zone_device_unregister(thermal->tzdev);
@@ -891,6 +1025,7 @@ void mlxsw_thermal_fini(struct mlxsw_thermal *thermal)
{
int i;
+ mlxsw_thermal_gearboxes_fini(thermal);
mlxsw_thermal_modules_fini(thermal);
if (thermal->tzdev) {
thermal_zone_device_unregister(thermal->tzdev);
--
2.20.1
^ permalink raw reply related
* [PATCH net-next v2 2/3] mlxsw: core: Add the hottest thermal zone detection
From: Ido Schimmel @ 2019-06-24 10:32 UTC (permalink / raw)
To: netdev; +Cc: davem, jiri, vadimp, andrew, mlxsw, Ido Schimmel
In-Reply-To: <20190624103203.22090-1-idosch@idosch.org>
From: Vadim Pasternak <vadimp@mellanox.com>
When multiple sensors are mapped to the same cooling device, the
cooling device should be set according the worst sensor from the
sensors associated with this cooling device.
Provide the hottest thermal zone detection and enforce cooling device
to follow the temperature trends of the hottest zone only.
Prevent competition for the cooling device control from others zones,
by "stable trend" indication. A cooling device will not perform any
actions associated with a zone with a "stable trend".
When other thermal zone is detected as a hottest, a cooling device is
to be switched to following temperature trends of new hottest zone.
Thermal zone score is represented by 32 bits unsigned integer and
calculated according to the next formula:
For T < TZ<t><i>, where t from {normal trip = 0, high trip = 1, hot
trip = 2, critical = 3}:
TZ<i> score = (T + (TZ<t><i> - T) / 2) / (TZ<t><i> - T) * 256 ** j;
Highest thermal zone score s is set as MAX(TZ<i>score);
Following this formula, if TZ<i> is in trip point higher than TZ<k>,
the higher score is to be always assigned to TZ<i>.
For two thermal zones located at the same kind of trip point, the higher
score will be assigned to the zone which is closer to the next trip
point. Thus, the highest score will always be assigned objectively to
the hottest thermal zone.
All the thermal zones initially are to be configured with mode
"enabled" with the "step_wise" governor.
Signed-off-by: Vadim Pasternak <vadimp@mellanox.com>
Acked-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/core_thermal.c | 75 +++++++++++++++----
1 file changed, 62 insertions(+), 13 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
index 88f43ad2cc4f..504a34d240f7 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core_thermal.c
@@ -23,6 +23,7 @@
#define MLXSW_THERMAL_HYSTERESIS_TEMP 5000 /* 5C */
#define MLXSW_THERMAL_MODULE_TEMP_SHIFT (MLXSW_THERMAL_HYSTERESIS_TEMP * 2)
#define MLXSW_THERMAL_ZONE_MAX_NAME 16
+#define MLXSW_THERMAL_TEMP_SCORE_MAX GENMASK(31, 0)
#define MLXSW_THERMAL_MAX_STATE 10
#define MLXSW_THERMAL_MAX_DUTY 255
/* Minimum and maximum fan allowed speed in percent: from 20% to 100%. Values
@@ -113,6 +114,8 @@ struct mlxsw_thermal {
struct mlxsw_thermal_module *tz_module_arr;
struct mlxsw_thermal_module *tz_gearbox_arr;
u8 tz_gearbox_num;
+ unsigned int tz_highest_score;
+ struct thermal_zone_device *tz_highest_dev;
};
static inline u8 mlxsw_state_to_duty(int state)
@@ -197,6 +200,34 @@ mlxsw_thermal_module_trips_update(struct device *dev, struct mlxsw_core *core,
return 0;
}
+static void mlxsw_thermal_tz_score_update(struct mlxsw_thermal *thermal,
+ struct thermal_zone_device *tzdev,
+ struct mlxsw_thermal_trip *trips,
+ int temp)
+{
+ struct mlxsw_thermal_trip *trip = trips;
+ unsigned int score, delta, i, shift = 1;
+
+ /* Calculate thermal zone score, if temperature is above the critical
+ * threshold score is set to MLXSW_THERMAL_TEMP_SCORE_MAX.
+ */
+ score = MLXSW_THERMAL_TEMP_SCORE_MAX;
+ for (i = MLXSW_THERMAL_TEMP_TRIP_NORM; i < MLXSW_THERMAL_NUM_TRIPS;
+ i++, trip++) {
+ if (temp < trip->temp) {
+ delta = DIV_ROUND_CLOSEST(temp, trip->temp - temp);
+ score = delta * shift;
+ break;
+ }
+ shift *= 256;
+ }
+
+ if (score > thermal->tz_highest_score) {
+ thermal->tz_highest_score = score;
+ thermal->tz_highest_dev = tzdev;
+ }
+}
+
static int mlxsw_thermal_bind(struct thermal_zone_device *tzdev,
struct thermal_cooling_device *cdev)
{
@@ -292,6 +323,9 @@ static int mlxsw_thermal_get_temp(struct thermal_zone_device *tzdev,
return err;
}
mlxsw_reg_mtmp_unpack(mtmp_pl, &temp, NULL, NULL);
+ if (temp > 0)
+ mlxsw_thermal_tz_score_update(thermal, tzdev, thermal->trips,
+ temp);
*p_temp = (int) temp;
return 0;
@@ -353,6 +387,22 @@ static int mlxsw_thermal_set_trip_hyst(struct thermal_zone_device *tzdev,
return 0;
}
+static int mlxsw_thermal_trend_get(struct thermal_zone_device *tzdev,
+ int trip, enum thermal_trend *trend)
+{
+ struct mlxsw_thermal_module *tz = tzdev->devdata;
+ struct mlxsw_thermal *thermal = tz->parent;
+
+ if (trip < 0 || trip >= MLXSW_THERMAL_NUM_TRIPS)
+ return -EINVAL;
+
+ if (tzdev == thermal->tz_highest_dev)
+ return 1;
+
+ *trend = THERMAL_TREND_STABLE;
+ return 0;
+}
+
static struct thermal_zone_device_ops mlxsw_thermal_ops = {
.bind = mlxsw_thermal_bind,
.unbind = mlxsw_thermal_unbind,
@@ -364,6 +414,7 @@ static struct thermal_zone_device_ops mlxsw_thermal_ops = {
.set_trip_temp = mlxsw_thermal_set_trip_temp,
.get_trip_hyst = mlxsw_thermal_get_trip_hyst,
.set_trip_hyst = mlxsw_thermal_set_trip_hyst,
+ .get_trend = mlxsw_thermal_trend_get,
};
static int mlxsw_thermal_module_bind(struct thermal_zone_device *tzdev,
@@ -474,7 +525,9 @@ static int mlxsw_thermal_module_temp_get(struct thermal_zone_device *tzdev,
return 0;
/* Update trip points. */
- mlxsw_thermal_module_trips_update(dev, thermal->core, tz);
+ err = mlxsw_thermal_module_trips_update(dev, thermal->core, tz);
+ if (!err)
+ mlxsw_thermal_tz_score_update(thermal, tzdev, tz->trips, temp);
return 0;
}
@@ -539,10 +592,6 @@ mlxsw_thermal_module_trip_hyst_set(struct thermal_zone_device *tzdev, int trip,
return 0;
}
-static struct thermal_zone_params mlxsw_thermal_module_params = {
- .governor_name = "user_space",
-};
-
static struct thermal_zone_device_ops mlxsw_thermal_module_ops = {
.bind = mlxsw_thermal_module_bind,
.unbind = mlxsw_thermal_module_unbind,
@@ -554,6 +603,7 @@ static struct thermal_zone_device_ops mlxsw_thermal_module_ops = {
.set_trip_temp = mlxsw_thermal_module_trip_temp_set,
.get_trip_hyst = mlxsw_thermal_module_trip_hyst_get,
.set_trip_hyst = mlxsw_thermal_module_trip_hyst_set,
+ .get_trend = mlxsw_thermal_trend_get,
};
static int mlxsw_thermal_gearbox_temp_get(struct thermal_zone_device *tzdev,
@@ -574,6 +624,8 @@ static int mlxsw_thermal_gearbox_temp_get(struct thermal_zone_device *tzdev,
return err;
mlxsw_reg_mtmp_unpack(mtmp_pl, &temp, NULL, NULL);
+ if (temp > 0)
+ mlxsw_thermal_tz_score_update(thermal, tzdev, tz->trips, temp);
*p_temp = (int) temp;
return 0;
@@ -590,10 +642,7 @@ static struct thermal_zone_device_ops mlxsw_thermal_gearbox_ops = {
.set_trip_temp = mlxsw_thermal_module_trip_temp_set,
.get_trip_hyst = mlxsw_thermal_module_trip_hyst_get,
.set_trip_hyst = mlxsw_thermal_module_trip_hyst_set,
-};
-
-static struct thermal_zone_params mlxsw_thermal_gearbox_params = {
- .governor_name = "user_space",
+ .get_trend = mlxsw_thermal_trend_get,
};
static int mlxsw_thermal_get_max_state(struct thermal_cooling_device *cdev,
@@ -709,13 +758,13 @@ mlxsw_thermal_module_tz_init(struct mlxsw_thermal_module *module_tz)
MLXSW_THERMAL_TRIP_MASK,
module_tz,
&mlxsw_thermal_module_ops,
- &mlxsw_thermal_module_params,
- 0, 0);
+ NULL, 0, 0);
if (IS_ERR(module_tz->tzdev)) {
err = PTR_ERR(module_tz->tzdev);
return err;
}
+ module_tz->mode = THERMAL_DEVICE_ENABLED;
return 0;
}
@@ -833,11 +882,11 @@ mlxsw_thermal_gearbox_tz_init(struct mlxsw_thermal_module *gearbox_tz)
MLXSW_THERMAL_TRIP_MASK,
gearbox_tz,
&mlxsw_thermal_gearbox_ops,
- &mlxsw_thermal_gearbox_params,
- 0, 0);
+ NULL, 0, 0);
if (IS_ERR(gearbox_tz->tzdev))
return PTR_ERR(gearbox_tz->tzdev);
+ gearbox_tz->mode = THERMAL_DEVICE_ENABLED;
return 0;
}
--
2.20.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox