* Re: [PATCH net-next] cxgb4: add new T5 pci device id
From: David Miller @ 2017-05-30 18:12 UTC (permalink / raw)
To: ganeshgr; +Cc: netdev, nirranjan
In-Reply-To: <1496125240-7439-1-git-send-email-ganeshgr@chelsio.com>
From: Ganesh Goudar <ganeshgr@chelsio.com>
Date: Tue, 30 May 2017 11:50:40 +0530
> Signed-off-by: Ganesh Goudar <ganeshgr@chelsio.com>
Applied.
^ permalink raw reply
* Re: [PATCH net-next] cxgb4: FW upgrade fixes
From: David Miller @ 2017-05-30 18:13 UTC (permalink / raw)
To: ganeshgr; +Cc: netdev, nirranjan, indranil, arjun, leedom
In-Reply-To: <1496147766-12948-1-git-send-email-ganeshgr@chelsio.com>
From: Ganesh Goudar <ganeshgr@chelsio.com>
Date: Tue, 30 May 2017 18:06:06 +0530
> From: Arjun Vynipadath <arjun@chelsio.com>
>
> Disable FW_OK flag while flashing Firmware. This will help to fix any
> potential mailbox timeouts during Firmware flash.
>
> Grab new devlog parameters after Firmware restart. When we FLASH new
> Firmware onto an adapter, the new Firmware may have the Firmware Device Log
> located at a different memory address or have a different size for it.
>
> Signed-off-by: Arjun Vynipadath <arjun@chelsio.com>
> Signed-off-by: Casey Leedom <leedom@chelsio.com>
> Signed-off-by: Ganesh Goudar <ganeshgr@chelsio.com>
> ---
> v2:
> -Fix spelling mistake, "send" -> "sent"
Applied.
^ permalink raw reply
* Re: [PATCH net-next] cxgb4: Fix netdev_features flag
From: David Miller @ 2017-05-30 18:16 UTC (permalink / raw)
To: ganeshgr; +Cc: netdev, nirranjan, indranil, arjun, swise
In-Reply-To: <1496131224-11688-1-git-send-email-ganeshgr@chelsio.com>
From: Ganesh Goudar <ganeshgr@chelsio.com>
Date: Tue, 30 May 2017 13:30:24 +0530
> From: Arjun Vynipadath <arjun@chelsio.com>
>
> GRO is not supported by Chelsio HW when rx_csum is disabled.
> Update the netdev features flag when rx_csum is modified.
>
> Signed-off-by: Arjun Vynipadath <arjun@chelsio.com>
> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> Signed-off-by: Ganesh Goudar <ganeshgr@chelsio.com>
Applied.
^ permalink raw reply
* Re: [PATCH net 0/3] MAINTAINERS updates
From: David Miller @ 2017-05-30 18:18 UTC (permalink / raw)
To: tariqt; +Cc: netdev, eranbe, majd, saeedm, yishaih
In-Reply-To: <1496133846-18343-1-git-send-email-tariqt@mellanox.com>
From: Tariq Toukan <tariqt@mellanox.com>
Date: Tue, 30 May 2017 11:44:03 +0300
> This patchset contains updates to the MAINTAINERS file.
> In the first patch, I replace Yishai as the maintainer of
> the mlx4_core driver.
>
> In the other two patches we move an RDMA header file from
> the list of the mlx4/mlx5 core driver into the respective
> IB driver, where it belongs.
>
> Series generated against net commit:
> 468b0df61a51 Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf
Series applied, thanks Tariq.
^ permalink raw reply
* Re: [PATCH V6 net-next iproute] ip: Add support for netdev events to monitor
From: Vlad Yasevich @ 2017-05-30 18:26 UTC (permalink / raw)
To: Stephen Hemminger, Vladislav Yasevich
Cc: netdev, dsahern, roopa, jiri, vfalico, andy
In-Reply-To: <20170530101221.1c5cb643@xeon-e3>
On 05/30/2017 01:12 PM, Stephen Hemminger wrote:
> On Sat, 27 May 2017 10:14:36 -0400
> Vladislav Yasevich <vyasevich@gmail.com> wrote:
>
>>
>> +static const char *netdev_events[] = {"NONE",
>> + "REBOOT",
>> + "FEATURE CHANGE",
>> + "BONDING FAILOVER",
>> + "NOTIFY PEERS",
>> + "RESEND IGMP",
>> + "BONDING OPTION"};
>
> Overall this looks fine, I will pickup the if_link.h from net-next.
>
> One stylistic change.
>
> Please add simple line break, and initialize by value:
>
> static const char *netdev_events[] = {
> [IFLA_EVENT_NONE] = "NONE",
> ...
>
> Do you want some prefix or bounding around the event output?
Don't really care about output from my side. If you think some prefix
would be good, I can surely add it.
> Also a little concerned that the output format change may break some program
> could the new output be at the end of the line?
>
I can try moving it to the end.
Thanks
-vlad
^ permalink raw reply
* [PATCH net-next v2 0/6] net: dsa: tagger simplification
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
This patchset removes the unused arguments of the taggers rcv function
and the unneeded labels in their implementations, and handles freeing of
the original SKB in the xmit caller.
Changes in v2:
- do not remove tagger function copies
- document hot path requirements
- make netdev_uses_dsa simpler
- add reviewers' tags
Vivien Didelot (6):
net: dsa: comment hot path requirements
net: dsa: do not cast dst
net: dsa: remove dsa_uses_tagged_protocol
net: dsa: remove unused arguments of tagger rcv
net: dsa: remove out_drop label in taggers rcv
net: dsa: factor skb freeing on xmit
include/net/dsa.h | 15 +++++----------
net/dsa/dsa.c | 2 +-
net/dsa/dsa2.c | 2 +-
net/dsa/dsa_priv.h | 5 ++---
net/dsa/legacy.c | 2 +-
net/dsa/slave.c | 8 ++++++--
net/dsa/tag_brcm.c | 21 ++++++---------------
net/dsa/tag_dsa.c | 25 ++++++++-----------------
net/dsa/tag_edsa.c | 25 ++++++++-----------------
net/dsa/tag_lan9303.c | 8 ++------
net/dsa/tag_mtk.c | 19 +++++--------------
net/dsa/tag_qca.c | 21 ++++++---------------
net/dsa/tag_trailer.c | 17 +++++------------
13 files changed, 56 insertions(+), 114 deletions(-)
--
2.13.0
^ permalink raw reply
* [PATCH net-next v2 1/6] net: dsa: comment hot path requirements
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
The DSA layer uses inline helpers and copies of the tagging functions
for faster access in hot path. Add comments to detail that.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
include/net/dsa.h | 3 +++
net/dsa/dsa_priv.h | 1 +
2 files changed, 4 insertions(+)
diff --git a/include/net/dsa.h b/include/net/dsa.h
index c0e567c0c824..58297e4c6b31 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -126,6 +126,8 @@ struct dsa_switch_tree {
* protocol to use.
*/
struct net_device *master_netdev;
+
+ /* Copy of tag_ops->rcv for faster access in hot path */
struct sk_buff * (*rcv)(struct sk_buff *skb,
struct net_device *dev,
struct packet_type *pt,
@@ -464,6 +466,7 @@ struct mii_bus *dsa_host_dev_to_mii_bus(struct device *dev);
struct net_device *dsa_dev_to_net_device(struct device *dev);
+/* Keep inline for faster access in hot path */
static inline bool dsa_uses_tagged_protocol(struct dsa_switch_tree *dst)
{
return dst->rcv != NULL;
diff --git a/net/dsa/dsa_priv.h b/net/dsa/dsa_priv.h
index c1d4180651af..2ce9fec9eec5 100644
--- a/net/dsa/dsa_priv.h
+++ b/net/dsa/dsa_priv.h
@@ -73,6 +73,7 @@ struct dsa_device_ops {
};
struct dsa_slave_priv {
+ /* Copy of dp->ds->dst->tag_ops->xmit for faster access in hot path */
struct sk_buff * (*xmit)(struct sk_buff *skb,
struct net_device *dev);
--
2.13.0
^ permalink raw reply related
* [PATCH net-next v2 2/6] net: dsa: do not cast dst
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
dsa_ptr is not a void pointer anymore since Nov 2011, as of cf50dcc24f82
("dsa: Change dsa_uses_{dsa, trailer}_tags() into inline functions"),
but an explicit dsa_switch_tree pointer, thus remove the (void *) cast.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
net/dsa/dsa2.c | 2 +-
net/dsa/legacy.c | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
index 4301f52e4f5a..c4afeca6f478 100644
--- a/net/dsa/dsa2.c
+++ b/net/dsa/dsa2.c
@@ -454,7 +454,7 @@ static int dsa_dst_apply(struct dsa_switch_tree *dst)
* sent to the tag format's receive function.
*/
wmb();
- dst->master_netdev->dsa_ptr = (void *)dst;
+ dst->master_netdev->dsa_ptr = dst;
dst->applied = true;
return 0;
diff --git a/net/dsa/legacy.c b/net/dsa/legacy.c
index ac4379b8d7ac..d70a1a788d17 100644
--- a/net/dsa/legacy.c
+++ b/net/dsa/legacy.c
@@ -651,7 +651,7 @@ static int dsa_setup_dst(struct dsa_switch_tree *dst, struct net_device *dev,
* sent to the tag format's receive function.
*/
wmb();
- dev->dsa_ptr = (void *)dst;
+ dev->dsa_ptr = dst;
return 0;
}
--
2.13.0
^ permalink raw reply related
* [PATCH net-next v2 3/6] net: dsa: remove dsa_uses_tagged_protocol
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
Since dev->dsa_ptr is a pointer to a dsa_switch_tree, there is no need
to have another inline helper just to check rcv.
Remove dsa_uses_tagged_protocol and check dsa_ptr && dsa_ptr->rcv
together at the same time.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
include/net/dsa.h | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 58297e4c6b31..4675b52c964c 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -467,16 +467,10 @@ struct mii_bus *dsa_host_dev_to_mii_bus(struct device *dev);
struct net_device *dsa_dev_to_net_device(struct device *dev);
/* Keep inline for faster access in hot path */
-static inline bool dsa_uses_tagged_protocol(struct dsa_switch_tree *dst)
-{
- return dst->rcv != NULL;
-}
-
static inline bool netdev_uses_dsa(struct net_device *dev)
{
#if IS_ENABLED(CONFIG_NET_DSA)
- if (dev->dsa_ptr != NULL)
- return dsa_uses_tagged_protocol(dev->dsa_ptr);
+ return dev->dsa_ptr && dev->dsa_ptr->rcv;
#endif
return false;
}
--
2.13.0
^ permalink raw reply related
* [PATCH net-next v2 4/6] net: dsa: remove unused arguments of tagger rcv
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
The struct dsa_device_ops defines the rcv function with 2 unused
arguments struct packet_type *pt, and struct net_device *orig_dev.
This patch removes them from the definition and implementations.
Reviewed-by: Andrew Lunn <andrew@lunn.ch>
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
include/net/dsa.h | 4 +---
net/dsa/dsa.c | 2 +-
net/dsa/dsa_priv.h | 4 +---
net/dsa/tag_brcm.c | 4 +---
net/dsa/tag_dsa.c | 4 +---
net/dsa/tag_edsa.c | 4 +---
net/dsa/tag_lan9303.c | 3 +--
net/dsa/tag_mtk.c | 4 +---
net/dsa/tag_qca.c | 4 +---
net/dsa/tag_trailer.c | 4 +---
10 files changed, 10 insertions(+), 27 deletions(-)
diff --git a/include/net/dsa.h b/include/net/dsa.h
index 4675b52c964c..9e128e447f0b 100644
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -129,9 +129,7 @@ struct dsa_switch_tree {
/* Copy of tag_ops->rcv for faster access in hot path */
struct sk_buff * (*rcv)(struct sk_buff *skb,
- struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev);
+ struct net_device *dev);
/*
* Original copy of the master netdev ethtool_ops
diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c
index 3288a80d4d6c..58594784ab94 100644
--- a/net/dsa/dsa.c
+++ b/net/dsa/dsa.c
@@ -204,7 +204,7 @@ static int dsa_switch_rcv(struct sk_buff *skb, struct net_device *dev,
if (!skb)
return 0;
- nskb = dst->rcv(skb, dev, pt, orig_dev);
+ nskb = dst->rcv(skb, dev);
if (!nskb) {
kfree_skb(skb);
return 0;
diff --git a/net/dsa/dsa_priv.h b/net/dsa/dsa_priv.h
index 2ce9fec9eec5..1a69d70fcd3a 100644
--- a/net/dsa/dsa_priv.h
+++ b/net/dsa/dsa_priv.h
@@ -67,9 +67,7 @@ struct dsa_notifier_vlan_info {
struct dsa_device_ops {
struct sk_buff *(*xmit)(struct sk_buff *skb, struct net_device *dev);
- struct sk_buff *(*rcv)(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev);
+ struct sk_buff *(*rcv)(struct sk_buff *skb, struct net_device *dev);
};
struct dsa_slave_priv {
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index 9f204f18ada3..c8bff4d7543d 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -92,9 +92,7 @@ static struct sk_buff *brcm_tag_xmit(struct sk_buff *skb, struct net_device *dev
return NULL;
}
-static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
diff --git a/net/dsa/tag_dsa.c b/net/dsa/tag_dsa.c
index 3b62a57956a3..41f78e42f9ba 100644
--- a/net/dsa/tag_dsa.c
+++ b/net/dsa/tag_dsa.c
@@ -68,9 +68,7 @@ static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
return NULL;
}
-static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
diff --git a/net/dsa/tag_edsa.c b/net/dsa/tag_edsa.c
index f95cafd05702..36389fc8fe06 100644
--- a/net/dsa/tag_edsa.c
+++ b/net/dsa/tag_edsa.c
@@ -81,9 +81,7 @@ static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
return NULL;
}
-static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
diff --git a/net/dsa/tag_lan9303.c b/net/dsa/tag_lan9303.c
index afd59330b5f1..82e150486497 100644
--- a/net/dsa/tag_lan9303.c
+++ b/net/dsa/tag_lan9303.c
@@ -71,8 +71,7 @@ static struct sk_buff *lan9303_xmit(struct sk_buff *skb, struct net_device *dev)
return NULL;
}
-static struct sk_buff *lan9303_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt, struct net_device *orig_dev)
+static struct sk_buff *lan9303_rcv(struct sk_buff *skb, struct net_device *dev)
{
u16 *lan9303_tag;
struct dsa_switch_tree *dst = dev->dsa_ptr;
diff --git a/net/dsa/tag_mtk.c b/net/dsa/tag_mtk.c
index d1258e84cd71..7c9764c8d61b 100644
--- a/net/dsa/tag_mtk.c
+++ b/net/dsa/tag_mtk.c
@@ -47,9 +47,7 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb,
return NULL;
}
-static struct sk_buff *mtk_tag_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *mtk_tag_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
diff --git a/net/dsa/tag_qca.c b/net/dsa/tag_qca.c
index 2451007699b7..d38bd5cbfad8 100644
--- a/net/dsa/tag_qca.c
+++ b/net/dsa/tag_qca.c
@@ -66,9 +66,7 @@ static struct sk_buff *qca_tag_xmit(struct sk_buff *skb, struct net_device *dev)
return NULL;
}
-static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
diff --git a/net/dsa/tag_trailer.c b/net/dsa/tag_trailer.c
index 7488ab2932ab..e1fd78197fc8 100644
--- a/net/dsa/tag_trailer.c
+++ b/net/dsa/tag_trailer.c
@@ -58,9 +58,7 @@ static struct sk_buff *trailer_xmit(struct sk_buff *skb, struct net_device *dev)
return nskb;
}
-static struct sk_buff *trailer_rcv(struct sk_buff *skb, struct net_device *dev,
- struct packet_type *pt,
- struct net_device *orig_dev)
+static struct sk_buff *trailer_rcv(struct sk_buff *skb, struct net_device *dev)
{
struct dsa_switch_tree *dst = dev->dsa_ptr;
struct dsa_switch *ds;
--
2.13.0
^ permalink raw reply related
* [PATCH net-next v2 5/6] net: dsa: remove out_drop label in taggers rcv
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
Many rcv functions from net/dsa/tag_*.c have a useless out_drop goto
label which simply returns NULL. Kill it in favor of the obvious.
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
net/dsa/tag_brcm.c | 11 ++++-------
net/dsa/tag_dsa.c | 13 +++++--------
net/dsa/tag_edsa.c | 13 +++++--------
net/dsa/tag_mtk.c | 9 +++------
net/dsa/tag_qca.c | 11 ++++-------
net/dsa/tag_trailer.c | 9 +++------
6 files changed, 24 insertions(+), 42 deletions(-)
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index c8bff4d7543d..10fa4c0ca46b 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -102,27 +102,27 @@ static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev)
ds = dst->cpu_dp->ds;
if (unlikely(!pskb_may_pull(skb, BRCM_TAG_LEN)))
- goto out_drop;
+ return NULL;
/* skb->data points to the EtherType, the tag is right before it */
brcm_tag = skb->data - 2;
/* The opcode should never be different than 0b000 */
if (unlikely((brcm_tag[0] >> BRCM_OPCODE_SHIFT) & BRCM_OPCODE_MASK))
- goto out_drop;
+ return NULL;
/* We should never see a reserved reason code without knowing how to
* handle it
*/
if (unlikely(brcm_tag[2] & BRCM_EG_RC_RSVD))
- goto out_drop;
+ return NULL;
/* Locate which port this is coming from */
source_port = brcm_tag[3] & BRCM_EG_PID_MASK;
/* Validate port against switch setup, either the port is totally */
if (source_port >= ds->num_ports || !ds->ports[source_port].netdev)
- goto out_drop;
+ return NULL;
/* Remove Broadcom tag and update checksum */
skb_pull_rcsum(skb, BRCM_TAG_LEN);
@@ -135,9 +135,6 @@ static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev)
skb->dev = ds->ports[source_port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops brcm_netdev_ops = {
diff --git a/net/dsa/tag_dsa.c b/net/dsa/tag_dsa.c
index 41f78e42f9ba..9f5fecaa4a93 100644
--- a/net/dsa/tag_dsa.c
+++ b/net/dsa/tag_dsa.c
@@ -77,7 +77,7 @@ static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
int source_port;
if (unlikely(!pskb_may_pull(skb, DSA_HLEN)))
- goto out_drop;
+ return NULL;
/*
* The ethertype field is part of the DSA header.
@@ -88,7 +88,7 @@ static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
* Check that frame type is either TO_CPU or FORWARD.
*/
if ((dsa_header[0] & 0xc0) != 0x00 && (dsa_header[0] & 0xc0) != 0xc0)
- goto out_drop;
+ return NULL;
/*
* Determine source device and port.
@@ -101,14 +101,14 @@ static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
* port is a registered DSA port.
*/
if (source_device >= DSA_MAX_SWITCHES)
- goto out_drop;
+ return NULL;
ds = dst->ds[source_device];
if (!ds)
- goto out_drop;
+ return NULL;
if (source_port >= ds->num_ports || !ds->ports[source_port].netdev)
- goto out_drop;
+ return NULL;
/*
* Convert the DSA header to an 802.1q header if the 'tagged'
@@ -159,9 +159,6 @@ static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
skb->dev = ds->ports[source_port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops dsa_netdev_ops = {
diff --git a/net/dsa/tag_edsa.c b/net/dsa/tag_edsa.c
index 36389fc8fe06..a9a06e19abeb 100644
--- a/net/dsa/tag_edsa.c
+++ b/net/dsa/tag_edsa.c
@@ -90,7 +90,7 @@ static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
int source_port;
if (unlikely(!pskb_may_pull(skb, EDSA_HLEN)))
- goto out_drop;
+ return NULL;
/*
* Skip the two null bytes after the ethertype.
@@ -101,7 +101,7 @@ static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
* Check that frame type is either TO_CPU or FORWARD.
*/
if ((edsa_header[0] & 0xc0) != 0x00 && (edsa_header[0] & 0xc0) != 0xc0)
- goto out_drop;
+ return NULL;
/*
* Determine source device and port.
@@ -114,14 +114,14 @@ static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
* port is a registered DSA port.
*/
if (source_device >= DSA_MAX_SWITCHES)
- goto out_drop;
+ return NULL;
ds = dst->ds[source_device];
if (!ds)
- goto out_drop;
+ return NULL;
if (source_port >= ds->num_ports || !ds->ports[source_port].netdev)
- goto out_drop;
+ return NULL;
/*
* If the 'tagged' bit is set, convert the DSA tag to a 802.1q
@@ -178,9 +178,6 @@ static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
skb->dev = ds->ports[source_port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops edsa_netdev_ops = {
diff --git a/net/dsa/tag_mtk.c b/net/dsa/tag_mtk.c
index 7c9764c8d61b..327dd7b596df 100644
--- a/net/dsa/tag_mtk.c
+++ b/net/dsa/tag_mtk.c
@@ -55,7 +55,7 @@ static struct sk_buff *mtk_tag_rcv(struct sk_buff *skb, struct net_device *dev)
__be16 *phdr, hdr;
if (unlikely(!pskb_may_pull(skb, MTK_HDR_LEN)))
- goto out_drop;
+ return NULL;
/* The MTK header is added by the switch between src addr
* and ethertype at this point, skb->data points to 2 bytes
@@ -77,19 +77,16 @@ static struct sk_buff *mtk_tag_rcv(struct sk_buff *skb, struct net_device *dev)
*/
ds = dst->ds[0];
if (!ds)
- goto out_drop;
+ return NULL;
/* Get source port information */
port = (hdr & MTK_HDR_RECV_SOURCE_PORT_MASK);
if (!ds->ports[port].netdev)
- goto out_drop;
+ return NULL;
skb->dev = ds->ports[port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops mtk_netdev_ops = {
diff --git a/net/dsa/tag_qca.c b/net/dsa/tag_qca.c
index d38bd5cbfad8..8080ad8f2abb 100644
--- a/net/dsa/tag_qca.c
+++ b/net/dsa/tag_qca.c
@@ -75,7 +75,7 @@ static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev)
__be16 *phdr, hdr;
if (unlikely(!pskb_may_pull(skb, QCA_HDR_LEN)))
- goto out_drop;
+ return NULL;
/* The QCA header is added by the switch between src addr and Ethertype
* At this point, skb->data points to ethertype so header should be
@@ -87,7 +87,7 @@ static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev)
/* Make sure the version is correct */
ver = (hdr & QCA_HDR_RECV_VERSION_MASK) >> QCA_HDR_RECV_VERSION_S;
if (unlikely(ver != QCA_HDR_VERSION))
- goto out_drop;
+ return NULL;
/* Remove QCA tag and recalculate checksum */
skb_pull_rcsum(skb, QCA_HDR_LEN);
@@ -99,20 +99,17 @@ static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev)
*/
ds = dst->cpu_dp->ds;
if (!ds)
- goto out_drop;
+ return NULL;
/* Get source port information */
port = (hdr & QCA_HDR_RECV_SOURCE_PORT_MASK);
if (!ds->ports[port].netdev)
- goto out_drop;
+ return NULL;
/* Update skb & forward the frame accordingly */
skb->dev = ds->ports[port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops qca_netdev_ops = {
diff --git a/net/dsa/tag_trailer.c b/net/dsa/tag_trailer.c
index e1fd78197fc8..7422b1329712 100644
--- a/net/dsa/tag_trailer.c
+++ b/net/dsa/tag_trailer.c
@@ -68,25 +68,22 @@ static struct sk_buff *trailer_rcv(struct sk_buff *skb, struct net_device *dev)
ds = dst->cpu_dp->ds;
if (skb_linearize(skb))
- goto out_drop;
+ return NULL;
trailer = skb_tail_pointer(skb) - 4;
if (trailer[0] != 0x80 || (trailer[1] & 0xf8) != 0x00 ||
(trailer[2] & 0xef) != 0x00 || trailer[3] != 0x00)
- goto out_drop;
+ return NULL;
source_port = trailer[1] & 7;
if (source_port >= ds->num_ports || !ds->ports[source_port].netdev)
- goto out_drop;
+ return NULL;
pskb_trim_rcsum(skb, skb->len - 4);
skb->dev = ds->ports[source_port].netdev;
return skb;
-
-out_drop:
- return NULL;
}
const struct dsa_device_ops trailer_netdev_ops = {
--
2.13.0
^ permalink raw reply related
* [PATCH net-next v2 6/6] net: dsa: factor skb freeing on xmit
From: Vivien Didelot @ 2017-05-30 18:33 UTC (permalink / raw)
To: netdev
Cc: linux-kernel, kernel, David S. Miller, Florian Fainelli,
Andrew Lunn, Vivien Didelot
In-Reply-To: <20170530183319.10718-1-vivien.didelot@savoirfairelinux.com>
The taggers are currently responsible to free the original SKB if they
made a copy of it, or in case of error.
This patch simplifies this by freeing the original SKB in the
dsa_slave_xmit caller, but only if an error (NULL) is returned.
It is still the responsibility of the tagger to free the original SKB if
it returned a copy of it.
At the same time, fix the checkpatch NULL comparison check:
CHECK: Comparison to NULL could be written "!nskb"
#208: FILE: net/dsa/tag_trailer.c:35:
+ if (nskb == NULL)
Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
---
net/dsa/slave.c | 8 ++++++--
net/dsa/tag_brcm.c | 6 +-----
net/dsa/tag_dsa.c | 8 ++------
net/dsa/tag_edsa.c | 8 ++------
net/dsa/tag_lan9303.c | 5 +----
net/dsa/tag_mtk.c | 6 +-----
net/dsa/tag_qca.c | 6 +-----
net/dsa/tag_trailer.c | 4 +---
8 files changed, 15 insertions(+), 36 deletions(-)
diff --git a/net/dsa/slave.c b/net/dsa/slave.c
index 887e26695519..4473aec9dcda 100644
--- a/net/dsa/slave.c
+++ b/net/dsa/slave.c
@@ -362,10 +362,14 @@ static netdev_tx_t dsa_slave_xmit(struct sk_buff *skb, struct net_device *dev)
dev->stats.tx_packets++;
dev->stats.tx_bytes += skb->len;
- /* Transmit function may have to reallocate the original SKB */
+ /* Transmit function may have to reallocate the original SKB,
+ * in which case it must have freed it. Only free it here on error.
+ */
nskb = p->xmit(skb, dev);
- if (!nskb)
+ if (!nskb) {
+ kfree_skb(skb);
return NETDEV_TX_OK;
+ }
/* SKB for netpoll still need to be mangled with the protocol-specific
* tag to be successfully transmitted
diff --git a/net/dsa/tag_brcm.c b/net/dsa/tag_brcm.c
index 10fa4c0ca46b..5db6bcfde025 100644
--- a/net/dsa/tag_brcm.c
+++ b/net/dsa/tag_brcm.c
@@ -65,7 +65,7 @@ static struct sk_buff *brcm_tag_xmit(struct sk_buff *skb, struct net_device *dev
u8 *brcm_tag;
if (skb_cow_head(skb, BRCM_TAG_LEN) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, BRCM_TAG_LEN);
@@ -86,10 +86,6 @@ static struct sk_buff *brcm_tag_xmit(struct sk_buff *skb, struct net_device *dev
brcm_tag[3] = (1 << p->dp->index) & BRCM_IG_DSTMAP1_MASK;
return skb;
-
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *brcm_tag_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_dsa.c b/net/dsa/tag_dsa.c
index 9f5fecaa4a93..6837ca9160a8 100644
--- a/net/dsa/tag_dsa.c
+++ b/net/dsa/tag_dsa.c
@@ -28,7 +28,7 @@ static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
*/
if (skb->protocol == htons(ETH_P_8021Q)) {
if (skb_cow_head(skb, 0) < 0)
- goto out_free;
+ return NULL;
/*
* Construct tagged FROM_CPU DSA tag from 802.1q tag.
@@ -46,7 +46,7 @@ static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
}
} else {
if (skb_cow_head(skb, DSA_HLEN) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, DSA_HLEN);
memmove(skb->data, skb->data + DSA_HLEN, 2 * ETH_ALEN);
@@ -62,10 +62,6 @@ static struct sk_buff *dsa_xmit(struct sk_buff *skb, struct net_device *dev)
}
return skb;
-
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *dsa_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_edsa.c b/net/dsa/tag_edsa.c
index a9a06e19abeb..96ddc90472a2 100644
--- a/net/dsa/tag_edsa.c
+++ b/net/dsa/tag_edsa.c
@@ -30,7 +30,7 @@ static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
*/
if (skb->protocol == htons(ETH_P_8021Q)) {
if (skb_cow_head(skb, DSA_HLEN) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, DSA_HLEN);
memmove(skb->data, skb->data + DSA_HLEN, 2 * ETH_ALEN);
@@ -55,7 +55,7 @@ static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
}
} else {
if (skb_cow_head(skb, EDSA_HLEN) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, EDSA_HLEN);
memmove(skb->data, skb->data + EDSA_HLEN, 2 * ETH_ALEN);
@@ -75,10 +75,6 @@ static struct sk_buff *edsa_xmit(struct sk_buff *skb, struct net_device *dev)
}
return skb;
-
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *edsa_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_lan9303.c b/net/dsa/tag_lan9303.c
index 82e150486497..e33348101e8f 100644
--- a/net/dsa/tag_lan9303.c
+++ b/net/dsa/tag_lan9303.c
@@ -52,7 +52,7 @@ static struct sk_buff *lan9303_xmit(struct sk_buff *skb, struct net_device *dev)
if (skb_cow_head(skb, LAN9303_TAG_LEN) < 0) {
dev_dbg(&dev->dev,
"Cannot make room for the special tag. Dropping packet\n");
- goto out_free;
+ return NULL;
}
/* provide 'LAN9303_TAG_LEN' bytes additional space */
@@ -66,9 +66,6 @@ static struct sk_buff *lan9303_xmit(struct sk_buff *skb, struct net_device *dev)
lan9303_tag[1] = htons(p->dp->index | BIT(4));
return skb;
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *lan9303_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_mtk.c b/net/dsa/tag_mtk.c
index 327dd7b596df..ee62ae5f03e1 100644
--- a/net/dsa/tag_mtk.c
+++ b/net/dsa/tag_mtk.c
@@ -27,7 +27,7 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb,
u8 *mtk_tag;
if (skb_cow_head(skb, MTK_HDR_LEN) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, MTK_HDR_LEN);
@@ -41,10 +41,6 @@ static struct sk_buff *mtk_tag_xmit(struct sk_buff *skb,
mtk_tag[3] = 0;
return skb;
-
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *mtk_tag_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_qca.c b/net/dsa/tag_qca.c
index 8080ad8f2abb..4d3bc2920477 100644
--- a/net/dsa/tag_qca.c
+++ b/net/dsa/tag_qca.c
@@ -45,7 +45,7 @@ static struct sk_buff *qca_tag_xmit(struct sk_buff *skb, struct net_device *dev)
dev->stats.tx_bytes += skb->len;
if (skb_cow_head(skb, 0) < 0)
- goto out_free;
+ return NULL;
skb_push(skb, QCA_HDR_LEN);
@@ -60,10 +60,6 @@ static struct sk_buff *qca_tag_xmit(struct sk_buff *skb, struct net_device *dev)
*phdr = htons(hdr);
return skb;
-
-out_free:
- kfree_skb(skb);
- return NULL;
}
static struct sk_buff *qca_tag_rcv(struct sk_buff *skb, struct net_device *dev)
diff --git a/net/dsa/tag_trailer.c b/net/dsa/tag_trailer.c
index 7422b1329712..8c9a7f5ecc66 100644
--- a/net/dsa/tag_trailer.c
+++ b/net/dsa/tag_trailer.c
@@ -32,10 +32,8 @@ static struct sk_buff *trailer_xmit(struct sk_buff *skb, struct net_device *dev)
padlen = 60 - skb->len;
nskb = alloc_skb(NET_IP_ALIGN + skb->len + padlen + 4, GFP_ATOMIC);
- if (nskb == NULL) {
- kfree_skb(skb);
+ if (!nskb)
return NULL;
- }
skb_reserve(nskb, NET_IP_ALIGN);
skb_reset_mac_header(nskb);
--
2.13.0
^ permalink raw reply related
* [PATCH v2 net-next 1/1] netvsc: Add #include's for csum_* function declarations
From: mikelley @ 2017-05-30 18:36 UTC (permalink / raw)
To: davem, netdev, linux-kernel, devel, olaf, apw, jasowang,
leann.ogasawara, marcelo.cerri, sthemmin
Cc: Michael Kelley
From: Michael Kelley <mikelley@microsoft.com>
Add direct #include statements for declarations of csum_tcpudp_magic()
and csum_ipv6_magic(). While the needed #include's are picked up
indirectly for the x86 architecture, they aren't on other
architectures, resulting in compile errors.
Signed-off-by: Michael Kelley <mikelley@microsoft.com>
---
drivers/net/hyperv/netvsc_drv.c | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
index 4421a6d..2564ac8 100644
--- a/drivers/net/hyperv/netvsc_drv.c
+++ b/drivers/net/hyperv/netvsc_drv.c
@@ -37,6 +37,8 @@
#include <net/route.h>
#include <net/sock.h>
#include <net/pkt_sched.h>
+#include <net/checksum.h>
+#include <net/ip6_checksum.h>
#include "hyperv_net.h"
--
1.7.1
^ permalink raw reply related
* Re: [PATCH V2 2/3] net-next: dsa: add multi cpu port support
From: John Crispin @ 2017-05-30 18:37 UTC (permalink / raw)
To: kbuild test robot
Cc: kbuild-all, Andrew Lunn, Vivien Didelot, Florian Fainelli,
David S . Miller, Sean Wang, netdev, linux-kernel
In-Reply-To: <201705302336.BC8366CF%fengguang.wu@intel.com>
Hi,
the patch series is based on net-next from 12 hours ago and works fine
on that tree. I compile and runtime tested it quite intensively on
various boards
John
On 30/05/17 17:38, kbuild test robot wrote:
> Hi John,
>
> [auto build test ERROR on net-next/master]
> [also build test ERROR on next-20170530]
> [cannot apply to v4.12-rc3]
> [if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
>
> url: https://github.com/0day-ci/linux/commits/John-Crispin/Documentation-devicetree-add-multiple-cpu-port-DSA-binding/20170530-224954
> config: x86_64-randconfig-x014-201722 (attached as .config)
> compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
> reproduce:
> # save the attached .config to linux build tree
> make ARCH=x86_64
>
> All error/warnings (new ones prefixed by >>):
>
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
> net//dsa/dsa2.c: In function 'dsa_ds_parse':
>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of 'dsa_is_normal_port' from incompatible pointer type [-Werror=incompatible-pointer-types]
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:160:30: note: in definition of macro '__trace_if'
> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but argument is of type 'struct dsa_port *'
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
>>> net//dsa/dsa2.c:574:12: error: too few arguments to function 'dsa_is_normal_port'
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:160:30: note: in definition of macro '__trace_if'
> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: declared here
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of 'dsa_is_normal_port' from incompatible pointer type [-Werror=incompatible-pointer-types]
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:160:42: note: in definition of macro '__trace_if'
> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but argument is of type 'struct dsa_port *'
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
>>> net//dsa/dsa2.c:574:12: error: too few arguments to function 'dsa_is_normal_port'
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:160:42: note: in definition of macro '__trace_if'
> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: declared here
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of 'dsa_is_normal_port' from incompatible pointer type [-Werror=incompatible-pointer-types]
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:171:16: note: in definition of macro '__trace_if'
> ______r = !!(cond); \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but argument is of type 'struct dsa_port *'
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> In file included from include/linux/ioport.h:12:0,
> from include/linux/device.h:16,
> from net//dsa/dsa2.c:13:
>>> net//dsa/dsa2.c:574:12: error: too few arguments to function 'dsa_is_normal_port'
> else if (dsa_is_normal_port(port))
> ^
> include/linux/compiler.h:171:16: note: in definition of macro '__trace_if'
> ______r = !!(cond); \
> ^~~~
>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
> else if (dsa_is_normal_port(port))
> ^~
> In file included from net//dsa/dsa_priv.h:17:0,
> from net//dsa/dsa2.c:22:
> include/net/dsa.h:264:20: note: declared here
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^~~~~~~~~~~~~~~~~~
> cc1: some warnings being treated as errors
>
> vim +/dsa_is_normal_port +574 net//dsa/dsa2.c
>
> 568 port = &ds->ports[index];
> 569 if (!dsa_port_is_valid(port))
> 570 continue;
> 571
> 572 if (dsa_port_is_cpu(port))
> 573 err = dsa_cpu_parse(port, index, dst, ds);
> > 574 else if (dsa_is_normal_port(port))
> 575 err = dsa_user_parse(port->dn, index, ds);
> 576
> 577 if (err)
>
> ---
> 0-DAY kernel test infrastructure Open Source Technology Center
> https://lists.01.org/pipermail/kbuild-all Intel Corporation
^ permalink raw reply
* Re: [PATCH v3 0/6] stmmac: pci: Refactor DMI probing
From: Andy Shevchenko @ 2017-05-30 18:48 UTC (permalink / raw)
To: Jan Kiszka
Cc: Giuseppe Cavallaro, Alexandre Torgue, David Miller, netdev,
linux-kernel@vger.kernel.org
In-Reply-To: <cover.1496129604.git.jan.kiszka@siemens.com>
On Tue, May 30, 2017 at 10:33 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote:
> Some cleanups of the way we probe DMI platforms in the driver. Reduces
> a bit of open-coding and makes the logic easier reusable for any
> potential DMI platform != Quark.
>
> Tested on IOT2000 and Galileo Gen2.
So, for patches 1-4,
Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com>
I'm not convinced that patch 6 is needed, so, my vote is not to include it.
Patch 5 I would like to review later. Main problem to me is {-1, -1}.
It looks non-standard and feels not good. I'll try to think on the
solution.
> Jan Kiszka (6):
> stmmac: pci: Make stmmac_pci_info structure constant
> stmmac: pci: Use stmmac_pci_info for all devices
> stmmac: pci: Make stmmac_pci_find_phy_addr truly generic
> stmmac: pci: Select quark_pci_dmi_data from quark_default_data
> stmmac: pci: Use dmi_system_id table for retrieving PHY addresses
> stmmac: pci: Remove setup handler indirection via stmmac_pci_info
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply
* Re: [PATCH net-next v2 4/6] net: dsa: remove unused arguments of tagger rcv
From: Florian Fainelli @ 2017-05-30 19:03 UTC (permalink / raw)
To: Vivien Didelot, netdev; +Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <20170530183319.10718-5-vivien.didelot@savoirfairelinux.com>
Hey Vivien,
On 05/30/2017 11:33 AM, Vivien Didelot wrote:
> The struct dsa_device_ops defines the rcv function with 2 unused
> arguments struct packet_type *pt, and struct net_device *orig_dev.
>
> This patch removes them from the definition and implementations.
>
> Reviewed-by: Andrew Lunn <andrew@lunn.ch>
> Signed-off-by: Vivien Didelot <vivien.didelot@savoirfairelinux.com>
I actually have a patch pending that adds support for HW
insertion/extraction of switch tags (broadcom HW supports that) which
require the orig_dev to be there so we know what features are supported
by the master network device.
Do you mind dropping this one from your patch series?
Thanks!
--
Florian
^ permalink raw reply
* Re: [PATCH v2 net-next 1/3] perf, bpf: Add BPF support to all perf_event types
From: Peter Zijlstra @ 2017-05-30 19:03 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: David S . Miller, Brendan Gregg, Daniel Borkmann, Teng Qin,
netdev, linux-kernel
In-Reply-To: <132719d0-a18c-d4a0-8e72-56658c0c344f@fb.com>
On Tue, May 30, 2017 at 10:37:46AM -0700, Alexei Starovoitov wrote:
> On 5/30/17 9:51 AM, Peter Zijlstra wrote:
> > I'm not entirely sure I see how that is required. Should per task not
> > already work? The WARN that's there will only trigger if you call them
> > on the wrong task, which is something you shouldn't do anyway.
>
> The kernel WARN is considered to be a bug of bpf infra. That's the
> reason we do all these checks at map update time and at run-time.
> The bpf program authors should be able to do all possible experiments
> until their scripts work. Dealing with kernel warns and reboots is not
> something user space folks like to do.
> Today bpf_perf_event_read() for per-task events isn't really
> working due to event->oncpu != cpu runtime check in there.
> If we convert warns to returns the existing scripts will continue
> to work as-is and per-task will be possible.
Ah yes.. I always forget that side of things (not ever having seen a
bpf thing working doesn't help either of course).
Let me consider that for a bit..
^ permalink raw reply
* Re: [PATCH V2 2/3] net-next: dsa: add multi cpu port support
From: Florian Fainelli @ 2017-05-30 19:15 UTC (permalink / raw)
To: John Crispin, kbuild test robot
Cc: kbuild-all, Andrew Lunn, Vivien Didelot, David S . Miller,
Sean Wang, netdev, linux-kernel
In-Reply-To: <7d8cdddb-f107-9945-f366-12d320ecb188@phrozen.org>
Hi John,
On 05/30/2017 11:37 AM, John Crispin wrote:
> Hi,
>
> the patch series is based on net-next from 12 hours ago and works fine
> on that tree. I compile and runtime tested it quite intensively on
> various boards
The warning is legit though:
572 if (dsa_port_is_cpu(port))
573 err = dsa_cpu_parse(port, index, dst, ds);
> 574 else if (dsa_is_normal_port(port))
575 err = dsa_user_parse(port->dn, index, ds);
and after applying your patches I also met it:
net/dsa/dsa2.c: In function 'dsa_ds_parse':
net/dsa/dsa2.c:574:3: warning: passing argument 1 of
'dsa_is_normal_port' from incompatible pointer type [enabled by default]
else if (dsa_is_normal_port(port))
^
In file included from net/dsa/dsa_priv.h:17:0,
from net/dsa/dsa2.c:22:
./include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but
argument is of type 'struct dsa_port *'
static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
^
net/dsa/dsa2.c:574:3: error: too few arguments to function
'dsa_is_normal_port'
else if (dsa_is_normal_port(port))
^
In file included from net/dsa/dsa_priv.h:17:0,
from net/dsa/dsa2.c:22:
./include/net/dsa.h:264:20: note: declared here
static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
^
CC net/bridge/br_stp.o
scripts/Makefile.build:302: recipe for target 'net/dsa/dsa2.o' failed
make[4]: *** [net/dsa/dsa2.o] Error 1
scripts/Makefile.build:561: recipe for target 'net/dsa' failed
make[3]: *** [net/dsa] Error 2
make[3]: *** Waiting for unfinished jobs....
We need something like this, I have some comments on your patches that I
will send shortly. Thanks!
diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
index 8b13aa735c40..124c5acfa123 100644
--- a/net/dsa/dsa2.c
+++ b/net/dsa/dsa2.c
@@ -539,14 +539,14 @@ static int dsa_cpu_parse(struct dsa_port *port,
u32 index,
return 0;
}
-static int dsa_user_parse(struct device_node *port, u32 index,
+static int dsa_user_parse(struct dsa_port *port, u32 index,
struct dsa_switch *ds)
{
struct device_node *cpu_port;
const unsigned int *cpu_port_reg;
int cpu_port_index;
- cpu_port = of_parse_phandle(port, "cpu", 0);
+ cpu_port = of_parse_phandle(port->dn, "cpu", 0);
if (cpu_port) {
cpu_port_reg = of_get_property(cpu_port, "reg", NULL);
if (!cpu_port_reg)
@@ -572,7 +572,7 @@ static int dsa_ds_parse(struct dsa_switch_tree *dst,
struct dsa_switch *ds)
if (dsa_port_is_cpu(port))
err = dsa_cpu_parse(port, index, dst, ds);
else if (dsa_is_normal_port(port))
- err = dsa_user_parse(port->dn, index, ds);
+ err = dsa_user_parse(port, index, ds);
if (err)
return err;
>
> John
>
>
> On 30/05/17 17:38, kbuild test robot wrote:
>> Hi John,
>>
>> [auto build test ERROR on net-next/master]
>> [also build test ERROR on next-20170530]
>> [cannot apply to v4.12-rc3]
>> [if your patch is applied to the wrong git tree, please drop us a note
>> to help improve the system]
>>
>> url:
>> https://github.com/0day-ci/linux/commits/John-Crispin/Documentation-devicetree-add-multiple-cpu-port-DSA-binding/20170530-224954
>>
>> config: x86_64-randconfig-x014-201722 (attached as .config)
>> compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
>> reproduce:
>> # save the attached .config to linux build tree
>> make ARCH=x86_64
>>
>> All error/warnings (new ones prefixed by >>):
>>
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>> net//dsa/dsa2.c: In function 'dsa_ds_parse':
>>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of
>>>> 'dsa_is_normal_port' from incompatible pointer type
>>>> [-Werror=incompatible-pointer-types]
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:160:30: note: in definition of macro
>> '__trace_if'
>> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but
>> argument is of type 'struct dsa_port *'
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>>>> net//dsa/dsa2.c:574:12: error: too few arguments to function
>>>> 'dsa_is_normal_port'
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:160:30: note: in definition of macro
>> '__trace_if'
>> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: declared here
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of
>>>> 'dsa_is_normal_port' from incompatible pointer type
>>>> [-Werror=incompatible-pointer-types]
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:160:42: note: in definition of macro
>> '__trace_if'
>> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but
>> argument is of type 'struct dsa_port *'
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>>>> net//dsa/dsa2.c:574:12: error: too few arguments to function
>>>> 'dsa_is_normal_port'
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:160:42: note: in definition of macro
>> '__trace_if'
>> if (__builtin_constant_p(!!(cond)) ? !!(cond) : \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: declared here
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>>>> net//dsa/dsa2.c:574:31: error: passing argument 1 of
>>>> 'dsa_is_normal_port' from incompatible pointer type
>>>> [-Werror=incompatible-pointer-types]
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:171:16: note: in definition of macro
>> '__trace_if'
>> ______r = !!(cond); \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but
>> argument is of type 'struct dsa_port *'
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> In file included from include/linux/ioport.h:12:0,
>> from include/linux/device.h:16,
>> from net//dsa/dsa2.c:13:
>>>> net//dsa/dsa2.c:574:12: error: too few arguments to function
>>>> 'dsa_is_normal_port'
>> else if (dsa_is_normal_port(port))
>> ^
>> include/linux/compiler.h:171:16: note: in definition of macro
>> '__trace_if'
>> ______r = !!(cond); \
>> ^~~~
>>>> net//dsa/dsa2.c:574:8: note: in expansion of macro 'if'
>> else if (dsa_is_normal_port(port))
>> ^~
>> In file included from net//dsa/dsa_priv.h:17:0,
>> from net//dsa/dsa2.c:22:
>> include/net/dsa.h:264:20: note: declared here
>> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
>> ^~~~~~~~~~~~~~~~~~
>> cc1: some warnings being treated as errors
>>
>> vim +/dsa_is_normal_port +574 net//dsa/dsa2.c
>>
>> 568 port = &ds->ports[index];
>> 569 if (!dsa_port_is_valid(port))
>> 570 continue;
>> 571
>> 572 if (dsa_port_is_cpu(port))
>> 573 err = dsa_cpu_parse(port, index, dst, ds);
>> > 574 else if (dsa_is_normal_port(port))
>> 575 err = dsa_user_parse(port->dn, index, ds);
>> 576
>> 577 if (err)
>>
>> ---
>> 0-DAY kernel test infrastructure Open Source Technology
>> Center
>> https://lists.01.org/pipermail/kbuild-all Intel
>> Corporation
>
--
Florian
^ permalink raw reply related
* Re: [PATCH v2] arm: eBPF JIT compiler
From: Kees Cook @ 2017-05-30 19:19 UTC (permalink / raw)
To: Shubham Bansal, Network Development, Daniel Borkmann,
David S. Miller, Alexei Starovoitov
Cc: Russell King, linux-arm-kernel@lists.infradead.org, LKML,
Andrew Lunn
In-Reply-To: <1495754003-21099-1-git-send-email-illusionist.neo@gmail.com>
Forwarding this to net-dev and eBPF folks, who weren't on CC...
-Kees
On Thu, May 25, 2017 at 4:13 PM, Shubham Bansal
<illusionist.neo@gmail.com> wrote:
> The JIT compiler emits ARM 32 bit instructions. Currently, It supports
> eBPF only. Classic BPF is supported because of the conversion by BPF
> core.
>
> This patch is essentially changing the current implementation of JIT
> compiler of Berkeley Packet Filter from classic to internal with almost
> all instructions from eBPF ISA supported except the following
> BPF_ALU64 | BPF_DIV | BPF_K
> BPF_ALU64 | BPF_DIV | BPF_X
> BPF_ALU64 | BPF_MOD | BPF_K
> BPF_ALU64 | BPF_MOD | BPF_X
> BPF_STX | BPF_XADD | BPF_W
> BPF_STX | BPF_XADD | BPF_DW
> BPF_JMP | BPF_CALL
>
> Implementation is using scratch space to emulate 64 bit eBPF ISA on 32 bit
> ARM because of deficiency of general purpose registers on ARM. Currently,
> only LITTLE ENDIAN machines are supported in this eBPF JIT Compiler.
>
> Tested on ARMv7 with QEMU by me (Shubham Bansal).
> Tested on ARMv5 by Andrew Lunn (andrew@lunn.ch).
> Expected to work on ARMv6 as well, as its a part ARMv7 and part ARMv5.
> Although, a proper testing is not done for ARMv6.
>
> Both of these testing are done with and without CONFIG_FRAME_POINTER
> separately for LITTLE ENDIAN machine.
>
> For testing:
>
> 1. JIT is enabled with
> echo 1 > /proc/sys/net/core/bpf_jit_enable
> 2. Constant Blinding can be enabled along with JIT using
> echo 1 > /proc/sys/net/core/bpf_jit_enable
> echo 2 > /proc/sys/net/core/bpf_jit_harden
>
> See Documentation/networking/filter.txt for more information.
>
> Result : test_bpf: Summary: 314 PASSED, 0 FAILED, [278/306 JIT'ed]
>
> Signed-off-by: Shubham Bansal <illusionist.neo@gmail.com>
> ---
> Documentation/networking/filter.txt | 4 +-
> arch/arm/Kconfig | 2 +-
> arch/arm/net/bpf_jit_32.c | 2404 ++++++++++++++++++++++++-----------
> arch/arm/net/bpf_jit_32.h | 108 +-
> 4 files changed, 1713 insertions(+), 805 deletions(-)
>
> diff --git a/Documentation/networking/filter.txt b/Documentation/networking/filter.txt
> index b69b205..01165ac 100644
> --- a/Documentation/networking/filter.txt
> +++ b/Documentation/networking/filter.txt
> @@ -596,8 +596,8 @@ skb pointer). All constraints and restrictions from bpf_check_classic() apply
> before a conversion to the new layout is being done behind the scenes!
>
> Currently, the classic BPF format is being used for JITing on most 32-bit
> -architectures, whereas x86-64, aarch64, s390x, powerpc64, sparc64 perform JIT
> -compilation from eBPF instruction set.
> +architectures, whereas x86-64, aarch64, arm, s390x, powerpc64, sparc64 perform
> +JIT compilation from eBPF instruction set.
>
> Some core changes of the new internal format:
>
> diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
> index 8a7ab5e..13ade46 100644
> --- a/arch/arm/Kconfig
> +++ b/arch/arm/Kconfig
> @@ -47,7 +47,7 @@ config ARM
> select HAVE_ARCH_SECCOMP_FILTER if (AEABI && !OABI_COMPAT)
> select HAVE_ARCH_TRACEHOOK
> select HAVE_ARM_SMCCC if CPU_V7
> - select HAVE_CBPF_JIT
> + select HAVE_EBPF_JIT
> select HAVE_CC_STACKPROTECTOR
> select HAVE_CONTEXT_TRACKING
> select HAVE_C_RECORDMCOUNT
> diff --git a/arch/arm/net/bpf_jit_32.c b/arch/arm/net/bpf_jit_32.c
> index 93d0b6d..c7476e5 100644
> --- a/arch/arm/net/bpf_jit_32.c
> +++ b/arch/arm/net/bpf_jit_32.c
> @@ -1,13 +1,15 @@
> /*
> - * Just-In-Time compiler for BPF filters on 32bit ARM
> + * Just-In-Time compiler for eBPF filters on 32bit ARM
> *
> * Copyright (c) 2011 Mircea Gherzan <mgherzan@gmail.com>
> + * Copyright (c) 2017 Shubham Bansal <illusionist.neo@gmail.com>
> *
> * This program is free software; you can redistribute it and/or modify it
> * under the terms of the GNU General Public License as published by the
> * Free Software Foundation; version 2 of the License.
> */
>
> +#include <linux/bpf.h>
> #include <linux/bitops.h>
> #include <linux/compiler.h>
> #include <linux/errno.h>
> @@ -23,44 +25,91 @@
>
> #include "bpf_jit_32.h"
>
> +int bpf_jit_enable __read_mostly;
> +
> +#define STACK_OFFSET(k) (k)
> +#define TMP_REG_1 (MAX_BPF_JIT_REG + 0) /* TEMP Register 1 */
> +#define TMP_REG_2 (MAX_BPF_JIT_REG + 1) /* TEMP Register 2 */
> +#define TCALL_CNT (MAX_BPF_JIT_REG + 2) /* Tail Call Count */
> +
> +/* Flags used for JIT optimization */
> +#define SEEN_CALL (1 << 0)
> +
> +#define FLAG_IMM_OVERFLOW (1 << 0)
> +
> /*
> - * ABI:
> + * Map eBPF registers to ARM 32bit registers or stack scratch space.
> + *
> + * 1. First argument is passed using the arm 32bit registers and rest of the
> + * arguments are passed on stack scratch space.
> + * 2. First callee-saved aregument is mapped to arm 32 bit registers and rest
> + * arguments are mapped to scratch space on stack.
> + * 3. We need two 64 bit temp registers to do complex operations on eBPF
> + * registers.
> + *
> + * As the eBPF registers are all 64 bit registers and arm has only 32 bit
> + * registers, we have to map each eBPF registers with two arm 32 bit regs or
> + * scratch memory space and we have to build eBPF 64 bit register from those.
> *
> - * r0 scratch register
> - * r4 BPF register A
> - * r5 BPF register X
> - * r6 pointer to the skb
> - * r7 skb->data
> - * r8 skb_headlen(skb)
> */
> +static const u8 bpf2a32[][2] = {
> + /* return value from in-kernel function, and exit value from eBPF */
> + [BPF_REG_0] = {ARM_R1, ARM_R0},
> + /* arguments from eBPF program to in-kernel function */
> + [BPF_REG_1] = {ARM_R3, ARM_R2},
> + /* Stored on stack scratch space */
> + [BPF_REG_2] = {STACK_OFFSET(0), STACK_OFFSET(4)},
> + [BPF_REG_3] = {STACK_OFFSET(8), STACK_OFFSET(12)},
> + [BPF_REG_4] = {STACK_OFFSET(16), STACK_OFFSET(20)},
> + [BPF_REG_5] = {STACK_OFFSET(24), STACK_OFFSET(28)},
> + /* callee saved registers that in-kernel function will preserve */
> + [BPF_REG_6] = {ARM_R5, ARM_R4},
> + /* Stored on stack scratch space */
> + [BPF_REG_7] = {STACK_OFFSET(32), STACK_OFFSET(36)},
> + [BPF_REG_8] = {STACK_OFFSET(40), STACK_OFFSET(44)},
> + [BPF_REG_9] = {STACK_OFFSET(48), STACK_OFFSET(52)},
> + /* Read only Frame Pointer to access Stack */
> + [BPF_REG_FP] = {STACK_OFFSET(56), STACK_OFFSET(60)},
> + /* Temporary Register for internal BPF JIT, can be used
> + * for constant blindings and others.
> + */
> + [TMP_REG_1] = {ARM_R7, ARM_R6},
> + [TMP_REG_2] = {ARM_R10, ARM_R8},
> + /* Tail call count. Stored on stack scratch space. */
> + [TCALL_CNT] = {STACK_OFFSET(64), STACK_OFFSET(68)},
> + /* temporary register for blinding constants.
> + * Stored on stack scratch space.
> + */
> + [BPF_REG_AX] = {STACK_OFFSET(72), STACK_OFFSET(76)},
> +};
>
> -#define r_scratch ARM_R0
> -/* r1-r3 are (also) used for the unaligned loads on the non-ARMv7 slowpath */
> -#define r_off ARM_R1
> -#define r_A ARM_R4
> -#define r_X ARM_R5
> -#define r_skb ARM_R6
> -#define r_skb_data ARM_R7
> -#define r_skb_hl ARM_R8
> -
> -#define SCRATCH_SP_OFFSET 0
> -#define SCRATCH_OFF(k) (SCRATCH_SP_OFFSET + 4 * (k))
> -
> -#define SEEN_MEM ((1 << BPF_MEMWORDS) - 1)
> -#define SEEN_MEM_WORD(k) (1 << (k))
> -#define SEEN_X (1 << BPF_MEMWORDS)
> -#define SEEN_CALL (1 << (BPF_MEMWORDS + 1))
> -#define SEEN_SKB (1 << (BPF_MEMWORDS + 2))
> -#define SEEN_DATA (1 << (BPF_MEMWORDS + 3))
> +#define dst_lo dst[1]
> +#define dst_hi dst[0]
> +#define src_lo src[1]
> +#define src_hi src[0]
>
> -#define FLAG_NEED_X_RESET (1 << 0)
> -#define FLAG_IMM_OVERFLOW (1 << 1)
> +/*
> + * JIT Context:
> + *
> + * prog : bpf_prog
> + * idx : index of current last JITed instruction.
> + * prologue_bytes : bytes used in prologue.
> + * epilogue_offset : offset of epilogue starting.
> + * seen : bit mask used for JIT optimization.
> + * offsets : array of eBPF instruction offsets in
> + * JITed code.
> + * target : final JITed code.
> + * epilogue_bytes : no of bytes used in epilogue.
> + * imm_count : no of immediate counts used for global
> + * variables.
> + * imms : array of global variable addresses.
> + */
>
> struct jit_ctx {
> - const struct bpf_prog *skf;
> - unsigned idx;
> - unsigned prologue_bytes;
> - int ret0_fp_idx;
> + const struct bpf_prog *prog;
> + unsigned int idx;
> + unsigned int prologue_bytes;
> + unsigned int epilogue_offset;
> u32 seen;
> u32 flags;
> u32 *offsets;
> @@ -72,68 +121,16 @@ struct jit_ctx {
> #endif
> };
>
> -int bpf_jit_enable __read_mostly;
> -
> -static inline int call_neg_helper(struct sk_buff *skb, int offset, void *ret,
> - unsigned int size)
> -{
> - void *ptr = bpf_internal_load_pointer_neg_helper(skb, offset, size);
> -
> - if (!ptr)
> - return -EFAULT;
> - memcpy(ret, ptr, size);
> - return 0;
> -}
> -
> -static u64 jit_get_skb_b(struct sk_buff *skb, int offset)
> -{
> - u8 ret;
> - int err;
> -
> - if (offset < 0)
> - err = call_neg_helper(skb, offset, &ret, 1);
> - else
> - err = skb_copy_bits(skb, offset, &ret, 1);
> -
> - return (u64)err << 32 | ret;
> -}
> -
> -static u64 jit_get_skb_h(struct sk_buff *skb, int offset)
> -{
> - u16 ret;
> - int err;
> -
> - if (offset < 0)
> - err = call_neg_helper(skb, offset, &ret, 2);
> - else
> - err = skb_copy_bits(skb, offset, &ret, 2);
> -
> - return (u64)err << 32 | ntohs(ret);
> -}
> -
> -static u64 jit_get_skb_w(struct sk_buff *skb, int offset)
> -{
> - u32 ret;
> - int err;
> -
> - if (offset < 0)
> - err = call_neg_helper(skb, offset, &ret, 4);
> - else
> - err = skb_copy_bits(skb, offset, &ret, 4);
> -
> - return (u64)err << 32 | ntohl(ret);
> -}
> -
> /*
> * Wrappers which handle both OABI and EABI and assures Thumb2 interworking
> * (where the assembly routines like __aeabi_uidiv could cause problems).
> */
> -static u32 jit_udiv(u32 dividend, u32 divisor)
> +static u32 jit_udiv32(u32 dividend, u32 divisor)
> {
> return dividend / divisor;
> }
>
> -static u32 jit_mod(u32 dividend, u32 divisor)
> +static u32 jit_mod32(u32 dividend, u32 divisor)
> {
> return dividend % divisor;
> }
> @@ -157,36 +154,22 @@ static inline void emit(u32 inst, struct jit_ctx *ctx)
> _emit(ARM_COND_AL, inst, ctx);
> }
>
> -static u16 saved_regs(struct jit_ctx *ctx)
> +/*
> + * Checks if immediate value can be converted to imm12(12 bits) value.
> + */
> +static int16_t imm8m(u32 x)
> {
> - u16 ret = 0;
> -
> - if ((ctx->skf->len > 1) ||
> - (ctx->skf->insns[0].code == (BPF_RET | BPF_A)))
> - ret |= 1 << r_A;
> -
> -#ifdef CONFIG_FRAME_POINTER
> - ret |= (1 << ARM_FP) | (1 << ARM_IP) | (1 << ARM_LR) | (1 << ARM_PC);
> -#else
> - if (ctx->seen & SEEN_CALL)
> - ret |= 1 << ARM_LR;
> -#endif
> - if (ctx->seen & (SEEN_DATA | SEEN_SKB))
> - ret |= 1 << r_skb;
> - if (ctx->seen & SEEN_DATA)
> - ret |= (1 << r_skb_data) | (1 << r_skb_hl);
> - if (ctx->seen & SEEN_X)
> - ret |= 1 << r_X;
> -
> - return ret;
> -}
> + u32 rot;
>
> -static inline int mem_words_used(struct jit_ctx *ctx)
> -{
> - /* yes, we do waste some stack space IF there are "holes" in the set" */
> - return fls(ctx->seen & SEEN_MEM);
> + for (rot = 0; rot < 16; rot++)
> + if ((x & ~ror32(0xff, 2 * rot)) == 0)
> + return rol32(x, 2 * rot) | (rot << 8);
> + return -1;
> }
>
> +/*
> + * Initializes the JIT space with undefined instructions.
> + */
> static void jit_fill_hole(void *area, unsigned int size)
> {
> u32 *ptr;
> @@ -195,88 +178,34 @@ static void jit_fill_hole(void *area, unsigned int size)
> *ptr++ = __opcode_to_mem_arm(ARM_INST_UDF);
> }
>
> -static void build_prologue(struct jit_ctx *ctx)
> -{
> - u16 reg_set = saved_regs(ctx);
> - u16 off;
> -
> -#ifdef CONFIG_FRAME_POINTER
> - emit(ARM_MOV_R(ARM_IP, ARM_SP), ctx);
> - emit(ARM_PUSH(reg_set), ctx);
> - emit(ARM_SUB_I(ARM_FP, ARM_IP, 4), ctx);
> -#else
> - if (reg_set)
> - emit(ARM_PUSH(reg_set), ctx);
> -#endif
> +/* Stack must be multiples of 16 Bytes */
> +#define STACK_ALIGN(sz) (((sz) + 15) & ~15)
>
> - if (ctx->seen & (SEEN_DATA | SEEN_SKB))
> - emit(ARM_MOV_R(r_skb, ARM_R0), ctx);
> -
> - if (ctx->seen & SEEN_DATA) {
> - off = offsetof(struct sk_buff, data);
> - emit(ARM_LDR_I(r_skb_data, r_skb, off), ctx);
> - /* headlen = len - data_len */
> - off = offsetof(struct sk_buff, len);
> - emit(ARM_LDR_I(r_skb_hl, r_skb, off), ctx);
> - off = offsetof(struct sk_buff, data_len);
> - emit(ARM_LDR_I(r_scratch, r_skb, off), ctx);
> - emit(ARM_SUB_R(r_skb_hl, r_skb_hl, r_scratch), ctx);
> - }
> -
> - if (ctx->flags & FLAG_NEED_X_RESET)
> - emit(ARM_MOV_I(r_X, 0), ctx);
> -
> - /* do not leak kernel data to userspace */
> - if (bpf_needs_clear_a(&ctx->skf->insns[0]))
> - emit(ARM_MOV_I(r_A, 0), ctx);
> -
> - /* stack space for the BPF_MEM words */
> - if (ctx->seen & SEEN_MEM)
> - emit(ARM_SUB_I(ARM_SP, ARM_SP, mem_words_used(ctx) * 4), ctx);
> -}
> -
> -static void build_epilogue(struct jit_ctx *ctx)
> -{
> - u16 reg_set = saved_regs(ctx);
> -
> - if (ctx->seen & SEEN_MEM)
> - emit(ARM_ADD_I(ARM_SP, ARM_SP, mem_words_used(ctx) * 4), ctx);
> -
> - reg_set &= ~(1 << ARM_LR);
> -
> -#ifdef CONFIG_FRAME_POINTER
> - /* the first instruction of the prologue was: mov ip, sp */
> - reg_set &= ~(1 << ARM_IP);
> - reg_set |= (1 << ARM_SP);
> - emit(ARM_LDM(ARM_SP, reg_set), ctx);
> -#else
> - if (reg_set) {
> - if (ctx->seen & SEEN_CALL)
> - reg_set |= 1 << ARM_PC;
> - emit(ARM_POP(reg_set), ctx);
> - }
> +/* Stack space for BPF_REG_2, BPF_REG_3, BPF_REG_4,
> + * BPF_REG_5, BPF_REG_7, BPF_REG_8, BPF_REG_9,
> + * BPF_REG_FP and Tail call counts.
> + */
> +#define SCRATCH_SIZE 80
>
> - if (!(ctx->seen & SEEN_CALL))
> - emit(ARM_BX(ARM_LR), ctx);
> -#endif
> -}
> +/* total stack size used in JITed code */
> +#define _STACK_SIZE \
> + (MAX_BPF_STACK + \
> + + SCRATCH_SIZE + \
> + + 4 /* extra for skb_copy_bits buffer */)
>
> -static int16_t imm8m(u32 x)
> -{
> - u32 rot;
> +#define STACK_SIZE STACK_ALIGN(_STACK_SIZE)
>
> - for (rot = 0; rot < 16; rot++)
> - if ((x & ~ror32(0xff, 2 * rot)) == 0)
> - return rol32(x, 2 * rot) | (rot << 8);
> +/* Get the offset of eBPF REGISTERs stored on scratch space. */
> +#define STACK_VAR(off) (STACK_SIZE-off-4)
>
> - return -1;
> -}
> +/* Offset of skb_copy_bits buffer */
> +#define SKB_BUFFER STACK_VAR(SCRATCH_SIZE)
>
> #if __LINUX_ARM_ARCH__ < 7
>
> static u16 imm_offset(u32 k, struct jit_ctx *ctx)
> {
> - unsigned i = 0, offset;
> + unsigned int i = 0, offset;
> u16 imm;
>
> /* on the "fake" run we just count them (duplicates included) */
> @@ -295,7 +224,7 @@ static u16 imm_offset(u32 k, struct jit_ctx *ctx)
> ctx->imms[i] = k;
>
> /* constants go just after the epilogue */
> - offset = ctx->offsets[ctx->skf->len];
> + offset = ctx->offsets[ctx->prog->len - 1] * 4;
> offset += ctx->prologue_bytes;
> offset += ctx->epilogue_bytes;
> offset += i * 4;
> @@ -319,10 +248,22 @@ static u16 imm_offset(u32 k, struct jit_ctx *ctx)
>
> #endif /* __LINUX_ARM_ARCH__ */
>
> +static inline int bpf2a32_offset(int bpf_to, int bpf_from,
> + const struct jit_ctx *ctx) {
> + int to, from;
> +
> + if (ctx->target == NULL)
> + return 0;
> + to = ctx->offsets[bpf_to];
> + from = ctx->offsets[bpf_from];
> +
> + return to - from - 1;
> +}
> +
> /*
> * Move an immediate that's not an imm8m to a core register.
> */
> -static inline void emit_mov_i_no8m(int rd, u32 val, struct jit_ctx *ctx)
> +static inline void emit_mov_i_no8m(const u8 rd, u32 val, struct jit_ctx *ctx)
> {
> #if __LINUX_ARM_ARCH__ < 7
> emit(ARM_LDR_I(rd, ARM_PC, imm_offset(val, ctx)), ctx);
> @@ -333,7 +274,7 @@ static inline void emit_mov_i_no8m(int rd, u32 val, struct jit_ctx *ctx)
> #endif
> }
>
> -static inline void emit_mov_i(int rd, u32 val, struct jit_ctx *ctx)
> +static inline void emit_mov_i(const u8 rd, u32 val, struct jit_ctx *ctx)
> {
> int imm12 = imm8m(val);
>
> @@ -343,676 +284,1553 @@ static inline void emit_mov_i(int rd, u32 val, struct jit_ctx *ctx)
> emit_mov_i_no8m(rd, val, ctx);
> }
>
> -#if __LINUX_ARM_ARCH__ < 6
> -
> -static void emit_load_be32(u8 cond, u8 r_res, u8 r_addr, struct jit_ctx *ctx)
> +static inline void emit_blx_r(u8 tgt_reg, struct jit_ctx *ctx)
> {
> - _emit(cond, ARM_LDRB_I(ARM_R3, r_addr, 1), ctx);
> - _emit(cond, ARM_LDRB_I(ARM_R1, r_addr, 0), ctx);
> - _emit(cond, ARM_LDRB_I(ARM_R2, r_addr, 3), ctx);
> - _emit(cond, ARM_LSL_I(ARM_R3, ARM_R3, 16), ctx);
> - _emit(cond, ARM_LDRB_I(ARM_R0, r_addr, 2), ctx);
> - _emit(cond, ARM_ORR_S(ARM_R3, ARM_R3, ARM_R1, SRTYPE_LSL, 24), ctx);
> - _emit(cond, ARM_ORR_R(ARM_R3, ARM_R3, ARM_R2), ctx);
> - _emit(cond, ARM_ORR_S(r_res, ARM_R3, ARM_R0, SRTYPE_LSL, 8), ctx);
> + ctx->seen |= SEEN_CALL;
> +#if __LINUX_ARM_ARCH__ < 5
> + emit(ARM_MOV_R(ARM_LR, ARM_PC), ctx);
> +
> + if (elf_hwcap & HWCAP_THUMB)
> + emit(ARM_BX(tgt_reg), ctx);
> + else
> + emit(ARM_MOV_R(ARM_PC, tgt_reg), ctx);
> +#else
> + emit(ARM_BLX_R(tgt_reg), ctx);
> +#endif
> }
>
> -static void emit_load_be16(u8 cond, u8 r_res, u8 r_addr, struct jit_ctx *ctx)
> +static inline int epilogue_offset(const struct jit_ctx *ctx)
> {
> - _emit(cond, ARM_LDRB_I(ARM_R1, r_addr, 0), ctx);
> - _emit(cond, ARM_LDRB_I(ARM_R2, r_addr, 1), ctx);
> - _emit(cond, ARM_ORR_S(r_res, ARM_R2, ARM_R1, SRTYPE_LSL, 8), ctx);
> + int to, from;
> + /* No need for 1st dummy run */
> + if (ctx->target == NULL)
> + return 0;
> + to = ctx->epilogue_offset;
> + from = ctx->idx;
> +
> + return to - from - 2;
> }
>
> -static inline void emit_swap16(u8 r_dst, u8 r_src, struct jit_ctx *ctx)
> +static inline void emit_udivmod(u8 rd, u8 rm, u8 rn, struct jit_ctx *ctx, u8 op)
> {
> - /* r_dst = (r_src << 8) | (r_src >> 8) */
> - emit(ARM_LSL_I(ARM_R1, r_src, 8), ctx);
> - emit(ARM_ORR_S(r_dst, ARM_R1, r_src, SRTYPE_LSR, 8), ctx);
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + s32 jmp_offset;
> +
> + /* checks if divisor is zero or not. If it is, then
> + * exit directly.
> + */
> + emit(ARM_CMP_I(rn, 0), ctx);
> + _emit(ARM_COND_EQ, ARM_MOV_I(ARM_R0, 0), ctx);
> + jmp_offset = epilogue_offset(ctx);
> + _emit(ARM_COND_EQ, ARM_B(jmp_offset), ctx);
> +#if __LINUX_ARM_ARCH__ == 7
> + if (elf_hwcap & HWCAP_IDIVA) {
> + if (op == BPF_DIV)
> + emit(ARM_UDIV(rd, rm, rn), ctx);
> + else {
> + emit(ARM_UDIV(ARM_IP, rm, rn), ctx);
> + emit(ARM_MLS(rd, rn, ARM_IP, rm), ctx);
> + }
> + return;
> + }
> +#endif
>
> /*
> - * we need to mask out the bits set in r_dst[23:16] due to
> - * the first shift instruction.
> - *
> - * note that 0x8ff is the encoded immediate 0x00ff0000.
> + * For BPF_ALU | BPF_DIV | BPF_K instructions
> + * As ARM_R1 and ARM_R0 contains 1st argument of bpf
> + * function, we need to save it on caller side to save
> + * it from getting destroyed within callee.
> + * After the return from the callee, we restore ARM_R0
> + * ARM_R1.
> */
> - emit(ARM_BIC_I(r_dst, r_dst, 0x8ff), ctx);
> -}
> + if (rn != ARM_R1) {
> + emit(ARM_MOV_R(tmp[0], ARM_R1), ctx);
> + emit(ARM_MOV_R(ARM_R1, rn), ctx);
> + }
> + if (rm != ARM_R0) {
> + emit(ARM_MOV_R(tmp[1], ARM_R0), ctx);
> + emit(ARM_MOV_R(ARM_R0, rm), ctx);
> + }
>
> -#else /* ARMv6+ */
> + /* Call appropriate function */
> + ctx->seen |= SEEN_CALL;
> + emit_mov_i(ARM_IP, op == BPF_DIV ?
> + (u32)jit_udiv32 : (u32)jit_mod32, ctx);
> + emit_blx_r(ARM_IP, ctx);
>
> -static void emit_load_be32(u8 cond, u8 r_res, u8 r_addr, struct jit_ctx *ctx)
> -{
> - _emit(cond, ARM_LDR_I(r_res, r_addr, 0), ctx);
> -#ifdef __LITTLE_ENDIAN
> - _emit(cond, ARM_REV(r_res, r_res), ctx);
> -#endif
> + /* Save return value */
> + if (rd != ARM_R0)
> + emit(ARM_MOV_R(rd, ARM_R0), ctx);
> +
> + /* Restore ARM_R0 and ARM_R1 */
> + if (rn != ARM_R1)
> + emit(ARM_MOV_R(ARM_R1, tmp[0]), ctx);
> + if (rm != ARM_R0)
> + emit(ARM_MOV_R(ARM_R0, tmp[1]), ctx);
> }
>
> -static void emit_load_be16(u8 cond, u8 r_res, u8 r_addr, struct jit_ctx *ctx)
> +/* Checks whether BPF register is on scratch stack space or not. */
> +static inline bool is_on_stack(u8 bpf_reg)
> {
> - _emit(cond, ARM_LDRH_I(r_res, r_addr, 0), ctx);
> -#ifdef __LITTLE_ENDIAN
> - _emit(cond, ARM_REV16(r_res, r_res), ctx);
> -#endif
> + static u8 stack_regs[] = {BPF_REG_AX, BPF_REG_3, BPF_REG_4, BPF_REG_5,
> + BPF_REG_7, BPF_REG_8, BPF_REG_9, TCALL_CNT,
> + BPF_REG_2, BPF_REG_FP};
> + int i, reg_len = sizeof(stack_regs);
> +
> + for (i = 0 ; i < reg_len ; i++) {
> + if (bpf_reg == stack_regs[i])
> + return true;
> + }
> + return false;
> }
>
> -static inline void emit_swap16(u8 r_dst __maybe_unused,
> - u8 r_src __maybe_unused,
> - struct jit_ctx *ctx __maybe_unused)
> +static inline void emit_a32_mov_i(const u8 dst, const u32 val,
> + bool dstk, struct jit_ctx *ctx)
> {
> -#ifdef __LITTLE_ENDIAN
> - emit(ARM_REV16(r_dst, r_src), ctx);
> -#endif
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> +
> + if (dstk) {
> + emit_mov_i(tmp[1], val, ctx);
> + emit(ARM_STR_I(tmp[1], ARM_SP, STACK_VAR(dst)), ctx);
> + } else {
> + emit_mov_i(dst, val, ctx);
> + }
> }
>
> -#endif /* __LINUX_ARM_ARCH__ < 6 */
> +/* Sign extended move */
> +static inline void emit_a32_mov_i64(const bool is64, const u8 dst[],
> + const u32 val, bool dstk,
> + struct jit_ctx *ctx) {
> + u32 hi = 0;
>
> + if (is64 && (val & (1<<31)))
> + hi = (u32)~0;
> + emit_a32_mov_i(dst_lo, val, dstk, ctx);
> + emit_a32_mov_i(dst_hi, hi, dstk, ctx);
> +}
>
> -/* Compute the immediate value for a PC-relative branch. */
> -static inline u32 b_imm(unsigned tgt, struct jit_ctx *ctx)
> -{
> - u32 imm;
> +static inline void emit_a32_add_r(const u8 dst, const u8 src,
> + const bool is64, const bool hi,
> + struct jit_ctx *ctx) {
> + /* 64 bit :
> + * adds dst_lo, dst_lo, src_lo
> + * adc dst_hi, dst_hi, src_hi
> + * 32 bit :
> + * add dst_lo, dst_lo, src_lo
> + */
> + if (!hi && is64)
> + emit(ARM_ADDS_R(dst, dst, src), ctx);
> + else if (hi && is64)
> + emit(ARM_ADC_R(dst, dst, src), ctx);
> + else
> + emit(ARM_ADD_R(dst, dst, src), ctx);
> +}
>
> - if (ctx->target == NULL)
> - return 0;
> - /*
> - * BPF allows only forward jumps and the offset of the target is
> - * still the one computed during the first pass.
> +static inline void emit_a32_sub_r(const u8 dst, const u8 src,
> + const bool is64, const bool hi,
> + struct jit_ctx *ctx) {
> + /* 64 bit :
> + * subs dst_lo, dst_lo, src_lo
> + * sbc dst_hi, dst_hi, src_hi
> + * 32 bit :
> + * sub dst_lo, dst_lo, src_lo
> */
> - imm = ctx->offsets[tgt] + ctx->prologue_bytes - (ctx->idx * 4 + 8);
> + if (!hi && is64)
> + emit(ARM_SUBS_R(dst, dst, src), ctx);
> + else if (hi && is64)
> + emit(ARM_SBC_R(dst, dst, src), ctx);
> + else
> + emit(ARM_SUB_R(dst, dst, src), ctx);
> +}
>
> - return imm >> 2;
> +static inline void emit_alu_r(const u8 dst, const u8 src, const bool is64,
> + const bool hi, const u8 op, struct jit_ctx *ctx){
> + switch (BPF_OP(op)) {
> + /* dst = dst + src */
> + case BPF_ADD:
> + emit_a32_add_r(dst, src, is64, hi, ctx);
> + break;
> + /* dst = dst - src */
> + case BPF_SUB:
> + emit_a32_sub_r(dst, src, is64, hi, ctx);
> + break;
> + /* dst = dst | src */
> + case BPF_OR:
> + emit(ARM_ORR_R(dst, dst, src), ctx);
> + break;
> + /* dst = dst & src */
> + case BPF_AND:
> + emit(ARM_AND_R(dst, dst, src), ctx);
> + break;
> + /* dst = dst ^ src */
> + case BPF_XOR:
> + emit(ARM_EOR_R(dst, dst, src), ctx);
> + break;
> + /* dst = dst * src */
> + case BPF_MUL:
> + emit(ARM_MUL(dst, dst, src), ctx);
> + break;
> + /* dst = dst << src */
> + case BPF_LSH:
> + emit(ARM_LSL_R(dst, dst, src), ctx);
> + break;
> + /* dst = dst >> src */
> + case BPF_RSH:
> + emit(ARM_LSR_R(dst, dst, src), ctx);
> + break;
> + /* dst = dst >> src (signed)*/
> + case BPF_ARSH:
> + emit(ARM_MOV_SR(dst, dst, SRTYPE_ASR, src), ctx);
> + break;
> + }
> }
>
> -#define OP_IMM3(op, r1, r2, imm_val, ctx) \
> - do { \
> - imm12 = imm8m(imm_val); \
> - if (imm12 < 0) { \
> - emit_mov_i_no8m(r_scratch, imm_val, ctx); \
> - emit(op ## _R((r1), (r2), r_scratch), ctx); \
> - } else { \
> - emit(op ## _I((r1), (r2), imm12), ctx); \
> - } \
> - } while (0)
> -
> -static inline void emit_err_ret(u8 cond, struct jit_ctx *ctx)
> -{
> - if (ctx->ret0_fp_idx >= 0) {
> - _emit(cond, ARM_B(b_imm(ctx->ret0_fp_idx, ctx)), ctx);
> - /* NOP to keep the size constant between passes */
> - emit(ARM_MOV_R(ARM_R0, ARM_R0), ctx);
> +/* ALU operation (32 bit)
> + * dst = dst (op) src
> + */
> +static inline void emit_a32_alu_r(const u8 dst, const u8 src,
> + bool dstk, bool sstk,
> + struct jit_ctx *ctx, const bool is64,
> + const bool hi, const u8 op) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rn = sstk ? tmp[1] : src;
> +
> + if (sstk)
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src)), ctx);
> +
> + /* ALU operation */
> + if (dstk) {
> + emit(ARM_LDR_I(tmp[0], ARM_SP, STACK_VAR(dst)), ctx);
> + emit_alu_r(tmp[0], rn, is64, hi, op, ctx);
> + emit(ARM_STR_I(tmp[0], ARM_SP, STACK_VAR(dst)), ctx);
> } else {
> - _emit(cond, ARM_MOV_I(ARM_R0, 0), ctx);
> - _emit(cond, ARM_B(b_imm(ctx->skf->len, ctx)), ctx);
> + emit_alu_r(dst, rn, is64, hi, op, ctx);
> }
> }
>
> -static inline void emit_blx_r(u8 tgt_reg, struct jit_ctx *ctx)
> -{
> -#if __LINUX_ARM_ARCH__ < 5
> - emit(ARM_MOV_R(ARM_LR, ARM_PC), ctx);
> +/* ALU operation (64 bit) */
> +static inline void emit_a32_alu_r64(const bool is64, const u8 dst[],
> + const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx,
> + const u8 op) {
> + emit_a32_alu_r(dst_lo, src_lo, dstk, sstk, ctx, is64, false, op);
> + if (is64)
> + emit_a32_alu_r(dst_hi, src_hi, dstk, sstk, ctx, is64, true, op);
> + else
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> +}
>
> - if (elf_hwcap & HWCAP_THUMB)
> - emit(ARM_BX(tgt_reg), ctx);
> +/* dst = imm (4 bytes)*/
> +static inline void emit_a32_mov_r(const u8 dst, const u8 src,
> + bool dstk, bool sstk,
> + struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rt = sstk ? tmp[0] : src;
> +
> + if (sstk)
> + emit(ARM_LDR_I(tmp[0], ARM_SP, STACK_VAR(src)), ctx);
> + if (dstk)
> + emit(ARM_STR_I(rt, ARM_SP, STACK_VAR(dst)), ctx);
> else
> - emit(ARM_MOV_R(ARM_PC, tgt_reg), ctx);
> -#else
> - emit(ARM_BLX_R(tgt_reg), ctx);
> -#endif
> + emit(ARM_MOV_R(dst, rt), ctx);
> }
>
> -static inline void emit_udivmod(u8 rd, u8 rm, u8 rn, struct jit_ctx *ctx,
> - int bpf_op)
> -{
> -#if __LINUX_ARM_ARCH__ == 7
> - if (elf_hwcap & HWCAP_IDIVA) {
> - if (bpf_op == BPF_DIV)
> - emit(ARM_UDIV(rd, rm, rn), ctx);
> - else {
> - emit(ARM_UDIV(ARM_R3, rm, rn), ctx);
> - emit(ARM_MLS(rd, rn, ARM_R3, rm), ctx);
> - }
> - return;
> +/* dst = src */
> +static inline void emit_a32_mov_r64(const bool is64, const u8 dst[],
> + const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx) {
> + emit_a32_mov_r(dst_lo, src_lo, dstk, sstk, ctx);
> + if (is64) {
> + /* complete 8 byte move */
> + emit_a32_mov_r(dst_hi, src_hi, dstk, sstk, ctx);
> + } else {
> + /* Zero out high 4 bytes */
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> }
> -#endif
> +}
>
> - /*
> - * For BPF_ALU | BPF_DIV | BPF_K instructions, rm is ARM_R4
> - * (r_A) and rn is ARM_R0 (r_scratch) so load rn first into
> - * ARM_R1 to avoid accidentally overwriting ARM_R0 with rm
> - * before using it as a source for ARM_R1.
> - *
> - * For BPF_ALU | BPF_DIV | BPF_X rm is ARM_R4 (r_A) and rn is
> - * ARM_R5 (r_X) so there is no particular register overlap
> - * issues.
> - */
> - if (rn != ARM_R1)
> - emit(ARM_MOV_R(ARM_R1, rn), ctx);
> - if (rm != ARM_R0)
> - emit(ARM_MOV_R(ARM_R0, rm), ctx);
> +/* Shift operations */
> +static inline void emit_a32_alu_i(const u8 dst, const u32 val, bool dstk,
> + struct jit_ctx *ctx, const u8 op) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rd = dstk ? tmp[0] : dst;
> +
> + if (dstk)
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst)), ctx);
> +
> + /* Do shift operation */
> + switch (op) {
> + case BPF_LSH:
> + emit(ARM_LSL_I(rd, rd, val), ctx);
> + break;
> + case BPF_RSH:
> + emit(ARM_LSR_I(rd, rd, val), ctx);
> + break;
> + case BPF_NEG:
> + emit(ARM_RSB_I(rd, rd, val), ctx);
> + break;
> + }
> +
> + if (dstk)
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst)), ctx);
> +}
> +
> +/* dst = ~dst (64 bit) */
> +static inline void emit_a32_neg64(const u8 dst[], bool dstk,
> + struct jit_ctx *ctx){
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rd = dstk ? tmp[1] : dst[1];
> + u8 rm = dstk ? tmp[0] : dst[0];
> +
> + /* Setup Operand */
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do Negate Operation */
> + emit(ARM_RSBS_I(rd, rd, 0), ctx);
> + emit(ARM_RSC_I(rm, rm, 0), ctx);
> +
> + if (dstk) {
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +}
>
> +/* dst = dst << src */
> +static inline void emit_a32_lsh_r64(const u8 dst[], const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> +
> + /* Setup Operands */
> + u8 rt = sstk ? tmp2[1] : src_lo;
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (sstk)
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(src_lo)), ctx);
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do LSH operation */
> + emit(ARM_SUB_I(ARM_IP, rt, 32), ctx);
> + emit(ARM_RSB_I(tmp2[0], rt, 32), ctx);
> + /* As we are using ARM_LR */
> ctx->seen |= SEEN_CALL;
> - emit_mov_i(ARM_R3, bpf_op == BPF_DIV ? (u32)jit_udiv : (u32)jit_mod,
> - ctx);
> - emit_blx_r(ARM_R3, ctx);
> + emit(ARM_MOV_SR(ARM_LR, rm, SRTYPE_ASL, rt), ctx);
> + emit(ARM_ORR_SR(ARM_LR, ARM_LR, rd, SRTYPE_ASL, ARM_IP), ctx);
> + emit(ARM_ORR_SR(ARM_IP, ARM_LR, rd, SRTYPE_LSR, tmp2[0]), ctx);
> + emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_ASL, rt), ctx);
> +
> + if (dstk) {
> + emit(ARM_STR_I(ARM_LR, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(ARM_IP, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + } else {
> + emit(ARM_MOV_R(rd, ARM_LR), ctx);
> + emit(ARM_MOV_R(rm, ARM_IP), ctx);
> + }
> +}
>
> - if (rd != ARM_R0)
> - emit(ARM_MOV_R(rd, ARM_R0), ctx);
> +/* dst = dst >> src (signed)*/
> +static inline void emit_a32_arsh_r64(const u8 dst[], const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup Operands */
> + u8 rt = sstk ? tmp2[1] : src_lo;
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (sstk)
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(src_lo)), ctx);
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do the ARSH operation */
> + emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
> + emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
> + /* As we are using ARM_LR */
> + ctx->seen |= SEEN_CALL;
> + emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
> + emit(ARM_ORR_SR(ARM_LR, ARM_LR, rm, SRTYPE_ASL, ARM_IP), ctx);
> + _emit(ARM_COND_MI, ARM_B(0), ctx);
> + emit(ARM_ORR_SR(ARM_LR, ARM_LR, rm, SRTYPE_ASR, tmp2[0]), ctx);
> + emit(ARM_MOV_SR(ARM_IP, rm, SRTYPE_ASR, rt), ctx);
> + if (dstk) {
> + emit(ARM_STR_I(ARM_LR, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(ARM_IP, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + } else {
> + emit(ARM_MOV_R(rd, ARM_LR), ctx);
> + emit(ARM_MOV_R(rm, ARM_IP), ctx);
> + }
> }
>
> -static inline void update_on_xread(struct jit_ctx *ctx)
> +/* dst = dst >> src */
> +static inline void emit_a32_lsr_r64(const u8 dst[], const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup Operands */
> + u8 rt = sstk ? tmp2[1] : src_lo;
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (sstk)
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(src_lo)), ctx);
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do LSH operation */
> + emit(ARM_RSB_I(ARM_IP, rt, 32), ctx);
> + emit(ARM_SUBS_I(tmp2[0], rt, 32), ctx);
> + /* As we are using ARM_LR */
> + ctx->seen |= SEEN_CALL;
> + emit(ARM_MOV_SR(ARM_LR, rd, SRTYPE_LSR, rt), ctx);
> + emit(ARM_ORR_SR(ARM_LR, ARM_LR, rm, SRTYPE_ASL, ARM_IP), ctx);
> + emit(ARM_ORR_SR(ARM_LR, ARM_LR, rm, SRTYPE_LSR, tmp2[0]), ctx);
> + emit(ARM_MOV_SR(ARM_IP, rm, SRTYPE_LSR, rt), ctx);
> + if (dstk) {
> + emit(ARM_STR_I(ARM_LR, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(ARM_IP, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + } else {
> + emit(ARM_MOV_R(rd, ARM_LR), ctx);
> + emit(ARM_MOV_R(rm, ARM_IP), ctx);
> + }
> +}
> +
> +/* dst = dst << val */
> +static inline void emit_a32_lsh_i64(const u8 dst[], bool dstk,
> + const u32 val, struct jit_ctx *ctx){
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup operands */
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do LSH operation */
> + if (val < 32) {
> + emit(ARM_MOV_SI(tmp2[0], rm, SRTYPE_ASL, val), ctx);
> + emit(ARM_ORR_SI(rm, tmp2[0], rd, SRTYPE_LSR, 32 - val), ctx);
> + emit(ARM_MOV_SI(rd, rd, SRTYPE_ASL, val), ctx);
> + } else {
> + if (val == 32)
> + emit(ARM_MOV_R(rm, rd), ctx);
> + else
> + emit(ARM_MOV_SI(rm, rd, SRTYPE_ASL, val - 32), ctx);
> + emit(ARM_EOR_R(rd, rd, rd), ctx);
> + }
> +
> + if (dstk) {
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +}
> +
> +/* dst = dst >> val */
> +static inline void emit_a32_lsr_i64(const u8 dst[], bool dstk,
> + const u32 val, struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup operands */
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do LSR operation */
> + if (val < 32) {
> + emit(ARM_MOV_SI(tmp2[1], rd, SRTYPE_LSR, val), ctx);
> + emit(ARM_ORR_SI(rd, tmp2[1], rm, SRTYPE_ASL, 32 - val), ctx);
> + emit(ARM_MOV_SI(rm, rm, SRTYPE_LSR, val), ctx);
> + } else if (val == 32) {
> + emit(ARM_MOV_R(rd, rm), ctx);
> + emit(ARM_MOV_I(rm, 0), ctx);
> + } else {
> + emit(ARM_MOV_SI(rd, rm, SRTYPE_LSR, val - 32), ctx);
> + emit(ARM_MOV_I(rm, 0), ctx);
> + }
> +
> + if (dstk) {
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +}
> +
> +/* dst = dst >> val (signed) */
> +static inline void emit_a32_arsh_i64(const u8 dst[], bool dstk,
> + const u32 val, struct jit_ctx *ctx){
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup operands */
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> +
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Do ARSH operation */
> + if (val < 32) {
> + emit(ARM_MOV_SI(tmp2[1], rd, SRTYPE_LSR, val), ctx);
> + emit(ARM_ORR_SI(rd, tmp2[1], rm, SRTYPE_ASL, 32 - val), ctx);
> + emit(ARM_MOV_SI(rm, rm, SRTYPE_ASR, val), ctx);
> + } else if (val == 32) {
> + emit(ARM_MOV_R(rd, rm), ctx);
> + emit(ARM_MOV_SI(rm, rm, SRTYPE_ASR, 31), ctx);
> + } else {
> + emit(ARM_MOV_SI(rd, rm, SRTYPE_ASR, val - 32), ctx);
> + emit(ARM_MOV_SI(rm, rm, SRTYPE_ASR, 31), ctx);
> + }
> +
> + if (dstk) {
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +}
> +
> +static inline void emit_a32_mul_r64(const u8 dst[], const u8 src[], bool dstk,
> + bool sstk, struct jit_ctx *ctx) {
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + /* Setup operands for multiplication */
> + u8 rd = dstk ? tmp[1] : dst_lo;
> + u8 rm = dstk ? tmp[0] : dst_hi;
> + u8 rt = sstk ? tmp2[1] : src_lo;
> + u8 rn = sstk ? tmp2[0] : src_hi;
> +
> + if (dstk) {
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> + if (sstk) {
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(src_lo)), ctx);
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src_hi)), ctx);
> + }
> +
> + /* Do Multiplication */
> + emit(ARM_MUL(ARM_IP, rd, rn), ctx);
> + emit(ARM_MUL(ARM_LR, rm, rt), ctx);
> + /* As we are using ARM_LR */
> + ctx->seen |= SEEN_CALL;
> + emit(ARM_ADD_R(ARM_LR, ARM_IP, ARM_LR), ctx);
> +
> + emit(ARM_UMULL(ARM_IP, rm, rd, rt), ctx);
> + emit(ARM_ADD_R(rm, ARM_LR, rm), ctx);
> + if (dstk) {
> + emit(ARM_STR_I(ARM_IP, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rm, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + } else {
> + emit(ARM_MOV_R(rd, ARM_IP), ctx);
> + }
> +}
> +
> +/* *(size *)(dst + off) = src */
> +static inline void emit_str_r(const u8 dst, const u8 src, bool dstk,
> + const s32 off, struct jit_ctx *ctx, const u8 sz){
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rd = dstk ? tmp[1] : dst;
> +
> + if (dstk)
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst)), ctx);
> + if (off) {
> + emit_a32_mov_i(tmp[0], off, false, ctx);
> + emit(ARM_ADD_R(tmp[0], rd, tmp[0]), ctx);
> + rd = tmp[0];
> + }
> + switch (sz) {
> + case BPF_W:
> + /* Store a Word */
> + emit(ARM_STR_I(src, rd, 0), ctx);
> + break;
> + case BPF_H:
> + /* Store a HalfWord */
> + emit(ARM_STRH_I(src, rd, 0), ctx);
> + break;
> + case BPF_B:
> + /* Store a Byte */
> + emit(ARM_STRB_I(src, rd, 0), ctx);
> + break;
> + }
> +}
> +
> +/* dst = *(size*)(src + off) */
> +static inline void emit_ldx_r(const u8 dst, const u8 src, bool dstk,
> + const s32 off, struct jit_ctx *ctx, const u8 sz){
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + u8 rd = dstk ? tmp[1] : dst;
> + u8 rm = src;
> +
> + if (off) {
> + emit_a32_mov_i(tmp[0], off, false, ctx);
> + emit(ARM_ADD_R(tmp[0], tmp[0], src), ctx);
> + rm = tmp[0];
> + }
> + switch (sz) {
> + case BPF_W:
> + /* Load a Word */
> + emit(ARM_LDR_I(rd, rm, 0), ctx);
> + break;
> + case BPF_H:
> + /* Load a HalfWord */
> + emit(ARM_LDRH_I(rd, rm, 0), ctx);
> + break;
> + case BPF_B:
> + /* Load a Byte */
> + emit(ARM_LDRB_I(rd, rm, 0), ctx);
> + break;
> + }
> + if (dstk)
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst)), ctx);
> +}
> +
> +/* Arithmatic Operation */
> +static inline void emit_ar_r(const u8 rd, const u8 rt, const u8 rm,
> + const u8 rn, struct jit_ctx *ctx, u8 op) {
> + switch (op) {
> + case BPF_JSET:
> + ctx->seen |= SEEN_CALL;
> + emit(ARM_AND_R(ARM_IP, rt, rn), ctx);
> + emit(ARM_AND_R(ARM_LR, rd, rm), ctx);
> + emit(ARM_ORRS_R(ARM_IP, ARM_LR, ARM_IP), ctx);
> + break;
> + case BPF_JEQ:
> + case BPF_JNE:
> + case BPF_JGT:
> + case BPF_JGE:
> + emit(ARM_CMP_R(rd, rm), ctx);
> + _emit(ARM_COND_EQ, ARM_CMP_R(rt, rn), ctx);
> + break;
> + case BPF_JSGT:
> + emit(ARM_CMP_R(rn, rt), ctx);
> + emit(ARM_SBCS_R(ARM_IP, rm, rd), ctx);
> + break;
> + case BPF_JSGE:
> + emit(ARM_CMP_R(rt, rn), ctx);
> + emit(ARM_SBCS_R(ARM_IP, rd, rm), ctx);
> + break;
> + }
> +}
> +
> +static int out_offset = -1; /* initialized on the first pass of build_body() */
> +static int emit_bpf_tail_call(struct jit_ctx *ctx)
> +{
> +
> + /* bpf_tail_call(void *prog_ctx, struct bpf_array *array, u64 index) */
> + const u8 *r2 = bpf2a32[BPF_REG_2];
> + const u8 *r3 = bpf2a32[BPF_REG_3];
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + const u8 *tcc = bpf2a32[TCALL_CNT];
> + const int idx0 = ctx->idx;
> +#define cur_offset (ctx->idx - idx0)
> +#define jmp_offset (out_offset - (cur_offset))
> + u32 off, lo, hi;
> +
> + /* if (index >= array->map.max_entries)
> + * goto out;
> + */
> + off = offsetof(struct bpf_array, map.max_entries);
> + /* array->map.max_entries */
> + emit_a32_mov_i(tmp[1], off, false, ctx);
> + emit(ARM_LDR_I(tmp2[1], ARM_SP, STACK_VAR(r2[1])), ctx);
> + emit(ARM_LDR_R(tmp[1], tmp2[1], tmp[1]), ctx);
> + /* index (64 bit) */
> + emit(ARM_LDR_I(tmp2[1], ARM_SP, STACK_VAR(r3[1])), ctx);
> + /* index >= array->map.max_entries */
> + emit(ARM_CMP_R(tmp2[1], tmp[1]), ctx);
> + _emit(ARM_COND_CS, ARM_B(jmp_offset), ctx);
> +
> + /* if (tail_call_cnt > MAX_TAIL_CALL_CNT)
> + * goto out;
> + * tail_call_cnt++;
> + */
> + lo = (u32)MAX_TAIL_CALL_CNT;
> + hi = (u32)((u64)MAX_TAIL_CALL_CNT >> 32);
> + emit(ARM_LDR_I(tmp[1], ARM_SP, STACK_VAR(tcc[1])), ctx);
> + emit(ARM_LDR_I(tmp[0], ARM_SP, STACK_VAR(tcc[0])), ctx);
> + emit(ARM_CMP_I(tmp[0], hi), ctx);
> + _emit(ARM_COND_EQ, ARM_CMP_I(tmp[1], lo), ctx);
> + _emit(ARM_COND_HI, ARM_B(jmp_offset), ctx);
> + emit(ARM_ADDS_I(tmp[1], tmp[1], 1), ctx);
> + emit(ARM_ADC_I(tmp[0], tmp[0], 0), ctx);
> + emit(ARM_STR_I(tmp[1], ARM_SP, STACK_VAR(tcc[1])), ctx);
> + emit(ARM_STR_I(tmp[0], ARM_SP, STACK_VAR(tcc[0])), ctx);
> +
> + /* prog = array->ptrs[index]
> + * if (prog == NULL)
> + * goto out;
> + */
> + off = offsetof(struct bpf_array, ptrs);
> + emit_a32_mov_i(tmp[1], off, false, ctx);
> + emit(ARM_LDR_I(tmp2[1], ARM_SP, STACK_VAR(r2[1])), ctx);
> + emit(ARM_LDR_R(tmp[1], tmp2[1], tmp[1]), ctx);
> + emit(ARM_LDR_I(tmp2[1], ARM_SP, STACK_VAR(r3[1])), ctx);
> + emit(ARM_MOV_SI(tmp[0], tmp2[1], SRTYPE_ASL, 2), ctx);
> + emit(ARM_LDR_R(tmp[1], tmp[1], tmp[0]), ctx);
> + emit(ARM_CMP_I(tmp[1], 0), ctx);
> + _emit(ARM_COND_EQ, ARM_B(jmp_offset), ctx);
> +
> + /* goto *(prog->bpf_func + prologue_size); */
> + off = offsetof(struct bpf_prog, bpf_func);
> + emit_a32_mov_i(tmp2[1], off, false, ctx);
> + emit(ARM_LDR_R(tmp[1], tmp[1], tmp2[1]), ctx);
> + emit(ARM_ADD_I(tmp[1], tmp[1], ctx->prologue_bytes), ctx);
> + emit(ARM_BX(tmp[1]), ctx);
> +
> + /* out: */
> + if (out_offset == -1)
> + out_offset = cur_offset;
> + if (cur_offset != out_offset) {
> + pr_err_once("tail_call out_offset = %d, expected %d!\n",
> + cur_offset, out_offset);
> + return -1;
> + }
> + return 0;
> +#undef cur_offset
> +#undef jmp_offset
> +}
> +
> +/* 0xabcd => 0xcdab */
> +static inline void emit_rev16(const u8 rd, const u8 rn, struct jit_ctx *ctx)
> {
> - if (!(ctx->seen & SEEN_X))
> - ctx->flags |= FLAG_NEED_X_RESET;
> +#if __LINUX_ARM_ARCH__ < 6
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> +
> + emit(ARM_AND_I(tmp2[1], rn, 0xff), ctx);
> + emit(ARM_MOV_SI(tmp2[0], rn, SRTYPE_LSR, 8), ctx);
> + emit(ARM_AND_I(tmp2[0], tmp2[0], 0xff), ctx);
> + emit(ARM_ORR_SI(rd, tmp2[0], tmp2[1], SRTYPE_LSL, 8), ctx);
> +#else /* ARMv6+ */
> + emit(ARM_REV16(rd, rn), ctx);
> +#endif
> +}
>
> - ctx->seen |= SEEN_X;
> +/* 0xabcdefgh => 0xghefcdab */
> +static inline void emit_rev32(const u8 rd, const u8 rn, struct jit_ctx *ctx)
> +{
> +#if __LINUX_ARM_ARCH__ < 6
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> +
> + emit(ARM_AND_I(tmp2[1], rn, 0xff), ctx);
> + emit(ARM_MOV_SI(tmp2[0], rn, SRTYPE_LSR, 24), ctx);
> + emit(ARM_ORR_SI(ARM_IP, tmp2[0], tmp2[1], SRTYPE_LSL, 24), ctx);
> +
> + emit(ARM_MOV_SI(tmp2[1], rn, SRTYPE_LSR, 8), ctx);
> + emit(ARM_AND_I(tmp2[1], tmp2[1], 0xff), ctx);
> + emit(ARM_MOV_SI(tmp2[0], rn, SRTYPE_LSR, 16), ctx);
> + emit(ARM_AND_I(tmp2[0], tmp2[0], 0xff), ctx);
> + emit(ARM_MOV_SI(tmp2[0], tmp2[0], SRTYPE_LSL, 8), ctx);
> + emit(ARM_ORR_SI(tmp2[0], tmp2[0], tmp2[1], SRTYPE_LSL, 16), ctx);
> + emit(ARM_ORR_R(rd, ARM_IP, tmp2[0]), ctx);
> +
> +#else /* ARMv6+ */
> + emit(ARM_REV(rd, rn), ctx);
> +#endif
> }
>
> -static int build_body(struct jit_ctx *ctx)
> +static void build_prologue(struct jit_ctx *ctx)
> {
> - void *load_func[] = {jit_get_skb_b, jit_get_skb_h, jit_get_skb_w};
> - const struct bpf_prog *prog = ctx->skf;
> - const struct sock_filter *inst;
> - unsigned i, load_order, off, condt;
> - int imm12;
> - u32 k;
> + const u8 r0 = bpf2a32[BPF_REG_0][1];
> + const u8 r2 = bpf2a32[BPF_REG_1][1];
> + const u8 r3 = bpf2a32[BPF_REG_1][0];
> + const u8 r4 = bpf2a32[BPF_REG_6][1];
> + const u8 r5 = bpf2a32[BPF_REG_6][0];
> + const u8 r6 = bpf2a32[TMP_REG_1][1];
> + const u8 r7 = bpf2a32[TMP_REG_1][0];
> + const u8 r8 = bpf2a32[TMP_REG_2][1];
> + const u8 r10 = bpf2a32[TMP_REG_2][0];
> + const u8 fplo = bpf2a32[BPF_REG_FP][1];
> + const u8 fphi = bpf2a32[BPF_REG_FP][0];
> + const u8 sp = ARM_SP;
> + const u8 *tcc = bpf2a32[TCALL_CNT];
> +
> + u16 reg_set = 0;
>
> - for (i = 0; i < prog->len; i++) {
> - u16 code;
> + /*
> + * eBPF prog stack layout
> + *
> + * high
> + * original ARM_SP => +-----+ eBPF prologue
> + * |FP/LR|
> + * current ARM_FP => +-----+
> + * | ... | callee saved registers
> + * eBPF fp register => +-----+ <= (BPF_FP)
> + * | ... | eBPF JIT scratch space
> + * | | eBPF prog stack
> + * +-----+
> + * |RSVD | JIT scratchpad
> + * current A64_SP => +-----+ <= (BPF_FP - STACK_SIZE)
> + * | |
> + * | ... | Function call stack
> + * | |
> + * +-----+
> + * low
> + */
>
> - inst = &(prog->insns[i]);
> - /* K as an immediate value operand */
> - k = inst->k;
> - code = bpf_anc_helper(inst);
> + /* Save callee saved registers. */
> + reg_set |= (1<<r4) | (1<<r5) | (1<<r6) | (1<<r7) | (1<<r8) | (1<<r10);
> +#ifdef CONFIG_FRAME_POINTER
> + reg_set |= (1<<ARM_FP) | (1<<ARM_IP) | (1<<ARM_LR) | (1<<ARM_PC);
> + emit(ARM_MOV_R(ARM_IP, sp), ctx);
> + emit(ARM_PUSH(reg_set), ctx);
> + emit(ARM_SUB_I(ARM_FP, ARM_IP, 4), ctx);
> +#else
> + /* Check if call instruction exists in BPF body */
> + if (ctx->seen & SEEN_CALL)
> + reg_set |= (1<<ARM_LR);
> + emit(ARM_PUSH(reg_set), ctx);
> +#endif
> + /* Save frame pointer for later */
> + emit(ARM_SUB_I(ARM_IP, sp, SCRATCH_SIZE), ctx);
> +
> + /* Set up function call stack */
> + emit(ARM_SUB_I(ARM_SP, ARM_SP, imm8m(STACK_SIZE)), ctx);
> +
> + /* Set up BPF prog stack base register */
> + emit_a32_mov_r(fplo, ARM_IP, true, false, ctx);
> + emit_a32_mov_i(fphi, 0, true, ctx);
> +
> + /* mov r4, 0 */
> + emit(ARM_MOV_I(r4, 0), ctx);
> + /* MOV bpf_ctx pointer to BPF_R1 */
> + emit(ARM_MOV_R(r3, r4), ctx);
> + emit(ARM_MOV_R(r2, r0), ctx);
> + /* Initialize Tail Count */
> + emit(ARM_STR_I(r4, ARM_SP, STACK_VAR(tcc[0])), ctx);
> + emit(ARM_STR_I(r4, ARM_SP, STACK_VAR(tcc[1])), ctx);
> + /* end of prologue */
> +}
>
> - /* compute offsets only in the fake pass */
> - if (ctx->target == NULL)
> - ctx->offsets[i] = ctx->idx * 4;
> +static void build_epilogue(struct jit_ctx *ctx)
> +{
> + const u8 r4 = bpf2a32[BPF_REG_6][1];
> + const u8 r5 = bpf2a32[BPF_REG_6][0];
> + const u8 r6 = bpf2a32[TMP_REG_1][1];
> + const u8 r7 = bpf2a32[TMP_REG_1][0];
> + const u8 r8 = bpf2a32[TMP_REG_2][1];
> + const u8 r10 = bpf2a32[TMP_REG_2][0];
> + u16 reg_set = 0;
> +
> + /* unwind function call stack */
> + emit(ARM_ADD_I(ARM_SP, ARM_SP, imm8m(STACK_SIZE)), ctx);
> +
> + /* restore callee saved registers. */
> + reg_set |= (1<<r4) | (1<<r5) | (1<<r6) | (1<<r7) | (1<<r8) | (1<<r10);
> +#ifdef CONFIG_FRAME_POINTER
> + /* the first instruction of the prologue was: mov ip, sp */
> + reg_set |= (1<<ARM_FP) | (1<<ARM_SP) | (1<<ARM_PC);
> + emit(ARM_LDM(ARM_SP, reg_set), ctx);
> +#else
> + if (ctx->seen & SEEN_CALL)
> + reg_set |= (1<<ARM_PC);
> + /* Restore callee saved registers. */
> + emit(ARM_POP(reg_set), ctx);
> + /* Return back to the callee function */
> + if (!(ctx->seen & SEEN_CALL))
> + emit(ARM_BX(ARM_LR), ctx);
> +#endif
> +}
>
> - switch (code) {
> - case BPF_LD | BPF_IMM:
> - emit_mov_i(r_A, k, ctx);
> +/*
> + * Convert an eBPF instruction to native instruction, i.e
> + * JITs an eBPF instruction.
> + * Returns :
> + * 0 - Successfully JITed an 8-byte eBPF instruction
> + * >0 - Successfully JITed a 16-byte eBPF instruction
> + * <0 - Failed to JIT.
> + */
> +static int build_insn(const struct bpf_insn *insn, struct jit_ctx *ctx)
> +{
> + const u8 code = insn->code;
> + const u8 *dst = bpf2a32[insn->dst_reg];
> + const u8 *src = bpf2a32[insn->src_reg];
> + const u8 *tmp = bpf2a32[TMP_REG_1];
> + const u8 *tmp2 = bpf2a32[TMP_REG_2];
> + const s16 off = insn->off;
> + const s32 imm = insn->imm;
> + const int i = insn - ctx->prog->insnsi;
> + const bool is64 = BPF_CLASS(code) == BPF_ALU64;
> + const bool dstk = is_on_stack(insn->dst_reg);
> + const bool sstk = is_on_stack(insn->src_reg);
> + u8 rd, rt, rm, rn;
> + s32 jmp_offset;
> +
> +#define check_imm(bits, imm) do { \
> + if ((((imm) > 0) && ((imm) >> (bits))) || \
> + (((imm) < 0) && (~(imm) >> (bits)))) { \
> + pr_info("[%2d] imm=%d(0x%x) out of range\n", \
> + i, imm, imm); \
> + return -EINVAL; \
> + } \
> +} while (0)
> +#define check_imm24(imm) check_imm(24, imm)
> +
> + switch (code) {
> + /* ALU operations */
> +
> + /* dst = src */
> + case BPF_ALU | BPF_MOV | BPF_K:
> + case BPF_ALU | BPF_MOV | BPF_X:
> + case BPF_ALU64 | BPF_MOV | BPF_K:
> + case BPF_ALU64 | BPF_MOV | BPF_X:
> + switch (BPF_SRC(code)) {
> + case BPF_X:
> + emit_a32_mov_r64(is64, dst, src, dstk, sstk, ctx);
> break;
> - case BPF_LD | BPF_W | BPF_LEN:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, len) != 4);
> - emit(ARM_LDR_I(r_A, r_skb,
> - offsetof(struct sk_buff, len)), ctx);
> + case BPF_K:
> + /* Sign-extend immediate value to destination reg */
> + emit_a32_mov_i64(is64, dst, imm, dstk, ctx);
> break;
> - case BPF_LD | BPF_MEM:
> - /* A = scratch[k] */
> - ctx->seen |= SEEN_MEM_WORD(k);
> - emit(ARM_LDR_I(r_A, ARM_SP, SCRATCH_OFF(k)), ctx);
> + }
> + break;
> + /* dst = dst + src/imm */
> + /* dst = dst - src/imm */
> + /* dst = dst | src/imm */
> + /* dst = dst & src/imm */
> + /* dst = dst ^ src/imm */
> + /* dst = dst * src/imm */
> + /* dst = dst << src */
> + /* dst = dst >> src */
> + case BPF_ALU | BPF_ADD | BPF_K:
> + case BPF_ALU | BPF_ADD | BPF_X:
> + case BPF_ALU | BPF_SUB | BPF_K:
> + case BPF_ALU | BPF_SUB | BPF_X:
> + case BPF_ALU | BPF_OR | BPF_K:
> + case BPF_ALU | BPF_OR | BPF_X:
> + case BPF_ALU | BPF_AND | BPF_K:
> + case BPF_ALU | BPF_AND | BPF_X:
> + case BPF_ALU | BPF_XOR | BPF_K:
> + case BPF_ALU | BPF_XOR | BPF_X:
> + case BPF_ALU | BPF_MUL | BPF_K:
> + case BPF_ALU | BPF_MUL | BPF_X:
> + case BPF_ALU | BPF_LSH | BPF_X:
> + case BPF_ALU | BPF_RSH | BPF_X:
> + case BPF_ALU | BPF_ARSH | BPF_K:
> + case BPF_ALU | BPF_ARSH | BPF_X:
> + case BPF_ALU64 | BPF_ADD | BPF_K:
> + case BPF_ALU64 | BPF_ADD | BPF_X:
> + case BPF_ALU64 | BPF_SUB | BPF_K:
> + case BPF_ALU64 | BPF_SUB | BPF_X:
> + case BPF_ALU64 | BPF_OR | BPF_K:
> + case BPF_ALU64 | BPF_OR | BPF_X:
> + case BPF_ALU64 | BPF_AND | BPF_K:
> + case BPF_ALU64 | BPF_AND | BPF_X:
> + case BPF_ALU64 | BPF_XOR | BPF_K:
> + case BPF_ALU64 | BPF_XOR | BPF_X:
> + switch (BPF_SRC(code)) {
> + case BPF_X:
> + emit_a32_alu_r64(is64, dst, src, dstk, sstk,
> + ctx, BPF_OP(code));
> break;
> - case BPF_LD | BPF_W | BPF_ABS:
> - load_order = 2;
> - goto load;
> - case BPF_LD | BPF_H | BPF_ABS:
> - load_order = 1;
> - goto load;
> - case BPF_LD | BPF_B | BPF_ABS:
> - load_order = 0;
> -load:
> - emit_mov_i(r_off, k, ctx);
> -load_common:
> - ctx->seen |= SEEN_DATA | SEEN_CALL;
> -
> - if (load_order > 0) {
> - emit(ARM_SUB_I(r_scratch, r_skb_hl,
> - 1 << load_order), ctx);
> - emit(ARM_CMP_R(r_scratch, r_off), ctx);
> - condt = ARM_COND_GE;
> - } else {
> - emit(ARM_CMP_R(r_skb_hl, r_off), ctx);
> - condt = ARM_COND_HI;
> - }
> -
> - /*
> - * test for negative offset, only if we are
> - * currently scheduled to take the fast
> - * path. this will update the flags so that
> - * the slowpath instruction are ignored if the
> - * offset is negative.
> - *
> - * for loard_order == 0 the HI condition will
> - * make loads at offset 0 take the slow path too.
> + case BPF_K:
> + /* Move immediate value to the temporary register
> + * and then do the ALU operation on the temporary
> + * register as this will sign-extend the immediate
> + * value into temporary reg and then it would be
> + * safe to do the operation on it.
> */
> - _emit(condt, ARM_CMP_I(r_off, 0), ctx);
> -
> - _emit(condt, ARM_ADD_R(r_scratch, r_off, r_skb_data),
> - ctx);
> -
> - if (load_order == 0)
> - _emit(condt, ARM_LDRB_I(r_A, r_scratch, 0),
> - ctx);
> - else if (load_order == 1)
> - emit_load_be16(condt, r_A, r_scratch, ctx);
> - else if (load_order == 2)
> - emit_load_be32(condt, r_A, r_scratch, ctx);
> -
> - _emit(condt, ARM_B(b_imm(i + 1, ctx)), ctx);
> -
> - /* the slowpath */
> - emit_mov_i(ARM_R3, (u32)load_func[load_order], ctx);
> - emit(ARM_MOV_R(ARM_R0, r_skb), ctx);
> - /* the offset is already in R1 */
> - emit_blx_r(ARM_R3, ctx);
> - /* check the result of skb_copy_bits */
> - emit(ARM_CMP_I(ARM_R1, 0), ctx);
> - emit_err_ret(ARM_COND_NE, ctx);
> - emit(ARM_MOV_R(r_A, ARM_R0), ctx);
> + emit_a32_mov_i64(is64, tmp2, imm, false, ctx);
> + emit_a32_alu_r64(is64, dst, tmp2, dstk, false,
> + ctx, BPF_OP(code));
> break;
> - case BPF_LD | BPF_W | BPF_IND:
> - load_order = 2;
> - goto load_ind;
> - case BPF_LD | BPF_H | BPF_IND:
> - load_order = 1;
> - goto load_ind;
> - case BPF_LD | BPF_B | BPF_IND:
> - load_order = 0;
> -load_ind:
> - update_on_xread(ctx);
> - OP_IMM3(ARM_ADD, r_off, r_X, k, ctx);
> - goto load_common;
> - case BPF_LDX | BPF_IMM:
> - ctx->seen |= SEEN_X;
> - emit_mov_i(r_X, k, ctx);
> + }
> + break;
> + /* dst = dst / src(imm) */
> + /* dst = dst % src(imm) */
> + case BPF_ALU | BPF_DIV | BPF_K:
> + case BPF_ALU | BPF_DIV | BPF_X:
> + case BPF_ALU | BPF_MOD | BPF_K:
> + case BPF_ALU | BPF_MOD | BPF_X:
> + rt = src_lo;
> + rd = dstk ? tmp2[1] : dst_lo;
> + if (dstk)
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + switch (BPF_SRC(code)) {
> + case BPF_X:
> + rt = sstk ? tmp2[0] : rt;
> + if (sstk)
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(src_lo)),
> + ctx);
> break;
> - case BPF_LDX | BPF_W | BPF_LEN:
> - ctx->seen |= SEEN_X | SEEN_SKB;
> - emit(ARM_LDR_I(r_X, r_skb,
> - offsetof(struct sk_buff, len)), ctx);
> + case BPF_K:
> + rt = tmp2[0];
> + emit_a32_mov_i(rt, imm, false, ctx);
> break;
> - case BPF_LDX | BPF_MEM:
> - ctx->seen |= SEEN_X | SEEN_MEM_WORD(k);
> - emit(ARM_LDR_I(r_X, ARM_SP, SCRATCH_OFF(k)), ctx);
> + }
> + emit_udivmod(rd, rd, rt, ctx, BPF_OP(code));
> + if (dstk)
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> + break;
> + case BPF_ALU64 | BPF_DIV | BPF_K:
> + case BPF_ALU64 | BPF_DIV | BPF_X:
> + case BPF_ALU64 | BPF_MOD | BPF_K:
> + case BPF_ALU64 | BPF_MOD | BPF_X:
> + goto notyet;
> + /* dst = dst >> imm */
> + /* dst = dst << imm */
> + case BPF_ALU | BPF_RSH | BPF_K:
> + case BPF_ALU | BPF_LSH | BPF_K:
> + if (unlikely(imm > 31))
> + return -EINVAL;
> + if (imm)
> + emit_a32_alu_i(dst_lo, imm, dstk, ctx, BPF_OP(code));
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> + break;
> + /* dst = dst << imm */
> + case BPF_ALU64 | BPF_LSH | BPF_K:
> + if (unlikely(imm > 63))
> + return -EINVAL;
> + emit_a32_lsh_i64(dst, dstk, imm, ctx);
> + break;
> + /* dst = dst >> imm */
> + case BPF_ALU64 | BPF_RSH | BPF_K:
> + if (unlikely(imm > 63))
> + return -EINVAL;
> + emit_a32_lsr_i64(dst, dstk, imm, ctx);
> + break;
> + /* dst = dst << src */
> + case BPF_ALU64 | BPF_LSH | BPF_X:
> + emit_a32_lsh_r64(dst, src, dstk, sstk, ctx);
> + break;
> + /* dst = dst >> src */
> + case BPF_ALU64 | BPF_RSH | BPF_X:
> + emit_a32_lsr_r64(dst, src, dstk, sstk, ctx);
> + break;
> + /* dst = dst >> src (signed) */
> + case BPF_ALU64 | BPF_ARSH | BPF_X:
> + emit_a32_arsh_r64(dst, src, dstk, sstk, ctx);
> + break;
> + /* dst = dst >> imm (signed) */
> + case BPF_ALU64 | BPF_ARSH | BPF_K:
> + if (unlikely(imm > 63))
> + return -EINVAL;
> + emit_a32_arsh_i64(dst, dstk, imm, ctx);
> + break;
> + /* dst = ~dst */
> + case BPF_ALU | BPF_NEG:
> + emit_a32_alu_i(dst_lo, 0, dstk, ctx, BPF_OP(code));
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> + break;
> + /* dst = ~dst (64 bit) */
> + case BPF_ALU64 | BPF_NEG:
> + emit_a32_neg64(dst, dstk, ctx);
> + break;
> + /* dst = dst * src/imm */
> + case BPF_ALU64 | BPF_MUL | BPF_X:
> + case BPF_ALU64 | BPF_MUL | BPF_K:
> + switch (BPF_SRC(code)) {
> + case BPF_X:
> + emit_a32_mul_r64(dst, src, dstk, sstk, ctx);
> break;
> - case BPF_LDX | BPF_B | BPF_MSH:
> - /* x = ((*(frame + k)) & 0xf) << 2; */
> - ctx->seen |= SEEN_X | SEEN_DATA | SEEN_CALL;
> - /* the interpreter should deal with the negative K */
> - if ((int)k < 0)
> - return -1;
> - /* offset in r1: we might have to take the slow path */
> - emit_mov_i(r_off, k, ctx);
> - emit(ARM_CMP_R(r_skb_hl, r_off), ctx);
> -
> - /* load in r0: common with the slowpath */
> - _emit(ARM_COND_HI, ARM_LDRB_R(ARM_R0, r_skb_data,
> - ARM_R1), ctx);
> - /*
> - * emit_mov_i() might generate one or two instructions,
> - * the same holds for emit_blx_r()
> + case BPF_K:
> + /* Move immediate value to the temporary register
> + * and then do the multiplication on it as this
> + * will sign-extend the immediate value into temp
> + * reg then it would be safe to do the operation
> + * on it.
> */
> - _emit(ARM_COND_HI, ARM_B(b_imm(i + 1, ctx) - 2), ctx);
> -
> - emit(ARM_MOV_R(ARM_R0, r_skb), ctx);
> - /* r_off is r1 */
> - emit_mov_i(ARM_R3, (u32)jit_get_skb_b, ctx);
> - emit_blx_r(ARM_R3, ctx);
> - /* check the return value of skb_copy_bits */
> - emit(ARM_CMP_I(ARM_R1, 0), ctx);
> - emit_err_ret(ARM_COND_NE, ctx);
> -
> - emit(ARM_AND_I(r_X, ARM_R0, 0x00f), ctx);
> - emit(ARM_LSL_I(r_X, r_X, 2), ctx);
> - break;
> - case BPF_ST:
> - ctx->seen |= SEEN_MEM_WORD(k);
> - emit(ARM_STR_I(r_A, ARM_SP, SCRATCH_OFF(k)), ctx);
> - break;
> - case BPF_STX:
> - update_on_xread(ctx);
> - ctx->seen |= SEEN_MEM_WORD(k);
> - emit(ARM_STR_I(r_X, ARM_SP, SCRATCH_OFF(k)), ctx);
> - break;
> - case BPF_ALU | BPF_ADD | BPF_K:
> - /* A += K */
> - OP_IMM3(ARM_ADD, r_A, r_A, k, ctx);
> - break;
> - case BPF_ALU | BPF_ADD | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_ADD_R(r_A, r_A, r_X), ctx);
> - break;
> - case BPF_ALU | BPF_SUB | BPF_K:
> - /* A -= K */
> - OP_IMM3(ARM_SUB, r_A, r_A, k, ctx);
> - break;
> - case BPF_ALU | BPF_SUB | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_SUB_R(r_A, r_A, r_X), ctx);
> - break;
> - case BPF_ALU | BPF_MUL | BPF_K:
> - /* A *= K */
> - emit_mov_i(r_scratch, k, ctx);
> - emit(ARM_MUL(r_A, r_A, r_scratch), ctx);
> - break;
> - case BPF_ALU | BPF_MUL | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_MUL(r_A, r_A, r_X), ctx);
> - break;
> - case BPF_ALU | BPF_DIV | BPF_K:
> - if (k == 1)
> - break;
> - emit_mov_i(r_scratch, k, ctx);
> - emit_udivmod(r_A, r_A, r_scratch, ctx, BPF_DIV);
> - break;
> - case BPF_ALU | BPF_DIV | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_CMP_I(r_X, 0), ctx);
> - emit_err_ret(ARM_COND_EQ, ctx);
> - emit_udivmod(r_A, r_A, r_X, ctx, BPF_DIV);
> - break;
> - case BPF_ALU | BPF_MOD | BPF_K:
> - if (k == 1) {
> - emit_mov_i(r_A, 0, ctx);
> - break;
> - }
> - emit_mov_i(r_scratch, k, ctx);
> - emit_udivmod(r_A, r_A, r_scratch, ctx, BPF_MOD);
> + emit_a32_mov_i64(is64, tmp2, imm, false, ctx);
> + emit_a32_mul_r64(dst, tmp2, dstk, false, ctx);
> break;
> - case BPF_ALU | BPF_MOD | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_CMP_I(r_X, 0), ctx);
> - emit_err_ret(ARM_COND_EQ, ctx);
> - emit_udivmod(r_A, r_A, r_X, ctx, BPF_MOD);
> - break;
> - case BPF_ALU | BPF_OR | BPF_K:
> - /* A |= K */
> - OP_IMM3(ARM_ORR, r_A, r_A, k, ctx);
> + }
> + break;
> + /* dst = htole(dst) */
> + /* dst = htobe(dst) */
> + case BPF_ALU | BPF_END | BPF_FROM_LE:
> + case BPF_ALU | BPF_END | BPF_FROM_BE:
> + rd = dstk ? tmp[0] : dst_hi;
> + rt = dstk ? tmp[1] : dst_lo;
> + if (dstk) {
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> + if (BPF_SRC(code) == BPF_FROM_LE)
> + goto emit_bswap_uxt;
> + switch (imm) {
> + case 16:
> + emit_rev16(rt, rt, ctx);
> + goto emit_bswap_uxt;
> + case 32:
> + emit_rev32(rt, rt, ctx);
> + goto emit_bswap_uxt;
> + case 64:
> + /* Because of the usage of ARM_LR */
> + ctx->seen |= SEEN_CALL;
> + emit_rev32(ARM_LR, rt, ctx);
> + emit_rev32(rt, rd, ctx);
> + emit(ARM_MOV_R(rd, ARM_LR), ctx);
> break;
> - case BPF_ALU | BPF_OR | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_ORR_R(r_A, r_A, r_X), ctx);
> + }
> + goto exit;
> +emit_bswap_uxt:
> + switch (imm) {
> + case 16:
> + /* zero-extend 16 bits into 64 bits */
> +#if __LINUX_ARM_ARCH__ < 6
> + emit_a32_mov_i(tmp2[1], 0xffff, false, ctx);
> + emit(ARM_AND_R(rt, rt, tmp2[1]), ctx);
> +#else /* ARMv6+ */
> + emit(ARM_UXTH(rt, rt), ctx);
> +#endif
> + emit(ARM_EOR_R(rd, rd, rd), ctx);
> break;
> - case BPF_ALU | BPF_XOR | BPF_K:
> - /* A ^= K; */
> - OP_IMM3(ARM_EOR, r_A, r_A, k, ctx);
> + case 32:
> + /* zero-extend 32 bits into 64 bits */
> + emit(ARM_EOR_R(rd, rd, rd), ctx);
> break;
> - case BPF_ANC | SKF_AD_ALU_XOR_X:
> - case BPF_ALU | BPF_XOR | BPF_X:
> - /* A ^= X */
> - update_on_xread(ctx);
> - emit(ARM_EOR_R(r_A, r_A, r_X), ctx);
> + case 64:
> + /* nop */
> break;
> - case BPF_ALU | BPF_AND | BPF_K:
> - /* A &= K */
> - OP_IMM3(ARM_AND, r_A, r_A, k, ctx);
> + }
> +exit:
> + if (dstk) {
> + emit(ARM_STR_I(rt, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_STR_I(rd, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> + break;
> + /* dst = imm64 */
> + case BPF_LD | BPF_IMM | BPF_DW:
> + {
> + const struct bpf_insn insn1 = insn[1];
> + u32 hi, lo = imm;
> +
> + if (insn1.code != 0 || insn1.src_reg != 0 ||
> + insn1.dst_reg != 0 || insn1.off != 0) {
> + /* Note: verifier in BPF core must catch invalid
> + * instruction.
> + */
> + pr_err_once("Invalid BPF_LD_IMM64 instruction\n");
> + return -EINVAL;
> + }
> + hi = insn1.imm;
> + emit_a32_mov_i(dst_lo, lo, dstk, ctx);
> + emit_a32_mov_i(dst_hi, hi, dstk, ctx);
> +
> + return 1;
> + }
> + /* LDX: dst = *(size *)(src + off) */
> + case BPF_LDX | BPF_MEM | BPF_W:
> + case BPF_LDX | BPF_MEM | BPF_H:
> + case BPF_LDX | BPF_MEM | BPF_B:
> + case BPF_LDX | BPF_MEM | BPF_DW:
> + rn = sstk ? tmp2[1] : src_lo;
> + if (sstk)
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + switch (BPF_SIZE(code)) {
> + case BPF_W:
> + /* Load a Word */
> + case BPF_H:
> + /* Load a Half-Word */
> + case BPF_B:
> + /* Load a Byte */
> + emit_ldx_r(dst_lo, rn, dstk, off, ctx, BPF_SIZE(code));
> + emit_a32_mov_i(dst_hi, 0, dstk, ctx);
> break;
> - case BPF_ALU | BPF_AND | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_AND_R(r_A, r_A, r_X), ctx);
> + case BPF_DW:
> + /* Load a double word */
> + emit_ldx_r(dst_lo, rn, dstk, off, ctx, BPF_W);
> + emit_ldx_r(dst_hi, rn, dstk, off+4, ctx, BPF_W);
> break;
> - case BPF_ALU | BPF_LSH | BPF_K:
> - if (unlikely(k > 31))
> - return -1;
> - emit(ARM_LSL_I(r_A, r_A, k), ctx);
> + }
> + break;
> + /* R0 = ntohx(*(size *)(((struct sk_buff *)R6)->data + imm)) */
> + case BPF_LD | BPF_ABS | BPF_W:
> + case BPF_LD | BPF_ABS | BPF_H:
> + case BPF_LD | BPF_ABS | BPF_B:
> + /* R0 = ntohx(*(size *)(((struct sk_buff *)R6)->data + src + imm)) */
> + case BPF_LD | BPF_IND | BPF_W:
> + case BPF_LD | BPF_IND | BPF_H:
> + case BPF_LD | BPF_IND | BPF_B:
> + {
> + const u8 r4 = bpf2a32[BPF_REG_6][1]; /* r4 = ptr to sk_buff */
> + const u8 r0 = bpf2a32[BPF_REG_0][1]; /*r0: struct sk_buff *skb*/
> + /* rtn value */
> + const u8 r1 = bpf2a32[BPF_REG_0][0]; /* r1: int k */
> + const u8 r2 = bpf2a32[BPF_REG_1][1]; /* r2: unsigned int size */
> + const u8 r3 = bpf2a32[BPF_REG_1][0]; /* r3: void *buffer */
> + const u8 r6 = bpf2a32[TMP_REG_1][1]; /* r6: void *(*func)(..) */
> + int size;
> +
> + /* Setting up first argument */
> + emit(ARM_MOV_R(r0, r4), ctx);
> +
> + /* Setting up second argument */
> + emit_a32_mov_i(r1, imm, false, ctx);
> + if (BPF_MODE(code) == BPF_IND)
> + emit_a32_alu_r(r1, src_lo, false, sstk, ctx,
> + false, false, BPF_ADD);
> +
> + /* Setting up third argument */
> + switch (BPF_SIZE(code)) {
> + case BPF_W:
> + size = 4;
> break;
> - case BPF_ALU | BPF_LSH | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_LSL_R(r_A, r_A, r_X), ctx);
> + case BPF_H:
> + size = 2;
> break;
> - case BPF_ALU | BPF_RSH | BPF_K:
> - if (unlikely(k > 31))
> - return -1;
> - if (k)
> - emit(ARM_LSR_I(r_A, r_A, k), ctx);
> + case BPF_B:
> + size = 1;
> break;
> - case BPF_ALU | BPF_RSH | BPF_X:
> - update_on_xread(ctx);
> - emit(ARM_LSR_R(r_A, r_A, r_X), ctx);
> + default:
> + return -EINVAL;
> + }
> + emit_a32_mov_i(r2, size, false, ctx);
> +
> + /* Setting up fourth argument */
> + emit(ARM_ADD_I(r3, ARM_SP, imm8m(SKB_BUFFER)), ctx);
> +
> + /* Setting up function pointer to call */
> + emit_a32_mov_i(r6, (unsigned int)bpf_load_pointer, false, ctx);
> + emit_blx_r(r6, ctx);
> +
> + emit(ARM_EOR_R(r1, r1, r1), ctx);
> + /* Check if return address is NULL or not.
> + * if NULL then jump to epilogue
> + * else continue to load the value from retn address
> + */
> + emit(ARM_CMP_I(r0, 0), ctx);
> + jmp_offset = epilogue_offset(ctx);
> + check_imm24(jmp_offset);
> + _emit(ARM_COND_EQ, ARM_B(jmp_offset), ctx);
> +
> + /* Load value from the address */
> + switch (BPF_SIZE(code)) {
> + case BPF_W:
> + emit(ARM_LDR_I(r0, r0, 0), ctx);
> + emit_rev32(r0, r0, ctx);
> break;
> - case BPF_ALU | BPF_NEG:
> - /* A = -A */
> - emit(ARM_RSB_I(r_A, r_A, 0), ctx);
> + case BPF_H:
> + emit(ARM_LDRH_I(r0, r0, 0), ctx);
> + emit_rev16(r0, r0, ctx);
> break;
> - case BPF_JMP | BPF_JA:
> - /* pc += K */
> - emit(ARM_B(b_imm(i + k + 1, ctx)), ctx);
> + case BPF_B:
> + emit(ARM_LDRB_I(r0, r0, 0), ctx);
> + /* No need to reverse */
> break;
> - case BPF_JMP | BPF_JEQ | BPF_K:
> - /* pc += (A == K) ? pc->jt : pc->jf */
> - condt = ARM_COND_EQ;
> - goto cmp_imm;
> - case BPF_JMP | BPF_JGT | BPF_K:
> - /* pc += (A > K) ? pc->jt : pc->jf */
> - condt = ARM_COND_HI;
> - goto cmp_imm;
> - case BPF_JMP | BPF_JGE | BPF_K:
> - /* pc += (A >= K) ? pc->jt : pc->jf */
> - condt = ARM_COND_HS;
> -cmp_imm:
> - imm12 = imm8m(k);
> - if (imm12 < 0) {
> - emit_mov_i_no8m(r_scratch, k, ctx);
> - emit(ARM_CMP_R(r_A, r_scratch), ctx);
> - } else {
> - emit(ARM_CMP_I(r_A, imm12), ctx);
> - }
> -cond_jump:
> - if (inst->jt)
> - _emit(condt, ARM_B(b_imm(i + inst->jt + 1,
> - ctx)), ctx);
> - if (inst->jf)
> - _emit(condt ^ 1, ARM_B(b_imm(i + inst->jf + 1,
> - ctx)), ctx);
> + }
> + break;
> + }
> + /* ST: *(size *)(dst + off) = imm */
> + case BPF_ST | BPF_MEM | BPF_W:
> + case BPF_ST | BPF_MEM | BPF_H:
> + case BPF_ST | BPF_MEM | BPF_B:
> + case BPF_ST | BPF_MEM | BPF_DW:
> + switch (BPF_SIZE(code)) {
> + case BPF_DW:
> + /* Sign-extend immediate value into temp reg */
> + emit_a32_mov_i64(true, tmp2, imm, false, ctx);
> + emit_str_r(dst_lo, tmp2[1], dstk, off, ctx, BPF_W);
> + emit_str_r(dst_lo, tmp2[0], dstk, off+4, ctx, BPF_W);
> break;
> - case BPF_JMP | BPF_JEQ | BPF_X:
> - /* pc += (A == X) ? pc->jt : pc->jf */
> - condt = ARM_COND_EQ;
> - goto cmp_x;
> - case BPF_JMP | BPF_JGT | BPF_X:
> - /* pc += (A > X) ? pc->jt : pc->jf */
> - condt = ARM_COND_HI;
> - goto cmp_x;
> - case BPF_JMP | BPF_JGE | BPF_X:
> - /* pc += (A >= X) ? pc->jt : pc->jf */
> - condt = ARM_COND_CS;
> -cmp_x:
> - update_on_xread(ctx);
> - emit(ARM_CMP_R(r_A, r_X), ctx);
> - goto cond_jump;
> - case BPF_JMP | BPF_JSET | BPF_K:
> - /* pc += (A & K) ? pc->jt : pc->jf */
> - condt = ARM_COND_NE;
> - /* not set iff all zeroes iff Z==1 iff EQ */
> -
> - imm12 = imm8m(k);
> - if (imm12 < 0) {
> - emit_mov_i_no8m(r_scratch, k, ctx);
> - emit(ARM_TST_R(r_A, r_scratch), ctx);
> - } else {
> - emit(ARM_TST_I(r_A, imm12), ctx);
> - }
> - goto cond_jump;
> - case BPF_JMP | BPF_JSET | BPF_X:
> - /* pc += (A & X) ? pc->jt : pc->jf */
> - update_on_xread(ctx);
> - condt = ARM_COND_NE;
> - emit(ARM_TST_R(r_A, r_X), ctx);
> - goto cond_jump;
> - case BPF_RET | BPF_A:
> - emit(ARM_MOV_R(ARM_R0, r_A), ctx);
> - goto b_epilogue;
> - case BPF_RET | BPF_K:
> - if ((k == 0) && (ctx->ret0_fp_idx < 0))
> - ctx->ret0_fp_idx = i;
> - emit_mov_i(ARM_R0, k, ctx);
> -b_epilogue:
> - if (i != ctx->skf->len - 1)
> - emit(ARM_B(b_imm(prog->len, ctx)), ctx);
> + case BPF_W:
> + case BPF_H:
> + case BPF_B:
> + emit_a32_mov_i(tmp2[1], imm, false, ctx);
> + emit_str_r(dst_lo, tmp2[1], dstk, off, ctx,
> + BPF_SIZE(code));
> break;
> - case BPF_MISC | BPF_TAX:
> - /* X = A */
> - ctx->seen |= SEEN_X;
> - emit(ARM_MOV_R(r_X, r_A), ctx);
> + }
> + break;
> + /* STX XADD: lock *(u32 *)(dst + off) += src */
> + case BPF_STX | BPF_XADD | BPF_W:
> + /* STX XADD: lock *(u64 *)(dst + off) += src */
> + case BPF_STX | BPF_XADD | BPF_DW:
> + goto notyet;
> + /* STX: *(size *)(dst + off) = src */
> + case BPF_STX | BPF_MEM | BPF_W:
> + case BPF_STX | BPF_MEM | BPF_H:
> + case BPF_STX | BPF_MEM | BPF_B:
> + case BPF_STX | BPF_MEM | BPF_DW:
> + {
> + u8 sz = BPF_SIZE(code);
> +
> + rn = sstk ? tmp2[1] : src_lo;
> + rm = sstk ? tmp2[0] : src_hi;
> + if (!sstk)
> + goto do_store;
> + switch (BPF_SIZE(code)) {
> + case BPF_W:
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + goto empty_hi;
> + case BPF_H:
> + emit(ARM_LDRH_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + goto empty_hi;
> + case BPF_B:
> + emit(ARM_LDRB_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + goto empty_hi;
> +empty_hi:
> + emit(ARM_EOR_R(rm, rm, rm), ctx);
> + case BPF_DW:
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(src_hi)), ctx);
> + sz = BPF_W;
> break;
> - case BPF_MISC | BPF_TXA:
> - /* A = X */
> - update_on_xread(ctx);
> - emit(ARM_MOV_R(r_A, r_X), ctx);
> + }
> +
> +do_store:
> + /* Clear higher word except for BPF_DW */
> + if (BPF_SIZE(code) != BPF_DW)
> + emit(ARM_EOR_R(rm, rm, rm), ctx);
> +
> + /* Store the value */
> + emit_str_r(dst_lo, rn, dstk, off, ctx, sz);
> + emit_str_r(dst_lo, rm, dstk, off+4, ctx, BPF_W);
> + break;
> + }
> + /* PC += off if dst == src */
> + /* PC += off if dst > src */
> + /* PC += off if dst >= src */
> + /* PC += off if dst != src */
> + /* PC += off if dst > src (signed) */
> + /* PC += off if dst >= src (signed) */
> + /* PC += off if dst & src */
> + case BPF_JMP | BPF_JEQ | BPF_X:
> + case BPF_JMP | BPF_JGT | BPF_X:
> + case BPF_JMP | BPF_JGE | BPF_X:
> + case BPF_JMP | BPF_JNE | BPF_X:
> + case BPF_JMP | BPF_JSGT | BPF_X:
> + case BPF_JMP | BPF_JSGE | BPF_X:
> + case BPF_JMP | BPF_JSET | BPF_X:
> + /* Setup source registers */
> + rm = sstk ? tmp2[0] : src_hi;
> + rn = sstk ? tmp2[1] : src_lo;
> + if (sstk) {
> + emit(ARM_LDR_I(rn, ARM_SP, STACK_VAR(src_lo)), ctx);
> + emit(ARM_LDR_I(rm, ARM_SP, STACK_VAR(src_hi)), ctx);
> + }
> + goto go_jmp;
> + /* PC += off if dst == imm */
> + /* PC += off if dst > imm */
> + /* PC += off if dst >= imm */
> + /* PC += off if dst != imm */
> + /* PC += off if dst > imm (signed) */
> + /* PC += off if dst >= imm (signed) */
> + /* PC += off if dst & imm */
> + case BPF_JMP | BPF_JEQ | BPF_K:
> + case BPF_JMP | BPF_JGT | BPF_K:
> + case BPF_JMP | BPF_JGE | BPF_K:
> + case BPF_JMP | BPF_JNE | BPF_K:
> + case BPF_JMP | BPF_JSGT | BPF_K:
> + case BPF_JMP | BPF_JSGE | BPF_K:
> + case BPF_JMP | BPF_JSET | BPF_K:
> + if (off == 0)
> break;
> - case BPF_ANC | SKF_AD_PROTOCOL:
> - /* A = ntohs(skb->protocol) */
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
> - protocol) != 2);
> - off = offsetof(struct sk_buff, protocol);
> - emit(ARM_LDRH_I(r_scratch, r_skb, off), ctx);
> - emit_swap16(r_A, r_scratch, ctx);
> + rm = tmp2[0];
> + rn = tmp2[1];
> + /* Sign-extend immediate value */
> + emit_a32_mov_i64(true, tmp2, imm, false, ctx);
> +go_jmp:
> + /* Setup destination register */
> + rd = dstk ? tmp[0] : dst_hi;
> + rt = dstk ? tmp[1] : dst_lo;
> + if (dstk) {
> + emit(ARM_LDR_I(rt, ARM_SP, STACK_VAR(dst_lo)), ctx);
> + emit(ARM_LDR_I(rd, ARM_SP, STACK_VAR(dst_hi)), ctx);
> + }
> +
> + /* Check for the condition */
> + emit_ar_r(rd, rt, rm, rn, ctx, BPF_OP(code));
> +
> + /* Setup JUMP instruction */
> + jmp_offset = bpf2a32_offset(i+off, i, ctx);
> + switch (BPF_OP(code)) {
> + case BPF_JNE:
> + case BPF_JSET:
> + _emit(ARM_COND_NE, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_CPU:
> - /* r_scratch = current_thread_info() */
> - OP_IMM3(ARM_BIC, r_scratch, ARM_SP, THREAD_SIZE - 1, ctx);
> - /* A = current_thread_info()->cpu */
> - BUILD_BUG_ON(FIELD_SIZEOF(struct thread_info, cpu) != 4);
> - off = offsetof(struct thread_info, cpu);
> - emit(ARM_LDR_I(r_A, r_scratch, off), ctx);
> + case BPF_JEQ:
> + _emit(ARM_COND_EQ, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_IFINDEX:
> - case BPF_ANC | SKF_AD_HATYPE:
> - /* A = skb->dev->ifindex */
> - /* A = skb->dev->type */
> - ctx->seen |= SEEN_SKB;
> - off = offsetof(struct sk_buff, dev);
> - emit(ARM_LDR_I(r_scratch, r_skb, off), ctx);
> -
> - emit(ARM_CMP_I(r_scratch, 0), ctx);
> - emit_err_ret(ARM_COND_EQ, ctx);
> -
> - BUILD_BUG_ON(FIELD_SIZEOF(struct net_device,
> - ifindex) != 4);
> - BUILD_BUG_ON(FIELD_SIZEOF(struct net_device,
> - type) != 2);
> -
> - if (code == (BPF_ANC | SKF_AD_IFINDEX)) {
> - off = offsetof(struct net_device, ifindex);
> - emit(ARM_LDR_I(r_A, r_scratch, off), ctx);
> - } else {
> - /*
> - * offset of field "type" in "struct
> - * net_device" is above what can be
> - * used in the ldrh rd, [rn, #imm]
> - * instruction, so load the offset in
> - * a register and use ldrh rd, [rn, rm]
> - */
> - off = offsetof(struct net_device, type);
> - emit_mov_i(ARM_R3, off, ctx);
> - emit(ARM_LDRH_R(r_A, r_scratch, ARM_R3), ctx);
> - }
> + case BPF_JGT:
> + _emit(ARM_COND_HI, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_MARK:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, mark) != 4);
> - off = offsetof(struct sk_buff, mark);
> - emit(ARM_LDR_I(r_A, r_skb, off), ctx);
> + case BPF_JGE:
> + _emit(ARM_COND_CS, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_RXHASH:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, hash) != 4);
> - off = offsetof(struct sk_buff, hash);
> - emit(ARM_LDR_I(r_A, r_skb, off), ctx);
> + case BPF_JSGT:
> + _emit(ARM_COND_LT, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_VLAN_TAG:
> - case BPF_ANC | SKF_AD_VLAN_TAG_PRESENT:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff, vlan_tci) != 2);
> - off = offsetof(struct sk_buff, vlan_tci);
> - emit(ARM_LDRH_I(r_A, r_skb, off), ctx);
> - if (code == (BPF_ANC | SKF_AD_VLAN_TAG))
> - OP_IMM3(ARM_AND, r_A, r_A, ~VLAN_TAG_PRESENT, ctx);
> - else {
> - OP_IMM3(ARM_LSR, r_A, r_A, 12, ctx);
> - OP_IMM3(ARM_AND, r_A, r_A, 0x1, ctx);
> - }
> + case BPF_JSGE:
> + _emit(ARM_COND_GE, ARM_B(jmp_offset), ctx);
> break;
> - case BPF_ANC | SKF_AD_PKTTYPE:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
> - __pkt_type_offset[0]) != 1);
> - off = PKT_TYPE_OFFSET();
> - emit(ARM_LDRB_I(r_A, r_skb, off), ctx);
> - emit(ARM_AND_I(r_A, r_A, PKT_TYPE_MAX), ctx);
> -#ifdef __BIG_ENDIAN_BITFIELD
> - emit(ARM_LSR_I(r_A, r_A, 5), ctx);
> -#endif
> + }
> + break;
> + /* JMP OFF */
> + case BPF_JMP | BPF_JA:
> + {
> + if (off == 0)
> break;
> - case BPF_ANC | SKF_AD_QUEUE:
> - ctx->seen |= SEEN_SKB;
> - BUILD_BUG_ON(FIELD_SIZEOF(struct sk_buff,
> - queue_mapping) != 2);
> - BUILD_BUG_ON(offsetof(struct sk_buff,
> - queue_mapping) > 0xff);
> - off = offsetof(struct sk_buff, queue_mapping);
> - emit(ARM_LDRH_I(r_A, r_skb, off), ctx);
> + jmp_offset = bpf2a32_offset(i+off, i, ctx);
> + check_imm24(jmp_offset);
> + emit(ARM_B(jmp_offset), ctx);
> + break;
> + }
> + /* tail call */
> + case BPF_JMP | BPF_CALL | BPF_X:
> + if (emit_bpf_tail_call(ctx))
> + return -EFAULT;
> + break;
> + /* function call */
> + case BPF_JMP | BPF_CALL:
> + goto notyet;
> + /* function return */
> + case BPF_JMP | BPF_EXIT:
> + /* Optimization: when last instruction is EXIT
> + * simply fallthrough to epilogue.
> + */
> + if (i == ctx->prog->len - 1)
> break;
> - case BPF_ANC | SKF_AD_PAY_OFFSET:
> - ctx->seen |= SEEN_SKB | SEEN_CALL;
> + jmp_offset = epilogue_offset(ctx);
> + check_imm24(jmp_offset);
> + emit(ARM_B(jmp_offset), ctx);
> + break;
> +notyet:
> + pr_info_once("*** NOT YET: opcode %02x ***\n", code);
> + return -EFAULT;
> + default:
> + pr_err_once("unknown opcode %02x\n", code);
> + return -EINVAL;
> + }
>
> - emit(ARM_MOV_R(ARM_R0, r_skb), ctx);
> - emit_mov_i(ARM_R3, (unsigned int)skb_get_poff, ctx);
> - emit_blx_r(ARM_R3, ctx);
> - emit(ARM_MOV_R(r_A, ARM_R0), ctx);
> - break;
> - case BPF_LDX | BPF_W | BPF_ABS:
> - /*
> - * load a 32bit word from struct seccomp_data.
> - * seccomp_check_filter() will already have checked
> - * that k is 32bit aligned and lies within the
> - * struct seccomp_data.
> - */
> - ctx->seen |= SEEN_SKB;
> - emit(ARM_LDR_I(r_A, r_skb, k), ctx);
> - break;
> - default:
> - return -1;
> + if (ctx->flags & FLAG_IMM_OVERFLOW)
> + /*
> + * this instruction generated an overflow when
> + * trying to access the literal pool, so
> + * delegate this filter to the kernel interpreter.
> + */
> + return -1;
> + return 0;
> +}
> +
> +static int build_body(struct jit_ctx *ctx)
> +{
> + const struct bpf_prog *prog = ctx->prog;
> + unsigned int i;
> +
> + for (i = 0; i < prog->len; i++) {
> + const struct bpf_insn *insn = &(prog->insnsi[i]);
> + int ret;
> +
> + ret = build_insn(insn, ctx);
> +
> + /* It's used with loading the 64 bit immediate value. */
> + if (ret > 0) {
> + i++;
> + if (ctx->target == NULL)
> + ctx->offsets[i] = ctx->idx;
> + continue;
> }
>
> - if (ctx->flags & FLAG_IMM_OVERFLOW)
> - /*
> - * this instruction generated an overflow when
> - * trying to access the literal pool, so
> - * delegate this filter to the kernel interpreter.
> - */
> - return -1;
> + if (ctx->target == NULL)
> + ctx->offsets[i] = ctx->idx;
> +
> + /* If unsuccesfull, return with error code */
> + if (ret)
> + return ret;
> }
> + return 0;
> +}
>
> - /* compute offsets only during the first pass */
> - if (ctx->target == NULL)
> - ctx->offsets[i] = ctx->idx * 4;
> +static int validate_code(struct jit_ctx *ctx)
> +{
> + int i;
> +
> + for (i = 0; i < ctx->idx; i++) {
> + u32 a32_insn = le32_to_cpu(ctx->target[i]);
> +
> + if (a32_insn == ARM_INST_UDF)
> + return -1;
> + }
>
> return 0;
> }
>
> +void bpf_jit_compile(struct bpf_prog *prog)
> +{
> + /* Nothing to do here. We support Internal BPF. */
> +}
>
> -void bpf_jit_compile(struct bpf_prog *fp)
> +struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog)
> {
> +#ifdef __LITTLE_ENDIAN
> + struct bpf_prog *tmp, *orig_prog = prog;
> struct bpf_binary_header *header;
> + bool tmp_blinded = false;
> struct jit_ctx ctx;
> - unsigned tmp_idx;
> - unsigned alloc_size;
> - u8 *target_ptr;
> + unsigned int tmp_idx;
> + unsigned int image_size;
> + u8 *image_ptr;
>
> + /* If BPF JIT was not enabled then we must fall back to
> + * the interpreter.
> + */
> if (!bpf_jit_enable)
> - return;
> + return orig_prog;
>
> - memset(&ctx, 0, sizeof(ctx));
> - ctx.skf = fp;
> - ctx.ret0_fp_idx = -1;
> + /* If constant blinding was enabled and we failed during blinding
> + * then we must fall back to the interpreter. Otherwise, we save
> + * the new JITed code.
> + */
> + tmp = bpf_jit_blind_constants(prog);
>
> - ctx.offsets = kzalloc(4 * (ctx.skf->len + 1), GFP_KERNEL);
> - if (ctx.offsets == NULL)
> - return;
> + if (IS_ERR(tmp))
> + return orig_prog;
> + if (tmp != prog) {
> + tmp_blinded = true;
> + prog = tmp;
> + }
> +
> + memset(&ctx, 0, sizeof(ctx));
> + ctx.prog = prog;
>
> - /* fake pass to fill in the ctx->seen */
> - if (unlikely(build_body(&ctx)))
> + /* Not able to allocate memory for offsets[] , then
> + * we must fall back to the interpreter
> + */
> + ctx.offsets = kcalloc(prog->len, sizeof(int), GFP_KERNEL);
> + if (ctx.offsets == NULL) {
> + prog = orig_prog;
> goto out;
> + }
> +
> + /* 1) fake pass to find in the length of the JITed code,
> + * to compute ctx->offsets and other context variables
> + * needed to compute final JITed code.
> + * Also, calculate random starting pointer/start of JITed code
> + * which is prefixed by random number of fault instructions.
> + *
> + * If the first pass fails then there is no chance of it
> + * being successful in the second pass, so just fall back
> + * to the interpreter.
> + */
> + if (build_body(&ctx)) {
> + prog = orig_prog;
> + goto out_off;
> + }
>
> tmp_idx = ctx.idx;
> build_prologue(&ctx);
> ctx.prologue_bytes = (ctx.idx - tmp_idx) * 4;
>
> + ctx.epilogue_offset = ctx.idx;
> +
> #if __LINUX_ARM_ARCH__ < 7
> tmp_idx = ctx.idx;
> build_epilogue(&ctx);
> @@ -1020,64 +1838,96 @@ void bpf_jit_compile(struct bpf_prog *fp)
>
> ctx.idx += ctx.imm_count;
> if (ctx.imm_count) {
> - ctx.imms = kzalloc(4 * ctx.imm_count, GFP_KERNEL);
> - if (ctx.imms == NULL)
> - goto out;
> + ctx.imms = kcalloc(ctx.imm_count, sizeof(u32), GFP_KERNEL);
> + if (ctx.imms == NULL) {
> + prog = orig_prog;
> + goto out_off;
> + }
> }
> #else
> - /* there's nothing after the epilogue on ARMv7 */
> + /* there's nothing about the epilogue on ARMv7 */
> build_epilogue(&ctx);
> #endif
> - alloc_size = 4 * ctx.idx;
> - header = bpf_jit_binary_alloc(alloc_size, &target_ptr,
> - 4, jit_fill_hole);
> - if (header == NULL)
> - goto out;
> + /* Now we can get the actual image size of the JITed arm code.
> + * Currently, we are not considering the THUMB-2 instructions
> + * for jit, although it can decrease the size of the image.
> + *
> + * As each arm instruction is of length 32bit, we are translating
> + * number of JITed intructions into the size required to store these
> + * JITed code.
> + */
> + image_size = sizeof(u32) * ctx.idx;
>
> - ctx.target = (u32 *) target_ptr;
> + /* Now we know the size of the structure to make */
> + header = bpf_jit_binary_alloc(image_size, &image_ptr,
> + sizeof(u32), jit_fill_hole);
> + /* Not able to allocate memory for the structure then
> + * we must fall back to the interpretation
> + */
> + if (header == NULL) {
> + prog = orig_prog;
> + goto out_imms;
> + }
> +
> + /* 2.) Actual pass to generate final JIT code */
> + ctx.target = (u32 *) image_ptr;
> ctx.idx = 0;
>
> build_prologue(&ctx);
> +
> + /* If building the body of the JITed code fails somehow,
> + * we fall back to the interpretation.
> + */
> if (build_body(&ctx) < 0) {
> -#if __LINUX_ARM_ARCH__ < 7
> - if (ctx.imm_count)
> - kfree(ctx.imms);
> -#endif
> + image_ptr = NULL;
> bpf_jit_binary_free(header);
> - goto out;
> + prog = orig_prog;
> + goto out_imms;
> }
> build_epilogue(&ctx);
>
> + /* 3.) Extra pass to validate JITed Code */
> + if (validate_code(&ctx)) {
> + image_ptr = NULL;
> + bpf_jit_binary_free(header);
> + prog = orig_prog;
> + goto out_imms;
> + }
> flush_icache_range((u32)header, (u32)(ctx.target + ctx.idx));
>
> -#if __LINUX_ARM_ARCH__ < 7
> - if (ctx.imm_count)
> - kfree(ctx.imms);
> -#endif
> -
> if (bpf_jit_enable > 1)
> /* there are 2 passes here */
> - bpf_jit_dump(fp->len, alloc_size, 2, ctx.target);
> + bpf_jit_dump(prog->len, image_size, 2, ctx.target);
>
> set_memory_ro((unsigned long)header, header->pages);
> - fp->bpf_func = (void *)ctx.target;
> - fp->jited = 1;
> -out:
> + prog->bpf_func = (void *)ctx.target;
> + prog->jited = 1;
> +out_imms:
> +#if __LINUX_ARM_ARCH__ < 7
> + if (ctx.imm_count)
> + kfree(ctx.imms);
> +#endif
> +out_off:
> kfree(ctx.offsets);
> - return;
> +out:
> + if (tmp_blinded)
> + bpf_jit_prog_release_other(prog, prog == orig_prog ?
> + tmp : orig_prog);
> +#endif /* __LITTLE_ENDIAN */
> + return prog;
> }
>
> -void bpf_jit_free(struct bpf_prog *fp)
> +void bpf_jit_free(struct bpf_prog *prog)
> {
> - unsigned long addr = (unsigned long)fp->bpf_func & PAGE_MASK;
> + unsigned long addr = (unsigned long)prog->bpf_func & PAGE_MASK;
> struct bpf_binary_header *header = (void *)addr;
>
> - if (!fp->jited)
> + if (!prog->jited)
> goto free_filter;
>
> set_memory_rw(addr, header->pages);
> bpf_jit_binary_free(header);
>
> free_filter:
> - bpf_prog_unlock_free(fp);
> + bpf_prog_unlock_free(prog);
> }
> diff --git a/arch/arm/net/bpf_jit_32.h b/arch/arm/net/bpf_jit_32.h
> index c46fca2..d5cf5f6 100644
> --- a/arch/arm/net/bpf_jit_32.h
> +++ b/arch/arm/net/bpf_jit_32.h
> @@ -11,6 +11,7 @@
> #ifndef PFILTER_OPCODES_ARM_H
> #define PFILTER_OPCODES_ARM_H
>
> +/* ARM 32bit Registers */
> #define ARM_R0 0
> #define ARM_R1 1
> #define ARM_R2 2
> @@ -22,38 +23,43 @@
> #define ARM_R8 8
> #define ARM_R9 9
> #define ARM_R10 10
> -#define ARM_FP 11
> -#define ARM_IP 12
> -#define ARM_SP 13
> -#define ARM_LR 14
> -#define ARM_PC 15
> -
> -#define ARM_COND_EQ 0x0
> -#define ARM_COND_NE 0x1
> -#define ARM_COND_CS 0x2
> +#define ARM_FP 11 /* Frame Pointer */
> +#define ARM_IP 12 /* Intra-procedure scratch register */
> +#define ARM_SP 13 /* Stack pointer: as load/store base reg */
> +#define ARM_LR 14 /* Link Register */
> +#define ARM_PC 15 /* Program counter */
> +
> +#define ARM_COND_EQ 0x0 /* == */
> +#define ARM_COND_NE 0x1 /* != */
> +#define ARM_COND_CS 0x2 /* unsigned >= */
> #define ARM_COND_HS ARM_COND_CS
> -#define ARM_COND_CC 0x3
> +#define ARM_COND_CC 0x3 /* unsigned < */
> #define ARM_COND_LO ARM_COND_CC
> -#define ARM_COND_MI 0x4
> -#define ARM_COND_PL 0x5
> -#define ARM_COND_VS 0x6
> -#define ARM_COND_VC 0x7
> -#define ARM_COND_HI 0x8
> -#define ARM_COND_LS 0x9
> -#define ARM_COND_GE 0xa
> -#define ARM_COND_LT 0xb
> -#define ARM_COND_GT 0xc
> -#define ARM_COND_LE 0xd
> -#define ARM_COND_AL 0xe
> +#define ARM_COND_MI 0x4 /* < 0 */
> +#define ARM_COND_PL 0x5 /* >= 0 */
> +#define ARM_COND_VS 0x6 /* Signed Overflow */
> +#define ARM_COND_VC 0x7 /* No Signed Overflow */
> +#define ARM_COND_HI 0x8 /* unsigned > */
> +#define ARM_COND_LS 0x9 /* unsigned <= */
> +#define ARM_COND_GE 0xa /* Signed >= */
> +#define ARM_COND_LT 0xb /* Signed < */
> +#define ARM_COND_GT 0xc /* Signed > */
> +#define ARM_COND_LE 0xd /* Signed <= */
> +#define ARM_COND_AL 0xe /* None */
>
> /* register shift types */
> #define SRTYPE_LSL 0
> #define SRTYPE_LSR 1
> #define SRTYPE_ASR 2
> #define SRTYPE_ROR 3
> +#define SRTYPE_ASL (SRTYPE_LSL)
>
> #define ARM_INST_ADD_R 0x00800000
> +#define ARM_INST_ADDS_R 0x00900000
> +#define ARM_INST_ADC_R 0x00a00000
> +#define ARM_INST_ADC_I 0x02a00000
> #define ARM_INST_ADD_I 0x02800000
> +#define ARM_INST_ADDS_I 0x02900000
>
> #define ARM_INST_AND_R 0x00000000
> #define ARM_INST_AND_I 0x02000000
> @@ -76,8 +82,10 @@
> #define ARM_INST_LDRH_I 0x01d000b0
> #define ARM_INST_LDRH_R 0x019000b0
> #define ARM_INST_LDR_I 0x05900000
> +#define ARM_INST_LDR_R 0x07900000
>
> #define ARM_INST_LDM 0x08900000
> +#define ARM_INST_LDM_IA 0x08b00000
>
> #define ARM_INST_LSL_I 0x01a00000
> #define ARM_INST_LSL_R 0x01a00010
> @@ -86,6 +94,7 @@
> #define ARM_INST_LSR_R 0x01a00030
>
> #define ARM_INST_MOV_R 0x01a00000
> +#define ARM_INST_MOVS_R 0x01b00000
> #define ARM_INST_MOV_I 0x03a00000
> #define ARM_INST_MOVW 0x03000000
> #define ARM_INST_MOVT 0x03400000
> @@ -96,17 +105,28 @@
> #define ARM_INST_PUSH 0x092d0000
>
> #define ARM_INST_ORR_R 0x01800000
> +#define ARM_INST_ORRS_R 0x01900000
> #define ARM_INST_ORR_I 0x03800000
>
> #define ARM_INST_REV 0x06bf0f30
> #define ARM_INST_REV16 0x06bf0fb0
>
> #define ARM_INST_RSB_I 0x02600000
> +#define ARM_INST_RSBS_I 0x02700000
> +#define ARM_INST_RSC_I 0x02e00000
>
> #define ARM_INST_SUB_R 0x00400000
> +#define ARM_INST_SUBS_R 0x00500000
> +#define ARM_INST_RSB_R 0x00600000
> #define ARM_INST_SUB_I 0x02400000
> +#define ARM_INST_SUBS_I 0x02500000
> +#define ARM_INST_SBC_I 0x02c00000
> +#define ARM_INST_SBC_R 0x00c00000
> +#define ARM_INST_SBCS_R 0x00d00000
>
> #define ARM_INST_STR_I 0x05800000
> +#define ARM_INST_STRB_I 0x05c00000
> +#define ARM_INST_STRH_I 0x01c000b0
>
> #define ARM_INST_TST_R 0x01100000
> #define ARM_INST_TST_I 0x03100000
> @@ -117,6 +137,8 @@
>
> #define ARM_INST_MLS 0x00600090
>
> +#define ARM_INST_UXTH 0x06ff0070
> +
> /*
> * Use a suitable undefined instruction to use for ARM/Thumb2 faulting.
> * We need to be careful not to conflict with those used by other modules
> @@ -135,9 +157,15 @@
> #define _AL3_R(op, rd, rn, rm) ((op ## _R) | (rd) << 12 | (rn) << 16 | (rm))
> /* immediate */
> #define _AL3_I(op, rd, rn, imm) ((op ## _I) | (rd) << 12 | (rn) << 16 | (imm))
> +/* register with register-shift */
> +#define _AL3_SR(inst) (inst | (1 << 4))
>
> #define ARM_ADD_R(rd, rn, rm) _AL3_R(ARM_INST_ADD, rd, rn, rm)
> +#define ARM_ADDS_R(rd, rn, rm) _AL3_R(ARM_INST_ADDS, rd, rn, rm)
> #define ARM_ADD_I(rd, rn, imm) _AL3_I(ARM_INST_ADD, rd, rn, imm)
> +#define ARM_ADDS_I(rd, rn, imm) _AL3_I(ARM_INST_ADDS, rd, rn, imm)
> +#define ARM_ADC_R(rd, rn, rm) _AL3_R(ARM_INST_ADC, rd, rn, rm)
> +#define ARM_ADC_I(rd, rn, imm) _AL3_I(ARM_INST_ADC, rd, rn, imm)
>
> #define ARM_AND_R(rd, rn, rm) _AL3_R(ARM_INST_AND, rd, rn, rm)
> #define ARM_AND_I(rd, rn, imm) _AL3_I(ARM_INST_AND, rd, rn, imm)
> @@ -156,7 +184,9 @@
> #define ARM_EOR_I(rd, rn, imm) _AL3_I(ARM_INST_EOR, rd, rn, imm)
>
> #define ARM_LDR_I(rt, rn, off) (ARM_INST_LDR_I | (rt) << 12 | (rn) << 16 \
> - | (off))
> + | ((off) & 0xfff))
> +#define ARM_LDR_R(rt, rn, rm) (ARM_INST_LDR_R | (rt) << 12 | (rn) << 16 \
> + | (rm))
> #define ARM_LDRB_I(rt, rn, off) (ARM_INST_LDRB_I | (rt) << 12 | (rn) << 16 \
> | (off))
> #define ARM_LDRB_R(rt, rn, rm) (ARM_INST_LDRB_R | (rt) << 12 | (rn) << 16 \
> @@ -167,15 +197,23 @@
> | (rm))
>
> #define ARM_LDM(rn, regs) (ARM_INST_LDM | (rn) << 16 | (regs))
> +#define ARM_LDM_IA(rn, regs) (ARM_INST_LDM_IA | (rn) << 16 | (regs))
>
> #define ARM_LSL_R(rd, rn, rm) (_AL3_R(ARM_INST_LSL, rd, 0, rn) | (rm) << 8)
> #define ARM_LSL_I(rd, rn, imm) (_AL3_I(ARM_INST_LSL, rd, 0, rn) | (imm) << 7)
>
> #define ARM_LSR_R(rd, rn, rm) (_AL3_R(ARM_INST_LSR, rd, 0, rn) | (rm) << 8)
> #define ARM_LSR_I(rd, rn, imm) (_AL3_I(ARM_INST_LSR, rd, 0, rn) | (imm) << 7)
> +#define ARM_ASR_R(rd, rn, rm) (_AL3_R(ARM_INST_ASR, rd, 0, rn) | (rm) << 8)
> +#define ARM_ASR_I(rd, rn, imm) (_AL3_I(ARM_INST_ASR, rd, 0, rn) | (imm) << 7)
>
> #define ARM_MOV_R(rd, rm) _AL3_R(ARM_INST_MOV, rd, 0, rm)
> +#define ARM_MOVS_R(rd, rm) _AL3_R(ARM_INST_MOVS, rd, 0, rm)
> #define ARM_MOV_I(rd, imm) _AL3_I(ARM_INST_MOV, rd, 0, imm)
> +#define ARM_MOV_SR(rd, rm, type, rs) \
> + (_AL3_SR(ARM_MOV_R(rd, rm)) | (type) << 5 | (rs) << 8)
> +#define ARM_MOV_SI(rd, rm, type, imm6) \
> + (ARM_MOV_R(rd, rm) | (type) << 5 | (imm6) << 7)
>
> #define ARM_MOVW(rd, imm) \
> (ARM_INST_MOVW | ((imm) >> 12) << 16 | (rd) << 12 | ((imm) & 0x0fff))
> @@ -190,19 +228,38 @@
>
> #define ARM_ORR_R(rd, rn, rm) _AL3_R(ARM_INST_ORR, rd, rn, rm)
> #define ARM_ORR_I(rd, rn, imm) _AL3_I(ARM_INST_ORR, rd, rn, imm)
> -#define ARM_ORR_S(rd, rn, rm, type, rs) \
> - (ARM_ORR_R(rd, rn, rm) | (type) << 5 | (rs) << 7)
> +#define ARM_ORR_SR(rd, rn, rm, type, rs) \
> + (_AL3_SR(ARM_ORR_R(rd, rn, rm)) | (type) << 5 | (rs) << 8)
> +#define ARM_ORRS_R(rd, rn, rm) _AL3_R(ARM_INST_ORRS, rd, rn, rm)
> +#define ARM_ORRS_SR(rd, rn, rm, type, rs) \
> + (_AL3_SR(ARM_ORRS_R(rd, rn, rm)) | (type) << 5 | (rs) << 8)
> +#define ARM_ORR_SI(rd, rn, rm, type, imm6) \
> + (ARM_ORR_R(rd, rn, rm) | (type) << 5 | (imm6) << 7)
> +#define ARM_ORRS_SI(rd, rn, rm, type, imm6) \
> + (ARM_ORRS_R(rd, rn, rm) | (type) << 5 | (imm6) << 7)
>
> #define ARM_REV(rd, rm) (ARM_INST_REV | (rd) << 12 | (rm))
> #define ARM_REV16(rd, rm) (ARM_INST_REV16 | (rd) << 12 | (rm))
>
> #define ARM_RSB_I(rd, rn, imm) _AL3_I(ARM_INST_RSB, rd, rn, imm)
> +#define ARM_RSBS_I(rd, rn, imm) _AL3_I(ARM_INST_RSBS, rd, rn, imm)
> +#define ARM_RSC_I(rd, rn, imm) _AL3_I(ARM_INST_RSC, rd, rn, imm)
>
> #define ARM_SUB_R(rd, rn, rm) _AL3_R(ARM_INST_SUB, rd, rn, rm)
> +#define ARM_SUBS_R(rd, rn, rm) _AL3_R(ARM_INST_SUBS, rd, rn, rm)
> +#define ARM_RSB_R(rd, rn, rm) _AL3_R(ARM_INST_RSB, rd, rn, rm)
> +#define ARM_SBC_R(rd, rn, rm) _AL3_R(ARM_INST_SBC, rd, rn, rm)
> +#define ARM_SBCS_R(rd, rn, rm) _AL3_R(ARM_INST_SBCS, rd, rn, rm)
> #define ARM_SUB_I(rd, rn, imm) _AL3_I(ARM_INST_SUB, rd, rn, imm)
> +#define ARM_SUBS_I(rd, rn, imm) _AL3_I(ARM_INST_SUBS, rd, rn, imm)
> +#define ARM_SBC_I(rd, rn, imm) _AL3_I(ARM_INST_SBC, rd, rn, imm)
>
> #define ARM_STR_I(rt, rn, off) (ARM_INST_STR_I | (rt) << 12 | (rn) << 16 \
> - | (off))
> + | ((off) & 0xfff))
> +#define ARM_STRH_I(rt, rn, off) (ARM_INST_STRH_I | (rt) << 12 | (rn) << 16 \
> + | (((off) & 0xf0) << 4) | ((off) & 0xf))
> +#define ARM_STRB_I(rt, rn, off) (ARM_INST_STRB_I | (rt) << 12 | (rn) << 16 \
> + | (((off) & 0xf0) << 4) | ((off) & 0xf))
>
> #define ARM_TST_R(rn, rm) _AL3_R(ARM_INST_TST, 0, rn, rm)
> #define ARM_TST_I(rn, imm) _AL3_I(ARM_INST_TST, 0, rn, imm)
> @@ -214,5 +271,6 @@
>
> #define ARM_MLS(rd, rn, rm, ra) (ARM_INST_MLS | (rd) << 16 | (rn) | (rm) << 8 \
> | (ra) << 12)
> +#define ARM_UXTH(rd, rm) (ARM_INST_UXTH | (rd) << 12 | (rm))
>
> #endif /* PFILTER_OPCODES_ARM_H */
> --
> 2.7.4
>
--
Kees Cook
Pixel Security
^ permalink raw reply
* Re: [PATCH linux-firmware] Mellanox: Add firmware for mlxsw_spectrum
From: Kyle McMartin @ 2017-05-30 19:22 UTC (permalink / raw)
To: Yotam Gigi; +Cc: idosch, jiri, eladr, arkadis, netdev, linux-firmware
In-Reply-To: <20170529104228.32947-1-yotamg@mellanox.com>
On Mon, May 29, 2017 at 01:42:28PM +0300, Yotam Gigi wrote:
> Add first firmware for the Mellanox Spectrum switch, as a followup to the
> recently added commit:
> 6b7421992b8d ("mlxsw: spectrum: Validate firmware revision on init")
>
> The version of the firmware release is 13.1420.122
>
> Signed-off-by: Yotam Gigi <yotamg@mellanox.com>
> ---
> mlxsw_spectrum-13.1420.122.mfa2 | Bin 0 -> 860424 bytes
> 1 file changed, 0 insertions(+), 0 deletions(-)
> create mode 100644 mlxsw_spectrum-13.1420.122.mfa2
>
> diff --git a/mlxsw_spectrum-13.1420.122.mfa2 b/mlxsw_spectrum-13.1420.122.mfa2
> new file mode 100644
Can we please put this in a subdirectory if we're going to be adding
more versions of this in the future?
--Kyle
^ permalink raw reply
* Re: [PATCH V2 2/3] net-next: dsa: add multi cpu port support
From: Florian Fainelli @ 2017-05-30 19:23 UTC (permalink / raw)
To: John Crispin, kbuild test robot
Cc: kbuild-all, Andrew Lunn, Vivien Didelot, David S . Miller,
Sean Wang, netdev, linux-kernel
In-Reply-To: <979a011e-0f41-a919-9d46-1f8258b645af@gmail.com>
On 05/30/2017 12:15 PM, Florian Fainelli wrote:
> Hi John,
>
> On 05/30/2017 11:37 AM, John Crispin wrote:
>> Hi,
>>
>> the patch series is based on net-next from 12 hours ago and works fine
>> on that tree. I compile and runtime tested it quite intensively on
>> various boards
>
> The warning is legit though:
>
> 572 if (dsa_port_is_cpu(port))
> 573 err = dsa_cpu_parse(port, index, dst, ds);
> > 574 else if (dsa_is_normal_port(port))
> 575 err = dsa_user_parse(port->dn, index, ds);
>
> and after applying your patches I also met it:
>
> net/dsa/dsa2.c: In function 'dsa_ds_parse':
> net/dsa/dsa2.c:574:3: warning: passing argument 1 of
> 'dsa_is_normal_port' from incompatible pointer type [enabled by default]
> else if (dsa_is_normal_port(port))
> ^
> In file included from net/dsa/dsa_priv.h:17:0,
> from net/dsa/dsa2.c:22:
> ./include/net/dsa.h:264:20: note: expected 'struct dsa_switch *' but
> argument is of type 'struct dsa_port *'
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^
> net/dsa/dsa2.c:574:3: error: too few arguments to function
> 'dsa_is_normal_port'
> else if (dsa_is_normal_port(port))
> ^
> In file included from net/dsa/dsa_priv.h:17:0,
> from net/dsa/dsa2.c:22:
> ./include/net/dsa.h:264:20: note: declared here
> static inline bool dsa_is_normal_port(struct dsa_switch *ds, int p)
> ^
> CC net/bridge/br_stp.o
> scripts/Makefile.build:302: recipe for target 'net/dsa/dsa2.o' failed
> make[4]: *** [net/dsa/dsa2.o] Error 1
> scripts/Makefile.build:561: recipe for target 'net/dsa' failed
> make[3]: *** [net/dsa] Error 2
> make[3]: *** Waiting for unfinished jobs....
>
> We need something like this, I have some comments on your patches that I
> will send shortly. Thanks!
>
> diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
> index 8b13aa735c40..124c5acfa123 100644
> --- a/net/dsa/dsa2.c
> +++ b/net/dsa/dsa2.c
> @@ -539,14 +539,14 @@ static int dsa_cpu_parse(struct dsa_port *port,
> u32 index,
> return 0;
> }
>
> -static int dsa_user_parse(struct device_node *port, u32 index,
> +static int dsa_user_parse(struct dsa_port *port, u32 index,
> struct dsa_switch *ds)
> {
> struct device_node *cpu_port;
> const unsigned int *cpu_port_reg;
> int cpu_port_index;
>
> - cpu_port = of_parse_phandle(port, "cpu", 0);
> + cpu_port = of_parse_phandle(port->dn, "cpu", 0);
> if (cpu_port) {
> cpu_port_reg = of_get_property(cpu_port, "reg", NULL);
> if (!cpu_port_reg)
> @@ -572,7 +572,7 @@ static int dsa_ds_parse(struct dsa_switch_tree *dst,
> struct dsa_switch *ds)
> if (dsa_port_is_cpu(port))
> err = dsa_cpu_parse(port, index, dst, ds);
> else if (dsa_is_normal_port(port))
> - err = dsa_user_parse(port->dn, index, ds);
> + err = dsa_user_parse(port, index, ds);
>
> if (err)
> return err;
here is a version that builds, I missed one hunk:
diff --git a/net/dsa/dsa2.c b/net/dsa/dsa2.c
index 8b13aa735c40..d71a2a610340 100644
--- a/net/dsa/dsa2.c
+++ b/net/dsa/dsa2.c
@@ -539,14 +539,14 @@ static int dsa_cpu_parse(struct dsa_port *port,
u32 index,
return 0;
}
-static int dsa_user_parse(struct device_node *port, u32 index,
+static int dsa_user_parse(struct dsa_port *port, u32 index,
struct dsa_switch *ds)
{
struct device_node *cpu_port;
const unsigned int *cpu_port_reg;
int cpu_port_index;
- cpu_port = of_parse_phandle(port, "cpu", 0);
+ cpu_port = of_parse_phandle(port->dn, "cpu", 0);
if (cpu_port) {
cpu_port_reg = of_get_property(cpu_port, "reg", NULL);
if (!cpu_port_reg)
@@ -571,8 +571,8 @@ static int dsa_ds_parse(struct dsa_switch_tree *dst,
struct dsa_switch *ds)
if (dsa_port_is_cpu(port))
err = dsa_cpu_parse(port, index, dst, ds);
- else if (dsa_is_normal_port(port))
- err = dsa_user_parse(port->dn, index, ds);
+ else if (dsa_is_normal_port(ds, index))
+ err = dsa_user_parse(port, index, ds);
if (err)
return err;
--
Florian
^ permalink raw reply related
* Re: [net-wireless-orinoco] question about potential null pointer dereference
From: Johannes Berg @ 2017-05-30 19:37 UTC (permalink / raw)
To: Gustavo A. R. Silva, Kalle Valo; +Cc: linux-wireless, netdev, linux-kernel
In-Reply-To: <20170530120032.Horde.GHW0X8-B795CdxPjQOfDPa9@gator4166.hostgator.com>
Hi,
> The issue here is that line 56 implies that pointer tfm_michael
> might be NULL. If this is the case, there is a potential NULL
> pointer dereference at line 52 once pointer tfm_michael is
> indirectly dereferenced inside macro SHASH_DESC_ON_STACK().
>
> My question is if there is any chance that pointer tfm_michael
> might be NULL when calling macro SHASH_DESC_ON_STACK() ?
>
> I'm trying to figure out if this is a false positive or something
> that needs to be fixed somehow.
Look, if you're just sending the coverity reports to the list without
reflecting and researching them, that's not actually useful - we can
look at them ourselves.
It took me at most a few minutes to figure this one out, so please just
do the same, look at the code, and figure out what the right answer is
here.
johannes
^ permalink raw reply
* Re: [PATCH V2 2/3] net-next: dsa: add multi cpu port support
From: Vivien Didelot @ 2017-05-30 19:45 UTC (permalink / raw)
To: John Crispin, Andrew Lunn, Florian Fainelli, David S . Miller,
Sean Wang
Cc: netdev, linux-kernel, John Crispin
In-Reply-To: <20170530104419.6052-2-john@phrozen.org>
Hi John,
John Crispin <john@phrozen.org> writes:
> +static inline bool dsa_is_upstream_port(struct dsa_switch *ds, int p)
> +{
> + return dsa_is_cpu_port(ds, p) || dsa_is_dsa_port(ds, p);
> +}
This looks confusing to me. What DSA calls an "upstream" port for the
moment is the port which goes to the CPU interface.
CPU0 (eth0)
|
| sw0 sw1 sw2
[p0 p1 p2]--[p0 p1 p2]--[p0 p1 p2]
| | | |
eth1 eth2 eth3 eth4
So in the example above, sw1p0 is an upstream port, but sw1p2 is not.
This is why dsa_upstream_port makes use of ds->rtable.
> @@ -1140,11 +1140,16 @@ int dsa_slave_create(struct dsa_switch *ds, struct device *parent,
> struct net_device *master;
> struct net_device *slave_dev;
> struct dsa_slave_priv *p;
> + int port_cpu = ds->ports[port].upstream;
ds->ports[port] is p->dp.
> int ret;
>
> - master = ds->dst->master_netdev;
> - if (ds->master_netdev)
> + if (port_cpu && ds->ports[port_cpu].ethernet)
0 is a valid port index for a CPU, e.g. Marvell 88E6390.
> + master = ds->ports[port_cpu].ethernet;
> + else if (ds->master_netdev)
> master = ds->master_netdev;
> + else
> + master = ds->dst->master_netdev;
> + master->dsa_ptr = (void *)ds->dst;
>
> slave_dev = alloc_netdev(sizeof(struct dsa_slave_priv), name,
> NET_NAME_UNKNOWN, ether_setup);
> @@ -1173,6 +1178,7 @@ int dsa_slave_create(struct dsa_switch *ds, struct device *parent,
> p->dp = &ds->ports[port];
> INIT_LIST_HEAD(&p->mall_tc_list);
> p->xmit = dst->tag_ops->xmit;
> + p->master = master;
I'm a bit confused why we need all these references to net devices. We
now have ds->master_netdev, dst->master_netdev, dp->ethernet and
p->master...
Wouldn't it be simpler if we only had dp->ethernet (or whichever more
explicit name) for the conduit interface used to send/receive frames?
Maybe I am missing something, in which case I'm sorry in advance.
Thanks,
Vivien
^ permalink raw reply
* Re: [PATCH V2 2/3] net-next: dsa: add multi cpu port support
From: Vivien Didelot @ 2017-05-30 19:50 UTC (permalink / raw)
To: John Crispin, Andrew Lunn, Florian Fainelli, David S . Miller,
Sean Wang
Cc: netdev, linux-kernel, John Crispin
In-Reply-To: <87inki5dsu.fsf@weeman.i-did-not-set--mail-host-address--so-tickle-me>
Hi John,
Vivien Didelot <vivien.didelot@savoirfairelinux.com> writes:
>> + int port_cpu = ds->ports[port].upstream;
>
> ds->ports[port] is p->dp.
I misread this part, p is not yet allocated in that chunk, please ignore
this one comment ;-)
Thanks,
Vivien
^ permalink raw reply
* Re: [PATCH net-next v2 4/6] net: dsa: remove unused arguments of tagger rcv
From: Vivien Didelot @ 2017-05-30 19:55 UTC (permalink / raw)
To: Florian Fainelli, netdev
Cc: linux-kernel, kernel, David S. Miller, Andrew Lunn
In-Reply-To: <b31811c6-d019-da1f-9df0-0ad00264204b@gmail.com>
Hi Florian,
Florian Fainelli <f.fainelli@gmail.com> writes:
> I actually have a patch pending that adds support for HW
> insertion/extraction of switch tags (broadcom HW supports that) which
> require the orig_dev to be there so we know what features are supported
> by the master network device.
Is orig_dev really needed in the tagging implementation, or only in the
layer above? (dsa_slave_xmit and dsa_switch_rcv.)
> Do you mind dropping this one from your patch series?
I don't mind if they are actually needed in taggers. I'd wait for
reviews of the other patches before respinning though.
Thanks,
Vivien
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox