* [PATCH net-next v2 00/10] net: dsa: lan9303: unicast offload, fdb,mdb,STP
From: Egil Hjelmeland @ 2017-07-25 16:15 UTC (permalink / raw)
To: corbet, andrew, vivien.didelot, f.fainelli, davem, kernel,
linux-doc, linux-kernel, netdev
Cc: Egil Hjelmeland
This series extends the LAN9303 3 port switch DSA driver. Highlights:
- Make the MDIO interface work
- Bridging: Unicast offload
- Bridging: Added fdb/mdb handling
- Bridging: STP support
- Documentation
Changes v1 -> v2:
- sorted out emailing issues, threading and date. And sent from private
account in order to avoid company disclaimer in emails.
- Removed the three last "work around" patches. But first moved one doc
paragraph to the document patch.
Egil Hjelmeland (10):
net: dsa: lan9303: Fixed MDIO interface
net: dsa: lan9303: Do not disable/enable switch fabric port 0 at
startup
net: dsa: lan9303: Refactor lan9303_enable_packet_processing()
net: dsa: lan9303: Added adjust_link() method
net: dsa: added dsa_net_device_to_dsa_port()
net: dsa: lan9303: added sysfs node swe_bcst_throt
net: dsa: lan9303: Added basic offloading of unicast traffic
net: dsa: lan9303: Added ALR/fdb/mdb handling
net: dsa: lan9303: Added Documentation/networking/dsa/lan9303.txt
net: dsa: lan9303: Only allocate 3 ports
Documentation/networking/dsa/lan9303.txt | 63 +++
drivers/net/dsa/lan9303-core.c | 709 ++++++++++++++++++++++++++++---
drivers/net/dsa/lan9303.h | 23 +
drivers/net/dsa/lan9303_i2c.c | 2 +
drivers/net/dsa/lan9303_mdio.c | 34 ++
include/net/dsa.h | 1 +
net/dsa/slave.c | 10 +
7 files changed, 772 insertions(+), 70 deletions(-)
create mode 100644 Documentation/networking/dsa/lan9303.txt
--
2.11.0
^ permalink raw reply
* [PATCH net] udp: preserve head state for IP_CMSG_PASSSEC
From: Paolo Abeni @ 2017-07-25 15:57 UTC (permalink / raw)
To: netdev; +Cc: David S. Miller, Eric Dumazet, Paul Moore
Paul Moore reported a SELinux/IP_PASSSEC regression
caused by missing skb->sp at recvmsg() time. We need to
preserve the skb head state to process the IP_CMSG_PASSSEC
cmsg.
With this commit we avoid releasing the skb head state in the
BH even if a secpath is attached to the current skb, and stores
the skb status (with/without head states) in the scratch area,
so that we can access it at skb deallocation time, without
incurring in cache-miss penalties.
This also avoids misusing the skb CB for ipv6 packets,
as introduced by the commit 0ddf3fb2c43d ("udp: preserve
skb->dst if required for IP options processing").
Clean a bit the scratch area helpers implementation, to
reduce the code differences between 32 and 64 bits build.
Reported-by: Paul Moore <paul@paul-moore.com>
Fixes: 0a463c78d25b ("udp: avoid a cache miss on dequeue")
Fixes: 0ddf3fb2c43d ("udp: preserve skb->dst if required for IP options processing")
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Tested-by: Paul Moore <paul@paul-moore.com>
---
include/net/udp.h | 33 ++++++++++++++++++++++-----------
net/ipv4/udp.c | 38 ++++++++++++++++++--------------------
2 files changed, 40 insertions(+), 31 deletions(-)
diff --git a/include/net/udp.h b/include/net/udp.h
index 972ce4baab6b..56ce2d2a612d 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -305,33 +305,44 @@ struct sock *udp6_lib_lookup_skb(struct sk_buff *skb,
/* UDP uses skb->dev_scratch to cache as much information as possible and avoid
* possibly multiple cache miss on dequeue()
*/
-#if BITS_PER_LONG == 64
-
-/* truesize, len and the bit needed to compute skb_csum_unnecessary will be on
- * cold cache lines at recvmsg time.
- * skb->len can be stored on 16 bits since the udp header has been already
- * validated and pulled.
- */
struct udp_dev_scratch {
- u32 truesize;
+ /* skb->truesize and the stateless bit are embedded in a single field;
+ * do not use a bitfield since the compiler emits better/smaller code
+ * this way
+ */
+ u32 _tsize_state;
+
+#if BITS_PER_LONG == 64
+ /* len and the bit needed to compute skb_csum_unnecessary
+ * will be on cold cache lines at recvmsg time.
+ * skb->len can be stored on 16 bits since the udp header has been
+ * already validated and pulled.
+ */
u16 len;
bool is_linear;
bool csum_unnecessary;
+#endif
};
+static inline struct udp_dev_scratch *udp_skb_scratch(struct sk_buff *skb)
+{
+ return (struct udp_dev_scratch *)&skb->dev_scratch;
+}
+
+#if BITS_PER_LONG == 64
static inline unsigned int udp_skb_len(struct sk_buff *skb)
{
- return ((struct udp_dev_scratch *)&skb->dev_scratch)->len;
+ return udp_skb_scratch(skb)->len;
}
static inline bool udp_skb_csum_unnecessary(struct sk_buff *skb)
{
- return ((struct udp_dev_scratch *)&skb->dev_scratch)->csum_unnecessary;
+ return udp_skb_scratch(skb)->csum_unnecessary;
}
static inline bool udp_skb_is_linear(struct sk_buff *skb)
{
- return ((struct udp_dev_scratch *)&skb->dev_scratch)->is_linear;
+ return udp_skb_scratch(skb)->is_linear;
}
#else
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index b057653ceca9..d243772f6efc 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -1163,34 +1163,32 @@ int udp_sendpage(struct sock *sk, struct page *page, int offset,
return ret;
}
-#if BITS_PER_LONG == 64
+#define UDP_SKB_IS_STATELESS 0x80000000
+
static void udp_set_dev_scratch(struct sk_buff *skb)
{
- struct udp_dev_scratch *scratch;
+ struct udp_dev_scratch *scratch = udp_skb_scratch(skb);
BUILD_BUG_ON(sizeof(struct udp_dev_scratch) > sizeof(long));
- scratch = (struct udp_dev_scratch *)&skb->dev_scratch;
- scratch->truesize = skb->truesize;
+ scratch->_tsize_state = skb->truesize;
+#if BITS_PER_LONG == 64
scratch->len = skb->len;
scratch->csum_unnecessary = !!skb_csum_unnecessary(skb);
scratch->is_linear = !skb_is_nonlinear(skb);
+#endif
+ if (likely(!skb->_skb_refdst))
+ scratch->_tsize_state |= UDP_SKB_IS_STATELESS;
}
static int udp_skb_truesize(struct sk_buff *skb)
{
- return ((struct udp_dev_scratch *)&skb->dev_scratch)->truesize;
-}
-#else
-static void udp_set_dev_scratch(struct sk_buff *skb)
-{
- skb->dev_scratch = skb->truesize;
+ return udp_skb_scratch(skb)->_tsize_state & ~UDP_SKB_IS_STATELESS;
}
-static int udp_skb_truesize(struct sk_buff *skb)
+static bool udp_skb_has_head_state(struct sk_buff *skb)
{
- return skb->dev_scratch;
+ return !(udp_skb_scratch(skb)->_tsize_state & UDP_SKB_IS_STATELESS);
}
-#endif
/* fully reclaim rmem/fwd memory allocated for skb */
static void udp_rmem_release(struct sock *sk, int size, int partial,
@@ -1388,10 +1386,10 @@ void skb_consume_udp(struct sock *sk, struct sk_buff *skb, int len)
unlock_sock_fast(sk, slow);
}
- /* we cleared the head states previously only if the skb lacks any IP
- * options, see __udp_queue_rcv_skb().
+ /* In the more common cases we cleared the head states previously,
+ * see __udp_queue_rcv_skb().
*/
- if (unlikely(IPCB(skb)->opt.optlen > 0))
+ if (unlikely(udp_skb_has_head_state(skb)))
skb_release_head_state(skb);
consume_stateless_skb(skb);
}
@@ -1784,11 +1782,11 @@ static int __udp_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
sk_mark_napi_id_once(sk, skb);
}
- /* At recvmsg() time we need skb->dst to process IP options-related
- * cmsg, elsewhere can we clear all pending head states while they are
- * hot in the cache
+ /* At recvmsg() time we may access skb->dst or skb->sp depending on
+ * the IP options and the cmsg flags, elsewhere can we clear all
+ * pending head states while they are hot in the cache
*/
- if (likely(IPCB(skb)->opt.optlen == 0))
+ if (likely(IPCB(skb)->opt.optlen == 0 && !skb->sp))
skb_release_head_state(skb);
rc = __udp_enqueue_schedule_skb(sk, skb);
--
2.13.3
^ permalink raw reply related
* [PATCH 6/8] net: mvpp2: add support for TX interrupts and RX queue distribution modes
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
This commit adds the support for two related features:
- Support for TX interrupts, with one interrupt for each CPU
- Support for different RX queue distribution modes
MVPP2_QDIST_SINGLE_MODE where a single interrupt, shared by all
CPUs, receives the RX events, and MVPP2_QDIST_MULTI_MODE, where the
per-CPU interrupts used for TX events are also used for RX events.
Since additional interrupts are needed, an update to the Device Tree
binding is needed. However, backward compatibility is preserved with
the old Device Tree binding, by gracefully degrading to the original
behavior, with only one RX interrupt, and TX completion being handled
by an hrtimer.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 275 +++++++++++++++++++++++++++++++----
1 file changed, 246 insertions(+), 29 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index a8d448b..cabdbd3 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -122,6 +122,9 @@
#define MVPP2_TXQ_DESC_ADDR_REG 0x2084
#define MVPP2_TXQ_DESC_SIZE_REG 0x2088
#define MVPP2_TXQ_DESC_SIZE_MASK 0x3ff0
+#define MVPP2_TXQ_THRESH_REG 0x2094
+#define MVPP2_TXQ_THRESH_OFFSET 16
+#define MVPP2_TXQ_THRESH_MASK 0x3fff
#define MVPP2_AGGR_TXQ_UPDATE_REG 0x2090
#define MVPP2_TXQ_INDEX_REG 0x2098
#define MVPP2_TXQ_PREF_BUF_REG 0x209c
@@ -185,6 +188,9 @@
#define MVPP22_AXI_CODE_DOMAIN_SYSTEM 3
/* Interrupt Cause and Mask registers */
+#define MVPP2_ISR_TX_THRESHOLD_REG(port) (0x5140 + 4 * (port))
+#define MVPP2_MAX_ISR_TX_THRESHOLD 0xfffff0
+
#define MVPP2_ISR_RX_THRESHOLD_REG(rxq) (0x5200 + 4 * (rxq))
#define MVPP2_MAX_ISR_RX_THRESHOLD 0xfffff0
#define MVPP21_ISR_RXQ_GROUP_REG(port) (0x5400 + 4 * (port))
@@ -208,6 +214,7 @@
#define MVPP2_ISR_RX_TX_CAUSE_REG(port) (0x5480 + 4 * (port))
#define MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK 0xffff
#define MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK 0xff0000
+#define MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_OFFSET 16
#define MVPP2_CAUSE_RX_FIFO_OVERRUN_MASK BIT(24)
#define MVPP2_CAUSE_FCS_ERR_MASK BIT(25)
#define MVPP2_CAUSE_TX_FIFO_UNDERRUN_MASK BIT(26)
@@ -435,6 +442,7 @@
/* Coalescing */
#define MVPP2_TXDONE_COAL_PKTS_THRESH 15
#define MVPP2_TXDONE_HRTIMER_PERIOD_NS 1000000UL
+#define MVPP2_TXDONE_COAL_USEC 1000
#define MVPP2_RX_COAL_PKTS 32
#define MVPP2_RX_COAL_USEC 100
@@ -881,6 +889,9 @@ struct mvpp2_port {
struct mvpp2_queue_vector qvecs[MVPP2_MAX_QVECS];
unsigned int nqvecs;
+ bool has_tx_irqs;
+
+ u32 tx_time_coal;
};
/* The mvpp2_tx_desc and mvpp2_rx_desc structures describe the
@@ -1146,6 +1157,15 @@ struct mvpp2_bm_pool {
u32 port_map;
};
+/* Queue modes */
+#define MVPP2_QDIST_SINGLE_MODE 0
+#define MVPP2_QDIST_MULTI_MODE 1
+
+static int queue_mode = MVPP2_QDIST_SINGLE_MODE;
+
+module_param(queue_mode, int, 0444);
+MODULE_PARM_DESC(queue_mode, "Set queue_mode (single=0, multi=1)");
+
#define MVPP2_DRIVER_NAME "mvpp2"
#define MVPP2_DRIVER_VERSION "1.0"
@@ -4257,11 +4277,40 @@ static void mvpp2_interrupts_mask(void *arg)
static void mvpp2_interrupts_unmask(void *arg)
{
struct mvpp2_port *port = arg;
+ u32 val;
+
+ val = MVPP2_CAUSE_MISC_SUM_MASK |
+ MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK;
+ if (port->has_tx_irqs)
+ val |= MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK;
mvpp2_percpu_write(port->priv, smp_processor_id(),
- MVPP2_ISR_RX_TX_MASK_REG(port->id),
- (MVPP2_CAUSE_MISC_SUM_MASK |
- MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK));
+ MVPP2_ISR_RX_TX_MASK_REG(port->id), val);
+}
+
+static void
+mvpp2_shared_interrupt_mask_unmask(struct mvpp2_port *port, bool mask)
+{
+ u32 val;
+ int i;
+
+ if (port->priv->hw_version != MVPP22)
+ return;
+
+ if (mask)
+ val = 0;
+ else
+ val = MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK;
+
+ for (i = 0; i < port->nqvecs; i++) {
+ struct mvpp2_queue_vector *v = port->qvecs + i;
+
+ if (v->type != MVPP2_QUEUE_VECTOR_SHARED)
+ continue;
+
+ mvpp2_percpu_write(port->priv, v->sw_thread_id,
+ MVPP2_ISR_RX_TX_MASK_REG(port->id), val);
+ }
}
/* Port configuration routines */
@@ -5115,6 +5164,23 @@ static void mvpp2_rx_pkts_coal_set(struct mvpp2_port *port,
put_cpu();
}
+/* For some reason in the LSP this is done on each CPU. Why ? */
+static void mvpp2_tx_pkts_coal_set(struct mvpp2_port *port,
+ struct mvpp2_tx_queue *txq)
+{
+ int cpu = get_cpu();
+ u32 val;
+
+ if (txq->done_pkts_coal > MVPP2_TXQ_THRESH_MASK)
+ txq->done_pkts_coal = MVPP2_TXQ_THRESH_MASK;
+
+ val = (txq->done_pkts_coal << MVPP2_TXQ_THRESH_OFFSET);
+ mvpp2_percpu_write(port->priv, cpu, MVPP2_TXQ_NUM_REG, txq->id);
+ mvpp2_percpu_write(port->priv, cpu, MVPP2_TXQ_THRESH_REG, val);
+
+ put_cpu();
+}
+
static u32 mvpp2_usec_to_cycles(u32 usec, unsigned long clk_hz)
{
u64 tmp = (u64)clk_hz * usec;
@@ -5151,6 +5217,22 @@ static void mvpp2_rx_time_coal_set(struct mvpp2_port *port,
mvpp2_write(port->priv, MVPP2_ISR_RX_THRESHOLD_REG(rxq->id), val);
}
+static void mvpp2_tx_time_coal_set(struct mvpp2_port *port)
+{
+ unsigned long freq = port->priv->tclk;
+ u32 val = mvpp2_usec_to_cycles(port->tx_time_coal, freq);
+
+ if (val > MVPP2_MAX_ISR_TX_THRESHOLD) {
+ port->tx_time_coal =
+ mvpp2_cycles_to_usec(MVPP2_MAX_ISR_TX_THRESHOLD, freq);
+
+ /* re-evaluate to get actual register value */
+ val = mvpp2_usec_to_cycles(port->tx_time_coal, freq);
+ }
+
+ mvpp2_write(port->priv, MVPP2_ISR_TX_THRESHOLD_REG(port->id), val);
+}
+
/* Free Tx queue skbuffs */
static void mvpp2_txq_bufs_free(struct mvpp2_port *port,
struct mvpp2_tx_queue *txq,
@@ -5209,7 +5291,8 @@ static void mvpp2_txq_done(struct mvpp2_port *port, struct mvpp2_tx_queue *txq,
netif_tx_wake_queue(nq);
}
-static unsigned int mvpp2_tx_done(struct mvpp2_port *port, u32 cause)
+static unsigned int mvpp2_tx_done(struct mvpp2_port *port, u32 cause,
+ int cpu)
{
struct mvpp2_tx_queue *txq;
struct mvpp2_txq_pcpu *txq_pcpu;
@@ -5220,7 +5303,7 @@ static unsigned int mvpp2_tx_done(struct mvpp2_port *port, u32 cause)
if (!txq)
break;
- txq_pcpu = this_cpu_ptr(txq->pcpu);
+ txq_pcpu = per_cpu_ptr(txq->pcpu, cpu);
if (txq_pcpu->count) {
mvpp2_txq_done(port, txq, txq_pcpu);
@@ -5608,6 +5691,14 @@ static int mvpp2_setup_txqs(struct mvpp2_port *port)
goto err_cleanup;
}
+ if (port->has_tx_irqs) {
+ mvpp2_tx_time_coal_set(port);
+ for (queue = 0; queue < port->ntxqs; queue++) {
+ txq = port->txqs[queue];
+ mvpp2_tx_pkts_coal_set(port, txq);
+ }
+ }
+
on_each_cpu(mvpp2_txq_sent_counter_clear, port, 1);
return 0;
@@ -5768,7 +5859,7 @@ static void mvpp2_tx_proc_cb(unsigned long data)
/* Process all the Tx queues */
cause = (1 << port->ntxqs) - 1;
- tx_todo = mvpp2_tx_done(port, cause);
+ tx_todo = mvpp2_tx_done(port, cause, smp_processor_id());
/* Set the timer in case not all the packets were processed */
if (tx_todo)
@@ -6144,7 +6235,8 @@ static int mvpp2_tx(struct sk_buff *skb, struct net_device *dev)
mvpp2_txq_done(port, txq, txq_pcpu);
/* Set the timer in case not all frags were processed */
- if (txq_pcpu->count <= frags && txq_pcpu->count > 0) {
+ if (!port->has_tx_irqs && txq_pcpu->count <= frags &&
+ txq_pcpu->count > 0) {
struct mvpp2_port_pcpu *port_pcpu = this_cpu_ptr(port->pcpu);
mvpp2_timer_set(port_pcpu);
@@ -6165,7 +6257,7 @@ static inline void mvpp2_cause_error(struct net_device *dev, int cause)
static int mvpp2_poll(struct napi_struct *napi, int budget)
{
- u32 cause_rx_tx, cause_rx, cause_misc;
+ u32 cause_rx_tx, cause_rx, cause_tx, cause_misc;
int rx_done = 0;
struct mvpp2_port *port = netdev_priv(napi->dev);
struct mvpp2_queue_vector *qv;
@@ -6183,11 +6275,10 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
*
* Each CPU has its own Rx/Tx cause register
*/
- cause_rx_tx = mvpp2_percpu_read(port->priv, cpu,
+ cause_rx_tx = mvpp2_percpu_read(port->priv, qv->sw_thread_id,
MVPP2_ISR_RX_TX_CAUSE_REG(port->id));
- cause_rx_tx &= ~MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK;
- cause_misc = cause_rx_tx & MVPP2_CAUSE_MISC_SUM_MASK;
+ cause_misc = cause_rx_tx & MVPP2_CAUSE_MISC_SUM_MASK;
if (cause_misc) {
mvpp2_cause_error(port->dev, cause_misc);
@@ -6198,9 +6289,15 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
cause_rx_tx & ~MVPP2_CAUSE_MISC_SUM_MASK);
}
- cause_rx = cause_rx_tx & MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK;
+ cause_tx = cause_rx_tx & MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK;
+ if (cause_tx) {
+ cause_tx >>= MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_OFFSET;
+ mvpp2_tx_done(port, cause_tx, qv->sw_thread_id);
+ }
/* Process RX packets */
+ cause_rx = cause_rx_tx & MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK;
+ cause_rx <<= qv->first_rxq;
cause_rx |= qv->pending_cause_rx;
while (cause_rx && budget > 0) {
int count;
@@ -6388,6 +6485,10 @@ static int mvpp2_irqs_init(struct mvpp2_port *port)
err = request_irq(qv->irq, mvpp2_isr, 0, port->dev->name, qv);
if (err)
goto err;
+
+ if (qv->type == MVPP2_QUEUE_VECTOR_PRIVATE)
+ irq_set_affinity_hint(qv->irq,
+ cpumask_of(qv->sw_thread_id));
}
return 0;
@@ -6395,6 +6496,7 @@ static int mvpp2_irqs_init(struct mvpp2_port *port)
for (i = 0; i < port->nqvecs; i++) {
struct mvpp2_queue_vector *qv = port->qvecs + i;
+ irq_set_affinity_hint(qv->irq, NULL);
free_irq(qv->irq, qv);
}
@@ -6408,6 +6510,7 @@ static void mvpp2_irqs_deinit(struct mvpp2_port *port)
for (i = 0; i < port->nqvecs; i++) {
struct mvpp2_queue_vector *qv = port->qvecs + i;
+ irq_set_affinity_hint(qv->irq, NULL);
free_irq(qv->irq, qv);
}
}
@@ -6482,6 +6585,7 @@ static int mvpp2_open(struct net_device *dev)
/* Unmask interrupts on all CPUs */
on_each_cpu(mvpp2_interrupts_unmask, port, 1);
+ mvpp2_shared_interrupt_mask_unmask(port, false);
mvpp2_start_dev(port);
@@ -6509,14 +6613,17 @@ static int mvpp2_stop(struct net_device *dev)
/* Mask interrupts on all CPUs */
on_each_cpu(mvpp2_interrupts_mask, port, 1);
+ mvpp2_shared_interrupt_mask_unmask(port, true);
mvpp2_irqs_deinit(port);
- for_each_present_cpu(cpu) {
- port_pcpu = per_cpu_ptr(port->pcpu, cpu);
+ if (!port->has_tx_irqs) {
+ for_each_present_cpu(cpu) {
+ port_pcpu = per_cpu_ptr(port->pcpu, cpu);
- hrtimer_cancel(&port_pcpu->tx_done_timer);
- port_pcpu->timer_scheduled = false;
- tasklet_kill(&port_pcpu->tx_done_tasklet);
+ hrtimer_cancel(&port_pcpu->tx_done_timer);
+ port_pcpu->timer_scheduled = false;
+ tasklet_kill(&port_pcpu->tx_done_tasklet);
+ }
}
mvpp2_cleanup_rxqs(port);
mvpp2_cleanup_txqs(port);
@@ -6700,10 +6807,18 @@ static int mvpp2_ethtool_set_coalesce(struct net_device *dev,
mvpp2_rx_time_coal_set(port, rxq);
}
+ if (port->has_tx_irqs) {
+ port->tx_time_coal = c->tx_coalesce_usecs;
+ mvpp2_tx_time_coal_set(port);
+ }
+
for (queue = 0; queue < port->ntxqs; queue++) {
struct mvpp2_tx_queue *txq = port->txqs[queue];
txq->done_pkts_coal = c->tx_max_coalesced_frames;
+
+ if (port->has_tx_irqs)
+ mvpp2_tx_pkts_coal_set(port, txq);
}
return 0;
@@ -6828,8 +6943,11 @@ static const struct ethtool_ops mvpp2_eth_tool_ops = {
.set_link_ksettings = phy_ethtool_set_link_ksettings,
};
-static int mvpp2_queue_vectors_init(struct mvpp2_port *port,
- struct device_node *port_node)
+/* Used for PPv2.1, or PPv2.2 with the old Device Tree binding that
+ * had a single IRQ defined per-port.
+ */
+static int mvpp2_simple_queue_vectors_init(struct mvpp2_port *port,
+ struct device_node *port_node)
{
struct mvpp2_queue_vector *v = &port->qvecs[0];
@@ -6850,6 +6968,66 @@ static int mvpp2_queue_vectors_init(struct mvpp2_port *port,
return 0;
}
+static int mvpp2_multi_queue_vectors_init(struct mvpp2_port *port,
+ struct device_node *port_node)
+{
+ struct mvpp2_queue_vector *v;
+ int i, ret;
+
+ port->nqvecs = num_possible_cpus();
+ if (queue_mode == MVPP2_QDIST_SINGLE_MODE)
+ port->nqvecs += 1;
+
+ for (i = 0; i < port->nqvecs; i++) {
+ char irqname[16];
+
+ v = port->qvecs + i;
+
+ v->port = port;
+ v->type = MVPP2_QUEUE_VECTOR_PRIVATE;
+ v->sw_thread_id = i;
+ v->sw_thread_mask = BIT(i);
+
+ snprintf(irqname, sizeof(irqname), "tx-cpu%d", i);
+
+ if (queue_mode == MVPP2_QDIST_MULTI_MODE) {
+ v->first_rxq = i * MVPP2_DEFAULT_RXQ;
+ v->nrxqs = MVPP2_DEFAULT_RXQ;
+ } else if (queue_mode == MVPP2_QDIST_SINGLE_MODE &&
+ i == (port->nqvecs - 1)) {
+ v->first_rxq = 0;
+ v->nrxqs = port->nrxqs;
+ v->type = MVPP2_QUEUE_VECTOR_SHARED;
+ strncpy(irqname, "rx-shared", sizeof(irqname));
+ }
+
+ v->irq = of_irq_get_byname(port_node, irqname);
+ if (v->irq <= 0) {
+ ret = -EINVAL;
+ goto err;
+ }
+
+ netif_napi_add(port->dev, &v->napi, mvpp2_poll,
+ NAPI_POLL_WEIGHT);
+ }
+
+ return 0;
+
+err:
+ for (i = 0; i < port->nqvecs; i++)
+ irq_dispose_mapping(port->qvecs[i].irq);
+ return ret;
+}
+
+static int mvpp2_queue_vectors_init(struct mvpp2_port *port,
+ struct device_node *port_node)
+{
+ if (port->has_tx_irqs)
+ return mvpp2_multi_queue_vectors_init(port, port_node);
+ else
+ return mvpp2_simple_queue_vectors_init(port, port_node);
+}
+
static void mvpp2_queue_vectors_deinit(struct mvpp2_port *port)
{
int i;
@@ -6909,6 +7087,8 @@ static int mvpp2_port_init(struct mvpp2_port *port)
mvpp2_egress_disable(port);
mvpp2_port_disable(port);
+ port->tx_time_coal = MVPP2_TXDONE_COAL_USEC;
+
port->txqs = devm_kcalloc(dev, port->ntxqs, sizeof(*port->txqs),
GFP_KERNEL);
if (!port->txqs)
@@ -7008,6 +7188,30 @@ static int mvpp2_port_init(struct mvpp2_port *port)
return err;
}
+/* Checks if the port DT description has the TX interrupts
+ * described. On PPv2.1, there are no such interrupts. On PPv2.2,
+ * there are available, but we need to keep support for old DTs.
+ */
+static bool mvpp2_port_has_tx_irqs(struct mvpp2 *priv,
+ struct device_node *port_node)
+{
+ char *irqs[5] = { "rx-shared", "tx-cpu0", "tx-cpu1",
+ "tx-cpu2", "tx-cpu3" };
+ int ret, i;
+
+ if (priv->hw_version == MVPP21)
+ return false;
+
+ for (i = 0; i < 5; i++) {
+ ret = of_property_match_string(port_node, "interrupt-names",
+ irqs[i]);
+ if (ret < 0)
+ return false;
+ }
+
+ return true;
+}
+
/* Ports initialization */
static int mvpp2_port_probe(struct platform_device *pdev,
struct device_node *port_node,
@@ -7022,13 +7226,22 @@ static int mvpp2_port_probe(struct platform_device *pdev,
const char *mac_from;
char hw_mac_addr[ETH_ALEN];
unsigned int ntxqs, nrxqs;
+ bool has_tx_irqs;
u32 id;
int features;
int phy_mode;
int err, i, cpu;
+ has_tx_irqs = mvpp2_port_has_tx_irqs(priv, port_node);
+
+ if (!has_tx_irqs)
+ queue_mode = MVPP2_QDIST_SINGLE_MODE;
+
ntxqs = MVPP2_MAX_TXQ;
- nrxqs = MVPP2_DEFAULT_RXQ;
+ if (priv->hw_version == MVPP22 && queue_mode == MVPP2_QDIST_MULTI_MODE)
+ nrxqs = MVPP2_DEFAULT_RXQ * num_possible_cpus();
+ else
+ nrxqs = MVPP2_DEFAULT_RXQ;
dev = alloc_etherdev_mqs(sizeof(*port), ntxqs, nrxqs);
if (!dev)
@@ -7057,6 +7270,8 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->dev = dev;
port->ntxqs = ntxqs;
port->nrxqs = nrxqs;
+ port->priv = priv;
+ port->has_tx_irqs = has_tx_irqs;
port->link_irq = of_irq_get_byname(port_node, "link");
if (port->link_irq == -EPROBE_DEFER) {
@@ -7074,7 +7289,6 @@ static int mvpp2_port_probe(struct platform_device *pdev,
if (of_property_read_bool(port_node, "marvell,loopback"))
port->flags |= MVPP2_F_LOOPBACK;
- port->priv = priv;
port->id = id;
if (priv->hw_version == MVPP21)
port->first_rxq = port->id * port->nrxqs;
@@ -7148,16 +7362,19 @@ static int mvpp2_port_probe(struct platform_device *pdev,
goto err_free_txq_pcpu;
}
- for_each_present_cpu(cpu) {
- port_pcpu = per_cpu_ptr(port->pcpu, cpu);
+ if (!port->has_tx_irqs) {
+ for_each_present_cpu(cpu) {
+ port_pcpu = per_cpu_ptr(port->pcpu, cpu);
- hrtimer_init(&port_pcpu->tx_done_timer, CLOCK_MONOTONIC,
- HRTIMER_MODE_REL_PINNED);
- port_pcpu->tx_done_timer.function = mvpp2_hr_timer_cb;
- port_pcpu->timer_scheduled = false;
+ hrtimer_init(&port_pcpu->tx_done_timer, CLOCK_MONOTONIC,
+ HRTIMER_MODE_REL_PINNED);
+ port_pcpu->tx_done_timer.function = mvpp2_hr_timer_cb;
+ port_pcpu->timer_scheduled = false;
- tasklet_init(&port_pcpu->tx_done_tasklet, mvpp2_tx_proc_cb,
- (unsigned long)dev);
+ tasklet_init(&port_pcpu->tx_done_tasklet,
+ mvpp2_tx_proc_cb,
+ (unsigned long)dev);
+ }
}
features = NETIF_F_SG | NETIF_F_IP_CSUM;
--
2.9.4
^ permalink raw reply related
* [PATCH 8/8] arm64: dts: marvell: add TX interrupts for PPv2.2
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
This commit updates the Marvell Armada 7K/8K Device Tree to describe
the TX interrupts of the Ethernet controllers, in both the master and
slave CP110s.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
Dave: please do *not* apply this patch. It will go through the ARM
mvebu maintainers. Thanks!
.../arm64/boot/dts/marvell/armada-cp110-master.dtsi | 21 ++++++++++++++++++---
arch/arm64/boot/dts/marvell/armada-cp110-slave.dtsi | 21 ++++++++++++++++++---
2 files changed, 36 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/boot/dts/marvell/armada-cp110-master.dtsi b/arch/arm64/boot/dts/marvell/armada-cp110-master.dtsi
index 9278ba6..d192cea 100644
--- a/arch/arm64/boot/dts/marvell/armada-cp110-master.dtsi
+++ b/arch/arm64/boot/dts/marvell/armada-cp110-master.dtsi
@@ -71,8 +71,13 @@
cpm_eth0: eth0 {
interrupts = <ICU_GRP_NSR 39 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 43 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 47 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 51 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 55 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 129 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <0>;
gop-port-id = <0>;
status = "disabled";
@@ -80,8 +85,13 @@
cpm_eth1: eth1 {
interrupts = <ICU_GRP_NSR 40 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 44 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 48 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 52 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 56 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 128 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <1>;
gop-port-id = <2>;
status = "disabled";
@@ -89,8 +99,13 @@
cpm_eth2: eth2 {
interrupts = <ICU_GRP_NSR 41 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 45 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 49 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 53 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 57 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 127 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <2>;
gop-port-id = <3>;
status = "disabled";
diff --git a/arch/arm64/boot/dts/marvell/armada-cp110-slave.dtsi b/arch/arm64/boot/dts/marvell/armada-cp110-slave.dtsi
index 3515817..8bde91b 100644
--- a/arch/arm64/boot/dts/marvell/armada-cp110-slave.dtsi
+++ b/arch/arm64/boot/dts/marvell/armada-cp110-slave.dtsi
@@ -78,8 +78,13 @@
cps_eth0: eth0 {
interrupts = <ICU_GRP_NSR 39 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 43 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 47 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 51 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 55 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 129 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <0>;
gop-port-id = <0>;
status = "disabled";
@@ -87,8 +92,13 @@
cps_eth1: eth1 {
interrupts = <ICU_GRP_NSR 40 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 44 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 48 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 52 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 56 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 128 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <1>;
gop-port-id = <2>;
status = "disabled";
@@ -96,8 +106,13 @@
cps_eth2: eth2 {
interrupts = <ICU_GRP_NSR 41 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 45 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 49 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 53 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 57 IRQ_TYPE_LEVEL_HIGH>,
<ICU_GRP_NSR 127 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared", "link";
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <2>;
gop-port-id = <3>;
status = "disabled";
--
2.9.4
^ permalink raw reply related
* [PATCH 4/8] net: mvpp2: move from cpu-centric naming to "software thread" naming
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
The PPv2.2 IP has a concept of "software thread", with all registers
of the PPv2.2 mapped 8 times, for concurrent accesses by 8 "software
threads". In addition, interrupts on RX queues are associated to such
"software thread".
For most cases, we map a "software thread" to the more conventional
concept of CPU, but we will soon have one exception: we will have a
model where we have one TX interrupt per CPU (each using one software
thread), and all RX events mapped to another software thread
(associated to another interrupt).
In preparation for this change, it makes sense to change the naming
from MVPP2_MAX_CPUS to MVPP2_MAX_THREADS, and plan for 8 software
threads instead of 4 currently.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 25 +++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index 8479269..7d37869 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -748,7 +748,7 @@ enum mvpp2_prs_l3_cast {
#define MVPP21_ADDR_SPACE_SZ 0
#define MVPP22_ADDR_SPACE_SZ SZ_64K
-#define MVPP2_MAX_CPUS 4
+#define MVPP2_MAX_THREADS 8
enum mvpp2_bm_type {
MVPP2_BM_FREE,
@@ -764,11 +764,12 @@ struct mvpp2 {
void __iomem *lms_base;
void __iomem *iface_base;
- /* On PPv2.2, each CPU can access the base register through a
- * separate address space, each 64 KB apart from each
- * other.
+ /* On PPv2.2, each "software thread" can access the base
+ * register through a separate address space, each 64 KB apart
+ * from each other. Typically, such address spaces will be
+ * used per CPU.
*/
- void __iomem *cpu_base[MVPP2_MAX_CPUS];
+ void __iomem *swth_base[MVPP2_MAX_THREADS];
/* On PPv2.2, some port control registers are located into the system
* controller space. These registers are accessible through a regmap.
@@ -1140,12 +1141,12 @@ struct mvpp2_bm_pool {
static void mvpp2_write(struct mvpp2 *priv, u32 offset, u32 data)
{
- writel(data, priv->cpu_base[0] + offset);
+ writel(data, priv->swth_base[0] + offset);
}
static u32 mvpp2_read(struct mvpp2 *priv, u32 offset)
{
- return readl(priv->cpu_base[0] + offset);
+ return readl(priv->swth_base[0] + offset);
}
/* These accessors should be used to access:
@@ -1187,13 +1188,13 @@ static u32 mvpp2_read(struct mvpp2 *priv, u32 offset)
static void mvpp2_percpu_write(struct mvpp2 *priv, int cpu,
u32 offset, u32 data)
{
- writel(data, priv->cpu_base[cpu] + offset);
+ writel(data, priv->swth_base[cpu] + offset);
}
static u32 mvpp2_percpu_read(struct mvpp2 *priv, int cpu,
u32 offset)
{
- return readl(priv->cpu_base[cpu] + offset);
+ return readl(priv->swth_base[cpu] + offset);
}
static dma_addr_t mvpp2_txdesc_dma_addr_get(struct mvpp2_port *port,
@@ -7282,7 +7283,7 @@ static int mvpp2_probe(struct platform_device *pdev)
struct mvpp2 *priv;
struct resource *res;
void __iomem *base;
- int port_count, cpu;
+ int port_count, i;
int err;
priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
@@ -7320,12 +7321,12 @@ static int mvpp2_probe(struct platform_device *pdev)
priv->sysctrl_base = NULL;
}
- for_each_present_cpu(cpu) {
+ for (i = 0; i < MVPP2_MAX_THREADS; i++) {
u32 addr_space_sz;
addr_space_sz = (priv->hw_version == MVPP21 ?
MVPP21_ADDR_SPACE_SZ : MVPP22_ADDR_SPACE_SZ);
- priv->cpu_base[cpu] = base + cpu * addr_space_sz;
+ priv->swth_base[i] = base + i * addr_space_sz;
}
if (priv->hw_version == MVPP21)
--
2.9.4
^ permalink raw reply related
* [PATCH 7/8] dt-bindings: net: marvell-pp2: update interrupt-names with TX interrupts
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
The PPv2.2 unit has several interrupts used for TX completion
notification. This commit updates the Device Tree binding describing
this HW block to mention such interrupts.
While at it, we update the example to use a recent Device Tree
example, that uses interrupts going through the ICU, and not to the
GIC directly.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
.../devicetree/bindings/net/marvell-pp2.txt | 33 +++++++++++++++++-----
1 file changed, 26 insertions(+), 7 deletions(-)
diff --git a/Documentation/devicetree/bindings/net/marvell-pp2.txt b/Documentation/devicetree/bindings/net/marvell-pp2.txt
index 553aadc..80c4d14 100644
--- a/Documentation/devicetree/bindings/net/marvell-pp2.txt
+++ b/Documentation/devicetree/bindings/net/marvell-pp2.txt
@@ -44,7 +44,8 @@ Optional properties (port):
- marvell,system-controller: a phandle to the system controller.
- interrupt-names: if more than a single interrupt for rx is given, must
be the name associated to the interrupts listed. Valid
- names are: "rx-shared", "link".
+ names are: "tx-cpu0", "tx-cpu1", "tx-cpu2", "tx-cpu3",
+ "rx-shared", "link".
Example for marvell,armada-375-pp2:
@@ -84,22 +85,40 @@ cpm_ethernet: ethernet@0 {
clock-names = "pp_clk", "gop_clk", "gp_clk";
eth0: eth0 {
- interrupts = <GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared";
+ interrupts = <ICU_GRP_NSR 39 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 43 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 47 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 51 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 55 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 129 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <0>;
gop-port-id = <0>;
};
eth1: eth1 {
- interrupts = <GIC_SPI 38 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared";
+ interrupts = <ICU_GRP_NSR 40 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 44 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 48 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 52 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 56 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 128 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <1>;
gop-port-id = <2>;
};
eth2: eth2 {
- interrupts = <GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>;
- interrupt-names = "rx-shared";
+ interrupts = <ICU_GRP_NSR 41 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 45 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 49 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 53 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 57 IRQ_TYPE_LEVEL_HIGH>,
+ <ICU_GRP_NSR 127 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-names = "tx-cpu0", "tx-cpu1", "tx-cpu2",
+ "tx-cpu3", "rx-shared", "link";
port-id = <2>;
gop-port-id = <3>;
};
--
2.9.4
^ permalink raw reply related
* [PATCH 5/8] net: mvpp2: introduce queue_vector concept
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
In preparation to the introduction of TX interrupts and improved RX
queue distribution, this commit introduces the concept of "queue
vector". A queue vector represents a number of RX and/or TX queues,
and an associated NAPI instance and interrupt.
This commit currently only creates a single queue_vector, so there are
no changes in behavior, but it paves the way for additional
queue_vector in the next commits.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 250 +++++++++++++++++++++++++----------
1 file changed, 178 insertions(+), 72 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index 7d37869..a8d448b 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -749,6 +749,7 @@ enum mvpp2_prs_l3_cast {
#define MVPP22_ADDR_SPACE_SZ SZ_64K
#define MVPP2_MAX_THREADS 8
+#define MVPP2_MAX_QVECS MVPP2_MAX_THREADS
enum mvpp2_bm_type {
MVPP2_BM_FREE,
@@ -821,6 +822,18 @@ struct mvpp2_port_pcpu {
struct tasklet_struct tx_done_tasklet;
};
+struct mvpp2_queue_vector {
+ int irq;
+ struct napi_struct napi;
+ enum { MVPP2_QUEUE_VECTOR_SHARED, MVPP2_QUEUE_VECTOR_PRIVATE } type;
+ int sw_thread_id;
+ u16 sw_thread_mask;
+ int first_rxq;
+ int nrxqs;
+ u32 pending_cause_rx;
+ struct mvpp2_port *port;
+};
+
struct mvpp2_port {
u8 id;
@@ -829,7 +842,6 @@ struct mvpp2_port {
*/
int gop_id;
- int irq;
int link_irq;
struct mvpp2 *priv;
@@ -845,9 +857,6 @@ struct mvpp2_port {
int pkt_size;
- u32 pending_cause_rx;
- struct napi_struct napi;
-
/* Per-CPU port control */
struct mvpp2_port_pcpu __percpu *pcpu;
@@ -869,6 +878,9 @@ struct mvpp2_port {
/* Index of first port's physical RXQ */
u8 first_rxq;
+
+ struct mvpp2_queue_vector qvecs[MVPP2_MAX_QVECS];
+ unsigned int nqvecs;
};
/* The mvpp2_tx_desc and mvpp2_rx_desc structures describe the
@@ -4190,22 +4202,40 @@ static int mvpp2_bm_update_mtu(struct net_device *dev, int mtu)
static inline void mvpp2_interrupts_enable(struct mvpp2_port *port)
{
- int cpu, cpu_mask = 0;
+ int i, sw_thread_mask = 0;
+
+ for (i = 0; i < port->nqvecs; i++)
+ sw_thread_mask |= port->qvecs[i].sw_thread_mask;
- for_each_present_cpu(cpu)
- cpu_mask |= 1 << cpu;
mvpp2_write(port->priv, MVPP2_ISR_ENABLE_REG(port->id),
- MVPP2_ISR_ENABLE_INTERRUPT(cpu_mask));
+ MVPP2_ISR_ENABLE_INTERRUPT(sw_thread_mask));
}
static inline void mvpp2_interrupts_disable(struct mvpp2_port *port)
{
- int cpu, cpu_mask = 0;
+ int i, sw_thread_mask = 0;
+
+ for (i = 0; i < port->nqvecs; i++)
+ sw_thread_mask |= port->qvecs[i].sw_thread_mask;
+
+ mvpp2_write(port->priv, MVPP2_ISR_ENABLE_REG(port->id),
+ MVPP2_ISR_DISABLE_INTERRUPT(sw_thread_mask));
+}
+
+static inline void mvpp2_qvec_interrupt_enable(struct mvpp2_queue_vector *qvec)
+{
+ struct mvpp2_port *port = qvec->port;
+
+ mvpp2_write(port->priv, MVPP2_ISR_ENABLE_REG(port->id),
+ MVPP2_ISR_ENABLE_INTERRUPT(qvec->sw_thread_mask));
+}
+
+static inline void mvpp2_qvec_interrupt_disable(struct mvpp2_queue_vector *qvec)
+{
+ struct mvpp2_port *port = qvec->port;
- for_each_present_cpu(cpu)
- cpu_mask |= 1 << cpu;
mvpp2_write(port->priv, MVPP2_ISR_ENABLE_REG(port->id),
- MVPP2_ISR_DISABLE_INTERRUPT(cpu_mask));
+ MVPP2_ISR_DISABLE_INTERRUPT(qvec->sw_thread_mask));
}
/* Mask the current CPU's Rx/Tx interrupts
@@ -5589,11 +5619,11 @@ static int mvpp2_setup_txqs(struct mvpp2_port *port)
/* The callback for per-port interrupt */
static irqreturn_t mvpp2_isr(int irq, void *dev_id)
{
- struct mvpp2_port *port = (struct mvpp2_port *)dev_id;
+ struct mvpp2_queue_vector *qv = dev_id;
- mvpp2_interrupts_disable(port);
+ mvpp2_qvec_interrupt_disable(qv);
- napi_schedule(&port->napi);
+ napi_schedule(&qv->napi);
return IRQ_HANDLED;
}
@@ -5850,8 +5880,8 @@ static u32 mvpp2_skb_tx_csum(struct mvpp2_port *port, struct sk_buff *skb)
}
/* Main rx processing */
-static int mvpp2_rx(struct mvpp2_port *port, int rx_todo,
- struct mvpp2_rx_queue *rxq)
+static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
+ int rx_todo, struct mvpp2_rx_queue *rxq)
{
struct net_device *dev = port->dev;
int rx_received;
@@ -5929,7 +5959,7 @@ static int mvpp2_rx(struct mvpp2_port *port, int rx_todo,
skb->protocol = eth_type_trans(skb, dev);
mvpp2_rx_csum(port, rx_status, skb);
- napi_gro_receive(&port->napi, skb);
+ napi_gro_receive(napi, skb);
}
if (rcvd_pkts) {
@@ -6138,8 +6168,11 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
u32 cause_rx_tx, cause_rx, cause_misc;
int rx_done = 0;
struct mvpp2_port *port = netdev_priv(napi->dev);
+ struct mvpp2_queue_vector *qv;
int cpu = smp_processor_id();
+ qv = container_of(napi, struct mvpp2_queue_vector, napi);
+
/* Rx/Tx cause register
*
* Bits 0-15: each bit indicates received packets on the Rx queue
@@ -6168,7 +6201,7 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
cause_rx = cause_rx_tx & MVPP2_CAUSE_RXQ_OCCUP_DESC_ALL_MASK;
/* Process RX packets */
- cause_rx |= port->pending_cause_rx;
+ cause_rx |= qv->pending_cause_rx;
while (cause_rx && budget > 0) {
int count;
struct mvpp2_rx_queue *rxq;
@@ -6177,7 +6210,7 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
if (!rxq)
break;
- count = mvpp2_rx(port, budget, rxq);
+ count = mvpp2_rx(port, napi, budget, rxq);
rx_done += count;
budget -= count;
if (budget > 0) {
@@ -6193,9 +6226,9 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
cause_rx = 0;
napi_complete_done(napi, rx_done);
- mvpp2_interrupts_enable(port);
+ mvpp2_qvec_interrupt_enable(qv);
}
- port->pending_cause_rx = cause_rx;
+ qv->pending_cause_rx = cause_rx;
return rx_done;
}
@@ -6203,11 +6236,13 @@ static int mvpp2_poll(struct napi_struct *napi, int budget)
static void mvpp2_start_dev(struct mvpp2_port *port)
{
struct net_device *ndev = port->dev;
+ int i;
mvpp2_gmac_max_rx_size_set(port);
mvpp2_txp_max_tx_size_set(port);
- napi_enable(&port->napi);
+ for (i = 0; i < port->nqvecs; i++)
+ napi_enable(&port->qvecs[i].napi);
/* Enable interrupts on all CPUs */
mvpp2_interrupts_enable(port);
@@ -6226,6 +6261,7 @@ static void mvpp2_start_dev(struct mvpp2_port *port)
static void mvpp2_stop_dev(struct mvpp2_port *port)
{
struct net_device *ndev = port->dev;
+ int i;
/* Stop new packets from arriving to RXQs */
mvpp2_ingress_disable(port);
@@ -6235,7 +6271,8 @@ static void mvpp2_stop_dev(struct mvpp2_port *port)
/* Disable interrupts on all CPUs */
mvpp2_interrupts_disable(port);
- napi_disable(&port->napi);
+ for (i = 0; i < port->nqvecs; i++)
+ napi_disable(&port->qvecs[i].napi);
netif_carrier_off(port->dev);
netif_tx_stop_all_queues(port->dev);
@@ -6341,6 +6378,40 @@ static void mvpp2_phy_disconnect(struct mvpp2_port *port)
phy_disconnect(ndev->phydev);
}
+static int mvpp2_irqs_init(struct mvpp2_port *port)
+{
+ int err, i;
+
+ for (i = 0; i < port->nqvecs; i++) {
+ struct mvpp2_queue_vector *qv = port->qvecs + i;
+
+ err = request_irq(qv->irq, mvpp2_isr, 0, port->dev->name, qv);
+ if (err)
+ goto err;
+ }
+
+ return 0;
+err:
+ for (i = 0; i < port->nqvecs; i++) {
+ struct mvpp2_queue_vector *qv = port->qvecs + i;
+
+ free_irq(qv->irq, qv);
+ }
+
+ return err;
+}
+
+static void mvpp2_irqs_deinit(struct mvpp2_port *port)
+{
+ int i;
+
+ for (i = 0; i < port->nqvecs; i++) {
+ struct mvpp2_queue_vector *qv = port->qvecs + i;
+
+ free_irq(qv->irq, qv);
+ }
+}
+
static int mvpp2_open(struct net_device *dev)
{
struct mvpp2_port *port = netdev_priv(dev);
@@ -6384,9 +6455,9 @@ static int mvpp2_open(struct net_device *dev)
goto err_cleanup_rxqs;
}
- err = request_irq(port->irq, mvpp2_isr, 0, dev->name, port);
+ err = mvpp2_irqs_init(port);
if (err) {
- netdev_err(port->dev, "cannot request IRQ %d\n", port->irq);
+ netdev_err(port->dev, "cannot init IRQs\n");
goto err_cleanup_txqs;
}
@@ -6419,7 +6490,7 @@ static int mvpp2_open(struct net_device *dev)
err_free_link_irq:
free_irq(port->link_irq, port);
err_free_irq:
- free_irq(port->irq, port);
+ mvpp2_irqs_deinit(port);
err_cleanup_txqs:
mvpp2_cleanup_txqs(port);
err_cleanup_rxqs:
@@ -6439,7 +6510,7 @@ static int mvpp2_stop(struct net_device *dev)
/* Mask interrupts on all CPUs */
on_each_cpu(mvpp2_interrupts_mask, port, 1);
- free_irq(port->irq, port);
+ mvpp2_irqs_deinit(port);
for_each_present_cpu(cpu) {
port_pcpu = per_cpu_ptr(port->pcpu, cpu);
@@ -6757,6 +6828,66 @@ static const struct ethtool_ops mvpp2_eth_tool_ops = {
.set_link_ksettings = phy_ethtool_set_link_ksettings,
};
+static int mvpp2_queue_vectors_init(struct mvpp2_port *port,
+ struct device_node *port_node)
+{
+ struct mvpp2_queue_vector *v = &port->qvecs[0];
+
+ v->first_rxq = 0;
+ v->nrxqs = port->nrxqs;
+ v->type = MVPP2_QUEUE_VECTOR_SHARED;
+ v->sw_thread_id = 0;
+ v->sw_thread_mask = *cpumask_bits(cpu_online_mask);
+ v->port = port;
+ v->irq = irq_of_parse_and_map(port_node, 0);
+ if (v->irq <= 0)
+ return -EINVAL;
+ netif_napi_add(port->dev, &v->napi, mvpp2_poll,
+ NAPI_POLL_WEIGHT);
+
+ port->nqvecs = 1;
+
+ return 0;
+}
+
+static void mvpp2_queue_vectors_deinit(struct mvpp2_port *port)
+{
+ int i;
+
+ for (i = 0; i < port->nqvecs; i++)
+ irq_dispose_mapping(port->qvecs[i].irq);
+}
+
+/* Configure Rx queue group interrupt for this port */
+static void mvpp2_rx_irqs_setup(struct mvpp2_port *port)
+{
+ struct mvpp2 *priv = port->priv;
+ u32 val;
+ int i;
+
+ if (priv->hw_version == MVPP21) {
+ mvpp2_write(priv, MVPP21_ISR_RXQ_GROUP_REG(port->id),
+ port->nrxqs);
+ return;
+ }
+
+ /* Handle the more complicated PPv2.2 case */
+ for (i = 0; i < port->nqvecs; i++) {
+ struct mvpp2_queue_vector *qv = port->qvecs + i;
+
+ if (!qv->nrxqs)
+ continue;
+
+ val = qv->sw_thread_id;
+ val |= port->id << MVPP22_ISR_RXQ_GROUP_INDEX_GROUP_OFFSET;
+ mvpp2_write(priv, MVPP22_ISR_RXQ_GROUP_INDEX_REG, val);
+
+ val = qv->first_rxq;
+ val |= qv->nrxqs << MVPP22_ISR_RXQ_SUB_GROUP_SIZE_OFFSET;
+ mvpp2_write(priv, MVPP22_ISR_RXQ_SUB_GROUP_CONFIG_REG, val);
+ }
+}
+
/* Initialize port HW */
static int mvpp2_port_init(struct mvpp2_port *port)
{
@@ -6838,19 +6969,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
port->rxqs[queue] = rxq;
}
- /* Configure Rx queue group interrupt for this port */
- if (priv->hw_version == MVPP21) {
- mvpp2_write(priv, MVPP21_ISR_RXQ_GROUP_REG(port->id),
- port->nrxqs);
- } else {
- u32 val;
-
- val = (port->id << MVPP22_ISR_RXQ_GROUP_INDEX_GROUP_OFFSET);
- mvpp2_write(priv, MVPP22_ISR_RXQ_GROUP_INDEX_REG, val);
-
- val = (port->nrxqs << MVPP22_ISR_RXQ_SUB_GROUP_SIZE_OFFSET);
- mvpp2_write(priv, MVPP22_ISR_RXQ_SUB_GROUP_CONFIG_REG, val);
- }
+ mvpp2_rx_irqs_setup(port);
/* Create Rx descriptor rings */
for (queue = 0; queue < port->nrxqs; queue++) {
@@ -6935,33 +7054,22 @@ static int mvpp2_port_probe(struct platform_device *pdev,
dev->ethtool_ops = &mvpp2_eth_tool_ops;
port = netdev_priv(dev);
+ port->dev = dev;
port->ntxqs = ntxqs;
port->nrxqs = nrxqs;
- if (of_get_property(port_node, "interrupt-names", NULL)) {
- port->irq = of_irq_get_byname(port_node, "rx-shared");
- if (port->irq <= 0) {
- err = (port->irq == -EPROBE_DEFER) ?
- -EPROBE_DEFER : -EINVAL;
- goto err_free_netdev;
- }
-
- port->link_irq = of_irq_get_byname(port_node, "link");
- if (port->link_irq == -EPROBE_DEFER) {
- err = -EPROBE_DEFER;
- goto err_free_irq;
- }
- if (port->link_irq <= 0)
- /* the link irq is optional */
- port->link_irq = 0;
- } else {
- /* kept for dt compatibility */
- port->irq = irq_of_parse_and_map(port_node, 0);
- if (port->irq <= 0) {
- err = -EINVAL;
- goto err_free_netdev;
- }
+ port->link_irq = of_irq_get_byname(port_node, "link");
+ if (port->link_irq == -EPROBE_DEFER) {
+ err = -EPROBE_DEFER;
+ goto err_free_netdev;
}
+ if (port->link_irq <= 0)
+ /* the link irq is optional */
+ port->link_irq = 0;
+
+ err = mvpp2_queue_vectors_init(port, port_node);
+ if (err)
+ goto err_free_netdev;
if (of_property_read_bool(port_node, "marvell,loopback"))
port->flags |= MVPP2_F_LOOPBACK;
@@ -6981,14 +7089,14 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(port->base)) {
err = PTR_ERR(port->base);
- goto err_free_irq;
+ goto err_deinit_qvecs;
}
} else {
if (of_property_read_u32(port_node, "gop-port-id",
&port->gop_id)) {
err = -EINVAL;
dev_err(&pdev->dev, "missing gop-port-id value\n");
- goto err_free_irq;
+ goto err_deinit_qvecs;
}
port->base = priv->iface_base + MVPP22_GMAC_BASE(port->gop_id);
@@ -6998,7 +7106,7 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->stats = netdev_alloc_pcpu_stats(struct mvpp2_pcpu_stats);
if (!port->stats) {
err = -ENOMEM;
- goto err_free_irq;
+ goto err_deinit_qvecs;
}
dt_mac_addr = of_get_mac_address(port_node);
@@ -7019,7 +7127,6 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->tx_ring_size = MVPP2_MAX_TXD;
port->rx_ring_size = MVPP2_MAX_RXD;
- port->dev = dev;
SET_NETDEV_DEV(dev, &pdev->dev);
err = mvpp2_port_init(port);
@@ -7053,7 +7160,6 @@ static int mvpp2_port_probe(struct platform_device *pdev,
(unsigned long)dev);
}
- netif_napi_add(dev, &port->napi, mvpp2_poll, NAPI_POLL_WEIGHT);
features = NETIF_F_SG | NETIF_F_IP_CSUM;
dev->features = features | NETIF_F_RXCSUM;
dev->hw_features |= features | NETIF_F_RXCSUM | NETIF_F_GRO;
@@ -7081,8 +7187,8 @@ static int mvpp2_port_probe(struct platform_device *pdev,
free_percpu(port->txqs[i]->pcpu);
err_free_stats:
free_percpu(port->stats);
-err_free_irq:
- irq_dispose_mapping(port->irq);
+err_deinit_qvecs:
+ mvpp2_queue_vectors_deinit(port);
err_free_netdev:
of_node_put(phy_node);
free_netdev(dev);
@@ -7100,7 +7206,7 @@ static void mvpp2_port_remove(struct mvpp2_port *port)
free_percpu(port->stats);
for (i = 0; i < port->ntxqs; i++)
free_percpu(port->txqs[i]->pcpu);
- irq_dispose_mapping(port->irq);
+ mvpp2_queue_vectors_deinit(port);
free_netdev(port->dev);
}
--
2.9.4
^ permalink raw reply related
* [PATCH 3/8] net: mvpp2: introduce per-port nrxqs/ntxqs variables
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
Currently, the global variables rxq_number and txq_number hold the
number of per-port TXQs and RXQs. Until now, such numbers were
constant regardless of the driver configuration. As we are going to
introduce different modes for TX and RX queues, these numbers will
depend on the configuration (PPv2.1 vs. PPv2.2, exact queue
distribution logic).
Therefore, as a preparation, we move the number of RXQs and TXQs in
the 'struct mvpp2_port' structure, next to the RXQs and TXQs
descriptor arrays.
For now, they remain initialized to the same default values as
rxq_number/txq_number used to be initialized, but this will change in
future commits.
The only non-mechanical change in this patch is that the check to
verify hardware constraints on the number of RXQs and TXQs is moved
from mvpp2_probe() to mvpp2_port_probe(), since it's now in
mvpp2_port_probe() that we initialize the per-port count of RXQ and
TXQ.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 83 ++++++++++++++++++------------------
1 file changed, 41 insertions(+), 42 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index eb8853f..8479269 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -837,7 +837,9 @@ struct mvpp2_port {
void __iomem *base;
struct mvpp2_rx_queue **rxqs;
+ unsigned int nrxqs;
struct mvpp2_tx_queue **txqs;
+ unsigned int ntxqs;
struct net_device *dev;
int pkt_size;
@@ -1131,13 +1133,6 @@ struct mvpp2_bm_pool {
u32 port_map;
};
-/* Static declaractions */
-
-/* Number of RXQs used by single port */
-static int rxq_number = MVPP2_DEFAULT_RXQ;
-/* Number of TXQs used by single port */
-static int txq_number = MVPP2_MAX_TXQ;
-
#define MVPP2_DRIVER_NAME "mvpp2"
#define MVPP2_DRIVER_VERSION "1.0"
@@ -4139,7 +4134,7 @@ static int mvpp2_swf_bm_pool_init(struct mvpp2_port *port)
port->pool_long->port_map |= (1 << port->id);
- for (rxq = 0; rxq < rxq_number; rxq++)
+ for (rxq = 0; rxq < port->nrxqs; rxq++)
mvpp2_rxq_long_pool_set(port, rxq, port->pool_long->id);
}
@@ -4153,7 +4148,7 @@ static int mvpp2_swf_bm_pool_init(struct mvpp2_port *port)
port->pool_short->port_map |= (1 << port->id);
- for (rxq = 0; rxq < rxq_number; rxq++)
+ for (rxq = 0; rxq < port->nrxqs; rxq++)
mvpp2_rxq_short_pool_set(port, rxq,
port->pool_short->id);
}
@@ -4678,7 +4673,7 @@ static void mvpp2_defaults_set(struct mvpp2_port *port)
MVPP2_RX_LOW_LATENCY_PKT_SIZE(256));
/* Enable Rx cache snoop */
- for (lrxq = 0; lrxq < rxq_number; lrxq++) {
+ for (lrxq = 0; lrxq < port->nrxqs; lrxq++) {
queue = port->rxqs[lrxq]->id;
val = mvpp2_read(port->priv, MVPP2_RXQ_CONFIG_REG(queue));
val |= MVPP2_SNOOP_PKT_SIZE_MASK |
@@ -4696,7 +4691,7 @@ static void mvpp2_ingress_enable(struct mvpp2_port *port)
u32 val;
int lrxq, queue;
- for (lrxq = 0; lrxq < rxq_number; lrxq++) {
+ for (lrxq = 0; lrxq < port->nrxqs; lrxq++) {
queue = port->rxqs[lrxq]->id;
val = mvpp2_read(port->priv, MVPP2_RXQ_CONFIG_REG(queue));
val &= ~MVPP2_RXQ_DISABLE_MASK;
@@ -4709,7 +4704,7 @@ static void mvpp2_ingress_disable(struct mvpp2_port *port)
u32 val;
int lrxq, queue;
- for (lrxq = 0; lrxq < rxq_number; lrxq++) {
+ for (lrxq = 0; lrxq < port->nrxqs; lrxq++) {
queue = port->rxqs[lrxq]->id;
val = mvpp2_read(port->priv, MVPP2_RXQ_CONFIG_REG(queue));
val |= MVPP2_RXQ_DISABLE_MASK;
@@ -4728,7 +4723,7 @@ static void mvpp2_egress_enable(struct mvpp2_port *port)
/* Enable all initialized TXs. */
qmap = 0;
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
struct mvpp2_tx_queue *txq = port->txqs[queue];
if (txq->descs)
@@ -5014,7 +5009,7 @@ static void mvpp2_txq_sent_counter_clear(void *arg)
struct mvpp2_port *port = arg;
int queue;
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
int id = port->txqs[queue]->id;
mvpp2_percpu_read(port->priv, smp_processor_id(),
@@ -5055,7 +5050,7 @@ static void mvpp2_txp_max_tx_size_set(struct mvpp2_port *port)
mvpp2_write(port->priv, MVPP2_TXP_SCHED_TOKEN_SIZE_REG, val);
}
- for (txq = 0; txq < txq_number; txq++) {
+ for (txq = 0; txq < port->ntxqs; txq++) {
val = mvpp2_read(port->priv,
MVPP2_TXQ_SCHED_TOKEN_SIZE_REG(txq));
size = val & MVPP2_TXQ_TOKEN_SIZE_MAX;
@@ -5531,7 +5526,7 @@ static void mvpp2_cleanup_txqs(struct mvpp2_port *port)
val |= MVPP2_TX_PORT_FLUSH_MASK(port->id);
mvpp2_write(port->priv, MVPP2_TX_PORT_FLUSH_REG, val);
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
txq = port->txqs[queue];
mvpp2_txq_clean(port, txq);
mvpp2_txq_deinit(port, txq);
@@ -5548,7 +5543,7 @@ static void mvpp2_cleanup_rxqs(struct mvpp2_port *port)
{
int queue;
- for (queue = 0; queue < rxq_number; queue++)
+ for (queue = 0; queue < port->nrxqs; queue++)
mvpp2_rxq_deinit(port, port->rxqs[queue]);
}
@@ -5557,7 +5552,7 @@ static int mvpp2_setup_rxqs(struct mvpp2_port *port)
{
int queue, err;
- for (queue = 0; queue < rxq_number; queue++) {
+ for (queue = 0; queue < port->nrxqs; queue++) {
err = mvpp2_rxq_init(port, port->rxqs[queue]);
if (err)
goto err_cleanup;
@@ -5575,7 +5570,7 @@ static int mvpp2_setup_txqs(struct mvpp2_port *port)
struct mvpp2_tx_queue *txq;
int queue, err;
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
txq = port->txqs[queue];
err = mvpp2_txq_init(port, txq);
if (err)
@@ -5741,7 +5736,7 @@ static void mvpp2_tx_proc_cb(unsigned long data)
port_pcpu->timer_scheduled = false;
/* Process all the Tx queues */
- cause = (1 << txq_number) - 1;
+ cause = (1 << port->ntxqs) - 1;
tx_todo = mvpp2_tx_done(port, cause);
/* Set the timer in case not all the packets were processed */
@@ -6624,7 +6619,7 @@ static int mvpp2_ethtool_set_coalesce(struct net_device *dev,
struct mvpp2_port *port = netdev_priv(dev);
int queue;
- for (queue = 0; queue < rxq_number; queue++) {
+ for (queue = 0; queue < port->nrxqs; queue++) {
struct mvpp2_rx_queue *rxq = port->rxqs[queue];
rxq->time_coal = c->rx_coalesce_usecs;
@@ -6633,7 +6628,7 @@ static int mvpp2_ethtool_set_coalesce(struct net_device *dev,
mvpp2_rx_time_coal_set(port, rxq);
}
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
struct mvpp2_tx_queue *txq = port->txqs[queue];
txq->done_pkts_coal = c->tx_max_coalesced_frames;
@@ -6769,15 +6764,20 @@ static int mvpp2_port_init(struct mvpp2_port *port)
struct mvpp2_txq_pcpu *txq_pcpu;
int queue, cpu, err;
- if (port->first_rxq + rxq_number >
+ /* Checks for hardware constraints */
+ if (port->first_rxq + port->nrxqs >
MVPP2_MAX_PORTS * priv->max_port_rxqs)
return -EINVAL;
+ if (port->nrxqs % 4 || (port->nrxqs > priv->max_port_rxqs) ||
+ (port->ntxqs > MVPP2_MAX_TXQ))
+ return -EINVAL;
+
/* Disable port */
mvpp2_egress_disable(port);
mvpp2_port_disable(port);
- port->txqs = devm_kcalloc(dev, txq_number, sizeof(*port->txqs),
+ port->txqs = devm_kcalloc(dev, port->ntxqs, sizeof(*port->txqs),
GFP_KERNEL);
if (!port->txqs)
return -ENOMEM;
@@ -6785,7 +6785,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
/* Associate physical Tx queues to this port and initialize.
* The mapping is predefined.
*/
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
int queue_phy_id = mvpp2_txq_phys(port->id, queue);
struct mvpp2_tx_queue *txq;
@@ -6812,7 +6812,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
port->txqs[queue] = txq;
}
- port->rxqs = devm_kcalloc(dev, rxq_number, sizeof(*port->rxqs),
+ port->rxqs = devm_kcalloc(dev, port->nrxqs, sizeof(*port->rxqs),
GFP_KERNEL);
if (!port->rxqs) {
err = -ENOMEM;
@@ -6820,7 +6820,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
}
/* Allocate and initialize Rx queue for this port */
- for (queue = 0; queue < rxq_number; queue++) {
+ for (queue = 0; queue < port->nrxqs; queue++) {
struct mvpp2_rx_queue *rxq;
/* Map physical Rx queue to port's logical Rx queue */
@@ -6840,19 +6840,19 @@ static int mvpp2_port_init(struct mvpp2_port *port)
/* Configure Rx queue group interrupt for this port */
if (priv->hw_version == MVPP21) {
mvpp2_write(priv, MVPP21_ISR_RXQ_GROUP_REG(port->id),
- rxq_number);
+ port->nrxqs);
} else {
u32 val;
val = (port->id << MVPP22_ISR_RXQ_GROUP_INDEX_GROUP_OFFSET);
mvpp2_write(priv, MVPP22_ISR_RXQ_GROUP_INDEX_REG, val);
- val = (rxq_number << MVPP22_ISR_RXQ_SUB_GROUP_SIZE_OFFSET);
+ val = (port->nrxqs << MVPP22_ISR_RXQ_SUB_GROUP_SIZE_OFFSET);
mvpp2_write(priv, MVPP22_ISR_RXQ_SUB_GROUP_CONFIG_REG, val);
}
/* Create Rx descriptor rings */
- for (queue = 0; queue < rxq_number; queue++) {
+ for (queue = 0; queue < port->nrxqs; queue++) {
struct mvpp2_rx_queue *rxq = port->rxqs[queue];
rxq->size = port->rx_ring_size;
@@ -6880,7 +6880,7 @@ static int mvpp2_port_init(struct mvpp2_port *port)
return 0;
err_free_percpu:
- for (queue = 0; queue < txq_number; queue++) {
+ for (queue = 0; queue < port->ntxqs; queue++) {
if (!port->txqs[queue])
continue;
free_percpu(port->txqs[queue]->pcpu);
@@ -6901,12 +6901,16 @@ static int mvpp2_port_probe(struct platform_device *pdev,
const char *dt_mac_addr;
const char *mac_from;
char hw_mac_addr[ETH_ALEN];
+ unsigned int ntxqs, nrxqs;
u32 id;
int features;
int phy_mode;
int err, i, cpu;
- dev = alloc_etherdev_mqs(sizeof(*port), txq_number, rxq_number);
+ ntxqs = MVPP2_MAX_TXQ;
+ nrxqs = MVPP2_DEFAULT_RXQ;
+
+ dev = alloc_etherdev_mqs(sizeof(*port), ntxqs, nrxqs);
if (!dev)
return -ENOMEM;
@@ -6930,6 +6934,8 @@ static int mvpp2_port_probe(struct platform_device *pdev,
dev->ethtool_ops = &mvpp2_eth_tool_ops;
port = netdev_priv(dev);
+ port->ntxqs = ntxqs;
+ port->nrxqs = nrxqs;
if (of_get_property(port_node, "interrupt-names", NULL)) {
port->irq = of_irq_get_byname(port_node, "rx-shared");
@@ -6962,7 +6968,7 @@ static int mvpp2_port_probe(struct platform_device *pdev,
port->priv = priv;
port->id = id;
if (priv->hw_version == MVPP21)
- port->first_rxq = port->id * rxq_number;
+ port->first_rxq = port->id * port->nrxqs;
else
port->first_rxq = port->id * priv->max_port_rxqs;
@@ -7070,7 +7076,7 @@ static int mvpp2_port_probe(struct platform_device *pdev,
err_free_port_pcpu:
free_percpu(port->pcpu);
err_free_txq_pcpu:
- for (i = 0; i < txq_number; i++)
+ for (i = 0; i < port->ntxqs; i++)
free_percpu(port->txqs[i]->pcpu);
err_free_stats:
free_percpu(port->stats);
@@ -7091,7 +7097,7 @@ static void mvpp2_port_remove(struct mvpp2_port *port)
of_node_put(port->phy_node);
free_percpu(port->pcpu);
free_percpu(port->stats);
- for (i = 0; i < txq_number; i++)
+ for (i = 0; i < port->ntxqs; i++)
free_percpu(port->txqs[i]->pcpu);
irq_dispose_mapping(port->irq);
free_netdev(port->dev);
@@ -7208,13 +7214,6 @@ static int mvpp2_init(struct platform_device *pdev, struct mvpp2 *priv)
int err, i;
u32 val;
- /* Checks for hardware constraints */
- if (rxq_number % 4 || (rxq_number > priv->max_port_rxqs) ||
- (txq_number > MVPP2_MAX_TXQ)) {
- dev_err(&pdev->dev, "invalid queue size parameter\n");
- return -EINVAL;
- }
-
/* MBUS windows configuration */
dram_target_info = mv_mbus_dram_info();
if (dram_target_info)
--
2.9.4
^ permalink raw reply related
* [PATCH 1/8] net: mvpp2: fix MVPP21_ISR_RXQ_GROUP_REG definition
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
The MVPP21_ISR_RXQ_GROUP_REG register is not indexed by rxq, but by
port, so we fix the parameter name accordingly. There are no
functional changes.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index 33a7eb8..f5c84f4 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -187,7 +187,7 @@
/* Interrupt Cause and Mask registers */
#define MVPP2_ISR_RX_THRESHOLD_REG(rxq) (0x5200 + 4 * (rxq))
#define MVPP2_MAX_ISR_RX_THRESHOLD 0xfffff0
-#define MVPP21_ISR_RXQ_GROUP_REG(rxq) (0x5400 + 4 * (rxq))
+#define MVPP21_ISR_RXQ_GROUP_REG(port) (0x5400 + 4 * (port))
#define MVPP22_ISR_RXQ_GROUP_INDEX_REG 0x5400
#define MVPP22_ISR_RXQ_GROUP_INDEX_SUBGROUP_MASK 0xf
--
2.9.4
^ permalink raw reply related
* [PATCH 2/8] net: mvpp2: remove RX queue group reset code
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
In-Reply-To: <20170725155509.10574-1-thomas.petazzoni@free-electrons.com>
The RX queue group allocation is anyway re-done later in
mvpp2_port_init(), so resetting it in mvpp2_init() is not very useful,
and will be annoying as we are going to rework the RX queue group
allocation logic.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
drivers/net/ethernet/marvell/mvpp2.c | 17 -----------------
1 file changed, 17 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2.c b/drivers/net/ethernet/marvell/mvpp2.c
index f5c84f4..eb8853f 100644
--- a/drivers/net/ethernet/marvell/mvpp2.c
+++ b/drivers/net/ethernet/marvell/mvpp2.c
@@ -7253,23 +7253,6 @@ static int mvpp2_init(struct platform_device *pdev, struct mvpp2 *priv)
/* Rx Fifo Init */
mvpp2_rx_fifo_init(priv);
- /* Reset Rx queue group interrupt configuration */
- for (i = 0; i < MVPP2_MAX_PORTS; i++) {
- if (priv->hw_version == MVPP21) {
- mvpp2_write(priv, MVPP21_ISR_RXQ_GROUP_REG(i),
- rxq_number);
- continue;
- } else {
- u32 val;
-
- val = (i << MVPP22_ISR_RXQ_GROUP_INDEX_GROUP_OFFSET);
- mvpp2_write(priv, MVPP22_ISR_RXQ_GROUP_INDEX_REG, val);
-
- val = (rxq_number << MVPP22_ISR_RXQ_SUB_GROUP_SIZE_OFFSET);
- mvpp2_write(priv, MVPP22_ISR_RXQ_SUB_GROUP_CONFIG_REG, val);
- }
- }
-
if (priv->hw_version == MVPP21)
writel(MVPP2_EXT_GLOBAL_CTRL_DEFAULT,
priv->lms_base + MVPP2_MNG_EXTENDED_GLOBAL_CTRL_REG);
--
2.9.4
^ permalink raw reply related
* [PATCH 0/8] net: mvpp2: add TX interrupts support
From: Thomas Petazzoni @ 2017-07-25 15:55 UTC (permalink / raw)
To: netdev, David S. Miller
Cc: Russell King, Antoine Tenart, Miquèl Raynal,
linux-arm-kernel, Jason Cooper, Andrew Lunn,
Sebastian Hesselbarth, Gregory Clement, Nadav Haklai, Hanna Hawa,
Yehuda Yitschak, Stefan Chulski, Marcin Wojtas, Thomas Petazzoni
Hello,
So far, the mvpp2 driver was using an hrtimer to handle TX
completion. This patch series adds support for using TX interrupts
(for each CPU) on PPv2.2, the variant of the IP used on Marvell Armada
7K/8K.
This series has been tested on Marvell Armada 7K (PPv2.2) and Armada
375 (PPv2.1).
Dave:
- This series depends on the previous series sent by Antoine Ténart
"net: mvpp2: MAC/GoP configuration and optional PHYs". Functionally
speaking there is no real dependency, but we touch in a few areas
the same piece of code, so I based my patch series on top of
Antoine's.
- Please do not apply the last patch of this series "arm64: dts:
marvell: add TX interrupts for PPv2.2", it will be taken by the ARM
mvebu maintainers.
Thanks!
Thomas
Thomas Petazzoni (8):
net: mvpp2: fix MVPP21_ISR_RXQ_GROUP_REG definition
net: mvpp2: remove RX queue group reset code
net: mvpp2: introduce per-port nrxqs/ntxqs variables
net: mvpp2: move from cpu-centric naming to "software thread" naming
net: mvpp2: introduce queue_vector concept
net: mvpp2: add support for TX interrupts and RX queue distribution
modes
dt-bindings: net: marvell-pp2: update interrupt-names with TX
interrupts
arm64: dts: marvell: add TX interrupts for PPv2.2
.../devicetree/bindings/net/marvell-pp2.txt | 33 +-
.../boot/dts/marvell/armada-cp110-master.dtsi | 21 +-
.../arm64/boot/dts/marvell/armada-cp110-slave.dtsi | 21 +-
drivers/net/ethernet/marvell/mvpp2.c | 638 +++++++++++++++------
4 files changed, 534 insertions(+), 179 deletions(-)
--
2.9.4
^ permalink raw reply
* [RFC PATCH 10/10] net: ipv6: Support for sockets bound to enslaved device
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Add support for sockets bound to a network interface enslaved to an
L3 Master device (e.g, VRF). Currently for VRF, skb->dev points to the
VRF device meaning socket lookups only consider this device index. The
real ingress device index is saved to IP6CB(skb)->iif and the VRF driver
marks the skb with IP6SKB_L3SLAVE to know that the real ingress device
is an enslaved one without having to lookup the iif.
Use those flags to add the enslaved device index to the socket lookup
and allow sk->sk_bound_dev_if to match either dif (VRF device) or sdif
(enslaved device).
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/linux/ipv6.h | 8 ++++++++
include/net/inet6_hashtables.h | 5 +++--
include/net/tcp.h | 7 +++++++
net/ipv6/inet6_hashtables.c | 6 +++---
net/ipv6/raw.c | 5 ++++-
net/ipv6/tcp_ipv6.c | 3 +++
net/ipv6/udp.c | 8 ++++++--
7 files changed, 34 insertions(+), 8 deletions(-)
diff --git a/include/linux/ipv6.h b/include/linux/ipv6.h
index e1b442996f81..094357907b45 100644
--- a/include/linux/ipv6.h
+++ b/include/linux/ipv6.h
@@ -153,6 +153,14 @@ static inline int inet6_iif(const struct sk_buff *skb)
}
/* can not be used in TCP layer after tcp_v6_fill_cb */
+static inline int inet6_sdif(const struct sk_buff *skb)
+{
+ bool l3_slave = ipv6_l3mdev_skb(IP6CB(skb)->flags);
+
+ return l3_slave ? IP6CB(skb)->iif : 0;
+}
+
+/* can not be used in TCP layer after tcp_v6_fill_cb */
static inline bool inet6_exact_dif_match(struct net *net, struct sk_buff *skb)
{
#if defined(CONFIG_NET_L3_MASTER_DEV)
diff --git a/include/net/inet6_hashtables.h b/include/net/inet6_hashtables.h
index 15db41272ff2..0fc5a2fe4ad3 100644
--- a/include/net/inet6_hashtables.h
+++ b/include/net/inet6_hashtables.h
@@ -94,13 +94,14 @@ struct sock *inet6_lookup(struct net *net, struct inet_hashinfo *hashinfo,
int inet6_hash(struct sock *sk);
#endif /* IS_ENABLED(CONFIG_IPV6) */
-#define INET6_MATCH(__sk, __net, __saddr, __daddr, __ports, __dif) \
+#define INET6_MATCH(__sk, __net, __saddr, __daddr, __ports, __dif, __sdif) \
(((__sk)->sk_portpair == (__ports)) && \
((__sk)->sk_family == AF_INET6) && \
ipv6_addr_equal(&(__sk)->sk_v6_daddr, (__saddr)) && \
ipv6_addr_equal(&(__sk)->sk_v6_rcv_saddr, (__daddr)) && \
(!(__sk)->sk_bound_dev_if || \
- ((__sk)->sk_bound_dev_if == (__dif))) && \
+ ((__sk)->sk_bound_dev_if == (__dif)) || \
+ ((__sk)->sk_bound_dev_if == (__sdif))) && \
net_eq(sock_net(__sk), (__net)))
#endif /* _INET6_HASHTABLES_H */
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 1a66ab82988b..3bc0bc4daa05 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -848,6 +848,13 @@ static inline int tcp_v6_iif(const struct sk_buff *skb)
return l3_slave ? skb->skb_iif : TCP_SKB_CB(skb)->header.h6.iif;
}
+
+static inline int tcp_v6_sdif(const struct sk_buff *skb)
+{
+ bool l3_slave = ipv6_l3mdev_skb(TCP_SKB_CB(skb)->header.h6.flags);
+
+ return l3_slave ? TCP_SKB_CB(skb)->header.h6.iif : 0;
+}
#endif
/* TCP_SKB_CB reference means this can not be used from early demux */
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index 878c03094f2e..2af34af36110 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -74,13 +74,13 @@ struct sock *__inet6_lookup_established(struct net *net,
if (sk->sk_hash != hash)
continue;
if (!INET6_MATCH(sk, net, saddr, daddr, ports,
- params->dif))
+ params->dif, params->sdif))
continue;
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
if (unlikely(!INET6_MATCH(sk, net, saddr, daddr, ports,
- params->dif))) {
+ params->dif, params->sdif))) {
sock_gen_put(sk);
goto begin;
}
@@ -205,7 +205,7 @@ static int __inet6_check_established(struct inet_timewait_death_row *death_row,
continue;
if (likely(INET6_MATCH(sk2, net, saddr, daddr, ports,
- dif))) {
+ dif, 0))) {
if (sk2->sk_state == TCP_TIME_WAIT) {
tw = inet_twsk(sk2);
if (twsk_unique(sk, sk2, twp))
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index 51e651f18ffb..bab365214d17 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -87,7 +87,8 @@ struct sock *__raw_v6_lookup(struct net *net, struct sock *sk,
continue;
if (sk->sk_bound_dev_if &&
- sk->sk_bound_dev_if != params->dif)
+ sk->sk_bound_dev_if != params->dif &&
+ sk->sk_bound_dev_if != params->sdif)
continue;
if (!ipv6_addr_any(&sk->sk_v6_rcv_saddr)) {
@@ -165,6 +166,7 @@ static bool ipv6_raw_deliver(struct sk_buff *skb, int nexthdr)
.daddr.ipv6 = &ipv6_hdr(skb)->daddr,
.hnum = nexthdr,
.dif = inet6_iif(skb),
+ .sdif = inet6_sdif(skb),
};
struct sock *sk;
bool delivered = false;
@@ -375,6 +377,7 @@ void raw6_icmp_error(struct sk_buff *skb, int nexthdr,
struct sk_lookup params = {
.hnum = nexthdr,
.dif = inet6_iif(skb),
+ .sdif = inet6_sdif(skb),
};
/* Note: ipv6_hdr(skb) != skb->data */
const struct ipv6hdr *ip6h = (const struct ipv6hdr *)skb->data;
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 154886daba7b..55a7256211ca 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -917,6 +917,7 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
.sport = th->source,
.hnum = ntohs(th->source),
.dif = tcp_v6_iif(skb),
+ .sdif = tcp_v6_sdif(skb),
};
/*
@@ -1414,6 +1415,7 @@ static int tcp_v6_rcv(struct sk_buff *skb)
struct net *net = dev_net(skb->dev);
struct sk_lookup params = {
.dif = inet6_iif(skb),
+ .sdif = inet6_sdif(skb),
};
if (skb->pkt_type != PACKET_HOST)
@@ -1577,6 +1579,7 @@ static int tcp_v6_rcv(struct sk_buff *skb)
.sport = th->source,
.hnum = ntohs(th->dest),
.dif = tcp_v6_iif(skb),
+ .sdif = tcp_v6_sdif(skb),
};
struct sock *sk2;
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 5c4fdbe52c24..bb72a480d169 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -220,6 +220,7 @@ struct sock *__udp6_lib_lookup(struct net *net, struct sk_lookup *params,
u32 hash = 0;
params->hnum = hnum;
+ params->sdif = inet6_sdif(skb);
params->exact_dif = udp6_lib_exact_dif_match(net, skb);
if (hslot->count > 10) {
@@ -673,7 +674,8 @@ static bool __udp_v6_is_mcast_sock(struct net *net, struct sock *sk,
(inet->inet_dport && inet->inet_dport != params->sport) ||
(!ipv6_addr_any(&sk->sk_v6_daddr) &&
!ipv6_addr_equal(&sk->sk_v6_daddr, rmt_addr)) ||
- (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif) ||
+ (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif &&
+ sk->sk_bound_dev_if != params->sdif) ||
(!ipv6_addr_any(&sk->sk_v6_rcv_saddr) &&
!ipv6_addr_equal(&sk->sk_v6_rcv_saddr, loc_addr)))
return false;
@@ -715,6 +717,7 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
.dport = uh->dest,
.hnum = hnum,
.dif = inet6_iif(skb),
+ .sdif = inet6_sdif(skb),
};
if (use_hash2) {
@@ -893,7 +896,7 @@ static struct sock *__udp6_lib_demux_lookup(struct net *net,
if (sk->sk_state == TCP_ESTABLISHED &&
INET6_MATCH(sk, net, params->saddr.ipv6,
params->daddr.ipv6, ports,
- params->dif))
+ params->dif, params->sdif))
return sk;
/* Only check first socket in chain */
@@ -910,6 +913,7 @@ static void udp_v6_early_demux(struct sk_buff *skb)
struct dst_entry *dst;
struct sk_lookup params = {
.dif = skb->dev->ifindex,
+ .sdif = inet6_sdif(skb),
};
if (skb->pkt_type != PACKET_HOST)
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 09/10] net: ipv4: Support for sockets bound to enslaved device
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Add support for sockets bound to a network interface enslaved to an
L3 Master device (e.g, VRF). Currently for VRF, skb->dev points to the
VRF device meaning socket lookups only consider this device index. The
real ingress device index is saved to IPCB(skb)->iif and the VRF driver
marks the skb with IPSKB_L3SLAVE to know that the real ingress device
is an enslaved one without having to lookup the iif.
Use those flags to add the enslaved device index to the socket lookup
and allow sk->sk_bound_dev_if to match either dif (VRF device) or sdif
(enslaved device).
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/linux/igmp.h | 3 ++-
include/net/inet_hashtables.h | 10 ++++++----
include/net/ip.h | 10 ++++++++++
include/net/tcp.h | 10 ++++++++++
net/ipv4/igmp.c | 6 ++++--
net/ipv4/inet_hashtables.c | 6 +++---
net/ipv4/raw.c | 7 +++++--
net/ipv4/tcp_ipv4.c | 6 ++++--
net/ipv4/udp.c | 11 ++++++++---
9 files changed, 52 insertions(+), 17 deletions(-)
diff --git a/include/linux/igmp.h b/include/linux/igmp.h
index 97caf1821de8..f8231854b5d6 100644
--- a/include/linux/igmp.h
+++ b/include/linux/igmp.h
@@ -118,7 +118,8 @@ extern int ip_mc_msfget(struct sock *sk, struct ip_msfilter *msf,
struct ip_msfilter __user *optval, int __user *optlen);
extern int ip_mc_gsfget(struct sock *sk, struct group_filter *gsf,
struct group_filter __user *optval, int __user *optlen);
-extern int ip_mc_sf_allow(struct sock *sk, __be32 local, __be32 rmt, int dif);
+extern int ip_mc_sf_allow(struct sock *sk, __be32 local, __be32 rmt,
+ int dif, int sdif);
extern void ip_mc_init_dev(struct in_device *);
extern void ip_mc_destroy_dev(struct in_device *);
extern void ip_mc_up(struct in_device *);
diff --git a/include/net/inet_hashtables.h b/include/net/inet_hashtables.h
index fabb8dd8fdb1..201f29d3c157 100644
--- a/include/net/inet_hashtables.h
+++ b/include/net/inet_hashtables.h
@@ -259,22 +259,24 @@ static inline struct sock *inet_lookup_listener(struct net *net,
(((__force __u64)(__be32)(__daddr)) << 32) | \
((__force __u64)(__be32)(__saddr)))
#endif /* __BIG_ENDIAN */
-#define INET_MATCH(__sk, __net, __cookie, __saddr, __daddr, __ports, __dif) \
+#define INET_MATCH(__sk, __net, __cookie, __saddr, __daddr, __ports, __dif, __sdif) \
(((__sk)->sk_portpair == (__ports)) && \
((__sk)->sk_addrpair == (__cookie)) && \
(!(__sk)->sk_bound_dev_if || \
- ((__sk)->sk_bound_dev_if == (__dif))) && \
+ ((__sk)->sk_bound_dev_if == (__dif)) || \
+ ((__sk)->sk_bound_dev_if == (__sdif))) && \
net_eq(sock_net(__sk), (__net)))
#else /* 32-bit arch */
#define INET_ADDR_COOKIE(__name, __saddr, __daddr) \
const int __name __deprecated __attribute__((unused))
-#define INET_MATCH(__sk, __net, __cookie, __saddr, __daddr, __ports, __dif) \
+#define INET_MATCH(__sk, __net, __cookie, __saddr, __daddr, __ports, __dif, __sdif) \
(((__sk)->sk_portpair == (__ports)) && \
((__sk)->sk_daddr == (__saddr)) && \
((__sk)->sk_rcv_saddr == (__daddr)) && \
(!(__sk)->sk_bound_dev_if || \
- ((__sk)->sk_bound_dev_if == (__dif))) && \
+ ((__sk)->sk_bound_dev_if == (__dif)) || \
+ ((__sk)->sk_bound_dev_if == (__sdif))) && \
net_eq(sock_net(__sk), (__net)))
#endif /* 64-bit arch */
diff --git a/include/net/ip.h b/include/net/ip.h
index 821cedcc8e73..e10da8814dba 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -78,6 +78,16 @@ struct ipcm_cookie {
#define IPCB(skb) ((struct inet_skb_parm*)((skb)->cb))
#define PKTINFO_SKB_CB(skb) ((struct in_pktinfo *)((skb)->cb))
+/* return enslaved device index if relevant */
+static inline int ip_sdif(struct sk_buff *skb)
+{
+#if IS_ENABLED(CONFIG_NET_L3_MASTER_DEV)
+ if (skb && ipv4_l3mdev_skb(IPCB(skb)->flags))
+ return IPCB(skb)->iif;
+#endif
+ return 0;
+}
+
struct ip_ra_chain {
struct ip_ra_chain __rcu *next;
struct sock *sk;
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 4f056ea79df2..1a66ab82988b 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -861,6 +861,16 @@ static inline bool inet_exact_dif_match(struct net *net, struct sk_buff *skb)
return false;
}
+/* TCP_SKB_CB reference means this can not be used from early demux */
+static inline int tcp_v4_sdif(struct sk_buff *skb)
+{
+#if IS_ENABLED(CONFIG_NET_L3_MASTER_DEV)
+ if (skb && ipv4_l3mdev_skb(TCP_SKB_CB(skb)->header.h4.flags))
+ return TCP_SKB_CB(skb)->header.h4.iif;
+#endif
+ return 0;
+}
+
/* Due to TSO, an SKB can be composed of multiple actual
* packets. To keep these tracked properly, we use this.
*/
diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c
index 28f14afd0dd3..0d5fb47743bf 100644
--- a/net/ipv4/igmp.c
+++ b/net/ipv4/igmp.c
@@ -2549,7 +2549,8 @@ int ip_mc_gsfget(struct sock *sk, struct group_filter *gsf,
/*
* check if a multicast source filter allows delivery for a given <src,dst,intf>
*/
-int ip_mc_sf_allow(struct sock *sk, __be32 loc_addr, __be32 rmt_addr, int dif)
+int ip_mc_sf_allow(struct sock *sk, __be32 loc_addr, __be32 rmt_addr,
+ int dif, int sdif)
{
struct inet_sock *inet = inet_sk(sk);
struct ip_mc_socklist *pmc;
@@ -2564,7 +2565,8 @@ int ip_mc_sf_allow(struct sock *sk, __be32 loc_addr, __be32 rmt_addr, int dif)
rcu_read_lock();
for_each_pmc_rcu(inet, pmc) {
if (pmc->multi.imr_multiaddr.s_addr == loc_addr &&
- pmc->multi.imr_ifindex == dif)
+ (pmc->multi.imr_ifindex == dif ||
+ pmc->multi.imr_ifindex == sdif))
break;
}
ret = inet->mc_all;
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index e581e200d01d..764da4302dac 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -291,12 +291,12 @@ struct sock *__inet_lookup_established(struct net *net,
if (sk->sk_hash != hash)
continue;
if (likely(INET_MATCH(sk, net, acookie, saddr, daddr,
- ports, params->dif))) {
+ ports, params->dif, params->sdif))) {
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
if (unlikely(!INET_MATCH(sk, net, acookie,
saddr, daddr, ports,
- params->dif))) {
+ params->dif, params->sdif))) {
sock_gen_put(sk);
goto begin;
}
@@ -345,7 +345,7 @@ static int __inet_check_established(struct inet_timewait_death_row *death_row,
continue;
if (likely(INET_MATCH(sk2, net, acookie,
- saddr, daddr, ports, dif))) {
+ saddr, daddr, ports, dif, 0))) {
if (sk2->sk_state == TCP_TIME_WAIT) {
tw = inet_twsk(sk2);
if (twsk_unique(sk, sk2, twp))
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index 4da5d87a61a5..a94f8f115b6e 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -132,7 +132,8 @@ struct sock *__raw_v4_lookup(struct net *net, struct sock *sk,
bool dev_match;
dev_match = (!sk->sk_bound_dev_if ||
- sk->sk_bound_dev_if == params->dif);
+ sk->sk_bound_dev_if == params->dif ||
+ sk->sk_bound_dev_if == params->sdif);
if (net_eq(sock_net(sk), net) &&
inet->inet_num == params->hnum &&
@@ -186,6 +187,7 @@ static int __raw_v4_input(struct sk_buff *skb, const struct iphdr *iph,
.daddr.ipv4 = iph->daddr,
.hnum = iph->protocol,
.dif = skb->dev->ifindex,
+ .sdif = ip_sdif(skb),
};
int delivered = 0;
struct sock *sk;
@@ -195,7 +197,7 @@ static int __raw_v4_input(struct sk_buff *skb, const struct iphdr *iph,
delivered = 1;
if ((iph->protocol != IPPROTO_ICMP || !icmp_filter(sk, skb)) &&
ip_mc_sf_allow(sk, iph->daddr, iph->saddr,
- skb->dev->ifindex)) {
+ skb->dev->ifindex, params.sdif)) {
struct sk_buff *clone = skb_clone(skb, GFP_ATOMIC);
/* Not releasing hash table! */
@@ -316,6 +318,7 @@ void raw_icmp_error(struct sk_buff *skb, int protocol, u32 info)
struct sk_lookup params = {
.hnum = protocol,
.dif = skb->dev->ifindex,
+ .sdif = ip_sdif(skb),
};
iph = (const struct iphdr *)skb->data;
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 89a0d166e677..d0f397dab3ed 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1664,7 +1664,9 @@ EXPORT_SYMBOL(tcp_filter);
int tcp_v4_rcv(struct sk_buff *skb)
{
struct net *net = dev_net(skb->dev);
- struct sk_lookup params = { };
+ struct sk_lookup params = {
+ .sdif = ip_sdif(skb),
+ };
const struct iphdr *iph;
const struct tcphdr *th;
bool refcounted;
@@ -1846,8 +1848,8 @@ int tcp_v4_rcv(struct sk_buff *skb)
.daddr.ipv4 = iph->daddr,
.sport = th->source,
.dport = th->dest,
- .hnum = ntohs(th->dest),
.dif = inet_iif(skb),
+ .sdif = tcp_v4_sdif(skb),
};
struct sock *sk2 = inet_lookup_listener(dev_net(skb->dev),
&tcp_hashinfo, skb,
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 132a8f070d16..5c9fffed9c4a 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -485,6 +485,7 @@ struct sock *__udp4_lib_lookup(struct net *net, struct sk_lookup *params,
u32 hash = 0;
params->hnum = hnum;
+ params->sdif = ip_sdif(skb);
params->exact_dif = udp_lib_exact_dif_match(net, skb);
if (hslot->count > 10) {
@@ -597,9 +598,10 @@ static inline bool __udp_is_mcast_sock(struct net *net, struct sock *sk,
(inet->inet_dport != params->sport && inet->inet_dport) ||
(inet->inet_rcv_saddr && inet->inet_rcv_saddr != loc_addr) ||
ipv6_only_sock(sk) ||
- (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif))
+ (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif &&
+ sk->sk_bound_dev_if != params->sdif))
return false;
- if (!ip_mc_sf_allow(sk, loc_addr, rmt_addr, params->dif))
+ if (!ip_mc_sf_allow(sk, loc_addr, rmt_addr, params->dif, params->sdif))
return false;
return true;
}
@@ -1970,6 +1972,7 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
.dport = uh->dest,
.hnum = hnum,
.dif = skb->dev->ifindex,
+ .sdif = ip_sdif(skb),
};
if (use_hash2) {
@@ -2210,7 +2213,8 @@ static struct sock *__udp4_lib_demux_lookup(struct net *net,
udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) {
if (INET_MATCH(sk, net, acookie, params->saddr.ipv4,
- params->daddr.ipv4, ports, params->dif))
+ params->daddr.ipv4, ports, params->dif,
+ params->sdif))
return sk;
/* Only check first socket in chain */
break;
@@ -2223,6 +2227,7 @@ void udp_v4_early_demux(struct sk_buff *skb)
struct net *net = dev_net(skb->dev);
struct sk_lookup params = {
.dif = skb->dev->ifindex,
+ .sdif = ip_sdif(skb),
};
const struct iphdr *iph;
const struct udphdr *uh;
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 08/10] net: Add sdif to sk_lookup
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Add a second device index, sdif, to the socket lookup struct. sdif
will be the device index for devices enslaved to an l3mdev. It allows
the lookups to consider the enslaved device as well as the L3 master
device when searching for a socket.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/sock.h | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/include/net/sock.h b/include/net/sock.h
index a2db5fd30192..c5d93a4bcd0a 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -507,23 +507,27 @@ struct sk_lookup {
unsigned short hnum;
int dif;
+ int sdif;
bool exact_dif;
};
-/* Compare sk_bound_dev_if to socket lookup dif
+/* Compare sk_bound_dev_if to socket lookup dif and sdif
* Returns:
* -1 exact dif required and not met
* 0 sk_bound_dev_if is either not set or does not match
- * 1 sk_bound_dev_if is set and matches dif
+ * 1 sk_bound_dev_if is set and matches dif or sdif
*/
static inline int sk_lookup_device_cmp(const struct sock *sk,
const struct sk_lookup *params)
{
+ bool dev_match = (sk->sk_bound_dev_if == params->dif ||
+ sk->sk_bound_dev_if == params->sdif);
+
/* exact_dif true == l3mdev case */
- if (params->exact_dif && sk->sk_bound_dev_if != params->dif)
+ if (params->exact_dif && !dev_match)
return -1;
- if (sk->sk_bound_dev_if && sk->sk_bound_dev_if == params->dif)
+ if (sk->sk_bound_dev_if && dev_match)
return 1;
return 0;
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 07/10] net: ipv6: Convert raw sockets to sk_lookup
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert __raw_v6_lookup to use the new sk_lookup struct
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/rawv6.h | 3 +--
net/ipv4/raw_diag.c | 15 ++++++++++-----
net/ipv6/raw.c | 41 +++++++++++++++++++++++------------------
3 files changed, 34 insertions(+), 25 deletions(-)
diff --git a/include/net/rawv6.h b/include/net/rawv6.h
index cbe4e9de1894..406268324d26 100644
--- a/include/net/rawv6.h
+++ b/include/net/rawv6.h
@@ -5,8 +5,7 @@
extern struct raw_hashinfo raw_v6_hashinfo;
struct sock *__raw_v6_lookup(struct net *net, struct sock *sk,
- unsigned short num, const struct in6_addr *loc_addr,
- const struct in6_addr *rmt_addr, int dif);
+ const struct sk_lookup *params);
int raw_abort(struct sock *sk, int err);
diff --git a/net/ipv4/raw_diag.c b/net/ipv4/raw_diag.c
index a708de070cc6..2314993f6294 100644
--- a/net/ipv4/raw_diag.c
+++ b/net/ipv4/raw_diag.c
@@ -53,11 +53,16 @@ static struct sock *raw_lookup(struct net *net, struct sock *from,
sk = __raw_v4_lookup(net, from, ¶ms);
}
#if IS_ENABLED(CONFIG_IPV6)
- else
- sk = __raw_v6_lookup(net, from, r->sdiag_raw_protocol,
- (const struct in6_addr *)r->id.idiag_src,
- (const struct in6_addr *)r->id.idiag_dst,
- r->id.idiag_if);
+ else {
+ struct sk_lookup params = {
+ .saddr.ipv6 = (const struct in6_addr *)r->id.idiag_dst,
+ .daddr.ipv6 = (const struct in6_addr *)r->id.idiag_src,
+ .hnum = r->sdiag_raw_protocol,
+ .dif = r->id.idiag_if,
+ };
+
+ sk = __raw_v6_lookup(net, from, ¶ms);
+ }
#endif
return sk;
}
diff --git a/net/ipv6/raw.c b/net/ipv6/raw.c
index 60be012fe708..51e651f18ffb 100644
--- a/net/ipv6/raw.c
+++ b/net/ipv6/raw.c
@@ -71,14 +71,14 @@ struct raw_hashinfo raw_v6_hashinfo = {
EXPORT_SYMBOL_GPL(raw_v6_hashinfo);
struct sock *__raw_v6_lookup(struct net *net, struct sock *sk,
- unsigned short num, const struct in6_addr *loc_addr,
- const struct in6_addr *rmt_addr, int dif)
+ const struct sk_lookup *params)
{
+ const struct in6_addr *loc_addr = params->daddr.ipv6;
+ const struct in6_addr *rmt_addr = params->saddr.ipv6;
bool is_multicast = ipv6_addr_is_multicast(loc_addr);
sk_for_each_from(sk)
- if (inet_sk(sk)->inet_num == num) {
-
+ if (inet_sk(sk)->inet_num == params->hnum) {
if (!net_eq(sock_net(sk), net))
continue;
@@ -86,7 +86,8 @@ struct sock *__raw_v6_lookup(struct net *net, struct sock *sk,
!ipv6_addr_equal(&sk->sk_v6_daddr, rmt_addr))
continue;
- if (sk->sk_bound_dev_if && sk->sk_bound_dev_if != dif)
+ if (sk->sk_bound_dev_if &&
+ sk->sk_bound_dev_if != params->dif)
continue;
if (!ipv6_addr_any(&sk->sk_v6_rcv_saddr)) {
@@ -159,15 +160,17 @@ EXPORT_SYMBOL(rawv6_mh_filter_unregister);
*/
static bool ipv6_raw_deliver(struct sk_buff *skb, int nexthdr)
{
- const struct in6_addr *saddr;
- const struct in6_addr *daddr;
+ struct sk_lookup params = {
+ .saddr.ipv6 = &ipv6_hdr(skb)->saddr,
+ .daddr.ipv6 = &ipv6_hdr(skb)->daddr,
+ .hnum = nexthdr,
+ .dif = inet6_iif(skb),
+ };
struct sock *sk;
bool delivered = false;
__u8 hash;
struct net *net;
- saddr = &ipv6_hdr(skb)->saddr;
- daddr = saddr + 1;
hash = nexthdr & (RAW_HTABLE_SIZE - 1);
@@ -178,7 +181,7 @@ static bool ipv6_raw_deliver(struct sk_buff *skb, int nexthdr)
goto out;
net = dev_net(skb->dev);
- sk = __raw_v6_lookup(net, sk, nexthdr, daddr, saddr, inet6_iif(skb));
+ sk = __raw_v6_lookup(net, sk, ¶ms);
while (sk) {
int filtered;
@@ -221,8 +224,7 @@ static bool ipv6_raw_deliver(struct sk_buff *skb, int nexthdr)
rawv6_rcv(sk, clone);
}
}
- sk = __raw_v6_lookup(net, sk_next(sk), nexthdr, daddr, saddr,
- inet6_iif(skb));
+ sk = __raw_v6_lookup(net, sk_next(sk), ¶ms);
}
out:
read_unlock(&raw_v6_hashinfo.lock);
@@ -362,23 +364,26 @@ void raw6_icmp_error(struct sk_buff *skb, int nexthdr,
u8 type, u8 code, int inner_offset, __be32 info)
{
struct sock *sk;
- int hash;
- const struct in6_addr *saddr, *daddr;
struct net *net;
+ int hash;
hash = nexthdr & (RAW_HTABLE_SIZE - 1);
read_lock(&raw_v6_hashinfo.lock);
sk = sk_head(&raw_v6_hashinfo.ht[hash]);
if (sk) {
+ struct sk_lookup params = {
+ .hnum = nexthdr,
+ .dif = inet6_iif(skb),
+ };
/* Note: ipv6_hdr(skb) != skb->data */
const struct ipv6hdr *ip6h = (const struct ipv6hdr *)skb->data;
- saddr = &ip6h->saddr;
- daddr = &ip6h->daddr;
+
+ params.daddr.ipv6 = &ip6h->saddr;
+ params.saddr.ipv6 = &ip6h->daddr;
net = dev_net(skb->dev);
- while ((sk = __raw_v6_lookup(net, sk, nexthdr, saddr, daddr,
- inet6_iif(skb)))) {
+ while ((sk = __raw_v6_lookup(net, sk, ¶ms))) {
rawv6_err(sk, skb, NULL, type, code,
inner_offset, info);
sk = sk_next(sk);
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 06/10] net: ipv6: Convert inet socket lookups to new struct
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert the various inet6_lookup functions to use the new sk_lookup
struct.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/inet6_hashtables.h | 39 +++++++-------------
net/dccp/ipv6.c | 22 ++++++++----
net/ipv4/inet_diag.c | 19 ++++++----
net/ipv4/udp_diag.c | 2 ++
net/ipv6/inet6_hashtables.c | 72 +++++++++++++++++++------------------
net/ipv6/netfilter/nf_socket_ipv6.c | 5 ++-
net/ipv6/tcp_ipv6.c | 60 +++++++++++++++++++++----------
net/netfilter/xt_TPROXY.c | 8 ++---
8 files changed, 125 insertions(+), 102 deletions(-)
diff --git a/include/net/inet6_hashtables.h b/include/net/inet6_hashtables.h
index b87becacd9d3..15db41272ff2 100644
--- a/include/net/inet6_hashtables.h
+++ b/include/net/inet6_hashtables.h
@@ -46,63 +46,50 @@ static inline unsigned int __inet6_ehashfn(const u32 lhash,
*/
struct sock *__inet6_lookup_established(struct net *net,
struct inet_hashinfo *hashinfo,
- const struct in6_addr *saddr,
- const __be16 sport,
- const struct in6_addr *daddr,
- const u16 hnum, const int dif);
+ const struct sk_lookup *params);
struct sock *inet6_lookup_listener(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const struct in6_addr *saddr,
- const __be16 sport,
- const struct in6_addr *daddr,
- const unsigned short hnum, const int dif);
+ struct sk_lookup *params);
static inline struct sock *__inet6_lookup(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const struct in6_addr *saddr,
- const __be16 sport,
- const struct in6_addr *daddr,
- const u16 hnum,
- const int dif,
+ struct sk_lookup *params,
bool *refcounted)
{
- struct sock *sk = __inet6_lookup_established(net, hashinfo, saddr,
- sport, daddr, hnum, dif);
+ struct sock *sk = __inet6_lookup_established(net, hashinfo, params);
+
*refcounted = true;
if (sk)
return sk;
*refcounted = false;
- return inet6_lookup_listener(net, hashinfo, skb, doff, saddr, sport,
- daddr, hnum, dif);
+ return inet6_lookup_listener(net, hashinfo, skb, doff, params);
}
static inline struct sock *__inet6_lookup_skb(struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const __be16 sport,
- const __be16 dport,
- int iif,
+ struct sk_lookup *params,
bool *refcounted)
{
struct sock *sk = skb_steal_sock(skb);
+ params->saddr.ipv6 = &ipv6_hdr(skb)->saddr,
+ params->daddr.ipv6 = &ipv6_hdr(skb)->daddr,
+ params->hnum = ntohs(params->dport),
+
*refcounted = true;
if (sk)
return sk;
return __inet6_lookup(dev_net(skb_dst(skb)->dev), hashinfo, skb,
- doff, &ipv6_hdr(skb)->saddr, sport,
- &ipv6_hdr(skb)->daddr, ntohs(dport),
- iif, refcounted);
+ doff, params, refcounted);
}
struct sock *inet6_lookup(struct net *net, struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const struct in6_addr *saddr, const __be16 sport,
- const struct in6_addr *daddr, const __be16 dport,
- const int dif);
+ struct sk_lookup *params);
int inet6_hash(struct sock *sk);
#endif /* IS_ENABLED(CONFIG_IPV6) */
diff --git a/net/dccp/ipv6.c b/net/dccp/ipv6.c
index c376af5bfdfb..e92f10a832dd 100644
--- a/net/dccp/ipv6.c
+++ b/net/dccp/ipv6.c
@@ -70,6 +70,11 @@ static void dccp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
u8 type, u8 code, int offset, __be32 info)
{
const struct ipv6hdr *hdr = (const struct ipv6hdr *)skb->data;
+ struct sk_lookup params = {
+ .saddr.ipv6 = &hdr->daddr,
+ .daddr.ipv6 = &hdr->saddr,
+ .dif = inet6_iif(skb),
+ };
const struct dccp_hdr *dh;
struct dccp_sock *dp;
struct ipv6_pinfo *np;
@@ -86,11 +91,10 @@ static void dccp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
BUILD_BUG_ON(offsetofend(struct dccp_hdr, dccph_dport) > 8);
dh = (struct dccp_hdr *)(skb->data + offset);
- sk = __inet6_lookup_established(net, &dccp_hashinfo,
- &hdr->daddr, dh->dccph_dport,
- &hdr->saddr, ntohs(dh->dccph_sport),
- inet6_iif(skb));
-
+ params.sport = dh->dccph_dport;
+ params.dport = dh->dccph_sport;
+ params.hnum = ntohs(dh->dccph_sport);
+ sk = __inet6_lookup_established(net, &dccp_hashinfo, ¶ms);
if (!sk) {
__ICMP6_INC_STATS(net, __in6_dev_get(skb->dev),
ICMP6_MIB_INERRORS);
@@ -656,6 +660,9 @@ static int dccp_v6_do_rcv(struct sock *sk, struct sk_buff *skb)
static int dccp_v6_rcv(struct sk_buff *skb)
{
+ struct sk_lookup params = {
+ .dif = inet6_iif(skb),
+ };
const struct dccp_hdr *dh;
bool refcounted;
struct sock *sk;
@@ -683,10 +690,11 @@ static int dccp_v6_rcv(struct sk_buff *skb)
else
DCCP_SKB_CB(skb)->dccpd_ack_seq = dccp_hdr_ack_seq(skb);
+ params.sport = dh->dccph_sport;
+ params.dport = dh->dccph_dport;
lookup:
sk = __inet6_lookup_skb(&dccp_hashinfo, skb, __dccp_hdr_len(dh),
- dh->dccph_sport, dh->dccph_dport,
- inet6_iif(skb), &refcounted);
+ ¶ms, &refcounted);
if (!sk) {
dccp_pr_debug("failed to look up flow ID in table and "
"get corresponding socket\n");
diff --git a/net/ipv4/inet_diag.c b/net/ipv4/inet_diag.c
index 6c3bc4e408d0..fa0d8531ce36 100644
--- a/net/ipv4/inet_diag.c
+++ b/net/ipv4/inet_diag.c
@@ -422,13 +422,18 @@ struct sock *inet_diag_find_one_icsk(struct net *net,
};
sk = inet_lookup(net, hashinfo, NULL, 0, ¶ms);
- } else
- sk = inet6_lookup(net, hashinfo, NULL, 0,
- (struct in6_addr *)req->id.idiag_dst,
- req->id.idiag_dport,
- (struct in6_addr *)req->id.idiag_src,
- req->id.idiag_sport,
- req->id.idiag_if);
+ } else {
+ struct sk_lookup params = {
+ .saddr.ipv6 = (struct in6_addr *)req->id.idiag_dst,
+ .daddr.ipv6 = (struct in6_addr *)req->id.idiag_src,
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
+ .dif = req->id.idiag_if,
+ };
+
+ sk = inet6_lookup(net, hashinfo, NULL, 0, ¶ms);
+ }
}
#endif
else {
diff --git a/net/ipv4/udp_diag.c b/net/ipv4/udp_diag.c
index 8c1221f5f2dd..a11be7b8b55d 100644
--- a/net/ipv4/udp_diag.c
+++ b/net/ipv4/udp_diag.c
@@ -60,6 +60,7 @@ static int udp_dump_one(struct udp_table *tbl, struct sk_buff *in_skb,
.daddr.ipv6 = (struct in6_addr *)req->id.idiag_dst,
.sport = req->id.idiag_sport,
.dport = req->id.idiag_dport,
+ .hnum = ntohs(req->id.idiag_dport),
.dif = req->id.idiag_if,
};
@@ -221,6 +222,7 @@ static int __udp_diag_destroy(struct sk_buff *in_skb,
.daddr.ipv6 = (struct in6_addr *)req->id.idiag_src,
.sport = req->id.idiag_dport,
.dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
.dif = req->id.idiag_if,
};
diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c
index b13b8f93079d..878c03094f2e 100644
--- a/net/ipv6/inet6_hashtables.c
+++ b/net/ipv6/inet6_hashtables.c
@@ -52,33 +52,35 @@ u32 inet6_ehashfn(const struct net *net,
*/
struct sock *__inet6_lookup_established(struct net *net,
struct inet_hashinfo *hashinfo,
- const struct in6_addr *saddr,
- const __be16 sport,
- const struct in6_addr *daddr,
- const u16 hnum,
- const int dif)
+ const struct sk_lookup *params)
{
+ const __portpair ports = INET_COMBINED_PORTS(params->sport,
+ params->hnum);
+ const struct in6_addr *saddr = params->saddr.ipv6;
+ const struct in6_addr *daddr = params->daddr.ipv6;
struct sock *sk;
const struct hlist_nulls_node *node;
- const __portpair ports = INET_COMBINED_PORTS(sport, hnum);
+
/* Optimize here for direct hit, only listening connections can
* have wildcards anyways.
*/
- unsigned int hash = inet6_ehashfn(net, daddr, hnum, saddr, sport);
+ unsigned int hash = inet6_ehashfn(net, daddr, params->hnum,
+ saddr, params->sport);
unsigned int slot = hash & hashinfo->ehash_mask;
struct inet_ehash_bucket *head = &hashinfo->ehash[slot];
-
begin:
sk_nulls_for_each_rcu(sk, node, &head->chain) {
if (sk->sk_hash != hash)
continue;
- if (!INET6_MATCH(sk, net, saddr, daddr, ports, dif))
+ if (!INET6_MATCH(sk, net, saddr, daddr, ports,
+ params->dif))
continue;
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
- if (unlikely(!INET6_MATCH(sk, net, saddr, daddr, ports, dif))) {
+ if (unlikely(!INET6_MATCH(sk, net, saddr, daddr, ports,
+ params->dif))) {
sock_gen_put(sk);
goto begin;
}
@@ -94,26 +96,27 @@ struct sock *__inet6_lookup_established(struct net *net,
EXPORT_SYMBOL(__inet6_lookup_established);
static inline int compute_score(struct sock *sk, struct net *net,
- const unsigned short hnum,
- const struct in6_addr *daddr,
- const int dif, bool exact_dif)
+ const struct sk_lookup *params)
{
int score = -1;
- if (net_eq(sock_net(sk), net) && inet_sk(sk)->inet_num == hnum &&
+ if (net_eq(sock_net(sk), net) &&
+ inet_sk(sk)->inet_num == params->hnum &&
sk->sk_family == PF_INET6) {
+ int rc;
score = 1;
if (!ipv6_addr_any(&sk->sk_v6_rcv_saddr)) {
- if (!ipv6_addr_equal(&sk->sk_v6_rcv_saddr, daddr))
+ if (!ipv6_addr_equal(&sk->sk_v6_rcv_saddr,
+ params->daddr.ipv6))
return -1;
score++;
}
- if (sk->sk_bound_dev_if || exact_dif) {
- if (sk->sk_bound_dev_if != dif)
- return -1;
+ rc = sk_lookup_device_cmp(sk, params);
+ if (rc < 0)
+ return -1;
+ if (rc > 0)
score++;
- }
if (sk->sk_incoming_cpu == raw_smp_processor_id())
score++;
}
@@ -122,26 +125,27 @@ static inline int compute_score(struct sock *sk, struct net *net,
/* called with rcu_read_lock() */
struct sock *inet6_lookup_listener(struct net *net,
- struct inet_hashinfo *hashinfo,
- struct sk_buff *skb, int doff,
- const struct in6_addr *saddr,
- const __be16 sport, const struct in6_addr *daddr,
- const unsigned short hnum, const int dif)
+ struct inet_hashinfo *hashinfo,
+ struct sk_buff *skb, int doff,
+ struct sk_lookup *params)
{
- unsigned int hash = inet_lhashfn(net, hnum);
+ unsigned int hash = inet_lhashfn(net, params->hnum);
struct inet_listen_hashbucket *ilb = &hashinfo->listening_hash[hash];
int score, hiscore = 0, matches = 0, reuseport = 0;
- bool exact_dif = inet6_exact_dif_match(net, skb);
struct sock *sk, *result = NULL;
u32 phash = 0;
+ params->exact_dif = inet6_exact_dif_match(net, skb);
+
sk_for_each(sk, &ilb->head) {
- score = compute_score(sk, net, hnum, daddr, dif, exact_dif);
+ score = compute_score(sk, net, params);
if (score > hiscore) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- phash = inet6_ehashfn(net, daddr, hnum,
- saddr, sport);
+ phash = inet6_ehashfn(net, params->daddr.ipv6,
+ params->hnum,
+ params->saddr.ipv6,
+ params->sport);
result = reuseport_select_sock(sk, phash,
skb, doff);
if (result)
@@ -163,15 +167,12 @@ EXPORT_SYMBOL_GPL(inet6_lookup_listener);
struct sock *inet6_lookup(struct net *net, struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const struct in6_addr *saddr, const __be16 sport,
- const struct in6_addr *daddr, const __be16 dport,
- const int dif)
+ struct sk_lookup *params)
{
struct sock *sk;
bool refcounted;
- sk = __inet6_lookup(net, hashinfo, skb, doff, saddr, sport, daddr,
- ntohs(dport), dif, &refcounted);
+ sk = __inet6_lookup(net, hashinfo, skb, doff, params, &refcounted);
if (sk && !refcounted && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
return sk;
@@ -203,7 +204,8 @@ static int __inet6_check_established(struct inet_timewait_death_row *death_row,
if (sk2->sk_hash != hash)
continue;
- if (likely(INET6_MATCH(sk2, net, saddr, daddr, ports, dif))) {
+ if (likely(INET6_MATCH(sk2, net, saddr, daddr, ports,
+ dif))) {
if (sk2->sk_state == TCP_TIME_WAIT) {
tw = inet_twsk(sk2);
if (twsk_unique(sk, sk2, twp))
diff --git a/net/ipv6/netfilter/nf_socket_ipv6.c b/net/ipv6/netfilter/nf_socket_ipv6.c
index 46e45b81094f..2918c9062e1a 100644
--- a/net/ipv6/netfilter/nf_socket_ipv6.c
+++ b/net/ipv6/netfilter/nf_socket_ipv6.c
@@ -91,14 +91,13 @@ nf_socket_get_sock_v6(struct net *net, struct sk_buff *skb, int doff,
.daddr.ipv6 = daddr,
.sport = sport,
.dport = dport,
+ .hnum = ntohs(dport),
.dif = in->ifindex,
};
switch (protocol) {
case IPPROTO_TCP:
- return inet6_lookup(net, &tcp_hashinfo, skb, doff,
- saddr, sport, daddr, dport,
- in->ifindex);
+ return inet6_lookup(net, &tcp_hashinfo, skb, doff, ¶ms);
case IPPROTO_UDP:
return udp6_lib_lookup(net, ¶ms);
}
diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
index 2521690d62d6..154886daba7b 100644
--- a/net/ipv6/tcp_ipv6.c
+++ b/net/ipv6/tcp_ipv6.c
@@ -45,6 +45,7 @@
#include <linux/random.h>
#include <net/tcp.h>
+#include <net/inet_hashtables.h>
#include <net/ndisc.h>
#include <net/inet6_hashtables.h>
#include <net/inet6_connection_sock.h>
@@ -338,6 +339,13 @@ static void tcp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
{
const struct ipv6hdr *hdr = (const struct ipv6hdr *)skb->data;
const struct tcphdr *th = (struct tcphdr *)(skb->data+offset);
+ struct sk_lookup params = {
+ .saddr.ipv6 = &hdr->daddr,
+ .daddr.ipv6 = &hdr->saddr,
+ .sport = th->dest,
+ .hnum = ntohs(th->source),
+ .dif = skb->dev->ifindex,
+ };
struct net *net = dev_net(skb->dev);
struct request_sock *fastopen;
struct ipv6_pinfo *np;
@@ -347,11 +355,7 @@ static void tcp_v6_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
bool fatal;
int err;
- sk = __inet6_lookup_established(net, &tcp_hashinfo,
- &hdr->daddr, th->dest,
- &hdr->saddr, ntohs(th->source),
- skb->dev->ifindex);
-
+ sk = __inet6_lookup_established(net, &tcp_hashinfo, ¶ms);
if (!sk) {
__ICMP6_INC_STATS(net, __in6_dev_get(skb->dev),
ICMP6_MIB_INERRORS);
@@ -907,6 +911,14 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
if (sk && sk_fullsock(sk)) {
key = tcp_v6_md5_do_lookup(sk, &ipv6h->saddr);
} else if (hash_location) {
+ struct sk_lookup params = {
+ .saddr.ipv6 = &ipv6h->saddr,
+ .daddr.ipv6 = &ipv6h->daddr,
+ .sport = th->source,
+ .hnum = ntohs(th->source),
+ .dif = tcp_v6_iif(skb),
+ };
+
/*
* active side is lost. Try to find listening socket through
* source port, and then find md5 key through listening socket.
@@ -915,10 +927,7 @@ static void tcp_v6_send_reset(const struct sock *sk, struct sk_buff *skb)
* no RST generated if md5 hash doesn't match.
*/
sk1 = inet6_lookup_listener(dev_net(skb_dst(skb)->dev),
- &tcp_hashinfo, NULL, 0,
- &ipv6h->saddr,
- th->source, &ipv6h->daddr,
- ntohs(th->source), tcp_v6_iif(skb));
+ &tcp_hashinfo, NULL, 0, ¶ms);
if (!sk1)
goto out;
@@ -1403,6 +1412,9 @@ static int tcp_v6_rcv(struct sk_buff *skb)
struct sock *sk;
int ret;
struct net *net = dev_net(skb->dev);
+ struct sk_lookup params = {
+ .dif = inet6_iif(skb),
+ };
if (skb->pkt_type != PACKET_HOST)
goto discard_it;
@@ -1428,10 +1440,11 @@ static int tcp_v6_rcv(struct sk_buff *skb)
th = (const struct tcphdr *)skb->data;
hdr = ipv6_hdr(skb);
+ params.sport = th->source;
+ params.dport = th->dest;
lookup:
sk = __inet6_lookup_skb(&tcp_hashinfo, skb, __tcp_hdrlen(th),
- th->source, th->dest, inet6_iif(skb),
- &refcounted);
+ ¶ms, &refcounted);
if (!sk)
goto no_tcp_socket;
@@ -1558,13 +1571,17 @@ static int tcp_v6_rcv(struct sk_buff *skb)
switch (tcp_timewait_state_process(inet_twsk(sk), skb, th)) {
case TCP_TW_SYN:
{
+ struct sk_lookup params = {
+ .saddr.ipv6 = &ipv6_hdr(skb)->saddr,
+ .daddr.ipv6 = &ipv6_hdr(skb)->daddr,
+ .sport = th->source,
+ .hnum = ntohs(th->dest),
+ .dif = tcp_v6_iif(skb),
+ };
struct sock *sk2;
sk2 = inet6_lookup_listener(dev_net(skb->dev), &tcp_hashinfo,
- skb, __tcp_hdrlen(th),
- &ipv6_hdr(skb)->saddr, th->source,
- &ipv6_hdr(skb)->daddr,
- ntohs(th->dest), tcp_v6_iif(skb));
+ skb, __tcp_hdrlen(th), ¶ms);
if (sk2) {
struct inet_timewait_sock *tw = inet_twsk(sk);
inet_twsk_deschedule_put(tw);
@@ -1591,6 +1608,10 @@ static int tcp_v6_rcv(struct sk_buff *skb)
static void tcp_v6_early_demux(struct sk_buff *skb)
{
+ /* Note : We use inet6_iif() here, not tcp_v6_iif() */
+ struct sk_lookup params = {
+ .dif = inet6_iif(skb),
+ };
const struct ipv6hdr *hdr;
const struct tcphdr *th;
struct sock *sk;
@@ -1607,11 +1628,12 @@ static void tcp_v6_early_demux(struct sk_buff *skb)
if (th->doff < sizeof(struct tcphdr) / 4)
return;
- /* Note : We use inet6_iif() here, not tcp_v6_iif() */
+ params.saddr.ipv6 = &hdr->saddr,
+ params.daddr.ipv6 = &hdr->daddr,
+ params.sport = th->source,
+ params.hnum = ntohs(th->dest),
sk = __inet6_lookup_established(dev_net(skb->dev), &tcp_hashinfo,
- &hdr->saddr, th->source,
- &hdr->daddr, ntohs(th->dest),
- inet6_iif(skb));
+ ¶ms);
if (sk) {
skb->sk = sk;
skb->destructor = sock_edemux;
diff --git a/net/netfilter/xt_TPROXY.c b/net/netfilter/xt_TPROXY.c
index 25843f741c0b..c031385369c4 100644
--- a/net/netfilter/xt_TPROXY.c
+++ b/net/netfilter/xt_TPROXY.c
@@ -193,6 +193,7 @@ nf_tproxy_get_sock_v6(struct net *net, struct sk_buff *skb, int thoff, void *hp,
.daddr.ipv6 = daddr,
.sport = sport,
.dport = dport,
+ .hnum = ntohs(dport),
.dif = in->ifindex,
};
struct sock *sk;
@@ -205,9 +206,7 @@ nf_tproxy_get_sock_v6(struct net *net, struct sk_buff *skb, int thoff, void *hp,
tcph = hp;
sk = inet6_lookup_listener(net, &tcp_hashinfo, skb,
thoff + __tcp_hdrlen(tcph),
- saddr, sport,
- daddr, ntohs(dport),
- in->ifindex);
+ ¶ms);
if (sk && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
@@ -219,8 +218,7 @@ nf_tproxy_get_sock_v6(struct net *net, struct sk_buff *skb, int thoff, void *hp,
break;
case NFT_LOOKUP_ESTABLISHED:
sk = __inet6_lookup_established(net, &tcp_hashinfo,
- saddr, sport, daddr, ntohs(dport),
- in->ifindex);
+ ¶ms);
break;
default:
BUG();
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 05/10] net: ipv6: Convert udp socket lookups to new struct
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert udp6_lib_lookup and __udp6_lib_lookup to use the new sk_lookup
struct.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/udp.h | 12 +--
net/ipv4/udp_diag.c | 33 ++++---
net/ipv6/netfilter/nf_socket_ipv6.c | 11 ++-
net/ipv6/udp.c | 177 +++++++++++++++++++-----------------
net/netfilter/xt_TPROXY.c | 10 +-
5 files changed, 135 insertions(+), 108 deletions(-)
diff --git a/include/net/udp.h b/include/net/udp.h
index 5e0ff095dc6d..c5a75e9422c6 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -288,15 +288,9 @@ struct sock *__udp4_lib_lookup(struct net *net, struct sk_lookup *params,
struct udp_table *tbl, struct sk_buff *skb);
struct sock *udp4_lib_lookup_skb(struct sk_buff *skb,
__be16 sport, __be16 dport);
-struct sock *udp6_lib_lookup(struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, __be16 dport,
- int dif);
-struct sock *__udp6_lib_lookup(struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, __be16 dport,
- int dif, struct udp_table *tbl,
- struct sk_buff *skb);
+struct sock *udp6_lib_lookup(struct net *net, struct sk_lookup *params);
+struct sock *__udp6_lib_lookup(struct net *net, struct sk_lookup *params,
+ struct udp_table *tbl, struct sk_buff *skb);
struct sock *udp6_lib_lookup_skb(struct sk_buff *skb,
__be16 sport, __be16 dport);
diff --git a/net/ipv4/udp_diag.c b/net/ipv4/udp_diag.c
index d7f6af42ebcc..8c1221f5f2dd 100644
--- a/net/ipv4/udp_diag.c
+++ b/net/ipv4/udp_diag.c
@@ -54,13 +54,17 @@ static int udp_dump_one(struct udp_table *tbl, struct sk_buff *in_skb,
sk = __udp4_lib_lookup(net, ¶ms, tbl, NULL);
}
#if IS_ENABLED(CONFIG_IPV6)
- else if (req->sdiag_family == AF_INET6)
- sk = __udp6_lib_lookup(net,
- (struct in6_addr *)req->id.idiag_src,
- req->id.idiag_sport,
- (struct in6_addr *)req->id.idiag_dst,
- req->id.idiag_dport,
- req->id.idiag_if, tbl, NULL);
+ else if (req->sdiag_family == AF_INET6) {
+ struct sk_lookup params = {
+ .saddr.ipv6 = (struct in6_addr *)req->id.idiag_src,
+ .daddr.ipv6 = (struct in6_addr *)req->id.idiag_dst,
+ .sport = req->id.idiag_sport,
+ .dport = req->id.idiag_dport,
+ .dif = req->id.idiag_if,
+ };
+
+ sk = __udp6_lib_lookup(net, ¶ms, tbl, NULL);
+ }
#endif
if (sk && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
@@ -212,12 +216,15 @@ static int __udp_diag_destroy(struct sk_buff *in_skb,
sk = __udp4_lib_lookup(net, ¶ms, tbl, NULL);
} else {
- sk = __udp6_lib_lookup(net,
- (struct in6_addr *)req->id.idiag_dst,
- req->id.idiag_dport,
- (struct in6_addr *)req->id.idiag_src,
- req->id.idiag_sport,
- req->id.idiag_if, tbl, NULL);
+ struct sk_lookup params = {
+ .saddr.ipv6 = (struct in6_addr *)req->id.idiag_dst,
+ .daddr.ipv6 = (struct in6_addr *)req->id.idiag_src,
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .dif = req->id.idiag_if,
+ };
+
+ sk = __udp6_lib_lookup(net, ¶ms, tbl, NULL);
}
}
#endif
diff --git a/net/ipv6/netfilter/nf_socket_ipv6.c b/net/ipv6/netfilter/nf_socket_ipv6.c
index ebb2bf84232a..46e45b81094f 100644
--- a/net/ipv6/netfilter/nf_socket_ipv6.c
+++ b/net/ipv6/netfilter/nf_socket_ipv6.c
@@ -86,14 +86,21 @@ nf_socket_get_sock_v6(struct net *net, struct sk_buff *skb, int doff,
const __be16 sport, const __be16 dport,
const struct net_device *in)
{
+ struct sk_lookup params = {
+ .saddr.ipv6 = saddr,
+ .daddr.ipv6 = daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = in->ifindex,
+ };
+
switch (protocol) {
case IPPROTO_TCP:
return inet6_lookup(net, &tcp_hashinfo, skb, doff,
saddr, sport, daddr, dport,
in->ifindex);
case IPPROTO_UDP:
- return udp6_lib_lookup(net, saddr, sport, daddr, dport,
- in->ifindex);
+ return udp6_lib_lookup(net, ¶ms);
}
return NULL;
diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c
index 4a3e65626e8b..5c4fdbe52c24 100644
--- a/net/ipv6/udp.c
+++ b/net/ipv6/udp.c
@@ -67,13 +67,14 @@ static bool udp6_lib_exact_dif_match(struct net *net, struct sk_buff *skb)
}
static u32 udp6_ehashfn(const struct net *net,
- const struct in6_addr *laddr,
- const u16 lport,
- const struct in6_addr *faddr,
- const __be16 fport)
+ const struct sk_lookup *params)
{
+ const struct in6_addr *laddr = params->daddr.ipv6;
+ const struct in6_addr *faddr = params->saddr.ipv6;
static u32 udp6_ehash_secret __read_mostly;
static u32 udp_ipv6_hash_secret __read_mostly;
+ const __be16 fport = params->sport;
+ const u16 lport = params->hnum;
u32 lhash, fhash;
@@ -127,15 +128,13 @@ static void udp_v6_rehash(struct sock *sk)
}
static int compute_score(struct sock *sk, struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, unsigned short hnum,
- int dif, bool exact_dif)
+ const struct sk_lookup *params)
{
- int score;
struct inet_sock *inet;
+ int score, rc;
if (!net_eq(sock_net(sk), net) ||
- udp_sk(sk)->udp_port_hash != hnum ||
+ udp_sk(sk)->udp_port_hash != params->hnum ||
sk->sk_family != PF_INET6)
return -1;
@@ -143,28 +142,28 @@ static int compute_score(struct sock *sk, struct net *net,
inet = inet_sk(sk);
if (inet->inet_dport) {
- if (inet->inet_dport != sport)
+ if (inet->inet_dport != params->sport)
return -1;
score++;
}
if (!ipv6_addr_any(&sk->sk_v6_rcv_saddr)) {
- if (!ipv6_addr_equal(&sk->sk_v6_rcv_saddr, daddr))
+ if (!ipv6_addr_equal(&sk->sk_v6_rcv_saddr, params->daddr.ipv6))
return -1;
score++;
}
if (!ipv6_addr_any(&sk->sk_v6_daddr)) {
- if (!ipv6_addr_equal(&sk->sk_v6_daddr, saddr))
+ if (!ipv6_addr_equal(&sk->sk_v6_daddr, params->saddr.ipv6))
return -1;
score++;
}
- if (sk->sk_bound_dev_if || exact_dif) {
- if (sk->sk_bound_dev_if != dif)
- return -1;
+ rc = sk_lookup_device_cmp(sk, params);
+ if (rc < 0)
+ return -1;
+ if (rc > 0)
score++;
- }
if (sk->sk_incoming_cpu == raw_smp_processor_id())
score++;
@@ -174,10 +173,9 @@ static int compute_score(struct sock *sk, struct net *net,
/* called with rcu_read_lock() */
static struct sock *udp6_lib_lookup2(struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, unsigned int hnum, int dif,
- bool exact_dif, struct udp_hslot *hslot2,
- struct sk_buff *skb)
+ const struct sk_lookup *params,
+ struct udp_hslot *hslot2,
+ struct sk_buff *skb)
{
struct sock *sk, *result;
int score, badness, matches = 0, reuseport = 0;
@@ -186,13 +184,11 @@ static struct sock *udp6_lib_lookup2(struct net *net,
result = NULL;
badness = -1;
udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) {
- score = compute_score(sk, net, saddr, sport,
- daddr, hnum, dif, exact_dif);
+ score = compute_score(sk, net, params);
if (score > badness) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- hash = udp6_ehashfn(net, daddr, hnum,
- saddr, sport);
+ hash = udp6_ehashfn(net, params);
result = reuseport_select_sock(sk, hash, skb,
sizeof(struct udphdr));
@@ -213,30 +209,27 @@ static struct sock *udp6_lib_lookup2(struct net *net,
}
/* rcu_read_lock() must be held */
-struct sock *__udp6_lib_lookup(struct net *net,
- const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, __be16 dport,
- int dif, struct udp_table *udptable,
- struct sk_buff *skb)
+struct sock *__udp6_lib_lookup(struct net *net, struct sk_lookup *params,
+ struct udp_table *udptable, struct sk_buff *skb)
{
struct sock *sk, *result;
- unsigned short hnum = ntohs(dport);
+ unsigned short hnum = ntohs(params->dport);
unsigned int hash2, slot2, slot = udp_hashfn(net, hnum, udptable->mask);
struct udp_hslot *hslot2, *hslot = &udptable->hash[slot];
- bool exact_dif = udp6_lib_exact_dif_match(net, skb);
int score, badness, matches = 0, reuseport = 0;
u32 hash = 0;
+ params->hnum = hnum;
+ params->exact_dif = udp6_lib_exact_dif_match(net, skb);
+
if (hslot->count > 10) {
- hash2 = udp6_portaddr_hash(net, daddr, hnum);
+ hash2 = udp6_portaddr_hash(net, params->daddr.ipv6, hnum);
slot2 = hash2 & udptable->mask;
hslot2 = &udptable->hash2[slot2];
if (hslot->count < hslot2->count)
goto begin;
- result = udp6_lib_lookup2(net, saddr, sport,
- daddr, hnum, dif, exact_dif,
- hslot2, skb);
+ result = udp6_lib_lookup2(net, params, hslot2, skb);
if (!result) {
unsigned int old_slot2 = slot2;
hash2 = udp6_portaddr_hash(net, &in6addr_any, hnum);
@@ -249,10 +242,7 @@ struct sock *__udp6_lib_lookup(struct net *net,
if (hslot->count < hslot2->count)
goto begin;
- result = udp6_lib_lookup2(net, saddr, sport,
- daddr, hnum, dif,
- exact_dif, hslot2,
- skb);
+ result = udp6_lib_lookup2(net, params, hslot2, skb);
}
return result;
}
@@ -260,13 +250,11 @@ struct sock *__udp6_lib_lookup(struct net *net,
result = NULL;
badness = -1;
sk_for_each_rcu(sk, &hslot->head) {
- score = compute_score(sk, net, saddr, sport, daddr, hnum, dif,
- exact_dif);
+ score = compute_score(sk, net, params);
if (score > badness) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- hash = udp6_ehashfn(net, daddr, hnum,
- saddr, sport);
+ hash = udp6_ehashfn(net, params);
result = reuseport_select_sock(sk, hash, skb,
sizeof(struct udphdr));
if (result)
@@ -292,23 +280,34 @@ static struct sock *__udp6_lib_lookup_skb(struct sk_buff *skb,
{
const struct ipv6hdr *iph = ipv6_hdr(skb);
struct sock *sk;
+ struct sk_lookup params = {
+ .saddr.ipv6 = &iph->saddr,
+ .daddr.ipv6 = &iph->daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = inet6_iif(skb),
+ };
sk = skb_steal_sock(skb);
if (unlikely(sk))
return sk;
- return __udp6_lib_lookup(dev_net(skb->dev), &iph->saddr, sport,
- &iph->daddr, dport, inet6_iif(skb),
- udptable, skb);
+
+ return __udp6_lib_lookup(dev_net(skb->dev), ¶ms, udptable, skb);
}
struct sock *udp6_lib_lookup_skb(struct sk_buff *skb,
__be16 sport, __be16 dport)
{
const struct ipv6hdr *iph = ipv6_hdr(skb);
-
- return __udp6_lib_lookup(dev_net(skb->dev), &iph->saddr, sport,
- &iph->daddr, dport, inet6_iif(skb),
- &udp_table, skb);
+ struct sk_lookup params = {
+ .saddr.ipv6 = &iph->saddr,
+ .daddr.ipv6 = &iph->daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = inet6_iif(skb),
+ };
+
+ return __udp6_lib_lookup(dev_net(skb->dev), ¶ms, &udp_table, skb);
}
EXPORT_SYMBOL_GPL(udp6_lib_lookup_skb);
@@ -318,13 +317,11 @@ EXPORT_SYMBOL_GPL(udp6_lib_lookup_skb);
#if IS_ENABLED(CONFIG_NETFILTER_XT_MATCH_SOCKET) || \
IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TPROXY) || \
IS_ENABLED(CONFIG_NF_SOCKET_IPV6)
-struct sock *udp6_lib_lookup(struct net *net, const struct in6_addr *saddr, __be16 sport,
- const struct in6_addr *daddr, __be16 dport, int dif)
+struct sock *udp6_lib_lookup(struct net *net, struct sk_lookup *params)
{
struct sock *sk;
- sk = __udp6_lib_lookup(net, saddr, sport, daddr, dport,
- dif, &udp_table, NULL);
+ sk = __udp6_lib_lookup(net, params, &udp_table, NULL);
if (sk && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
return sk;
@@ -487,16 +484,20 @@ void __udp6_lib_err(struct sk_buff *skb, struct inet6_skb_parm *opt,
{
struct ipv6_pinfo *np;
const struct ipv6hdr *hdr = (const struct ipv6hdr *)skb->data;
- const struct in6_addr *saddr = &hdr->saddr;
- const struct in6_addr *daddr = &hdr->daddr;
struct udphdr *uh = (struct udphdr *)(skb->data+offset);
+ struct sk_lookup params = {
+ .saddr.ipv6 = &hdr->daddr,
+ .daddr.ipv6 = &hdr->saddr,
+ .sport = uh->dest,
+ .dport = uh->source,
+ .dif = inet6_iif(skb),
+ };
struct sock *sk;
int harderr;
int err;
struct net *net = dev_net(skb->dev);
- sk = __udp6_lib_lookup(net, daddr, uh->dest, saddr, uh->source,
- inet6_iif(skb), udptable, skb);
+ sk = __udp6_lib_lookup(net, ¶ms, udptable, skb);
if (!sk) {
__ICMP6_INC_STATS(net, __in6_dev_get(skb->dev),
ICMP6_MIB_INERRORS);
@@ -658,21 +659,21 @@ static int udpv6_queue_rcv_skb(struct sock *sk, struct sk_buff *skb)
}
static bool __udp_v6_is_mcast_sock(struct net *net, struct sock *sk,
- __be16 loc_port, const struct in6_addr *loc_addr,
- __be16 rmt_port, const struct in6_addr *rmt_addr,
- int dif, unsigned short hnum)
+ struct sk_lookup *params)
{
+ const struct in6_addr *loc_addr = params->daddr.ipv6;
+ const struct in6_addr *rmt_addr = params->saddr.ipv6;
struct inet_sock *inet = inet_sk(sk);
if (!net_eq(sock_net(sk), net))
return false;
- if (udp_sk(sk)->udp_port_hash != hnum ||
+ if (udp_sk(sk)->udp_port_hash != params->hnum ||
sk->sk_family != PF_INET6 ||
- (inet->inet_dport && inet->inet_dport != rmt_port) ||
+ (inet->inet_dport && inet->inet_dport != params->sport) ||
(!ipv6_addr_any(&sk->sk_v6_daddr) &&
!ipv6_addr_equal(&sk->sk_v6_daddr, rmt_addr)) ||
- (sk->sk_bound_dev_if && sk->sk_bound_dev_if != dif) ||
+ (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif) ||
(!ipv6_addr_any(&sk->sk_v6_rcv_saddr) &&
!ipv6_addr_equal(&sk->sk_v6_rcv_saddr, loc_addr)))
return false;
@@ -705,9 +706,16 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
struct udp_hslot *hslot = udp_hashslot(udptable, net, hnum);
unsigned int offset = offsetof(typeof(*sk), sk_node);
unsigned int hash2 = 0, hash2_any = 0, use_hash2 = (hslot->count > 10);
- int dif = inet6_iif(skb);
struct hlist_node *node;
struct sk_buff *nskb;
+ struct sk_lookup params = {
+ .saddr.ipv6 = saddr,
+ .daddr.ipv6 = daddr,
+ .sport = uh->source,
+ .dport = uh->dest,
+ .hnum = hnum,
+ .dif = inet6_iif(skb),
+ };
if (use_hash2) {
hash2_any = udp6_portaddr_hash(net, &in6addr_any, hnum) &
@@ -719,8 +727,7 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
}
sk_for_each_entry_offset_rcu(sk, node, &hslot->head, offset) {
- if (!__udp_v6_is_mcast_sock(net, sk, uh->dest, daddr,
- uh->source, saddr, dif, hnum))
+ if (!__udp_v6_is_mcast_sock(net, sk, ¶ms))
continue;
/* If zero checksum and no_check is not on for
* the socket then skip it.
@@ -873,21 +880,22 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
static struct sock *__udp6_lib_demux_lookup(struct net *net,
- __be16 loc_port, const struct in6_addr *loc_addr,
- __be16 rmt_port, const struct in6_addr *rmt_addr,
- int dif)
+ const struct sk_lookup *params)
{
- unsigned short hnum = ntohs(loc_port);
- unsigned int hash2 = udp6_portaddr_hash(net, loc_addr, hnum);
+ unsigned short hnum = params->hnum;
+ unsigned int hash2 = udp6_portaddr_hash(net, params->daddr.ipv6, hnum);
unsigned int slot2 = hash2 & udp_table.mask;
struct udp_hslot *hslot2 = &udp_table.hash2[slot2];
- const __portpair ports = INET_COMBINED_PORTS(rmt_port, hnum);
+ const __portpair ports = INET_COMBINED_PORTS(params->sport, hnum);
struct sock *sk;
udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) {
if (sk->sk_state == TCP_ESTABLISHED &&
- INET6_MATCH(sk, net, rmt_addr, loc_addr, ports, dif))
+ INET6_MATCH(sk, net, params->saddr.ipv6,
+ params->daddr.ipv6, ports,
+ params->dif))
return sk;
+
/* Only check first socket in chain */
break;
}
@@ -900,7 +908,12 @@ static void udp_v6_early_demux(struct sk_buff *skb)
const struct udphdr *uh;
struct sock *sk;
struct dst_entry *dst;
- int dif = skb->dev->ifindex;
+ struct sk_lookup params = {
+ .dif = skb->dev->ifindex,
+ };
+
+ if (skb->pkt_type != PACKET_HOST)
+ return;
if (!pskb_may_pull(skb, skb_transport_offset(skb) +
sizeof(struct udphdr)))
@@ -908,13 +921,13 @@ static void udp_v6_early_demux(struct sk_buff *skb)
uh = udp_hdr(skb);
- if (skb->pkt_type == PACKET_HOST)
- sk = __udp6_lib_demux_lookup(net, uh->dest,
- &ipv6_hdr(skb)->daddr,
- uh->source, &ipv6_hdr(skb)->saddr,
- dif);
- else
- return;
+ params.daddr.ipv6 = &ipv6_hdr(skb)->daddr;
+ params.dport = uh->dest;
+ params.hnum = ntohs(uh->dest);
+ params.saddr.ipv6 = &ipv6_hdr(skb)->saddr;
+ params.sport = uh->source;
+
+ sk = __udp6_lib_demux_lookup(net, ¶ms);
if (!sk || !refcount_inc_not_zero(&sk->sk_refcnt))
return;
diff --git a/net/netfilter/xt_TPROXY.c b/net/netfilter/xt_TPROXY.c
index 5cce7eb7dea2..25843f741c0b 100644
--- a/net/netfilter/xt_TPROXY.c
+++ b/net/netfilter/xt_TPROXY.c
@@ -188,6 +188,13 @@ nf_tproxy_get_sock_v6(struct net *net, struct sk_buff *skb, int thoff, void *hp,
const struct net_device *in,
const enum nf_tproxy_lookup_t lookup_type)
{
+ struct sk_lookup params = {
+ .saddr.ipv6 = saddr,
+ .daddr.ipv6 = daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = in->ifindex,
+ };
struct sock *sk;
struct tcphdr *tcph;
@@ -220,8 +227,7 @@ nf_tproxy_get_sock_v6(struct net *net, struct sk_buff *skb, int thoff, void *hp,
}
break;
case IPPROTO_UDP:
- sk = udp6_lib_lookup(net, saddr, sport, daddr, dport,
- in->ifindex);
+ sk = udp6_lib_lookup(net, ¶ms);
if (sk) {
int connected = (sk->sk_state == TCP_ESTABLISHED);
int wildcard = ipv6_addr_any(&sk->sk_v6_rcv_saddr);
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 04/10] net: ipv4: Convert raw sockets to sk_lookup
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert __raw_v4_lookup to use the new sk_lookup struct
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/raw.h | 3 +--
net/ipv4/raw.c | 72 ++++++++++++++++++++++++++++++++++-------------------
net/ipv4/raw_diag.c | 15 +++++++----
3 files changed, 58 insertions(+), 32 deletions(-)
diff --git a/include/net/raw.h b/include/net/raw.h
index 57c33dd22ec4..8d0f0e5d013b 100644
--- a/include/net/raw.h
+++ b/include/net/raw.h
@@ -25,8 +25,7 @@ extern struct proto raw_prot;
extern struct raw_hashinfo raw_v4_hashinfo;
struct sock *__raw_v4_lookup(struct net *net, struct sock *sk,
- unsigned short num, __be32 raddr,
- __be32 laddr, int dif);
+ const struct sk_lookup *params);
int raw_abort(struct sock *sk, int err);
void raw_icmp_error(struct sk_buff *, int, u32);
diff --git a/net/ipv4/raw.c b/net/ipv4/raw.c
index b0bb5d0a30bd..4da5d87a61a5 100644
--- a/net/ipv4/raw.c
+++ b/net/ipv4/raw.c
@@ -122,15 +122,23 @@ void raw_unhash_sk(struct sock *sk)
EXPORT_SYMBOL_GPL(raw_unhash_sk);
struct sock *__raw_v4_lookup(struct net *net, struct sock *sk,
- unsigned short num, __be32 raddr, __be32 laddr, int dif)
+ const struct sk_lookup *params)
{
+ __be32 raddr = params->saddr.ipv4;
+ __be32 laddr = params->daddr.ipv4;
+
sk_for_each_from(sk) {
struct inet_sock *inet = inet_sk(sk);
+ bool dev_match;
+
+ dev_match = (!sk->sk_bound_dev_if ||
+ sk->sk_bound_dev_if == params->dif);
- if (net_eq(sock_net(sk), net) && inet->inet_num == num &&
- !(inet->inet_daddr && inet->inet_daddr != raddr) &&
+ if (net_eq(sock_net(sk), net) &&
+ inet->inet_num == params->hnum &&
+ !(inet->inet_daddr && inet->inet_daddr != raddr) &&
!(inet->inet_rcv_saddr && inet->inet_rcv_saddr != laddr) &&
- !(sk->sk_bound_dev_if && sk->sk_bound_dev_if != dif))
+ dev_match)
goto found; /* gotcha */
}
sk = NULL;
@@ -169,23 +177,20 @@ static int icmp_filter(const struct sock *sk, const struct sk_buff *skb)
* RFC 1122: SHOULD pass TOS value up to the transport layer.
* -> It does. And not only TOS, but all IP header.
*/
-static int raw_v4_input(struct sk_buff *skb, const struct iphdr *iph, int hash)
+static int __raw_v4_input(struct sk_buff *skb, const struct iphdr *iph,
+ struct hlist_head *head)
{
- struct sock *sk;
- struct hlist_head *head;
+ struct net *net = dev_net(skb->dev);
+ const struct sk_lookup params = {
+ .saddr.ipv4 = iph->saddr,
+ .daddr.ipv4 = iph->daddr,
+ .hnum = iph->protocol,
+ .dif = skb->dev->ifindex,
+ };
int delivered = 0;
- struct net *net;
-
- read_lock(&raw_v4_hashinfo.lock);
- head = &raw_v4_hashinfo.ht[hash];
- if (hlist_empty(head))
- goto out;
-
- net = dev_net(skb->dev);
- sk = __raw_v4_lookup(net, __sk_head(head), iph->protocol,
- iph->saddr, iph->daddr,
- skb->dev->ifindex);
+ struct sock *sk;
+ sk = __raw_v4_lookup(net, __sk_head(head), ¶ms);
while (sk) {
delivered = 1;
if ((iph->protocol != IPPROTO_ICMP || !icmp_filter(sk, skb)) &&
@@ -197,11 +202,22 @@ static int raw_v4_input(struct sk_buff *skb, const struct iphdr *iph, int hash)
if (clone)
raw_rcv(sk, clone);
}
- sk = __raw_v4_lookup(net, sk_next(sk), iph->protocol,
- iph->saddr, iph->daddr,
- skb->dev->ifindex);
+ sk = __raw_v4_lookup(net, sk_next(sk), ¶ms);
}
-out:
+
+ return delivered;
+}
+
+static int raw_v4_input(struct sk_buff *skb, const struct iphdr *iph, int hash)
+{
+ struct hlist_head *head;
+ int delivered = 0;
+
+ read_lock(&raw_v4_hashinfo.lock);
+ head = &raw_v4_hashinfo.ht[hash];
+ if (!hlist_empty(head))
+ delivered = __raw_v4_input(skb, iph, head);
+
read_unlock(&raw_v4_hashinfo.lock);
return delivered;
}
@@ -297,12 +313,18 @@ void raw_icmp_error(struct sk_buff *skb, int protocol, u32 info)
read_lock(&raw_v4_hashinfo.lock);
raw_sk = sk_head(&raw_v4_hashinfo.ht[hash]);
if (raw_sk) {
+ struct sk_lookup params = {
+ .hnum = protocol,
+ .dif = skb->dev->ifindex,
+ };
+
iph = (const struct iphdr *)skb->data;
net = dev_net(skb->dev);
- while ((raw_sk = __raw_v4_lookup(net, raw_sk, protocol,
- iph->daddr, iph->saddr,
- skb->dev->ifindex)) != NULL) {
+ params.saddr.ipv4 = iph->daddr;
+ params.daddr.ipv4 = iph->saddr;
+ while ((raw_sk = __raw_v4_lookup(net, raw_sk,
+ ¶ms)) != NULL) {
raw_err(raw_sk, skb, info);
raw_sk = sk_next(raw_sk);
iph = (const struct iphdr *)skb->data;
diff --git a/net/ipv4/raw_diag.c b/net/ipv4/raw_diag.c
index e1a51ca68d23..a708de070cc6 100644
--- a/net/ipv4/raw_diag.c
+++ b/net/ipv4/raw_diag.c
@@ -42,11 +42,16 @@ static struct sock *raw_lookup(struct net *net, struct sock *from,
struct inet_diag_req_raw *r = (void *)req;
struct sock *sk = NULL;
- if (r->sdiag_family == AF_INET)
- sk = __raw_v4_lookup(net, from, r->sdiag_raw_protocol,
- r->id.idiag_dst[0],
- r->id.idiag_src[0],
- r->id.idiag_if);
+ if (r->sdiag_family == AF_INET) {
+ const struct sk_lookup params = {
+ .saddr.ipv4 = r->id.idiag_dst[0],
+ .daddr.ipv4 = r->id.idiag_src[0],
+ .hnum = r->sdiag_raw_protocol,
+ .dif = r->id.idiag_if,
+ };
+
+ sk = __raw_v4_lookup(net, from, ¶ms);
+ }
#if IS_ENABLED(CONFIG_IPV6)
else
sk = __raw_v6_lookup(net, from, r->sdiag_raw_protocol,
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 03/10] net: ipv4: Convert inet socket lookups to new struct
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert the various inet_lookup functions to use the new sk_lookup
struct.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/inet_hashtables.h | 57 ++++++++++++++--------------------
net/dccp/ipv4.c | 19 +++++++++---
net/ipv4/inet_diag.c | 33 ++++++++++++++------
net/ipv4/inet_hashtables.c | 54 ++++++++++++++++++--------------
net/ipv4/netfilter/nf_socket_ipv4.c | 5 ++-
net/ipv4/tcp_ipv4.c | 62 +++++++++++++++++++++++++++----------
net/ipv4/udp_diag.c | 3 ++
net/netfilter/xt_TPROXY.c | 10 +++---
8 files changed, 145 insertions(+), 98 deletions(-)
diff --git a/include/net/inet_hashtables.h b/include/net/inet_hashtables.h
index 5026b1f08bb8..fabb8dd8fdb1 100644
--- a/include/net/inet_hashtables.h
+++ b/include/net/inet_hashtables.h
@@ -218,19 +218,16 @@ void inet_unhash(struct sock *sk);
struct sock *__inet_lookup_listener(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr,
- const unsigned short hnum,
- const int dif);
+ struct sk_lookup *params);
static inline struct sock *inet_lookup_listener(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- __be32 saddr, __be16 sport,
- __be32 daddr, __be16 dport, int dif)
+ struct sk_lookup *params)
{
- return __inet_lookup_listener(net, hashinfo, skb, doff, saddr, sport,
- daddr, ntohs(dport), dif);
+ params->hnum = ntohs(params->dport);
+
+ return __inet_lookup_listener(net, hashinfo, skb, doff, params);
}
/* Socket demux engine toys. */
@@ -286,53 +283,44 @@ static inline struct sock *inet_lookup_listener(struct net *net,
*/
struct sock *__inet_lookup_established(struct net *net,
struct inet_hashinfo *hashinfo,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr, const u16 hnum,
- const int dif);
+ const struct sk_lookup *params);
static inline struct sock *
inet_lookup_established(struct net *net, struct inet_hashinfo *hashinfo,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr, const __be16 dport,
- const int dif)
+ struct sk_lookup *params)
{
- return __inet_lookup_established(net, hashinfo, saddr, sport, daddr,
- ntohs(dport), dif);
+ params->hnum = ntohs(params->dport);
+
+ return __inet_lookup_established(net, hashinfo, params);
}
static inline struct sock *__inet_lookup(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr, const __be16 dport,
- const int dif,
+ struct sk_lookup *params,
bool *refcounted)
{
- u16 hnum = ntohs(dport);
struct sock *sk;
- sk = __inet_lookup_established(net, hashinfo, saddr, sport,
- daddr, hnum, dif);
+ params->hnum = ntohs(params->dport);
+
+ sk = __inet_lookup_established(net, hashinfo, params);
*refcounted = true;
if (sk)
return sk;
*refcounted = false;
- return __inet_lookup_listener(net, hashinfo, skb, doff, saddr,
- sport, daddr, hnum, dif);
+ return __inet_lookup_listener(net, hashinfo, skb, doff, params);
}
static inline struct sock *inet_lookup(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr, const __be16 dport,
- const int dif)
+ struct sk_lookup *params)
{
struct sock *sk;
bool refcounted;
- sk = __inet_lookup(net, hashinfo, skb, doff, saddr, sport, daddr,
- dport, dif, &refcounted);
+ sk = __inet_lookup(net, hashinfo, skb, doff, params, &refcounted);
if (sk && !refcounted && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
@@ -342,21 +330,22 @@ static inline struct sock *inet_lookup(struct net *net,
static inline struct sock *__inet_lookup_skb(struct inet_hashinfo *hashinfo,
struct sk_buff *skb,
int doff,
- const __be16 sport,
- const __be16 dport,
+ struct sk_lookup *params,
bool *refcounted)
{
struct sock *sk = skb_steal_sock(skb);
const struct iphdr *iph = ip_hdr(skb);
+ params->dif = inet_iif(skb),
+ params->saddr.ipv4 = iph->saddr,
+ params->daddr.ipv4 = iph->daddr,
+
*refcounted = true;
if (sk)
return sk;
return __inet_lookup(dev_net(skb_dst(skb)->dev), hashinfo, skb,
- doff, iph->saddr, sport,
- iph->daddr, dport, inet_iif(skb),
- refcounted);
+ doff, params, refcounted);
}
u32 inet6_ehashfn(const struct net *net,
diff --git a/net/dccp/ipv4.c b/net/dccp/ipv4.c
index f85d901f4e3f..f98a65fa5f5e 100644
--- a/net/dccp/ipv4.c
+++ b/net/dccp/ipv4.c
@@ -244,6 +244,11 @@ static void dccp_v4_err(struct sk_buff *skb, u32 info)
__u64 seq;
int err;
struct net *net = dev_net(skb->dev);
+ struct sk_lookup params = {
+ .saddr.ipv4 = iph->daddr,
+ .daddr.ipv4 = iph->saddr,
+ .dif = inet_iif(skb),
+ };
/* Only need dccph_dport & dccph_sport which are the first
* 4 bytes in dccp header.
@@ -253,10 +258,11 @@ static void dccp_v4_err(struct sk_buff *skb, u32 info)
BUILD_BUG_ON(offsetofend(struct dccp_hdr, dccph_dport) > 8);
dh = (struct dccp_hdr *)(skb->data + offset);
- sk = __inet_lookup_established(net, &dccp_hashinfo,
- iph->daddr, dh->dccph_dport,
- iph->saddr, ntohs(dh->dccph_sport),
- inet_iif(skb));
+ params.sport = dh->dccph_dport;
+ params.dport = dh->dccph_sport;
+ params.hnum = ntohs(dh->dccph_sport);
+
+ sk = inet_lookup_established(net, &dccp_hashinfo, ¶ms);
if (!sk) {
__ICMP_INC_STATS(net, ICMP_MIB_INERRORS);
return;
@@ -763,6 +769,7 @@ EXPORT_SYMBOL_GPL(dccp_invalid_packet);
/* this is called when real data arrives */
static int dccp_v4_rcv(struct sk_buff *skb)
{
+ struct sk_lookup params = {};
const struct dccp_hdr *dh;
const struct iphdr *iph;
bool refcounted;
@@ -801,9 +808,11 @@ static int dccp_v4_rcv(struct sk_buff *skb)
DCCP_SKB_CB(skb)->dccpd_ack_seq);
}
+ params.sport = dh->dccph_sport;
+ params.dport = dh->dccph_dport;
lookup:
sk = __inet_lookup_skb(&dccp_hashinfo, skb, __dccp_hdr_len(dh),
- dh->dccph_sport, dh->dccph_dport, &refcounted);
+ ¶ms, &refcounted);
if (!sk) {
dccp_pr_debug("failed to look up flow ID in table and "
"get corresponding socket\n");
diff --git a/net/ipv4/inet_diag.c b/net/ipv4/inet_diag.c
index 3828b3a805cd..6c3bc4e408d0 100644
--- a/net/ipv4/inet_diag.c
+++ b/net/ipv4/inet_diag.c
@@ -396,18 +396,33 @@ struct sock *inet_diag_find_one_icsk(struct net *net,
struct sock *sk;
rcu_read_lock();
- if (req->sdiag_family == AF_INET)
- sk = inet_lookup(net, hashinfo, NULL, 0, req->id.idiag_dst[0],
- req->id.idiag_dport, req->id.idiag_src[0],
- req->id.idiag_sport, req->id.idiag_if);
+ if (req->sdiag_family == AF_INET) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = req->id.idiag_dst[0],
+ .daddr.ipv4 = req->id.idiag_src[0],
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
+ .dif = req->id.idiag_if,
+ };
+
+ sk = inet_lookup(net, hashinfo, NULL, 0, ¶ms);
+ }
#if IS_ENABLED(CONFIG_IPV6)
else if (req->sdiag_family == AF_INET6) {
if (ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_dst) &&
- ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_src))
- sk = inet_lookup(net, hashinfo, NULL, 0, req->id.idiag_dst[3],
- req->id.idiag_dport, req->id.idiag_src[3],
- req->id.idiag_sport, req->id.idiag_if);
- else
+ ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_src)) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = req->id.idiag_dst[3],
+ .daddr.ipv4 = req->id.idiag_src[3],
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
+ .dif = req->id.idiag_if,
+ };
+
+ sk = inet_lookup(net, hashinfo, NULL, 0, ¶ms);
+ } else
sk = inet6_lookup(net, hashinfo, NULL, 0,
(struct in6_addr *)req->id.idiag_dst,
req->id.idiag_dport,
diff --git a/net/ipv4/inet_hashtables.c b/net/ipv4/inet_hashtables.c
index 2e3389d614d1..e581e200d01d 100644
--- a/net/ipv4/inet_hashtables.c
+++ b/net/ipv4/inet_hashtables.c
@@ -169,26 +169,28 @@ int __inet_inherit_port(const struct sock *sk, struct sock *child)
EXPORT_SYMBOL_GPL(__inet_inherit_port);
static inline int compute_score(struct sock *sk, struct net *net,
- const unsigned short hnum, const __be32 daddr,
- const int dif, bool exact_dif)
+ const struct sk_lookup *params)
{
int score = -1;
struct inet_sock *inet = inet_sk(sk);
- if (net_eq(sock_net(sk), net) && inet->inet_num == hnum &&
- !ipv6_only_sock(sk)) {
+ if (net_eq(sock_net(sk), net) &&
+ inet->inet_num == params->hnum &&
+ !ipv6_only_sock(sk)) {
__be32 rcv_saddr = inet->inet_rcv_saddr;
+ int rc;
+
score = sk->sk_family == PF_INET ? 2 : 1;
if (rcv_saddr) {
- if (rcv_saddr != daddr)
+ if (rcv_saddr != params->daddr.ipv4)
return -1;
score += 4;
}
- if (sk->sk_bound_dev_if || exact_dif) {
- if (sk->sk_bound_dev_if != dif)
- return -1;
+ rc = sk_lookup_device_cmp(sk, params);
+ if (rc < 0)
+ return -1;
+ if (rc > 0)
score += 4;
- }
if (sk->sk_incoming_cpu == raw_smp_processor_id())
score++;
}
@@ -206,24 +208,25 @@ static inline int compute_score(struct sock *sk, struct net *net,
struct sock *__inet_lookup_listener(struct net *net,
struct inet_hashinfo *hashinfo,
struct sk_buff *skb, int doff,
- const __be32 saddr, __be16 sport,
- const __be32 daddr, const unsigned short hnum,
- const int dif)
+ struct sk_lookup *params)
{
- unsigned int hash = inet_lhashfn(net, hnum);
+ unsigned int hash = inet_lhashfn(net, params->hnum);
struct inet_listen_hashbucket *ilb = &hashinfo->listening_hash[hash];
int score, hiscore = 0, matches = 0, reuseport = 0;
- bool exact_dif = inet_exact_dif_match(net, skb);
struct sock *sk, *result = NULL;
u32 phash = 0;
+ params->exact_dif = inet_exact_dif_match(net, skb);
+
sk_for_each_rcu(sk, &ilb->head) {
- score = compute_score(sk, net, hnum, daddr, dif, exact_dif);
+ score = compute_score(sk, net, params);
if (score > hiscore) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- phash = inet_ehashfn(net, daddr, hnum,
- saddr, sport);
+ phash = inet_ehashfn(net, params->daddr.ipv4,
+ params->hnum,
+ params->saddr.ipv4,
+ params->sport);
result = reuseport_select_sock(sk, phash,
skb, doff);
if (result)
@@ -265,11 +268,13 @@ void sock_edemux(struct sk_buff *skb)
EXPORT_SYMBOL(sock_edemux);
struct sock *__inet_lookup_established(struct net *net,
- struct inet_hashinfo *hashinfo,
- const __be32 saddr, const __be16 sport,
- const __be32 daddr, const u16 hnum,
- const int dif)
+ struct inet_hashinfo *hashinfo,
+ const struct sk_lookup *params)
{
+ const __be32 saddr = params->saddr.ipv4;
+ const __be32 daddr = params->daddr.ipv4;
+ const __be16 sport = params->sport;
+ const u16 hnum = params->hnum;
INET_ADDR_COOKIE(acookie, saddr, daddr);
const __portpair ports = INET_COMBINED_PORTS(sport, hnum);
struct sock *sk;
@@ -285,12 +290,13 @@ struct sock *__inet_lookup_established(struct net *net,
sk_nulls_for_each_rcu(sk, node, &head->chain) {
if (sk->sk_hash != hash)
continue;
- if (likely(INET_MATCH(sk, net, acookie,
- saddr, daddr, ports, dif))) {
+ if (likely(INET_MATCH(sk, net, acookie, saddr, daddr,
+ ports, params->dif))) {
if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt)))
goto out;
if (unlikely(!INET_MATCH(sk, net, acookie,
- saddr, daddr, ports, dif))) {
+ saddr, daddr, ports,
+ params->dif))) {
sock_gen_put(sk);
goto begin;
}
diff --git a/net/ipv4/netfilter/nf_socket_ipv4.c b/net/ipv4/netfilter/nf_socket_ipv4.c
index 121767b36763..b0f9954712f9 100644
--- a/net/ipv4/netfilter/nf_socket_ipv4.c
+++ b/net/ipv4/netfilter/nf_socket_ipv4.c
@@ -86,14 +86,13 @@ nf_socket_get_sock_v4(struct net *net, struct sk_buff *skb, const int doff,
.daddr.ipv4 = daddr,
.sport = sport,
.dport = dport,
+ .hnum = ntohs(dport),
.dif = in->ifindex,
};
switch (protocol) {
case IPPROTO_TCP:
- return inet_lookup(net, &tcp_hashinfo, skb, doff,
- saddr, sport, daddr, dport,
- in->ifindex);
+ return inet_lookup(net, &tcp_hashinfo, skb, doff, ¶ms);
case IPPROTO_UDP:
return udp4_lib_lookup(net, ¶ms);
}
diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index a20e7f03d5f7..89a0d166e677 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -382,10 +382,16 @@ void tcp_v4_err(struct sk_buff *icmp_skb, u32 info)
u32 delta_us;
int err;
struct net *net = dev_net(icmp_skb->dev);
-
- sk = __inet_lookup_established(net, &tcp_hashinfo, iph->daddr,
- th->dest, iph->saddr, ntohs(th->source),
- inet_iif(icmp_skb));
+ struct sk_lookup params = {
+ .daddr.ipv4 = iph->saddr,
+ .saddr.ipv4 = iph->daddr,
+ .sport = th->dest,
+ .dport = th->source,
+ .hnum = ntohs(th->source),
+ .dif = inet_iif(icmp_skb),
+ };
+
+ sk = inet_lookup_established(net, &tcp_hashinfo, ¶ms);
if (!sk) {
__ICMP_INC_STATS(net, ICMP_MIB_INERRORS);
return;
@@ -651,6 +657,14 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
key = tcp_md5_do_lookup(sk, (union tcp_md5_addr *)
&ip_hdr(skb)->saddr, AF_INET);
} else if (hash_location) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = ip_hdr(skb)->saddr,
+ .daddr.ipv4 = ip_hdr(skb)->daddr,
+ .hnum = ntohs(th->source),
+ .sport = th->source,
+ .dif = inet_iif(skb),
+ };
+
/*
* active side is lost. Try to find listening socket through
* source port, and then find md5 key through listening socket.
@@ -658,10 +672,8 @@ static void tcp_v4_send_reset(const struct sock *sk, struct sk_buff *skb)
* Incoming packet is checked with md5 hash with finding key,
* no RST generated if md5 hash doesn't match.
*/
- sk1 = __inet_lookup_listener(net, &tcp_hashinfo, NULL, 0,
- ip_hdr(skb)->saddr,
- th->source, ip_hdr(skb)->daddr,
- ntohs(th->source), inet_iif(skb));
+ sk1 = inet_lookup_listener(net, &tcp_hashinfo, NULL, 0,
+ ¶ms);
/* don't send rst if it can't find key */
if (!sk1)
goto out;
@@ -1509,6 +1521,10 @@ void tcp_v4_early_demux(struct sk_buff *skb)
const struct iphdr *iph;
const struct tcphdr *th;
struct sock *sk;
+ struct sk_lookup params = {
+ .dif = skb->skb_iif,
+ };
+
if (skb->pkt_type != PACKET_HOST)
return;
@@ -1522,10 +1538,13 @@ void tcp_v4_early_demux(struct sk_buff *skb)
if (th->doff < sizeof(struct tcphdr) / 4)
return;
- sk = __inet_lookup_established(dev_net(skb->dev), &tcp_hashinfo,
- iph->saddr, th->source,
- iph->daddr, ntohs(th->dest),
- skb->skb_iif);
+ params.saddr.ipv4 = iph->saddr;
+ params.daddr.ipv4 = iph->daddr;
+ params.sport = th->source;
+ params.dport = th->dest;
+ params.hnum = ntohs(th->dest),
+
+ sk = inet_lookup_established(dev_net(skb->dev), &tcp_hashinfo, ¶ms);
if (sk) {
skb->sk = sk;
skb->destructor = sock_edemux;
@@ -1645,6 +1664,7 @@ EXPORT_SYMBOL(tcp_filter);
int tcp_v4_rcv(struct sk_buff *skb)
{
struct net *net = dev_net(skb->dev);
+ struct sk_lookup params = { };
const struct iphdr *iph;
const struct tcphdr *th;
bool refcounted;
@@ -1693,9 +1713,11 @@ int tcp_v4_rcv(struct sk_buff *skb)
TCP_SKB_CB(skb)->ip_dsfield = ipv4_get_dsfield(iph);
TCP_SKB_CB(skb)->sacked = 0;
+ params.sport = th->source;
+ params.dport = th->dest;
lookup:
- sk = __inet_lookup_skb(&tcp_hashinfo, skb, __tcp_hdrlen(th), th->source,
- th->dest, &refcounted);
+ sk = __inet_lookup_skb(&tcp_hashinfo, skb, __tcp_hdrlen(th), ¶ms,
+ &refcounted);
if (!sk)
goto no_tcp_socket;
@@ -1819,12 +1841,18 @@ int tcp_v4_rcv(struct sk_buff *skb)
}
switch (tcp_timewait_state_process(inet_twsk(sk), skb, th)) {
case TCP_TW_SYN: {
+ struct sk_lookup params = {
+ .saddr.ipv4 = iph->saddr,
+ .daddr.ipv4 = iph->daddr,
+ .sport = th->source,
+ .dport = th->dest,
+ .hnum = ntohs(th->dest),
+ .dif = inet_iif(skb),
+ };
struct sock *sk2 = inet_lookup_listener(dev_net(skb->dev),
&tcp_hashinfo, skb,
__tcp_hdrlen(th),
- iph->saddr, th->source,
- iph->daddr, th->dest,
- inet_iif(skb));
+ ¶ms);
if (sk2) {
inet_twsk_deschedule_put(inet_twsk(sk));
sk = sk2;
diff --git a/net/ipv4/udp_diag.c b/net/ipv4/udp_diag.c
index 5e0640877536..d7f6af42ebcc 100644
--- a/net/ipv4/udp_diag.c
+++ b/net/ipv4/udp_diag.c
@@ -47,6 +47,7 @@ static int udp_dump_one(struct udp_table *tbl, struct sk_buff *in_skb,
.daddr.ipv4 = req->id.idiag_dst[0],
.sport = req->id.idiag_sport,
.dport = req->id.idiag_dport,
+ .hnum = ntohs(req->id.idiag_dport),
.dif = req->id.idiag_if,
};
@@ -190,6 +191,7 @@ static int __udp_diag_destroy(struct sk_buff *in_skb,
.daddr.ipv4 = req->id.idiag_src[0],
.sport = req->id.idiag_dport,
.dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
.dif = req->id.idiag_if,
};
@@ -204,6 +206,7 @@ static int __udp_diag_destroy(struct sk_buff *in_skb,
.daddr.ipv4 = req->id.idiag_src[3],
.sport = req->id.idiag_dport,
.dport = req->id.idiag_sport,
+ .hnum = ntohs(req->id.idiag_sport),
.dif = req->id.idiag_if,
};
diff --git a/net/netfilter/xt_TPROXY.c b/net/netfilter/xt_TPROXY.c
index 972a0e40c59a..5cce7eb7dea2 100644
--- a/net/netfilter/xt_TPROXY.c
+++ b/net/netfilter/xt_TPROXY.c
@@ -117,6 +117,7 @@ nf_tproxy_get_sock_v4(struct net *net, struct sk_buff *skb, void *hp,
.daddr.ipv4 = daddr,
.sport = sport,
.dport = dport,
+ .hnum = ntohs(dport),
.dif = in->ifindex,
};
@@ -129,11 +130,9 @@ nf_tproxy_get_sock_v4(struct net *net, struct sk_buff *skb, void *hp,
case NFT_LOOKUP_LISTENER:
tcph = hp;
sk = inet_lookup_listener(net, &tcp_hashinfo, skb,
- ip_hdrlen(skb) +
+ ip_hdrlen(skb) +
__tcp_hdrlen(tcph),
- saddr, sport,
- daddr, dport,
- in->ifindex);
+ ¶ms);
if (sk && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
@@ -145,8 +144,7 @@ nf_tproxy_get_sock_v4(struct net *net, struct sk_buff *skb, void *hp,
break;
case NFT_LOOKUP_ESTABLISHED:
sk = inet_lookup_established(net, &tcp_hashinfo,
- saddr, sport, daddr, dport,
- in->ifindex);
+ ¶ms);
break;
default:
BUG();
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 02/10] net: ipv4: Convert udp socket lookups to new struct
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Convert udp4_lib_lookup and __udp4_lib_lookup to use the new sk_lookup
struct.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/udp.h | 6 +-
net/ipv4/netfilter/nf_socket_ipv4.c | 11 ++-
net/ipv4/udp.c | 170 +++++++++++++++++++-----------------
net/ipv4/udp_diag.c | 51 +++++++----
net/netfilter/xt_TPROXY.c | 11 ++-
5 files changed, 144 insertions(+), 105 deletions(-)
diff --git a/include/net/udp.h b/include/net/udp.h
index 972ce4baab6b..5e0ff095dc6d 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -283,10 +283,8 @@ int udp_lib_getsockopt(struct sock *sk, int level, int optname,
int udp_lib_setsockopt(struct sock *sk, int level, int optname,
char __user *optval, unsigned int optlen,
int (*push_pending_frames)(struct sock *));
-struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
- __be32 daddr, __be16 dport, int dif);
-struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
- __be32 daddr, __be16 dport, int dif,
+struct sock *udp4_lib_lookup(struct net *net, struct sk_lookup *params);
+struct sock *__udp4_lib_lookup(struct net *net, struct sk_lookup *params,
struct udp_table *tbl, struct sk_buff *skb);
struct sock *udp4_lib_lookup_skb(struct sk_buff *skb,
__be16 sport, __be16 dport);
diff --git a/net/ipv4/netfilter/nf_socket_ipv4.c b/net/ipv4/netfilter/nf_socket_ipv4.c
index e9293bdebba0..121767b36763 100644
--- a/net/ipv4/netfilter/nf_socket_ipv4.c
+++ b/net/ipv4/netfilter/nf_socket_ipv4.c
@@ -81,14 +81,21 @@ nf_socket_get_sock_v4(struct net *net, struct sk_buff *skb, const int doff,
const __be16 sport, const __be16 dport,
const struct net_device *in)
{
+ struct sk_lookup params = {
+ .saddr.ipv4 = saddr,
+ .daddr.ipv4 = daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = in->ifindex,
+ };
+
switch (protocol) {
case IPPROTO_TCP:
return inet_lookup(net, &tcp_hashinfo, skb, doff,
saddr, sport, daddr, dport,
in->ifindex);
case IPPROTO_UDP:
- return udp4_lib_lookup(net, saddr, sport, daddr, dport,
- in->ifindex);
+ return udp4_lib_lookup(net, ¶ms);
}
return NULL;
}
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index b057653ceca9..132a8f070d16 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -379,15 +379,13 @@ int udp_v4_get_port(struct sock *sk, unsigned short snum)
}
static int compute_score(struct sock *sk, struct net *net,
- __be32 saddr, __be16 sport,
- __be32 daddr, unsigned short hnum, int dif,
- bool exact_dif)
+ const struct sk_lookup *params)
{
- int score;
struct inet_sock *inet;
+ int score, rc;
if (!net_eq(sock_net(sk), net) ||
- udp_sk(sk)->udp_port_hash != hnum ||
+ udp_sk(sk)->udp_port_hash != params->hnum ||
ipv6_only_sock(sk))
return -1;
@@ -395,28 +393,28 @@ static int compute_score(struct sock *sk, struct net *net,
inet = inet_sk(sk);
if (inet->inet_rcv_saddr) {
- if (inet->inet_rcv_saddr != daddr)
+ if (inet->inet_rcv_saddr != params->daddr.ipv4)
return -1;
score += 4;
}
if (inet->inet_daddr) {
- if (inet->inet_daddr != saddr)
+ if (inet->inet_daddr != params->saddr.ipv4)
return -1;
score += 4;
}
if (inet->inet_dport) {
- if (inet->inet_dport != sport)
+ if (inet->inet_dport != params->sport)
return -1;
score += 4;
}
- if (sk->sk_bound_dev_if || exact_dif) {
- if (sk->sk_bound_dev_if != dif)
- return -1;
+ rc = sk_lookup_device_cmp(sk, params);
+ if (rc < 0)
+ return -1;
+ if (rc > 0)
score += 4;
- }
if (sk->sk_incoming_cpu == raw_smp_processor_id())
score++;
return score;
@@ -436,10 +434,9 @@ static u32 udp_ehashfn(const struct net *net, const __be32 laddr,
/* called with rcu_read_lock() */
static struct sock *udp4_lib_lookup2(struct net *net,
- __be32 saddr, __be16 sport,
- __be32 daddr, unsigned int hnum, int dif, bool exact_dif,
- struct udp_hslot *hslot2,
- struct sk_buff *skb)
+ const struct sk_lookup *params,
+ struct udp_hslot *hslot2,
+ struct sk_buff *skb)
{
struct sock *sk, *result;
int score, badness, matches = 0, reuseport = 0;
@@ -448,13 +445,14 @@ static struct sock *udp4_lib_lookup2(struct net *net,
result = NULL;
badness = 0;
udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) {
- score = compute_score(sk, net, saddr, sport,
- daddr, hnum, dif, exact_dif);
+ score = compute_score(sk, net, params);
if (score > badness) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- hash = udp_ehashfn(net, daddr, hnum,
- saddr, sport);
+ hash = udp_ehashfn(net, params->daddr.ipv4,
+ params->hnum,
+ params->saddr.ipv4,
+ params->sport);
result = reuseport_select_sock(sk, hash, skb,
sizeof(struct udphdr));
if (result)
@@ -476,28 +474,27 @@ static struct sock *udp4_lib_lookup2(struct net *net,
/* UDP is nearly always wildcards out the wazoo, it makes no sense to try
* harder than this. -DaveM
*/
-struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
- __be16 sport, __be32 daddr, __be16 dport,
- int dif, struct udp_table *udptable, struct sk_buff *skb)
+struct sock *__udp4_lib_lookup(struct net *net, struct sk_lookup *params,
+ struct udp_table *udptable, struct sk_buff *skb)
{
struct sock *sk, *result;
- unsigned short hnum = ntohs(dport);
+ unsigned short hnum = ntohs(params->dport);
unsigned int hash2, slot2, slot = udp_hashfn(net, hnum, udptable->mask);
struct udp_hslot *hslot2, *hslot = &udptable->hash[slot];
- bool exact_dif = udp_lib_exact_dif_match(net, skb);
int score, badness, matches = 0, reuseport = 0;
u32 hash = 0;
+ params->hnum = hnum;
+ params->exact_dif = udp_lib_exact_dif_match(net, skb);
+
if (hslot->count > 10) {
- hash2 = udp4_portaddr_hash(net, daddr, hnum);
+ hash2 = udp4_portaddr_hash(net, params->daddr.ipv4, hnum);
slot2 = hash2 & udptable->mask;
hslot2 = &udptable->hash2[slot2];
if (hslot->count < hslot2->count)
goto begin;
- result = udp4_lib_lookup2(net, saddr, sport,
- daddr, hnum, dif,
- exact_dif, hslot2, skb);
+ result = udp4_lib_lookup2(net, params, hslot2, skb);
if (!result) {
unsigned int old_slot2 = slot2;
hash2 = udp4_portaddr_hash(net, htonl(INADDR_ANY), hnum);
@@ -510,9 +507,7 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
if (hslot->count < hslot2->count)
goto begin;
- result = udp4_lib_lookup2(net, saddr, sport,
- daddr, hnum, dif,
- exact_dif, hslot2, skb);
+ result = udp4_lib_lookup2(net, params, hslot2, skb);
}
return result;
}
@@ -520,13 +515,14 @@ struct sock *__udp4_lib_lookup(struct net *net, __be32 saddr,
result = NULL;
badness = 0;
sk_for_each_rcu(sk, &hslot->head) {
- score = compute_score(sk, net, saddr, sport,
- daddr, hnum, dif, exact_dif);
+ score = compute_score(sk, net, params);
if (score > badness) {
reuseport = sk->sk_reuseport;
if (reuseport) {
- hash = udp_ehashfn(net, daddr, hnum,
- saddr, sport);
+ hash = udp_ehashfn(net, params->daddr.ipv4,
+ params->hnum,
+ params->saddr.ipv4,
+ params->sport);
result = reuseport_select_sock(sk, hash, skb,
sizeof(struct udphdr));
if (result)
@@ -551,10 +547,16 @@ static inline struct sock *__udp4_lib_lookup_skb(struct sk_buff *skb,
struct udp_table *udptable)
{
const struct iphdr *iph = ip_hdr(skb);
+ struct net *net = dev_net(skb->dev);
+ struct sk_lookup params = {
+ .saddr.ipv4 = iph->saddr,
+ .daddr.ipv4 = iph->daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = inet_iif(skb),
+ };
- return __udp4_lib_lookup(dev_net(skb->dev), iph->saddr, sport,
- iph->daddr, dport, inet_iif(skb),
- udptable, skb);
+ return __udp4_lib_lookup(net, ¶ms, udptable, skb);
}
struct sock *udp4_lib_lookup_skb(struct sk_buff *skb,
@@ -570,13 +572,11 @@ EXPORT_SYMBOL_GPL(udp4_lib_lookup_skb);
#if IS_ENABLED(CONFIG_NETFILTER_XT_MATCH_SOCKET) || \
IS_ENABLED(CONFIG_NETFILTER_XT_TARGET_TPROXY) || \
IS_ENABLED(CONFIG_NF_SOCKET_IPV4)
-struct sock *udp4_lib_lookup(struct net *net, __be32 saddr, __be16 sport,
- __be32 daddr, __be16 dport, int dif)
+struct sock *udp4_lib_lookup(struct net *net, struct sk_lookup *params)
{
struct sock *sk;
- sk = __udp4_lib_lookup(net, saddr, sport, daddr, dport,
- dif, &udp_table, NULL);
+ sk = __udp4_lib_lookup(net, params, &udp_table, NULL);
if (sk && !refcount_inc_not_zero(&sk->sk_refcnt))
sk = NULL;
return sk;
@@ -585,21 +585,21 @@ EXPORT_SYMBOL_GPL(udp4_lib_lookup);
#endif
static inline bool __udp_is_mcast_sock(struct net *net, struct sock *sk,
- __be16 loc_port, __be32 loc_addr,
- __be16 rmt_port, __be32 rmt_addr,
- int dif, unsigned short hnum)
+ const struct sk_lookup *params)
{
struct inet_sock *inet = inet_sk(sk);
+ __be32 loc_addr = params->daddr.ipv4;
+ __be32 rmt_addr = params->saddr.ipv4;
if (!net_eq(sock_net(sk), net) ||
- udp_sk(sk)->udp_port_hash != hnum ||
+ udp_sk(sk)->udp_port_hash != params->hnum ||
(inet->inet_daddr && inet->inet_daddr != rmt_addr) ||
- (inet->inet_dport != rmt_port && inet->inet_dport) ||
+ (inet->inet_dport != params->sport && inet->inet_dport) ||
(inet->inet_rcv_saddr && inet->inet_rcv_saddr != loc_addr) ||
ipv6_only_sock(sk) ||
- (sk->sk_bound_dev_if && sk->sk_bound_dev_if != dif))
+ (sk->sk_bound_dev_if && sk->sk_bound_dev_if != params->dif))
return false;
- if (!ip_mc_sf_allow(sk, loc_addr, rmt_addr, dif))
+ if (!ip_mc_sf_allow(sk, loc_addr, rmt_addr, params->dif))
return false;
return true;
}
@@ -626,10 +626,15 @@ void __udp4_lib_err(struct sk_buff *skb, u32 info, struct udp_table *udptable)
int harderr;
int err;
struct net *net = dev_net(skb->dev);
-
- sk = __udp4_lib_lookup(net, iph->daddr, uh->dest,
- iph->saddr, uh->source, skb->dev->ifindex, udptable,
- NULL);
+ struct sk_lookup params = {
+ .saddr.ipv4 = iph->daddr,
+ .daddr.ipv4 = iph->saddr,
+ .sport = uh->dest,
+ .dport = uh->source,
+ .dif = skb->dev->ifindex,
+ };
+
+ sk = __udp4_lib_lookup(net, ¶ms, udptable, NULL);
if (!sk) {
__ICMP_INC_STATS(net, ICMP_MIB_INERRORS);
return; /* No socket for error */
@@ -1956,9 +1961,16 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
struct udp_hslot *hslot = udp_hashslot(udptable, net, hnum);
unsigned int hash2 = 0, hash2_any = 0, use_hash2 = (hslot->count > 10);
unsigned int offset = offsetof(typeof(*sk), sk_node);
- int dif = skb->dev->ifindex;
struct hlist_node *node;
struct sk_buff *nskb;
+ struct sk_lookup params = {
+ .saddr.ipv4 = saddr,
+ .daddr.ipv4 = daddr,
+ .sport = uh->source,
+ .dport = uh->dest,
+ .hnum = hnum,
+ .dif = skb->dev->ifindex,
+ };
if (use_hash2) {
hash2_any = udp4_portaddr_hash(net, htonl(INADDR_ANY), hnum) &
@@ -1970,8 +1982,7 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb,
}
sk_for_each_entry_offset_rcu(sk, node, &hslot->head, offset) {
- if (!__udp_is_mcast_sock(net, sk, uh->dest, daddr,
- uh->source, saddr, dif, hnum))
+ if (!__udp_is_mcast_sock(net, sk, ¶ms))
continue;
if (!first) {
@@ -2159,13 +2170,10 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
* If more than one socket found returns NULL
*/
static struct sock *__udp4_lib_mcast_demux_lookup(struct net *net,
- __be16 loc_port, __be32 loc_addr,
- __be16 rmt_port, __be32 rmt_addr,
- int dif)
+ const struct sk_lookup *params)
{
struct sock *sk, *result;
- unsigned short hnum = ntohs(loc_port);
- unsigned int slot = udp_hashfn(net, hnum, udp_table.mask);
+ unsigned int slot = udp_hashfn(net, params->hnum, udp_table.mask);
struct udp_hslot *hslot = &udp_table.hash[slot];
/* Do not bother scanning a too big list */
@@ -2174,8 +2182,7 @@ static struct sock *__udp4_lib_mcast_demux_lookup(struct net *net,
result = NULL;
sk_for_each_rcu(sk, &hslot->head) {
- if (__udp_is_mcast_sock(net, sk, loc_port, loc_addr,
- rmt_port, rmt_addr, dif, hnum)) {
+ if (__udp_is_mcast_sock(net, sk, params)) {
if (result)
return NULL;
result = sk;
@@ -2190,21 +2197,20 @@ static struct sock *__udp4_lib_mcast_demux_lookup(struct net *net,
* if the first socket is an exact match and if not move on.
*/
static struct sock *__udp4_lib_demux_lookup(struct net *net,
- __be16 loc_port, __be32 loc_addr,
- __be16 rmt_port, __be32 rmt_addr,
- int dif)
+ const struct sk_lookup *params)
{
- unsigned short hnum = ntohs(loc_port);
- unsigned int hash2 = udp4_portaddr_hash(net, loc_addr, hnum);
+ unsigned int hash2 = udp4_portaddr_hash(net, params->daddr.ipv4,
+ params->hnum);
unsigned int slot2 = hash2 & udp_table.mask;
struct udp_hslot *hslot2 = &udp_table.hash2[slot2];
- INET_ADDR_COOKIE(acookie, rmt_addr, loc_addr);
- const __portpair ports = INET_COMBINED_PORTS(rmt_port, hnum);
+ INET_ADDR_COOKIE(acookie, params->saddr.ipv4, params->daddr.ipv4);
+ const __portpair ports = INET_COMBINED_PORTS(params->sport,
+ params->hnum);
struct sock *sk;
udp_portaddr_for_each_entry_rcu(sk, &hslot2->head) {
- if (INET_MATCH(sk, net, acookie, rmt_addr,
- loc_addr, ports, dif))
+ if (INET_MATCH(sk, net, acookie, params->saddr.ipv4,
+ params->daddr.ipv4, ports, params->dif))
return sk;
/* Only check first socket in chain */
break;
@@ -2215,11 +2221,13 @@ static struct sock *__udp4_lib_demux_lookup(struct net *net,
void udp_v4_early_demux(struct sk_buff *skb)
{
struct net *net = dev_net(skb->dev);
+ struct sk_lookup params = {
+ .dif = skb->dev->ifindex,
+ };
const struct iphdr *iph;
const struct udphdr *uh;
struct sock *sk = NULL;
struct dst_entry *dst;
- int dif = skb->dev->ifindex;
int ours;
/* validate the packet */
@@ -2228,6 +2236,11 @@ void udp_v4_early_demux(struct sk_buff *skb)
iph = ip_hdr(skb);
uh = udp_hdr(skb);
+ params.saddr.ipv4 = iph->saddr;
+ params.daddr.ipv4 = iph->daddr;
+ params.sport = uh->source;
+ params.dport = uh->dest;
+ params.hnum = ntohs(uh->dest);
if (skb->pkt_type == PACKET_BROADCAST ||
skb->pkt_type == PACKET_MULTICAST) {
@@ -2244,12 +2257,9 @@ void udp_v4_early_demux(struct sk_buff *skb)
return;
}
- sk = __udp4_lib_mcast_demux_lookup(net, uh->dest, iph->daddr,
- uh->source, iph->saddr, dif);
- } else if (skb->pkt_type == PACKET_HOST) {
- sk = __udp4_lib_demux_lookup(net, uh->dest, iph->daddr,
- uh->source, iph->saddr, dif);
- }
+ sk = __udp4_lib_mcast_demux_lookup(net, ¶ms);
+ } else if (skb->pkt_type == PACKET_HOST)
+ sk = __udp4_lib_demux_lookup(net, ¶ms);
if (!sk || !refcount_inc_not_zero(&sk->sk_refcnt))
return;
diff --git a/net/ipv4/udp_diag.c b/net/ipv4/udp_diag.c
index 4515836d2a3a..5e0640877536 100644
--- a/net/ipv4/udp_diag.c
+++ b/net/ipv4/udp_diag.c
@@ -41,11 +41,17 @@ static int udp_dump_one(struct udp_table *tbl, struct sk_buff *in_skb,
struct net *net = sock_net(in_skb->sk);
rcu_read_lock();
- if (req->sdiag_family == AF_INET)
- sk = __udp4_lib_lookup(net,
- req->id.idiag_src[0], req->id.idiag_sport,
- req->id.idiag_dst[0], req->id.idiag_dport,
- req->id.idiag_if, tbl, NULL);
+ if (req->sdiag_family == AF_INET) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = req->id.idiag_src[0],
+ .daddr.ipv4 = req->id.idiag_dst[0],
+ .sport = req->id.idiag_sport,
+ .dport = req->id.idiag_dport,
+ .dif = req->id.idiag_if,
+ };
+
+ sk = __udp4_lib_lookup(net, ¶ms, tbl, NULL);
+ }
#if IS_ENABLED(CONFIG_IPV6)
else if (req->sdiag_family == AF_INET6)
sk = __udp6_lib_lookup(net,
@@ -178,27 +184,38 @@ static int __udp_diag_destroy(struct sk_buff *in_skb,
rcu_read_lock();
- if (req->sdiag_family == AF_INET)
- sk = __udp4_lib_lookup(net,
- req->id.idiag_dst[0], req->id.idiag_dport,
- req->id.idiag_src[0], req->id.idiag_sport,
- req->id.idiag_if, tbl, NULL);
+ if (req->sdiag_family == AF_INET) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = req->id.idiag_dst[0],
+ .daddr.ipv4 = req->id.idiag_src[0],
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .dif = req->id.idiag_if,
+ };
+
+ sk = __udp4_lib_lookup(net, ¶ms, tbl, NULL);
+ }
#if IS_ENABLED(CONFIG_IPV6)
else if (req->sdiag_family == AF_INET6) {
if (ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_dst) &&
- ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_src))
- sk = __udp4_lib_lookup(net,
- req->id.idiag_dst[3], req->id.idiag_dport,
- req->id.idiag_src[3], req->id.idiag_sport,
- req->id.idiag_if, tbl, NULL);
-
- else
+ ipv6_addr_v4mapped((struct in6_addr *)req->id.idiag_src)) {
+ struct sk_lookup params = {
+ .saddr.ipv4 = req->id.idiag_dst[3],
+ .daddr.ipv4 = req->id.idiag_src[3],
+ .sport = req->id.idiag_dport,
+ .dport = req->id.idiag_sport,
+ .dif = req->id.idiag_if,
+ };
+
+ sk = __udp4_lib_lookup(net, ¶ms, tbl, NULL);
+ } else {
sk = __udp6_lib_lookup(net,
(struct in6_addr *)req->id.idiag_dst,
req->id.idiag_dport,
(struct in6_addr *)req->id.idiag_src,
req->id.idiag_sport,
req->id.idiag_if, tbl, NULL);
+ }
}
#endif
else {
diff --git a/net/netfilter/xt_TPROXY.c b/net/netfilter/xt_TPROXY.c
index d767e35fff6b..972a0e40c59a 100644
--- a/net/netfilter/xt_TPROXY.c
+++ b/net/netfilter/xt_TPROXY.c
@@ -112,6 +112,14 @@ nf_tproxy_get_sock_v4(struct net *net, struct sk_buff *skb, void *hp,
const struct net_device *in,
const enum nf_tproxy_lookup_t lookup_type)
{
+ struct sk_lookup params = {
+ .saddr.ipv4 = saddr,
+ .daddr.ipv4 = daddr,
+ .sport = sport,
+ .dport = dport,
+ .dif = in->ifindex,
+ };
+
struct sock *sk;
struct tcphdr *tcph;
@@ -145,8 +153,7 @@ nf_tproxy_get_sock_v4(struct net *net, struct sk_buff *skb, void *hp,
}
break;
case IPPROTO_UDP:
- sk = udp4_lib_lookup(net, saddr, sport, daddr, dport,
- in->ifindex);
+ sk = udp4_lib_lookup(net, ¶ms);
if (sk) {
int connected = (sk->sk_state == TCP_ESTABLISHED);
int wildcard = (inet_sk(sk)->inet_rcv_saddr == 0);
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 01/10] net: Add sk_lookup struct and helper
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
In-Reply-To: <1500997121-3218-1-git-send-email-dsahern@gmail.com>
Consolidate the socket lookup args into a struct.
Add helper that compares sk_bound_dev_if for a socket to the lookup
parameters.
Signed-off-by: David Ahern <dsahern@gmail.com>
---
include/net/sock.h | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/include/net/sock.h b/include/net/sock.h
index 7c0632c7e870..a2db5fd30192 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -491,6 +491,44 @@ enum sk_pacing {
#define rcu_dereference_sk_user_data(sk) rcu_dereference(__sk_user_data((sk)))
#define rcu_assign_sk_user_data(sk, ptr) rcu_assign_pointer(__sk_user_data((sk)), ptr)
+/* used for socket lookups */
+struct sk_lookup {
+ union {
+ const struct in6_addr *ipv6;
+ __be32 ipv4;
+ } saddr;
+ union {
+ const struct in6_addr *ipv6;
+ __be32 ipv4;
+ } daddr;
+
+ __be16 sport;
+ __be16 dport;
+ unsigned short hnum;
+
+ int dif;
+ bool exact_dif;
+};
+
+/* Compare sk_bound_dev_if to socket lookup dif
+ * Returns:
+ * -1 exact dif required and not met
+ * 0 sk_bound_dev_if is either not set or does not match
+ * 1 sk_bound_dev_if is set and matches dif
+ */
+static inline int sk_lookup_device_cmp(const struct sock *sk,
+ const struct sk_lookup *params)
+{
+ /* exact_dif true == l3mdev case */
+ if (params->exact_dif && sk->sk_bound_dev_if != params->dif)
+ return -1;
+
+ if (sk->sk_bound_dev_if && sk->sk_bound_dev_if == params->dif)
+ return 1;
+
+ return 0;
+}
+
/*
* SK_CAN_REUSE and SK_NO_REUSE on a socket mean that the socket is OK
* or not whether his port will be reused by someone else. SK_FORCE_REUSE
--
2.1.4
^ permalink raw reply related
* [RFC PATCH 00/10] net: l3mdev: Support for sockets bound to enslaved device
From: David Ahern @ 2017-07-25 15:38 UTC (permalink / raw)
To: netdev; +Cc: David Ahern
A missing piece to the VRF puzzle is the ability to bind sockets to
devices enslaved to a VRF. This patch set adds the enslaved device
index, sdif, to IPv4 and IPv6 socket lookups. The end result for users
is the following scope options for services:
1. "global" services - sockets not bound to any device
Allows 1 service to work across all network interfaces with
connected sockets bound to the VRF the connection originates
(Requires net.ipv4.tcp_l3mdev_accept=1 for TCP and
net.ipv4.udp_l3mdev_accept=1 for UDP)
2. "VRF" local services - sockets bound to a VRF
Sockets work across all network interfaces enslaved to a VRF but
are limited to just the one VRF.
3. "device" services - sockets bound to a specific network interface
Service works only through the one specific interface.
Existing code for socket lookups already pass in 6+ arguments. Rather
than add another for the enslaved device index, the existing lookups
are converted to use a new sk_lookup struct. From there, the enslaved
device index becomes another element of the struct.
Patch 1 introduces sk_lookup struct and helper.
Patches 2-4 convert udp, inet and socket lookups for IPv4 to use the
new sk_lookup struct. Meant to be a conversion of IPv4 code only; no
functional change intended.
Patches 5-7 convert udp, inet and socket lookups for IPv6 to use the
new sk_lookup struct. Meant to be a conversion of IPv6 code only; no
functional change intended.
Patch 8 adds sdif to the sk_lookup struct allowing lookups to consider
a second device index.
Patches 9-10 add support for the enslaved device index to ipv4 and ipv6
socket lookups.
David Ahern (10):
net: Add sk_lookup struct and helper
net: ipv4: Convert udp socket lookups to new struct
net: ipv4: Convert inet socket lookups to new struct
net: ipv4: Convert raw sockets to sk_lookup
net: ipv6: Convert udp socket lookups to new struct
net: ipv6: Convert inet socket lookups to new struct
net: ipv6: Convert raw sockets to sk_lookup
net: Add sdif to sk_lookup
net: ipv4: Support for sockets bound to enslaved device
net: ipv6: Support for sockets bound to enslaved device
include/linux/igmp.h | 3 +-
include/linux/ipv6.h | 8 ++
include/net/inet6_hashtables.h | 44 ++++-----
include/net/inet_hashtables.h | 67 ++++++-------
include/net/ip.h | 10 ++
include/net/raw.h | 3 +-
include/net/rawv6.h | 3 +-
include/net/sock.h | 42 +++++++++
include/net/tcp.h | 17 ++++
include/net/udp.h | 18 +---
net/dccp/ipv4.c | 19 +++-
net/dccp/ipv6.c | 22 +++--
net/ipv4/igmp.c | 6 +-
net/ipv4/inet_diag.c | 50 +++++++---
net/ipv4/inet_hashtables.c | 56 ++++++-----
net/ipv4/netfilter/nf_socket_ipv4.c | 16 +++-
net/ipv4/raw.c | 77 +++++++++------
net/ipv4/raw_diag.c | 30 ++++--
net/ipv4/tcp_ipv4.c | 64 +++++++++----
net/ipv4/udp.c | 175 ++++++++++++++++++----------------
net/ipv4/udp_diag.c | 89 ++++++++++++------
net/ipv6/inet6_hashtables.c | 72 +++++++-------
net/ipv6/netfilter/nf_socket_ipv6.c | 16 +++-
net/ipv6/raw.c | 44 +++++----
net/ipv6/tcp_ipv6.c | 63 +++++++++----
net/ipv6/udp.c | 181 ++++++++++++++++++++----------------
net/netfilter/xt_TPROXY.c | 39 +++++---
27 files changed, 754 insertions(+), 480 deletions(-)
--
2.1.4
^ permalink raw reply
* Re: SELinux/IP_PASSSEC regression in 4.13-rcX
From: Paolo Abeni @ 2017-07-25 15:36 UTC (permalink / raw)
To: Paul Moore; +Cc: netdev, selinux
In-Reply-To: <CAHC9VhTe71CM4i+c86ckbV1GUPdNs1R5sByamZRqLwdYaWcMmw@mail.gmail.com>
On Tue, 2017-07-25 at 10:45 -0400, Paul Moore wrote:
> On Tue, Jul 25, 2017 at 5:59 AM, Paolo Abeni <pabeni@redhat.com> wrote:
> > On Mon, 2017-07-24 at 22:00 -0400, Paul Moore wrote:
> > > > I'm happy to test this, but if you are curious, you can find the
> > > > selinux-testsuite at the link below; the "inet_socket" tests are the
> > > > ones relevant to this problem.
> > > >
> > > > * https://github.com/SELinuxProject/selinux-testsuite
> >
> > Thanks, I'll have a look.
> >
> > > > However, I believe there is a problem with this patch, see below.
> >
> > [...]
> >
> > > > > -#if BITS_PER_LONG == 64
> > > > > +#define UDP_SKB_IS_STATELESS 0x80000000
> > > > > +
> > > > > static void udp_set_dev_scratch(struct sk_buff *skb)
> > > > > {
> > > > > - struct udp_dev_scratch *scratch;
> > > > > + struct udp_dev_scratch *scratch = udp_skb_scratch(skb);
> > > > >
> > > > > BUILD_BUG_ON(sizeof(struct udp_dev_scratch) > sizeof(long));
> > > >
> > > > The BUILD_BUG_ON() assertion no longer appears to be correct with this patch.
> > >
> > > Nevermind, I just took a closer look at this and realized I made a
> > > mistake when applying your patch (had to apply manually for some
> > > reason). I'm building a test kernel now.
> >
> > Yup, I compile-tested the code, plus some basic sanity checks, so the
> > build breakage felt unexpected.
> >
> > Thanks for testing,
>
> I just did a quick run through the selinux-testsuite and the
> regression would appear to be fixed, thanks! I'm guessing you'll send
> this to DaveM so we can get this fixed before v4.13 is released?
>
> Tested-by: Paul Moore <paul@paul-moore.com>
Sure. I'll submit soon for -net.
Cheers,
Paolo
^ permalink raw reply
* [PATCH net-next] virtio-net: mark PM functions as __maybe_unused
From: Arnd Bergmann @ 2017-07-25 15:35 UTC (permalink / raw)
To: Michael S. Tsirkin, Jason Wang
Cc: Arnd Bergmann, David S. Miller, John Fastabend, Willem de Bruijn,
Sebastian Andrzej Siewior, Eric Dumazet, virtualization, netdev,
linux-kernel
After removing the reset function, the freeze and restore functions
are now unused when CONFIG_PM_SLEEP is disabled:
drivers/net/virtio_net.c:1881:12: error: 'virtnet_restore_up' defined but not used [-Werror=unused-function]
static int virtnet_restore_up(struct virtio_device *vdev)
drivers/net/virtio_net.c:1859:13: error: 'virtnet_freeze_down' defined but not used [-Werror=unused-function]
static void virtnet_freeze_down(struct virtio_device *vdev)
A more robust way to do this is to remove the #ifdef around the callers
and instead mark them as __maybe_unused. The compiler will now just
silently drop the unused code.
Fixes: 4941d472bf95 ("virtio-net: do not reset during XDP set")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
drivers/net/virtio_net.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index d4751ce23b4f..1902701e15a9 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -2702,8 +2702,7 @@ static void virtnet_remove(struct virtio_device *vdev)
free_netdev(vi->dev);
}
-#ifdef CONFIG_PM_SLEEP
-static int virtnet_freeze(struct virtio_device *vdev)
+static __maybe_unused int virtnet_freeze(struct virtio_device *vdev)
{
struct virtnet_info *vi = vdev->priv;
@@ -2714,7 +2713,7 @@ static int virtnet_freeze(struct virtio_device *vdev)
return 0;
}
-static int virtnet_restore(struct virtio_device *vdev)
+static __maybe_unused int virtnet_restore(struct virtio_device *vdev)
{
struct virtnet_info *vi = vdev->priv;
int err;
@@ -2730,7 +2729,6 @@ static int virtnet_restore(struct virtio_device *vdev)
return 0;
}
-#endif
static struct virtio_device_id id_table[] = {
{ VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID },
--
2.9.0
^ permalink raw reply related
* TC-pedit man page examples error
From: Tyler Bautista @ 2017-07-25 15:33 UTC (permalink / raw)
To: netdev
To whom it may concern,
I recently attempted to use simple tc action pedit commands on the man
page and I ran into some errors. The following is some information
about my version of iproute and my machine:
----------------------------
the following is my iproute package information
Loaded plugins: fastestmirror, langpacks
iproute-3.10.0-74.el7.x86_64
changed_by = 0
checksum_data =
f18d9a19e6531081a605086e5003422a87846854a10a306d122b59ff7bebb9d7
checksum_type = sha256
command_line = update
from_repo = base
from_repo_revision = 1480942829
from_repo_timestamp = 1480942901
installed_by = 4294967295
origin_url =
http://mirror.confluxtech.com/centos/7/os/x86_64/Packages/iproute-3.10.0-74.el7.x86_64.rpm
reason = user
releasever = 7
var_uuid = df8acb3e-a9df-4ffe-b95f-0311b19b2d33
---------------------------------------------------------------------------------
I am running the latest version of Centos 7 with kernel version:
3.10.0-514.26.2.el7.x86_64
-------------
The following are the first three commands of the man page that I run
and their output
---------------------------------------------------------------------------------
tc qdisc replace dev eth0 root handle 1: htb
tc qdisc add dev eth0 ingress handle ffff:
tc filter add dev eth0 parent 1: u32 \
> match ip dport 23 0xffff \
> action pedit pedit munge ip dport set 22
bad pedit parsing
Illegal pedit construct (ip)
Usage: ... pedit munge <MUNGE>
Where: MUNGE := <RAW>|<LAYERED>
<RAW>:= <OFFSETC>[ATC]<CMD>
OFFSETC:= offset <offval> <u8|u16|u32>
ATC:= at <atval> offmask <maskval> shift <shiftval>
NOTE: offval is byte offset, must be multiple of 4
NOTE: maskval is a 32 bit hex number
NOTE: shiftval is a is a shift value
CMD:= clear | invert | set <setval>| retain
<LAYERED>:= ip <ipdata> | ip6 <ip6data>
| udp <udpdata> | tcp <tcpdata> | icmp <icmpdata>
For Example usage look at the examples directory
bad action parsing
parse_action: bad value (7:pedit)!
Illegal "action"
---------------------------------------------
The first two commands run without error. However the third command fails.
It apparently does not recognize the ip command. This seems strange to
me since the commands I run are on the tc-pedit man page. Any help on
this issue would be appreciated.
Sincerely,
Tyler Bautista
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox