* Re: [PATCH net-next 1/4] siphash: add cryptographically secure PRF
From: Jason A. Donenfeld @ 2017-01-06 20:03 UTC (permalink / raw)
To: David Miller
Cc: Netdev, LKML, Jean-Philippe Aumasson, Linus Torvalds,
Eric Biggers, David Laight, Eric Dumazet
In-Reply-To: <20170106.145737.1436197697379516563.davem@davemloft.net>
Will resubmit. Sorry. I had this in earlier series and dropped it in
this one. Apologies. Give me 30 minutes and you'll have a beautiful
and conformant patch series.
^ permalink raw reply
* [PATCH v2 2/7] net: ethernet: ti: cpdma: fix desc re-queuing
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
The currently processing cpdma descriptor with EOQ flag set may
contain two values in Next Descriptor Pointer field:
- valid pointer: means CPDMA missed addition of new desc in queue;
- null: no more descriptors in queue.
In the later case, it's not required to write to HDP register, but now
CPDMA does it.
Hence, add additional check for Next Descriptor Pointer != null in
cpdma_chan_process() function before writing in HDP register.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index d6f0ded..a53b384 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -1159,7 +1159,7 @@ static int __cpdma_chan_process(struct cpdma_chan *chan)
chan->count--;
chan->stats.good_dequeue++;
- if (status & CPDMA_DESC_EOQ) {
+ if ((status & CPDMA_DESC_EOQ) && chan->head) {
chan->stats.requeue++;
chan_write(chan, hdp, desc_phys(pool, chan->head));
}
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 5/7] net: ethernet: ti: cpsw: add support for descs pool size configuration
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
The CPSW CPDMA can process buffer descriptors placed as in internal
CPPI RAM as in DDR. This patch adds support in CPSW and CPDMA for
descs_pool_size mudule parameter, which defines total number of CPDMA CPPI
descriptors to be used for both ingress/egress packets processing:
- memory size, required for CPDMA descriptor pool, is calculated basing
on number of descriptors specified by user in descs_pool_size and
CPDMA descriptor size and allocated from coherent memory (CMA area);
- CPDMA descriptor pool will be allocated in DDR if pool memory size >
internal CPPI RAM or use internal CPPI RAM otherwise;
- if descs_pool_size not specified in DT - the default value 256 will
be used which will allow to place CPDMA descriptors pool into the
internal CPPI RAM (current default behaviour);
- CPDMA will ignore descs_pool_size if descs_pool_size = 0 for
backward comaptiobility with davinci_emac.
descs_pool_size is boot time setting and can't be changed once
CPSW/CPDMA is initialized.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/cpsw.c | 12 +++++++++---
drivers/net/ethernet/ti/davinci_cpdma.c | 12 ++++++++++++
drivers/net/ethernet/ti/davinci_cpdma.h | 1 +
3 files changed, 22 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index b203143..d39875e 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -145,6 +145,7 @@ do { \
cpsw->data.active_slave)
#define IRQ_NUM 2
#define CPSW_MAX_QUEUES 8
+#define CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT 256
static int debug_level;
module_param(debug_level, int, 0);
@@ -158,6 +159,10 @@ static int rx_packet_max = CPSW_MAX_PACKET_SIZE;
module_param(rx_packet_max, int, 0);
MODULE_PARM_DESC(rx_packet_max, "maximum receive packet size (bytes)");
+static int descs_pool_size = CPSW_CPDMA_DESCS_POOL_SIZE_DEFAULT;
+module_param(descs_pool_size, int, 0444);
+MODULE_PARM_DESC(descs_pool_size, "Number of CPDMA CPPI descriptors in pool");
+
struct cpsw_wr_regs {
u32 id_ver;
u32 soft_reset;
@@ -2969,6 +2974,7 @@ static int cpsw_probe(struct platform_device *pdev)
dma_params.has_ext_regs = true;
dma_params.desc_hw_addr = dma_params.desc_mem_phys;
dma_params.bus_freq_mhz = cpsw->bus_freq_mhz;
+ dma_params.descs_pool_size = descs_pool_size;
cpsw->dma = cpdma_ctlr_create(&dma_params);
if (!cpsw->dma) {
@@ -3072,9 +3078,9 @@ static int cpsw_probe(struct platform_device *pdev)
goto clean_ale_ret;
}
- cpsw_notice(priv, probe, "initialized device (regs %pa, irq %d)\n",
- &ss_res->start, ndev->irq);
-
+ cpsw_notice(priv, probe,
+ "initialized device (regs %pa, irq %d, pool size %d)\n",
+ &ss_res->start, ndev->irq, dma_params.descs_pool_size);
if (cpsw->data.dual_emac) {
ret = cpsw_probe_dual_emac(priv);
if (ret) {
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index b229bf3..65e2f12 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -219,6 +219,18 @@ int cpdma_desc_pool_create(struct cpdma_ctlr *ctlr)
cpdma_params->desc_align);
pool->num_desc = pool->mem_size / pool->desc_size;
+ if (cpdma_params->descs_pool_size) {
+ /* recalculate memory size required cpdma descriptor pool
+ * basing on number of descriptors specified by user and
+ * if memory size > CPPI internal RAM size (desc_mem_size)
+ * then switch to use DDR
+ */
+ pool->num_desc = cpdma_params->descs_pool_size;
+ pool->mem_size = pool->desc_size * pool->num_desc;
+ if (pool->mem_size > cpdma_params->desc_mem_size)
+ cpdma_params->desc_mem_phys = 0;
+ }
+
pool->gen_pool = devm_gen_pool_create(ctlr->dev, ilog2(pool->desc_size),
-1, "cpdma");
if (IS_ERR(pool->gen_pool)) {
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.h b/drivers/net/ethernet/ti/davinci_cpdma.h
index 4a167db..cb45f8f 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.h
+++ b/drivers/net/ethernet/ti/davinci_cpdma.h
@@ -37,6 +37,7 @@ struct cpdma_params {
int desc_mem_size;
int desc_align;
u32 bus_freq_mhz;
+ u32 descs_pool_size;
/*
* Some instances of embedded cpdma controllers have extra control and
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 6/7] net: ethernet: ti: cpsw: add support for ringparam configuration
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
The CPDMA uses one pool of descriptors for both RX and TX which by default
split between all channels proportionally depending on total number of
CPDMA channels and number of TX and RX channels. As result, more
descriptors will be consumed by TX path if there are more TX channels and
there is no way now to dedicate more descriptors for RX path.
So, add the ability to re-split CPDMA pool of descriptors between RX and TX
path via ethtool '-G' command wich will allow to configure and fix number
of descriptors used by RX and TX path, which, then, will be split between
RX/TX channels proportionally depending on RX/TX channels number and
weight. ethtool '-G' command will accept only number of RX entries and rest
of descriptors will be arranged for TX automatically.
Command:
ethtool -G <devname> rx <number of descriptors>
defaults and limitations:
- minimum number of rx descriptors is 10% of total number of descriptors in
CPDMA pool
- maximum number of rx descriptors is 90% of total number of descriptors in
CPDMA pool
- by default, descriptors will be split equally between RX/TX path
- any values passed in "tx" parameter will be ignored
Usage:
# ethtool -g eth0
Pre-set maximums:
RX: 7372
RX Mini: 0
RX Jumbo: 0
TX: 0
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
# ethtool -G eth0 rx 7372
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 7372
RX Mini: 0
RX Jumbo: 0
TX: 0
Current hardware settings:
RX: 7372
RX Mini: 0
RX Jumbo: 0
TX: 820
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/cpsw.c | 86 +++++++++++++++++++++++++++++++++
drivers/net/ethernet/ti/davinci_cpdma.c | 40 ++++++++++++---
drivers/net/ethernet/ti/davinci_cpdma.h | 4 ++
3 files changed, 122 insertions(+), 8 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index d39875e..f339268 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -2484,6 +2484,90 @@ static int cpsw_nway_reset(struct net_device *ndev)
return -EOPNOTSUPP;
}
+static void cpsw_get_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering)
+{
+ struct cpsw_priv *priv = netdev_priv(ndev);
+ struct cpsw_common *cpsw = priv->cpsw;
+
+ /* not supported */
+ ering->tx_max_pending = 0;
+ ering->tx_pending = cpdma_get_num_tx_descs(cpsw->dma);
+ /* Max 90% RX buffers */
+ ering->rx_max_pending = (descs_pool_size * 9) / 10;
+ ering->rx_pending = cpdma_get_num_rx_descs(cpsw->dma);
+}
+
+static int cpsw_set_ringparam(struct net_device *ndev,
+ struct ethtool_ringparam *ering)
+{
+ struct cpsw_priv *priv = netdev_priv(ndev);
+ struct cpsw_common *cpsw = priv->cpsw;
+ struct cpsw_slave *slave;
+ int i, ret;
+
+ /* ignore ering->tx_pending - only rx_pending adjustment is supported */
+
+ if (ering->rx_mini_pending || ering->rx_jumbo_pending ||
+ ering->rx_pending < (descs_pool_size / 10) ||
+ ering->rx_pending > ((descs_pool_size * 9) / 10))
+ return -EINVAL;
+
+ if (ering->rx_pending == cpdma_get_num_rx_descs(cpsw->dma))
+ return 0;
+
+ /* Disable NAPI scheduling */
+ cpsw_intr_disable(cpsw);
+
+ /* Stop all transmit queues for every network device.
+ * Disable re-using rx descriptors with dormant_on.
+ */
+ for (i = cpsw->data.slaves, slave = cpsw->slaves; i; i--, slave++) {
+ if (!(slave->ndev && netif_running(slave->ndev)))
+ continue;
+
+ netif_tx_stop_all_queues(slave->ndev);
+ netif_dormant_on(slave->ndev);
+ }
+
+ /* Handle rest of tx packets and stop cpdma channels */
+ cpdma_ctlr_stop(cpsw->dma);
+
+ cpdma_set_num_rx_descs(cpsw->dma, ering->rx_pending);
+
+ for (i = cpsw->data.slaves, slave = cpsw->slaves; i; i--, slave++) {
+ if (!(slave->ndev && netif_running(slave->ndev)))
+ continue;
+
+ /* Enable rx packets handling */
+ netif_dormant_off(slave->ndev);
+ }
+
+ if (cpsw_common_res_usage_state(cpsw)) {
+ cpdma_chan_split_pool(cpsw->dma);
+
+ ret = cpsw_fill_rx_channels(priv);
+ if (ret)
+ goto err;
+
+ /* After this receive is started */
+ cpdma_ctlr_start(cpsw->dma);
+ cpsw_intr_enable(cpsw);
+ }
+
+ /* Resume transmit for every affected interface */
+ for (i = cpsw->data.slaves, slave = cpsw->slaves; i; i--, slave++) {
+ if (!(slave->ndev && netif_running(slave->ndev)))
+ continue;
+ netif_tx_start_all_queues(slave->ndev);
+ }
+ return 0;
+err:
+ dev_err(priv->dev, "cannot set ring params, closing device\n");
+ dev_close(ndev);
+ return ret;
+}
+
static const struct ethtool_ops cpsw_ethtool_ops = {
.get_drvinfo = cpsw_get_drvinfo,
.get_msglevel = cpsw_get_msglevel,
@@ -2510,6 +2594,8 @@ static const struct ethtool_ops cpsw_ethtool_ops = {
.get_eee = cpsw_get_eee,
.set_eee = cpsw_set_eee,
.nway_reset = cpsw_nway_reset,
+ .get_ringparam = cpsw_get_ringparam,
+ .set_ringparam = cpsw_set_ringparam,
};
static void cpsw_slave_init(struct cpsw_slave *slave, struct cpsw_common *cpsw,
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 65e2f12..d80bff1 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -108,6 +108,8 @@ struct cpdma_ctlr {
spinlock_t lock;
struct cpdma_chan *channels[2 * CPDMA_MAX_CHANNELS];
int chan_num;
+ int num_rx_desc; /* RX descriptors number */
+ int num_tx_desc; /* TX descriptors number */
};
struct cpdma_chan {
@@ -518,6 +520,9 @@ struct cpdma_ctlr *cpdma_ctlr_create(struct cpdma_params *params)
if (cpdma_desc_pool_create(ctlr))
return NULL;
+ /* split pool equally between RX/TX by default */
+ ctlr->num_tx_desc = ctlr->pool->num_desc / 2;
+ ctlr->num_rx_desc = ctlr->pool->num_desc - ctlr->num_tx_desc;
if (WARN_ON(ctlr->num_chan > CPDMA_MAX_CHANNELS))
ctlr->num_chan = CPDMA_MAX_CHANNELS;
@@ -717,22 +722,22 @@ static void cpdma_chan_set_descs(struct cpdma_ctlr *ctlr,
}
}
/* use remains */
- most_chan->desc_num += desc_cnt;
+ if (most_chan)
+ most_chan->desc_num += desc_cnt;
}
/**
* cpdma_chan_split_pool - Splits ctrl pool between all channels.
* Has to be called under ctlr lock
*/
-static int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr)
+int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr)
{
int tx_per_ch_desc = 0, rx_per_ch_desc = 0;
- struct cpdma_desc_pool *pool = ctlr->pool;
int free_rx_num = 0, free_tx_num = 0;
int rx_weight = 0, tx_weight = 0;
int tx_desc_num, rx_desc_num;
struct cpdma_chan *chan;
- int i, tx_num = 0;
+ int i;
if (!ctlr->chan_num)
return 0;
@@ -750,15 +755,14 @@ static int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr)
if (!chan->weight)
free_tx_num++;
tx_weight += chan->weight;
- tx_num++;
}
}
if (rx_weight > 100 || tx_weight > 100)
return -EINVAL;
- tx_desc_num = (tx_num * pool->num_desc) / ctlr->chan_num;
- rx_desc_num = pool->num_desc - tx_desc_num;
+ tx_desc_num = ctlr->num_tx_desc;
+ rx_desc_num = ctlr->num_rx_desc;
if (free_tx_num) {
tx_per_ch_desc = tx_desc_num - (tx_weight * tx_desc_num) / 100;
@@ -774,6 +778,8 @@ static int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr)
return 0;
}
+EXPORT_SYMBOL_GPL(cpdma_chan_split_pool);
+
/* cpdma_chan_set_weight - set weight of a channel in percentage.
* Tx and Rx channels have separate weights. That is 100% for RX
@@ -907,7 +913,6 @@ struct cpdma_chan *cpdma_chan_create(struct cpdma_ctlr *ctlr, int chan_num,
chan->chan_num = chan_num;
chan->handler = handler;
chan->rate = 0;
- chan->desc_num = ctlr->pool->num_desc / 2;
chan->weight = 0;
if (is_rx_chan(chan)) {
@@ -1329,4 +1334,23 @@ int cpdma_control_set(struct cpdma_ctlr *ctlr, int control, int value)
}
EXPORT_SYMBOL_GPL(cpdma_control_set);
+int cpdma_get_num_rx_descs(struct cpdma_ctlr *ctlr)
+{
+ return ctlr->num_rx_desc;
+}
+EXPORT_SYMBOL_GPL(cpdma_get_num_rx_descs);
+
+int cpdma_get_num_tx_descs(struct cpdma_ctlr *ctlr)
+{
+ return ctlr->num_tx_desc;
+}
+EXPORT_SYMBOL_GPL(cpdma_get_num_tx_descs);
+
+void cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc)
+{
+ ctlr->num_rx_desc = num_rx_desc;
+ ctlr->num_tx_desc = ctlr->pool->num_desc - ctlr->num_rx_desc;
+}
+EXPORT_SYMBOL_GPL(cpdma_set_num_rx_descs);
+
MODULE_LICENSE("GPL");
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.h b/drivers/net/ethernet/ti/davinci_cpdma.h
index cb45f8f..fd65ce2 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.h
+++ b/drivers/net/ethernet/ti/davinci_cpdma.h
@@ -114,5 +114,9 @@ enum cpdma_control {
int cpdma_control_get(struct cpdma_ctlr *ctlr, int control);
int cpdma_control_set(struct cpdma_ctlr *ctlr, int control, int value);
+int cpdma_get_num_rx_descs(struct cpdma_ctlr *ctlr);
+void cpdma_set_num_rx_descs(struct cpdma_ctlr *ctlr, int num_rx_desc);
+int cpdma_get_num_tx_descs(struct cpdma_ctlr *ctlr);
+int cpdma_chan_split_pool(struct cpdma_ctlr *ctlr);
#endif
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 7/7] Documentation: DT: net: cpsw: remove no_bd_ram property
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko,
'Rob Herring '
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
Even if no_bd_ram property is described in TI CPSW bindings the support for
it has never been introduced in CPSW driver, so there are no real users of
it. Hence, remove no_bd_ram property from documentation and DT files.
Cc: 'Rob Herring <robh+dt@kernel.org>'
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
Documentation/devicetree/bindings/net/cpsw.txt | 3 ---
arch/arm/boot/dts/am33xx.dtsi | 1 -
arch/arm/boot/dts/am4372.dtsi | 1 -
arch/arm/boot/dts/dm814x.dtsi | 1 -
arch/arm/boot/dts/dra7.dtsi | 1 -
5 files changed, 7 deletions(-)
diff --git a/Documentation/devicetree/bindings/net/cpsw.txt b/Documentation/devicetree/bindings/net/cpsw.txt
index ebda7c9..7cc15c9 100644
--- a/Documentation/devicetree/bindings/net/cpsw.txt
+++ b/Documentation/devicetree/bindings/net/cpsw.txt
@@ -23,7 +23,6 @@ Required properties:
Optional properties:
- ti,hwmods : Must be "cpgmac0"
-- no_bd_ram : Must be 0 or 1
- dual_emac : Specifies Switch to act as Dual EMAC
- syscon : Phandle to the system control device node, which is
the control module device of the am33x
@@ -70,7 +69,6 @@ Examples:
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
rx_descs = <64>;
mac_control = <0x20>;
slaves = <2>;
@@ -99,7 +97,6 @@ Examples:
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
rx_descs = <64>;
mac_control = <0x20>;
slaves = <2>;
diff --git a/arch/arm/boot/dts/am33xx.dtsi b/arch/arm/boot/dts/am33xx.dtsi
index 64c8aa9..d458ceb 100644
--- a/arch/arm/boot/dts/am33xx.dtsi
+++ b/arch/arm/boot/dts/am33xx.dtsi
@@ -781,7 +781,6 @@
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
mac_control = <0x20>;
slaves = <2>;
active_slave = <0>;
diff --git a/arch/arm/boot/dts/am4372.dtsi b/arch/arm/boot/dts/am4372.dtsi
index ac55f93..837aff1 100644
--- a/arch/arm/boot/dts/am4372.dtsi
+++ b/arch/arm/boot/dts/am4372.dtsi
@@ -669,7 +669,6 @@
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
mac_control = <0x20>;
slaves = <2>;
active_slave = <0>;
diff --git a/arch/arm/boot/dts/dm814x.dtsi b/arch/arm/boot/dts/dm814x.dtsi
index 1facc5f..4cbeb5c 100644
--- a/arch/arm/boot/dts/dm814x.dtsi
+++ b/arch/arm/boot/dts/dm814x.dtsi
@@ -509,7 +509,6 @@
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
mac_control = <0x20>;
slaves = <2>;
active_slave = <0>;
diff --git a/arch/arm/boot/dts/dra7.dtsi b/arch/arm/boot/dts/dra7.dtsi
index addb753..b69df91 100644
--- a/arch/arm/boot/dts/dra7.dtsi
+++ b/arch/arm/boot/dts/dra7.dtsi
@@ -1707,7 +1707,6 @@
cpdma_channels = <8>;
ale_entries = <1024>;
bd_ram_size = <0x2000>;
- no_bd_ram = <0>;
mac_control = <0x20>;
slaves = <2>;
active_slave = <0>;
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 4/7] net: ethernet: ti: cpdma: use devm_ioremap
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
Use devm_ioremap() and simplify the code.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 7080eb9..b229bf3 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -195,8 +195,6 @@ static void cpdma_desc_pool_destroy(struct cpdma_ctlr *ctlr)
if (pool->cpumap)
dma_free_coherent(ctlr->dev, pool->mem_size, pool->cpumap,
pool->phys);
- else
- iounmap(pool->iomap);
}
/*
@@ -231,7 +229,8 @@ int cpdma_desc_pool_create(struct cpdma_ctlr *ctlr)
if (cpdma_params->desc_mem_phys) {
pool->phys = cpdma_params->desc_mem_phys;
- pool->iomap = ioremap(pool->phys, pool->mem_size);
+ pool->iomap = devm_ioremap(ctlr->dev, pool->phys,
+ pool->mem_size);
pool->hw_addr = cpdma_params->desc_hw_addr;
} else {
pool->cpumap = dma_alloc_coherent(ctlr->dev, pool->mem_size,
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 1/7] net: ethernet: ti: cpdma: am437x: allow descs to be plased in ddr
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
It's observed that cpsw/cpdma is not working properly when CPPI
descriptors are placed in DDR instead of internal CPPI RAM on am437x
SoC:
- rx/tx silently stops processing packets;
- or - after boot it's working for sometime, but stuck once Network
load is increased (ping is working, but iperf is not).
(The same issue has not been reproduced on am335x and am57xx).
It seems that write to HDP register processed faster by interconnect
than writing of descriptor memory buffer in DDR, which is probably
caused by store buffer / write buffer differences as these functions
are implemented differently across devices. So, to fix this i come up
with two minimal, required changes:
1) all accesses to the channel register HDP/CP/RXFREE registers should
be done using sync IO accessors readl()/writel(), because all previous
memory writes writes have to be completed before starting channel
(write to HDP) or completing desc processing.
2) the change 1 only doesn't work on am437x and additional reading of
desc's field is required right after the new descriptor was filled
with data and before pointer on it will be stored in
prev_desc->hw_next field or HDP register.
In addition, to above changes this patch eliminates all relaxed ordering
I/O accessors in this driver as suggested by David Miller to avoid such
kind of issues in the future, but with one exception - relaxed IO accessors
will still be used to fill desc in cpdma_chan_submit(), which is safe as
there is read barrier at the end of write sequence, and because sync IO
accessors usage here will affect on net performance.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 40 ++++++++++++++++++---------------
1 file changed, 22 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 36518fc..d6f0ded 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -166,12 +166,12 @@ static struct cpdma_control_info controls[] = {
#define num_chan params.num_chan
/* various accessors */
-#define dma_reg_read(ctlr, ofs) __raw_readl((ctlr)->dmaregs + (ofs))
-#define chan_read(chan, fld) __raw_readl((chan)->fld)
-#define desc_read(desc, fld) __raw_readl(&(desc)->fld)
-#define dma_reg_write(ctlr, ofs, v) __raw_writel(v, (ctlr)->dmaregs + (ofs))
-#define chan_write(chan, fld, v) __raw_writel(v, (chan)->fld)
-#define desc_write(desc, fld, v) __raw_writel((u32)(v), &(desc)->fld)
+#define dma_reg_read(ctlr, ofs) readl((ctlr)->dmaregs + (ofs))
+#define chan_read(chan, fld) readl((chan)->fld)
+#define desc_read(desc, fld) readl(&(desc)->fld)
+#define dma_reg_write(ctlr, ofs, v) writel(v, (ctlr)->dmaregs + (ofs))
+#define chan_write(chan, fld, v) writel(v, (chan)->fld)
+#define desc_write(desc, fld, v) writel((u32)(v), &(desc)->fld)
#define cpdma_desc_to_port(chan, mode, directed) \
do { \
@@ -542,10 +542,10 @@ int cpdma_ctlr_start(struct cpdma_ctlr *ctlr)
}
for (i = 0; i < ctlr->num_chan; i++) {
- __raw_writel(0, ctlr->params.txhdp + 4 * i);
- __raw_writel(0, ctlr->params.rxhdp + 4 * i);
- __raw_writel(0, ctlr->params.txcp + 4 * i);
- __raw_writel(0, ctlr->params.rxcp + 4 * i);
+ writel(0, ctlr->params.txhdp + 4 * i);
+ writel(0, ctlr->params.rxhdp + 4 * i);
+ writel(0, ctlr->params.txcp + 4 * i);
+ writel(0, ctlr->params.rxcp + 4 * i);
}
dma_reg_write(ctlr, CPDMA_RXINTMASKCLEAR, 0xffffffff);
@@ -1061,13 +1061,17 @@ int cpdma_chan_submit(struct cpdma_chan *chan, void *token, void *data,
mode = CPDMA_DESC_OWNER | CPDMA_DESC_SOP | CPDMA_DESC_EOP;
cpdma_desc_to_port(chan, mode, directed);
- desc_write(desc, hw_next, 0);
- desc_write(desc, hw_buffer, buffer);
- desc_write(desc, hw_len, len);
- desc_write(desc, hw_mode, mode | len);
- desc_write(desc, sw_token, token);
- desc_write(desc, sw_buffer, buffer);
- desc_write(desc, sw_len, len);
+ /* Relaxed IO accessors can be used here as there is read barrier
+ * at the end of write sequence.
+ */
+ writel_relaxed(0, &desc->hw_next);
+ writel_relaxed(buffer, &desc->hw_buffer);
+ writel_relaxed(len, &desc->hw_len);
+ writel_relaxed(mode | len, &desc->hw_mode);
+ writel_relaxed(token, &desc->sw_token);
+ writel_relaxed(buffer, &desc->sw_buffer);
+ writel_relaxed(len, &desc->sw_len);
+ desc_read(desc, sw_len);
__cpdma_chan_submit(chan, desc);
@@ -1136,7 +1140,7 @@ static int __cpdma_chan_process(struct cpdma_chan *chan)
}
desc_dma = desc_phys(pool, desc);
- status = __raw_readl(&desc->hw_mode);
+ status = desc_read(desc, hw_mode);
outlen = status & 0x7ff;
if (status & CPDMA_DESC_OWNER) {
chan->stats.busy_dequeue++;
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 3/7] net: ethernet: ti: cpdma: minimize number of parameters in cpdma_desc_pool_create/destroy()
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
In-Reply-To: <20170106200735.4210-1-grygorii.strashko@ti.com>
Update cpdma_desc_pool_create/destroy() to accept only one parameter
struct cpdma_ctlr*, as this structure contains all required
information for pool creation/destruction.
Signed-off-by: Grygorii Strashko <grygorii.strashko@ti.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 62 ++++++++++++++++-----------------
1 file changed, 30 insertions(+), 32 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index a53b384..7080eb9 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -181,8 +181,10 @@ static struct cpdma_control_info controls[] = {
(directed << CPDMA_TO_PORT_SHIFT)); \
} while (0)
-static void cpdma_desc_pool_destroy(struct cpdma_desc_pool *pool)
+static void cpdma_desc_pool_destroy(struct cpdma_ctlr *ctlr)
{
+ struct cpdma_desc_pool *pool = ctlr->pool;
+
if (!pool)
return;
@@ -191,7 +193,7 @@ static void cpdma_desc_pool_destroy(struct cpdma_desc_pool *pool)
gen_pool_size(pool->gen_pool),
gen_pool_avail(pool->gen_pool));
if (pool->cpumap)
- dma_free_coherent(pool->dev, pool->mem_size, pool->cpumap,
+ dma_free_coherent(ctlr->dev, pool->mem_size, pool->cpumap,
pool->phys);
else
iounmap(pool->iomap);
@@ -203,37 +205,37 @@ static void cpdma_desc_pool_destroy(struct cpdma_desc_pool *pool)
* devices (e.g. cpsw switches) use plain old memory. Descriptor pools
* abstract out these details
*/
-static struct cpdma_desc_pool *
-cpdma_desc_pool_create(struct device *dev, u32 phys, dma_addr_t hw_addr,
- int size, int align)
+int cpdma_desc_pool_create(struct cpdma_ctlr *ctlr)
{
+ struct cpdma_params *cpdma_params = &ctlr->params;
struct cpdma_desc_pool *pool;
- int ret;
+ int ret = -ENOMEM;
- pool = devm_kzalloc(dev, sizeof(*pool), GFP_KERNEL);
+ pool = devm_kzalloc(ctlr->dev, sizeof(*pool), GFP_KERNEL);
if (!pool)
goto gen_pool_create_fail;
+ ctlr->pool = pool;
- pool->dev = dev;
- pool->mem_size = size;
- pool->desc_size = ALIGN(sizeof(struct cpdma_desc), align);
- pool->num_desc = size / pool->desc_size;
+ pool->mem_size = cpdma_params->desc_mem_size;
+ pool->desc_size = ALIGN(sizeof(struct cpdma_desc),
+ cpdma_params->desc_align);
+ pool->num_desc = pool->mem_size / pool->desc_size;
- pool->gen_pool = devm_gen_pool_create(dev, ilog2(pool->desc_size), -1,
- "cpdma");
+ pool->gen_pool = devm_gen_pool_create(ctlr->dev, ilog2(pool->desc_size),
+ -1, "cpdma");
if (IS_ERR(pool->gen_pool)) {
- dev_err(dev, "pool create failed %ld\n",
- PTR_ERR(pool->gen_pool));
+ ret = PTR_ERR(pool->gen_pool);
+ dev_err(ctlr->dev, "pool create failed %d\n", ret);
goto gen_pool_create_fail;
}
- if (phys) {
- pool->phys = phys;
- pool->iomap = ioremap(phys, size); /* should be memremap? */
- pool->hw_addr = hw_addr;
+ if (cpdma_params->desc_mem_phys) {
+ pool->phys = cpdma_params->desc_mem_phys;
+ pool->iomap = ioremap(pool->phys, pool->mem_size);
+ pool->hw_addr = cpdma_params->desc_hw_addr;
} else {
- pool->cpumap = dma_alloc_coherent(dev, size, &pool->hw_addr,
- GFP_KERNEL);
+ pool->cpumap = dma_alloc_coherent(ctlr->dev, pool->mem_size,
+ &pool->hw_addr, GFP_KERNEL);
pool->iomap = (void __iomem __force *)pool->cpumap;
pool->phys = pool->hw_addr; /* assumes no IOMMU, don't use this value */
}
@@ -244,16 +246,17 @@ cpdma_desc_pool_create(struct device *dev, u32 phys, dma_addr_t hw_addr,
ret = gen_pool_add_virt(pool->gen_pool, (unsigned long)pool->iomap,
pool->phys, pool->mem_size, -1);
if (ret < 0) {
- dev_err(dev, "pool add failed %d\n", ret);
+ dev_err(ctlr->dev, "pool add failed %d\n", ret);
goto gen_pool_add_virt_fail;
}
- return pool;
+ return 0;
gen_pool_add_virt_fail:
- cpdma_desc_pool_destroy(pool);
+ cpdma_desc_pool_destroy(ctlr);
gen_pool_create_fail:
- return NULL;
+ ctlr->pool = NULL;
+ return ret;
}
static inline dma_addr_t desc_phys(struct cpdma_desc_pool *pool,
@@ -502,12 +505,7 @@ struct cpdma_ctlr *cpdma_ctlr_create(struct cpdma_params *params)
ctlr->chan_num = 0;
spin_lock_init(&ctlr->lock);
- ctlr->pool = cpdma_desc_pool_create(ctlr->dev,
- ctlr->params.desc_mem_phys,
- ctlr->params.desc_hw_addr,
- ctlr->params.desc_mem_size,
- ctlr->params.desc_align);
- if (!ctlr->pool)
+ if (cpdma_desc_pool_create(ctlr))
return NULL;
if (WARN_ON(ctlr->num_chan > CPDMA_MAX_CHANNELS))
@@ -623,7 +621,7 @@ int cpdma_ctlr_destroy(struct cpdma_ctlr *ctlr)
for (i = 0; i < ARRAY_SIZE(ctlr->channels); i++)
cpdma_chan_destroy(ctlr->channels[i]);
- cpdma_desc_pool_destroy(ctlr->pool);
+ cpdma_desc_pool_destroy(ctlr);
return ret;
}
EXPORT_SYMBOL_GPL(cpdma_ctlr_destroy);
--
2.10.1.dirty
^ permalink raw reply related
* [PATCH v2 0/7] net: ethernet: ti: cpsw: support placing CPDMA descriptors into DDR
From: Grygorii Strashko @ 2017-01-06 20:07 UTC (permalink / raw)
To: David S. Miller, netdev, Mugunthan V N, Ivan Khoronzhuk
Cc: Sekhar Nori, linux-kernel, linux-omap, Grygorii Strashko
This series intended to add support for placing CPDMA descriptors into DDR by
introducing new module parameter "descs_pool_size" to specify size of descriptor's
pool. The "descs_pool_size" defines total number of CPDMA
CPPI descriptors to be used for both ingress/egress packets
processing. If not specified - the default value 256 will be used
which will allow to place descriptor's pool into the internal CPPI
RAM.
In addition, added ability to re-split CPDMA pool of descriptors between RX and TX
path via ethtool '-G' command wich will allow to configure and fix number
of descriptors used by RX and TX path, which, then, will be split between
RX/TX channels proportionally depending on number of RX/TX channels and
its weight.
This allows significantly to reduce UDP packets drop rate
for bandwidth >301 Mbits/sec (am57x).
Before enabling this feature, the am437x SoC has to be fixed as it's proved
that it's not working when CPDMA descriptors placed in DDR.
So, the patch 1 fixes this issue.
Grygorii Strashko (7):
net: ethernet: ti: cpdma: am437x: allow descs to be plased in ddr
net: ethernet: ti: cpdma: fix desc re-queuing
net: ethernet: ti: cpdma: minimize number of parameters in
cpdma_desc_pool_create/destroy()
net: ethernet: ti: cpdma: use devm_ioremap
net: ethernet: ti: cpsw: add support for descs pool size configuration
net: ethernet: ti: cpsw: add support for ringparam configuration
Documentation: DT: net: cpsw: remove no_bd_ram property
Documentation/devicetree/bindings/net/cpsw.txt | 3 -
arch/arm/boot/dts/am33xx.dtsi | 1 -
arch/arm/boot/dts/am4372.dtsi | 1 -
arch/arm/boot/dts/dm814x.dtsi | 1 -
arch/arm/boot/dts/dra7.dtsi | 1 -
drivers/net/ethernet/ti/cpsw.c | 98 ++++++++++++++-
drivers/net/ethernet/ti/davinci_cpdma.c | 161 +++++++++++++++----------
drivers/net/ethernet/ti/davinci_cpdma.h | 5 +
8 files changed, 199 insertions(+), 72 deletions(-)
--
2.10.1.dirty
^ permalink raw reply
* [PATCH net-next 3/4] secure_seq: use SipHash in place of MD5
From: Jason A. Donenfeld @ 2017-01-06 20:10 UTC (permalink / raw)
To: davem, netdev, linux-kernel
Cc: Jason A. Donenfeld, Andi Kleen, David Laight, Tom Herbert,
Hannes Frederic Sowa, Eric Dumazet
In-Reply-To: <20170106201055.13765-1-Jason@zx2c4.com>
This gives a clear speed and security improvement. Siphash is both
faster and is more solid crypto than the aging MD5.
Rather than manually filling MD5 buffers, for IPv6, we simply create
a layout by a simple anonymous struct, for which gcc generates
rather efficient code. For IPv4, we pass the values directly to the
short input convenience functions.
64-bit x86_64:
[ 1.683628] secure_tcpv6_sequence_number_md5# cycles: 99563527
[ 1.717350] secure_tcp_sequence_number_md5# cycles: 92890502
[ 1.741968] secure_tcpv6_sequence_number_siphash# cycles: 67825362
[ 1.762048] secure_tcp_sequence_number_siphash# cycles: 67485526
32-bit x86:
[ 1.600012] secure_tcpv6_sequence_number_md5# cycles: 103227892
[ 1.634219] secure_tcp_sequence_number_md5# cycles: 94732544
[ 1.669102] secure_tcpv6_sequence_number_siphash# cycles: 96299384
[ 1.700165] secure_tcp_sequence_number_siphash# cycles: 86015473
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: David Miller <davem@davemloft.net>
Cc: David Laight <David.Laight@aculab.com>
Cc: Tom Herbert <tom@herbertland.com>
Cc: Hannes Frederic Sowa <hannes@stressinduktion.org>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
---
net/core/secure_seq.c | 135 ++++++++++++++++++++------------------------------
1 file changed, 54 insertions(+), 81 deletions(-)
diff --git a/net/core/secure_seq.c b/net/core/secure_seq.c
index 88a8e429fc3e..3dc2689bcc64 100644
--- a/net/core/secure_seq.c
+++ b/net/core/secure_seq.c
@@ -1,3 +1,5 @@
+/* Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved. */
+
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/cryptohash.h>
@@ -8,14 +10,14 @@
#include <linux/ktime.h>
#include <linux/string.h>
#include <linux/net.h>
-
+#include <linux/siphash.h>
#include <net/secure_seq.h>
#if IS_ENABLED(CONFIG_IPV6) || IS_ENABLED(CONFIG_INET)
+#include <linux/in6.h>
#include <net/tcp.h>
-#define NET_SECRET_SIZE (MD5_MESSAGE_BYTES / 4)
-static u32 net_secret[NET_SECRET_SIZE] ____cacheline_aligned;
+static siphash_key_t net_secret;
static __always_inline void net_secret_init(void)
{
@@ -44,80 +46,65 @@ static u32 seq_scale(u32 seq)
u32 secure_tcpv6_sequence_number(const __be32 *saddr, const __be32 *daddr,
__be16 sport, __be16 dport, u32 *tsoff)
{
- u32 secret[MD5_MESSAGE_BYTES / 4];
- u32 hash[MD5_DIGEST_WORDS];
- u32 i;
-
+ const struct {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ __be16 sport;
+ __be16 dport;
+ } __aligned(SIPHASH_ALIGNMENT) combined = {
+ .saddr = *(struct in6_addr *)saddr,
+ .daddr = *(struct in6_addr *)daddr,
+ .sport = sport,
+ .dport = dport
+ };
+ u64 hash;
net_secret_init();
- memcpy(hash, saddr, 16);
- for (i = 0; i < 4; i++)
- secret[i] = net_secret[i] + (__force u32)daddr[i];
- secret[4] = net_secret[4] +
- (((__force u16)sport << 16) + (__force u16)dport);
- for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
- secret[i] = net_secret[i];
-
- md5_transform(hash, secret);
-
- *tsoff = sysctl_tcp_timestamps == 1 ? hash[1] : 0;
- return seq_scale(hash[0]);
+ hash = siphash(&combined, offsetofend(typeof(combined), dport), net_secret);
+ *tsoff = sysctl_tcp_timestamps == 1 ? (hash >> 32) : 0;
+ return seq_scale(hash);
}
EXPORT_SYMBOL(secure_tcpv6_sequence_number);
u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
__be16 dport)
{
- u32 secret[MD5_MESSAGE_BYTES / 4];
- u32 hash[MD5_DIGEST_WORDS];
- u32 i;
-
+ const struct {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ __be16 dport;
+ } __aligned(SIPHASH_ALIGNMENT) combined = {
+ .saddr = *(struct in6_addr *)saddr,
+ .daddr = *(struct in6_addr *)daddr,
+ .dport = dport
+ };
net_secret_init();
- memcpy(hash, saddr, 16);
- for (i = 0; i < 4; i++)
- secret[i] = net_secret[i] + (__force u32) daddr[i];
- secret[4] = net_secret[4] + (__force u32)dport;
- for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
- secret[i] = net_secret[i];
-
- md5_transform(hash, secret);
-
- return hash[0];
+ return siphash(&combined, offsetofend(typeof(combined), dport), net_secret);
}
EXPORT_SYMBOL(secure_ipv6_port_ephemeral);
#endif
#ifdef CONFIG_INET
+/* secure_tcp_sequence_number(a, b, 0, d) == secure_ipv4_port_ephemeral(a, b, d),
+ * but fortunately, `sport' cannot be 0 in any circumstances. If this changes,
+ * it would be easy enough to have the former function use siphash_4u32, passing
+ * the arguments as separate u32.
+ */
+
u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
__be16 sport, __be16 dport, u32 *tsoff)
{
- u32 hash[MD5_DIGEST_WORDS];
-
+ u64 hash;
net_secret_init();
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
- hash[3] = net_secret[15];
-
- md5_transform(hash, net_secret);
-
- *tsoff = sysctl_tcp_timestamps == 1 ? hash[1] : 0;
- return seq_scale(hash[0]);
+ hash = siphash_3u32(saddr, daddr, (u32)sport << 16 | dport, net_secret);
+ *tsoff = sysctl_tcp_timestamps == 1 ? (hash >> 32) : 0;
+ return seq_scale(hash);
}
u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport)
{
- u32 hash[MD5_DIGEST_WORDS];
-
net_secret_init();
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = (__force u32)dport ^ net_secret[14];
- hash[3] = net_secret[15];
-
- md5_transform(hash, net_secret);
-
- return hash[0];
+ return siphash_3u32(saddr, daddr, dport, net_secret);
}
EXPORT_SYMBOL_GPL(secure_ipv4_port_ephemeral);
#endif
@@ -126,21 +113,11 @@ EXPORT_SYMBOL_GPL(secure_ipv4_port_ephemeral);
u64 secure_dccp_sequence_number(__be32 saddr, __be32 daddr,
__be16 sport, __be16 dport)
{
- u32 hash[MD5_DIGEST_WORDS];
u64 seq;
-
net_secret_init();
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
- hash[3] = net_secret[15];
-
- md5_transform(hash, net_secret);
-
- seq = hash[0] | (((u64)hash[1]) << 32);
+ seq = siphash_3u32(saddr, daddr, (u32)sport << 16 | dport, net_secret);
seq += ktime_get_real_ns();
seq &= (1ull << 48) - 1;
-
return seq;
}
EXPORT_SYMBOL(secure_dccp_sequence_number);
@@ -149,26 +126,22 @@ EXPORT_SYMBOL(secure_dccp_sequence_number);
u64 secure_dccpv6_sequence_number(__be32 *saddr, __be32 *daddr,
__be16 sport, __be16 dport)
{
- u32 secret[MD5_MESSAGE_BYTES / 4];
- u32 hash[MD5_DIGEST_WORDS];
+ const struct {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ __be16 sport;
+ __be16 dport;
+ } __aligned(SIPHASH_ALIGNMENT) combined = {
+ .saddr = *(struct in6_addr *)saddr,
+ .daddr = *(struct in6_addr *)daddr,
+ .sport = sport,
+ .dport = dport
+ };
u64 seq;
- u32 i;
-
net_secret_init();
- memcpy(hash, saddr, 16);
- for (i = 0; i < 4; i++)
- secret[i] = net_secret[i] + (__force u32)daddr[i];
- secret[4] = net_secret[4] +
- (((__force u16)sport << 16) + (__force u16)dport);
- for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
- secret[i] = net_secret[i];
-
- md5_transform(hash, secret);
-
- seq = hash[0] | (((u64)hash[1]) << 32);
+ seq = siphash(&combined, offsetofend(typeof(combined), dport), net_secret);
seq += ktime_get_real_ns();
seq &= (1ull << 48) - 1;
-
return seq;
}
EXPORT_SYMBOL(secure_dccpv6_sequence_number);
--
2.11.0
^ permalink raw reply related
* [PATCH net-next 0/4] Introduce The SipHash PRF
From: Jason A. Donenfeld @ 2017-01-06 20:10 UTC (permalink / raw)
To: davem, netdev, linux-kernel; +Cc: Jason A. Donenfeld
In-Reply-To: <20170106.145737.1436197697379516563.davem@davemloft.net>
This patch series introduces SipHash into the kernel. SipHash is a
cryptographically secure PRF, which serves a variety of functions, and is
introduced in patch #1. The following patch #2 introduces HalfSipHash,
an optimization suitable for hash tables only. Finally, the last two patches
in this series show two usages of the introduced siphash function family.
It is expected that after this initial introductin, other usages will follow.
Please read the extensive descriptions in patch #1 and patch #2 of what these
functions do and the various levels of assurances. They're products of intense
cryptographic research, and I believe they're suitable for the uses outlined
herein.
The use of SipHash is not limited to the networking subsystem -- indeed I
would like to use it in other places too in the kernel. But after discussing
with a few on this list and at Linus' suggestion, the initial import of these
functions is coming through the networking tree. After these are merged, it
will then be easier to expand use elsewhere.
Jason A. Donenfeld (4):
siphash: add cryptographically secure PRF
siphash: implement HalfSipHash1-3 for hash tables
secure_seq: use SipHash in place of MD5
syncookies: use SipHash in place of SHA1
Documentation/siphash.txt | 154 +++++++++++++
MAINTAINERS | 7 +
include/linux/siphash.h | 133 +++++++++++
lib/Kconfig.debug | 6 +-
lib/Makefile | 5 +-
lib/siphash.c | 548 ++++++++++++++++++++++++++++++++++++++++++++++
lib/test_siphash.c | 208 ++++++++++++++++++
net/core/secure_seq.c | 135 +++++-------
net/ipv4/syncookies.c | 20 +-
net/ipv6/syncookies.c | 37 ++--
10 files changed, 1130 insertions(+), 123 deletions(-)
create mode 100644 Documentation/siphash.txt
create mode 100644 include/linux/siphash.h
create mode 100644 lib/siphash.c
create mode 100644 lib/test_siphash.c
--
2.11.0
^ permalink raw reply
* [PATCH net-next 1/4] siphash: add cryptographically secure PRF
From: Jason A. Donenfeld @ 2017-01-06 20:10 UTC (permalink / raw)
To: davem, netdev, linux-kernel
Cc: Jason A. Donenfeld, Jean-Philippe Aumasson, Linus Torvalds,
Eric Biggers, David Laight, Eric Dumazet
In-Reply-To: <20170106201055.13765-1-Jason@zx2c4.com>
SipHash is a 64-bit keyed hash function that is actually a
cryptographically secure PRF, like HMAC. Except SipHash is super fast,
and is meant to be used as a hashtable keyed lookup function, or as a
general PRF for short input use cases, such as sequence numbers or RNG
chaining.
For the first usage:
There are a variety of attacks known as "hashtable poisoning" in which an
attacker forms some data such that the hash of that data will be the
same, and then preceeds to fill up all entries of a hashbucket. This is
a realistic and well-known denial-of-service vector. Currently
hashtables use jhash, which is fast but not secure, and some kind of
rotating key scheme (or none at all, which isn't good). SipHash is meant
as a replacement for jhash in these cases.
There are a modicum of places in the kernel that are vulnerable to
hashtable poisoning attacks, either via userspace vectors or network
vectors, and there's not a reliable mechanism inside the kernel at the
moment to fix it. The first step toward fixing these issues is actually
getting a secure primitive into the kernel for developers to use. Then
we can, bit by bit, port things over to it as deemed appropriate.
While SipHash is extremely fast for a cryptographically secure function,
it is likely a bit slower than the insecure jhash, and so replacements
will be evaluated on a case-by-case basis based on whether or not the
difference in speed is negligible and whether or not the current jhash usage
poses a real security risk.
For the second usage:
A few places in the kernel are using MD5 or SHA1 for creating secure
sequence numbers, syn cookies, port numbers, or fast random numbers.
SipHash is a faster and more fitting, and more secure replacement for MD5
in those situations. Replacing MD5 and SHA1 with SipHash for these uses is
obvious and straight-forward, and so is submitted along with this patch
series. There shouldn't be much of a debate over its efficacy.
Dozens of languages are already using this internally for their hash
tables and PRFs. Some of the BSDs already use this in their kernels.
SipHash is a widely known high-speed solution to a widely known set of
problems, and it's time we catch-up.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Eric Biggers <ebiggers3@gmail.com>
Cc: David Laight <David.Laight@aculab.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
---
Documentation/siphash.txt | 79 ++++++++++++++++
MAINTAINERS | 7 ++
include/linux/siphash.h | 79 ++++++++++++++++
lib/Kconfig.debug | 6 +-
lib/Makefile | 5 +-
lib/siphash.c | 232 ++++++++++++++++++++++++++++++++++++++++++++++
lib/test_siphash.c | 119 ++++++++++++++++++++++++
7 files changed, 522 insertions(+), 5 deletions(-)
create mode 100644 Documentation/siphash.txt
create mode 100644 include/linux/siphash.h
create mode 100644 lib/siphash.c
create mode 100644 lib/test_siphash.c
diff --git a/Documentation/siphash.txt b/Documentation/siphash.txt
new file mode 100644
index 000000000000..39ff7f0438e7
--- /dev/null
+++ b/Documentation/siphash.txt
@@ -0,0 +1,79 @@
+ SipHash - a short input PRF
+-----------------------------------------------
+Written by Jason A. Donenfeld <jason@zx2c4.com>
+
+SipHash is a cryptographically secure PRF -- a keyed hash function -- that
+performs very well for short inputs, hence the name. It was designed by
+cryptographers Daniel J. Bernstein and Jean-Philippe Aumasson. It is intended
+as a replacement for some uses of: `jhash`, `md5_transform`, `sha_transform`,
+and so forth.
+
+SipHash takes a secret key filled with randomly generated numbers and either
+an input buffer or several input integers. It spits out an integer that is
+indistinguishable from random. You may then use that integer as part of secure
+sequence numbers, secure cookies, or mask it off for use in a hash table.
+
+1. Generating a key
+
+Keys should always be generated from a cryptographically secure source of
+random numbers, either using get_random_bytes or get_random_once:
+
+siphash_key_t key;
+get_random_bytes(key, sizeof(key));
+
+If you're not deriving your key from here, you're doing it wrong.
+
+2. Using the functions
+
+There are two variants of the function, one that takes a list of integers, and
+one that takes a buffer:
+
+u64 siphash(const void *data, size_t len, siphash_key_t key);
+
+And:
+
+u64 siphash_1u64(u64, siphash_key_t key);
+u64 siphash_2u64(u64, u64, siphash_key_t key);
+u64 siphash_3u64(u64, u64, u64, siphash_key_t key);
+u64 siphash_4u64(u64, u64, u64, u64, siphash_key_t key);
+u64 siphash_1u32(u32, siphash_key_t key);
+u64 siphash_2u32(u32, u32, siphash_key_t key);
+u64 siphash_3u32(u32, u32, u32, siphash_key_t key);
+u64 siphash_4u32(u32, u32, u32, u32, siphash_key_t key);
+
+If you pass the generic siphash function something of a constant length, it
+will constant fold at compile-time and automatically choose one of the
+optimized functions.
+
+3. Hashtable key function usage:
+
+struct some_hashtable {
+ DECLARE_HASHTABLE(hashtable, 8);
+ siphash_key_t key;
+};
+
+void init_hashtable(struct some_hashtable *table)
+{
+ get_random_bytes(table->key, sizeof(table->key));
+}
+
+static inline hlist_head *some_hashtable_bucket(struct some_hashtable *table, struct interesting_input *input)
+{
+ return &table->hashtable[siphash(input, sizeof(*input), table->key) & (HASH_SIZE(table->hashtable) - 1)];
+}
+
+You may then iterate like usual over the returned hash bucket.
+
+4. Security
+
+SipHash has a very high security margin, with its 128-bit key. So long as the
+key is kept secret, it is impossible for an attacker to guess the outputs of
+the function, even if being able to observe many outputs, since 2^128 outputs
+is significant.
+
+Linux implements the "2-4" variant of SipHash.
+
+5. Resources
+
+Read the SipHash paper if you're interested in learning more:
+https://131002.net/siphash/siphash.pdf
diff --git a/MAINTAINERS b/MAINTAINERS
index cfff2c9e3d94..e1384ae37344 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11291,6 +11291,13 @@ F: arch/arm/mach-s3c24xx/mach-bast.c
F: arch/arm/mach-s3c24xx/bast-ide.c
F: arch/arm/mach-s3c24xx/bast-irq.c
+SIPHASH PRF ROUTINES
+M: Jason A. Donenfeld <Jason@zx2c4.com>
+S: Maintained
+F: lib/siphash.c
+F: lib/test_siphash.c
+F: include/linux/siphash.h
+
TI DAVINCI MACHINE SUPPORT
M: Sekhar Nori <nsekhar@ti.com>
M: Kevin Hilman <khilman@kernel.org>
diff --git a/include/linux/siphash.h b/include/linux/siphash.h
new file mode 100644
index 000000000000..7aa666eb00d9
--- /dev/null
+++ b/include/linux/siphash.h
@@ -0,0 +1,79 @@
+/* Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.
+ *
+ * SipHash: a fast short-input PRF
+ * https://131002.net/siphash/
+ *
+ * This implementation is specifically for SipHash2-4.
+ */
+
+#ifndef _LINUX_SIPHASH_H
+#define _LINUX_SIPHASH_H
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+
+#define SIPHASH_ALIGNMENT __alignof__(u64)
+typedef u64 siphash_key_t[2];
+
+u64 __siphash_aligned(const void *data, size_t len, const siphash_key_t key);
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+u64 __siphash_unaligned(const void *data, size_t len, const siphash_key_t key);
+#endif
+
+u64 siphash_1u64(const u64 a, const siphash_key_t key);
+u64 siphash_2u64(const u64 a, const u64 b, const siphash_key_t key);
+u64 siphash_3u64(const u64 a, const u64 b, const u64 c,
+ const siphash_key_t key);
+u64 siphash_4u64(const u64 a, const u64 b, const u64 c, const u64 d,
+ const siphash_key_t key);
+u64 siphash_1u32(const u32 a, const siphash_key_t key);
+u64 siphash_3u32(const u32 a, const u32 b, const u32 c, const siphash_key_t key);
+
+static inline u64 siphash_2u32(const u32 a, const u32 b, const siphash_key_t key)
+{
+ return siphash_1u64((u64)b << 32 | a, key);
+}
+static inline u64 siphash_4u32(const u32 a, const u32 b, const u32 c, const u32 d,
+ const siphash_key_t key)
+{
+ return siphash_2u64((u64)b << 32 | a, (u64)d << 32 | c, key);
+}
+
+
+static inline u64 ___siphash_aligned(const __le64 *data, size_t len, const siphash_key_t key)
+{
+ if (__builtin_constant_p(len) && len == 4)
+ return siphash_1u32(le32_to_cpu(data[0]), key);
+ if (__builtin_constant_p(len) && len == 8)
+ return siphash_1u64(le64_to_cpu(data[0]), key);
+ if (__builtin_constant_p(len) && len == 16)
+ return siphash_2u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
+ key);
+ if (__builtin_constant_p(len) && len == 24)
+ return siphash_3u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
+ le64_to_cpu(data[2]), key);
+ if (__builtin_constant_p(len) && len == 32)
+ return siphash_4u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
+ le64_to_cpu(data[2]), le64_to_cpu(data[3]),
+ key);
+ return __siphash_aligned(data, len, key);
+}
+
+/**
+ * siphash - compute 64-bit siphash PRF value
+ * @data: buffer to hash
+ * @size: size of @data
+ * @key: the siphash key
+ */
+static inline u64 siphash(const void *data, size_t len, const siphash_key_t key)
+{
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+ if (!IS_ALIGNED((unsigned long)data, SIPHASH_ALIGNMENT))
+ return __siphash_unaligned(data, len, key);
+#endif
+ return ___siphash_aligned(data, len, key);
+}
+
+#endif /* _LINUX_SIPHASH_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index b06848a104e6..3d2515a770c3 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1819,9 +1819,9 @@ config TEST_HASH
tristate "Perform selftest on hash functions"
default n
help
- Enable this option to test the kernel's integer (<linux/hash,h>)
- and string (<linux/stringhash.h>) hash functions on boot
- (or module load).
+ Enable this option to test the kernel's integer (<linux/hash.h>),
+ string (<linux/stringhash.h>), and siphash (<linux/siphash.h>)
+ hash functions on boot (or module load).
This is intended to help people writing architecture-specific
optimized versions. If unsure, say N.
diff --git a/lib/Makefile b/lib/Makefile
index bc4073a8cd08..7b3008d58600 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -22,7 +22,8 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
sha1.o chacha20.o md5.o irq_regs.o argv_split.o \
flex_proportions.o ratelimit.o show_mem.o \
is_single_threaded.o plist.o decompress.o kobject_uevent.o \
- earlycpio.o seq_buf.o nmi_backtrace.o nodemask.o win_minmax.o
+ earlycpio.o seq_buf.o siphash.o \
+ nmi_backtrace.o nodemask.o win_minmax.o
lib-$(CONFIG_MMU) += ioremap.o
lib-$(CONFIG_SMP) += cpumask.o
@@ -44,7 +45,7 @@ obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
obj-y += kstrtox.o
obj-$(CONFIG_TEST_BPF) += test_bpf.o
obj-$(CONFIG_TEST_FIRMWARE) += test_firmware.o
-obj-$(CONFIG_TEST_HASH) += test_hash.o
+obj-$(CONFIG_TEST_HASH) += test_hash.o test_siphash.o
obj-$(CONFIG_TEST_KASAN) += test_kasan.o
obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o
obj-$(CONFIG_TEST_LKM) += test_module.o
diff --git a/lib/siphash.c b/lib/siphash.c
new file mode 100644
index 000000000000..ff2151313667
--- /dev/null
+++ b/lib/siphash.c
@@ -0,0 +1,232 @@
+/* Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.
+ *
+ * SipHash: a fast short-input PRF
+ * https://131002.net/siphash/
+ *
+ * This implementation is specifically for SipHash2-4.
+ */
+
+#include <linux/siphash.h>
+#include <asm/unaligned.h>
+
+#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
+#include <linux/dcache.h>
+#include <asm/word-at-a-time.h>
+#endif
+
+#define SIPROUND \
+ do { \
+ v0 += v1; v1 = rol64(v1, 13); v1 ^= v0; v0 = rol64(v0, 32); \
+ v2 += v3; v3 = rol64(v3, 16); v3 ^= v2; \
+ v0 += v3; v3 = rol64(v3, 21); v3 ^= v0; \
+ v2 += v1; v1 = rol64(v1, 17); v1 ^= v2; v2 = rol64(v2, 32); \
+ } while(0)
+
+#define PREAMBLE(len) \
+ u64 v0 = 0x736f6d6570736575ULL; \
+ u64 v1 = 0x646f72616e646f6dULL; \
+ u64 v2 = 0x6c7967656e657261ULL; \
+ u64 v3 = 0x7465646279746573ULL; \
+ u64 b = ((u64)len) << 56; \
+ v3 ^= key[1]; \
+ v2 ^= key[0]; \
+ v1 ^= key[1]; \
+ v0 ^= key[0];
+
+#define POSTAMBLE \
+ v3 ^= b; \
+ SIPROUND; \
+ SIPROUND; \
+ v0 ^= b; \
+ v2 ^= 0xff; \
+ SIPROUND; \
+ SIPROUND; \
+ SIPROUND; \
+ SIPROUND; \
+ return (v0 ^ v1) ^ (v2 ^ v3);
+
+u64 __siphash_aligned(const void *data, size_t len, const siphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u64));
+ const u8 left = len & (sizeof(u64) - 1);
+ u64 m;
+ PREAMBLE(len)
+ for (; data != end; data += sizeof(u64)) {
+ m = le64_to_cpup(data);
+ v3 ^= m;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= m;
+ }
+#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
+ if (left)
+ b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
+ bytemask_from_count(left)));
+#else
+ switch (left) {
+ case 7: b |= ((u64)end[6]) << 48;
+ case 6: b |= ((u64)end[5]) << 40;
+ case 5: b |= ((u64)end[4]) << 32;
+ case 4: b |= le32_to_cpup(data); break;
+ case 3: b |= ((u64)end[2]) << 16;
+ case 2: b |= le16_to_cpup(data); break;
+ case 1: b |= end[0];
+ }
+#endif
+ POSTAMBLE
+}
+EXPORT_SYMBOL(__siphash_aligned);
+
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+u64 __siphash_unaligned(const void *data, size_t len, const siphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u64));
+ const u8 left = len & (sizeof(u64) - 1);
+ u64 m;
+ PREAMBLE(len)
+ for (; data != end; data += sizeof(u64)) {
+ m = get_unaligned_le64(data);
+ v3 ^= m;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= m;
+ }
+#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
+ if (left)
+ b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
+ bytemask_from_count(left)));
+#else
+ switch (left) {
+ case 7: b |= ((u64)end[6]) << 48;
+ case 6: b |= ((u64)end[5]) << 40;
+ case 5: b |= ((u64)end[4]) << 32;
+ case 4: b |= get_unaligned_le32(end); break;
+ case 3: b |= ((u64)end[2]) << 16;
+ case 2: b |= get_unaligned_le16(end); break;
+ case 1: b |= end[0];
+ }
+#endif
+ POSTAMBLE
+}
+EXPORT_SYMBOL(__siphash_unaligned);
+#endif
+
+/**
+ * siphash_1u64 - compute 64-bit siphash PRF value of a u64
+ * @first: first u64
+ * @key: the siphash key
+ */
+u64 siphash_1u64(const u64 first, const siphash_key_t key)
+{
+ PREAMBLE(8)
+ v3 ^= first;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= first;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_1u64);
+
+/**
+ * siphash_2u64 - compute 64-bit siphash PRF value of 2 u64
+ * @first: first u64
+ * @second: second u64
+ * @key: the siphash key
+ */
+u64 siphash_2u64(const u64 first, const u64 second, const siphash_key_t key)
+{
+ PREAMBLE(16)
+ v3 ^= first;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= second;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_2u64);
+
+/**
+ * siphash_3u64 - compute 64-bit siphash PRF value of 3 u64
+ * @first: first u64
+ * @second: second u64
+ * @third: third u64
+ * @key: the siphash key
+ */
+u64 siphash_3u64(const u64 first, const u64 second, const u64 third,
+ const siphash_key_t key)
+{
+ PREAMBLE(24)
+ v3 ^= first;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= second;
+ v3 ^= third;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= third;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_3u64);
+
+/**
+ * siphash_4u64 - compute 64-bit siphash PRF value of 4 u64
+ * @first: first u64
+ * @second: second u64
+ * @third: third u64
+ * @forth: forth u64
+ * @key: the siphash key
+ */
+u64 siphash_4u64(const u64 first, const u64 second, const u64 third,
+ const u64 forth, const siphash_key_t key)
+{
+ PREAMBLE(32)
+ v3 ^= first;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= second;
+ v3 ^= third;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= third;
+ v3 ^= forth;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= forth;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_4u64);
+
+u64 siphash_1u32(const u32 first, const siphash_key_t key)
+{
+ PREAMBLE(4)
+ b |= first;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_1u32);
+
+u64 siphash_3u32(const u32 first, const u32 second, const u32 third,
+ const siphash_key_t key)
+{
+ u64 combined = (u64)second << 32 | first;
+ PREAMBLE(12)
+ v3 ^= combined;
+ SIPROUND;
+ SIPROUND;
+ v0 ^= combined;
+ b |= third;
+ POSTAMBLE
+}
+EXPORT_SYMBOL(siphash_3u32);
diff --git a/lib/test_siphash.c b/lib/test_siphash.c
new file mode 100644
index 000000000000..e0ba2cf8dc67
--- /dev/null
+++ b/lib/test_siphash.c
@@ -0,0 +1,119 @@
+/* Test cases for siphash.c
+ *
+ * Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
+ *
+ * This file is provided under a dual BSD/GPLv2 license.
+ *
+ * SipHash: a fast short-input PRF
+ * https://131002.net/siphash/
+ *
+ * This implementation is specifically for SipHash2-4.
+ */
+
+#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+
+#include <linux/siphash.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/module.h>
+
+/* Test vectors taken from official reference source available at:
+ * https://131002.net/siphash/siphash24.c
+ */
+static const u64 test_vectors[64] = {
+ 0x726fdb47dd0e0e31ULL, 0x74f839c593dc67fdULL, 0x0d6c8009d9a94f5aULL,
+ 0x85676696d7fb7e2dULL, 0xcf2794e0277187b7ULL, 0x18765564cd99a68dULL,
+ 0xcbc9466e58fee3ceULL, 0xab0200f58b01d137ULL, 0x93f5f5799a932462ULL,
+ 0x9e0082df0ba9e4b0ULL, 0x7a5dbbc594ddb9f3ULL, 0xf4b32f46226bada7ULL,
+ 0x751e8fbc860ee5fbULL, 0x14ea5627c0843d90ULL, 0xf723ca908e7af2eeULL,
+ 0xa129ca6149be45e5ULL, 0x3f2acc7f57c29bdbULL, 0x699ae9f52cbe4794ULL,
+ 0x4bc1b3f0968dd39cULL, 0xbb6dc91da77961bdULL, 0xbed65cf21aa2ee98ULL,
+ 0xd0f2cbb02e3b67c7ULL, 0x93536795e3a33e88ULL, 0xa80c038ccd5ccec8ULL,
+ 0xb8ad50c6f649af94ULL, 0xbce192de8a85b8eaULL, 0x17d835b85bbb15f3ULL,
+ 0x2f2e6163076bcfadULL, 0xde4daaaca71dc9a5ULL, 0xa6a2506687956571ULL,
+ 0xad87a3535c49ef28ULL, 0x32d892fad841c342ULL, 0x7127512f72f27cceULL,
+ 0xa7f32346f95978e3ULL, 0x12e0b01abb051238ULL, 0x15e034d40fa197aeULL,
+ 0x314dffbe0815a3b4ULL, 0x027990f029623981ULL, 0xcadcd4e59ef40c4dULL,
+ 0x9abfd8766a33735cULL, 0x0e3ea96b5304a7d0ULL, 0xad0c42d6fc585992ULL,
+ 0x187306c89bc215a9ULL, 0xd4a60abcf3792b95ULL, 0xf935451de4f21df2ULL,
+ 0xa9538f0419755787ULL, 0xdb9acddff56ca510ULL, 0xd06c98cd5c0975ebULL,
+ 0xe612a3cb9ecba951ULL, 0xc766e62cfcadaf96ULL, 0xee64435a9752fe72ULL,
+ 0xa192d576b245165aULL, 0x0a8787bf8ecb74b2ULL, 0x81b3e73d20b49b6fULL,
+ 0x7fa8220ba3b2eceaULL, 0x245731c13ca42499ULL, 0xb78dbfaf3a8d83bdULL,
+ 0xea1ad565322a1a0bULL, 0x60e61c23a3795013ULL, 0x6606d7e446282b93ULL,
+ 0x6ca4ecb15c5f91e1ULL, 0x9f626da15c9625f3ULL, 0xe51b38608ef25f57ULL,
+ 0x958a324ceb064572ULL
+};
+static const siphash_key_t test_key =
+ { 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
+
+static int __init siphash_test_init(void)
+{
+ u8 in[64] __aligned(SIPHASH_ALIGNMENT);
+ u8 in_unaligned[65];
+ u8 i;
+ int ret = 0;
+
+ for (i = 0; i < 64; ++i) {
+ in[i] = i;
+ in_unaligned[i + 1] = i;
+ if (siphash(in, i, test_key) != test_vectors[i]) {
+ pr_info("self-test aligned %u: FAIL\n", i + 1);
+ ret = -EINVAL;
+ }
+ if (siphash(in_unaligned + 1, i, test_key) != test_vectors[i]) {
+ pr_info("self-test unaligned %u: FAIL\n", i + 1);
+ ret = -EINVAL;
+ }
+ }
+ if (siphash_1u64(0x0706050403020100ULL, test_key) != test_vectors[8]) {
+ pr_info("self-test 1u64: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key) != test_vectors[16]) {
+ pr_info("self-test 2u64: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_3u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
+ 0x1716151413121110ULL, test_key) != test_vectors[24]) {
+ pr_info("self-test 3u64: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_4u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
+ 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key) != test_vectors[32]) {
+ pr_info("self-test 4u64: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_1u32(0x03020100U, test_key) != test_vectors[4]) {
+ pr_info("self-test 1u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_2u32(0x03020100U, 0x07060504U, test_key) != test_vectors[8]) {
+ pr_info("self-test 2u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_3u32(0x03020100U, 0x07060504U,
+ 0x0b0a0908U, test_key) != test_vectors[12]) {
+ pr_info("self-test 3u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (siphash_4u32(0x03020100U, 0x07060504U,
+ 0x0b0a0908U, 0x0f0e0d0cU, test_key) != test_vectors[16]) {
+ pr_info("self-test 4u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (!ret)
+ pr_info("self-tests: pass\n");
+ return ret;
+}
+
+static void __exit siphash_test_exit(void)
+{
+}
+
+module_init(siphash_test_init);
+module_exit(siphash_test_exit);
+
+MODULE_AUTHOR("Jason A. Donenfeld <Jason@zx2c4.com>");
+MODULE_LICENSE("Dual BSD/GPL");
--
2.11.0
^ permalink raw reply related
* [PATCH net-next 2/4] siphash: implement HalfSipHash1-3 for hash tables
From: Jason A. Donenfeld @ 2017-01-06 20:10 UTC (permalink / raw)
To: davem, netdev, linux-kernel; +Cc: Jason A. Donenfeld, Jean-Philippe Aumasson
In-Reply-To: <20170106201055.13765-1-Jason@zx2c4.com>
HalfSipHash, or hsiphash, is a shortened version of SipHash, which
generates 32-bit outputs using a weaker 64-bit key. It has *much* lower
security margins, and shouldn't be used for anything too sensitive, but
it could be used as a hashtable key function replacement, if the output
is never exposed, and if the security requirement is not too high.
The goal is to make this something that performance-critical jhash users
would be willing to use.
On 64-bit machines, HalfSipHash1-3 is slower than SipHash1-3, so we alias
SipHash1-3 to HalfSipHash1-3 on those systems.
64-bit x86_64:
[ 0.509409] test_siphash: SipHash2-4 cycles: 4049181
[ 0.510650] test_siphash: SipHash1-3 cycles: 2512884
[ 0.512205] test_siphash: HalfSipHash1-3 cycles: 3429920
[ 0.512904] test_siphash: JenkinsHash cycles: 978267
So, we map hsiphash() -> SipHash1-3
32-bit x86:
[ 0.509868] test_siphash: SipHash2-4 cycles: 14812892
[ 0.513601] test_siphash: SipHash1-3 cycles: 9510710
[ 0.515263] test_siphash: HalfSipHash1-3 cycles: 3856157
[ 0.515952] test_siphash: JenkinsHash cycles: 1148567
So, we map hsiphash() -> HalfSipHash1-3
hsiphash() is roughly 3 times slower than jhash(), but comes with a
considerable security improvement.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
---
Documentation/siphash.txt | 75 +++++++++++
include/linux/siphash.h | 56 +++++++-
lib/siphash.c | 318 +++++++++++++++++++++++++++++++++++++++++++++-
lib/test_siphash.c | 139 ++++++++++++++++----
4 files changed, 561 insertions(+), 27 deletions(-)
diff --git a/Documentation/siphash.txt b/Documentation/siphash.txt
index 39ff7f0438e7..f93c1d7104c4 100644
--- a/Documentation/siphash.txt
+++ b/Documentation/siphash.txt
@@ -77,3 +77,78 @@ Linux implements the "2-4" variant of SipHash.
Read the SipHash paper if you're interested in learning more:
https://131002.net/siphash/siphash.pdf
+
+
+~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
+
+HalfSipHash - SipHash's insecure younger cousin
+-----------------------------------------------
+Written by Jason A. Donenfeld <jason@zx2c4.com>
+
+On the off-chance that SipHash is not fast enough for your needs, you might be
+able to justify using HalfSipHash, a terrifying but potentially useful
+possibility. HalfSipHash cuts SipHash's rounds down from "2-4" to "1-3" and,
+even scarier, uses an easily brute-forcable 64-bit key (with a 32-bit output)
+instead of SipHash's 128-bit key. However, this may appeal to some
+high-performance `jhash` users.
+
+Danger!
+
+Do not ever use HalfSipHash except for as a hashtable key function, and only
+then when you can be absolutely certain that the outputs will never be
+transmitted out of the kernel. This is only remotely useful over `jhash` as a
+means of mitigating hashtable flooding denial of service attacks.
+
+1. Generating a key
+
+Keys should always be generated from a cryptographically secure source of
+random numbers, either using get_random_bytes or get_random_once:
+
+hsiphash_key_t key;
+get_random_bytes(key, sizeof(key));
+
+If you're not deriving your key from here, you're doing it wrong.
+
+2. Using the functions
+
+There are two variants of the function, one that takes a list of integers, and
+one that takes a buffer:
+
+u32 hsiphash(const void *data, size_t len, siphash_key_t key);
+
+And:
+
+u32 hsiphash_1u32(u32, hsiphash_key_t key);
+u32 hsiphash_2u32(u32, u32, hsiphash_key_t key);
+u32 hsiphash_3u32(u32, u32, u32, hsiphash_key_t key);
+u32 hsiphash_4u32(u32, u32, u32, u32, hsiphash_key_t key);
+
+If you pass the generic hsiphash function something of a constant length, it
+will constant fold at compile-time and automatically choose one of the
+optimized functions.
+
+3. Hashtable key function usage:
+
+struct some_hashtable {
+ DECLARE_HASHTABLE(hashtable, 8);
+ hsiphash_key_t key;
+};
+
+void init_hashtable(struct some_hashtable *table)
+{
+ get_random_bytes(table->key, sizeof(table->key));
+}
+
+static inline hlist_head *some_hashtable_bucket(struct some_hashtable *table, struct interesting_input *input)
+{
+ return &table->hashtable[hsiphash(input, sizeof(*input), table->key) & (HASH_SIZE(table->hashtable) - 1)];
+}
+
+You may then iterate like usual over the returned hash bucket.
+
+4. Performance
+
+HalfSipHash is roughly 3 times slower than JenkinsHash. For many replacements,
+this will not be a problem, as the hashtable lookup isn't the bottleneck. And
+in general, this is probably a good sacrifice to make for the security and DoS
+resistance of HalfSipHash.
diff --git a/include/linux/siphash.h b/include/linux/siphash.h
index 7aa666eb00d9..efab44c654f3 100644
--- a/include/linux/siphash.h
+++ b/include/linux/siphash.h
@@ -5,7 +5,9 @@
* SipHash: a fast short-input PRF
* https://131002.net/siphash/
*
- * This implementation is specifically for SipHash2-4.
+ * This implementation is specifically for SipHash2-4 for a secure PRF
+ * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
+ * hashtables.
*/
#ifndef _LINUX_SIPHASH_H
@@ -76,4 +78,56 @@ static inline u64 siphash(const void *data, size_t len, const siphash_key_t key)
return ___siphash_aligned(data, len, key);
}
+#if BITS_PER_LONG == 64
+typedef siphash_key_t hsiphash_key_t;
+#define HSIPHASH_ALIGNMENT SIPHASH_ALIGNMENT
+#else
+typedef u32 hsiphash_key_t[2];
+#define HSIPHASH_ALIGNMENT __alignof__(u32)
+#endif
+
+u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key);
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key);
+#endif
+
+u32 hsiphash_1u32(const u32 a, const hsiphash_key_t key);
+u32 hsiphash_2u32(const u32 a, const u32 b, const hsiphash_key_t key);
+u32 hsiphash_3u32(const u32 a, const u32 b, const u32 c,
+ const hsiphash_key_t key);
+u32 hsiphash_4u32(const u32 a, const u32 b, const u32 c, const u32 d,
+ const hsiphash_key_t key);
+
+static inline u32 ___hsiphash_aligned(const __le32 *data, size_t len, const hsiphash_key_t key)
+{
+ if (__builtin_constant_p(len) && len == 4)
+ return hsiphash_1u32(le32_to_cpu(data[0]), key);
+ if (__builtin_constant_p(len) && len == 8)
+ return hsiphash_2u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
+ key);
+ if (__builtin_constant_p(len) && len == 12)
+ return hsiphash_3u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
+ le32_to_cpu(data[2]), key);
+ if (__builtin_constant_p(len) && len == 16)
+ return hsiphash_4u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
+ le32_to_cpu(data[2]), le32_to_cpu(data[3]),
+ key);
+ return __hsiphash_aligned(data, len, key);
+}
+
+/**
+ * hsiphash - compute 32-bit hsiphash PRF value
+ * @data: buffer to hash
+ * @size: size of @data
+ * @key: the hsiphash key
+ */
+static inline u32 hsiphash(const void *data, size_t len, const hsiphash_key_t key)
+{
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+ if (!IS_ALIGNED((unsigned long)data, HSIPHASH_ALIGNMENT))
+ return __hsiphash_unaligned(data, len, key);
+#endif
+ return ___hsiphash_aligned(data, len, key);
+}
+
#endif /* _LINUX_SIPHASH_H */
diff --git a/lib/siphash.c b/lib/siphash.c
index ff2151313667..e2481226d96c 100644
--- a/lib/siphash.c
+++ b/lib/siphash.c
@@ -5,7 +5,9 @@
* SipHash: a fast short-input PRF
* https://131002.net/siphash/
*
- * This implementation is specifically for SipHash2-4.
+ * This implementation is specifically for SipHash2-4 for a secure PRF
+ * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
+ * hashtables.
*/
#include <linux/siphash.h>
@@ -230,3 +232,317 @@ u64 siphash_3u32(const u32 first, const u32 second, const u32 third,
POSTAMBLE
}
EXPORT_SYMBOL(siphash_3u32);
+
+#if BITS_PER_LONG == 64
+/* Note that this HalfSipHash1-3 implementation on 64-bit
+ * isn't actually HalfSipHash1-3 but rather SipHash1-3. */
+
+#define HSIPROUND SIPROUND
+#define HPREAMBLE(len) PREAMBLE(len)
+#define HPOSTAMBLE \
+ v3 ^= b; \
+ HSIPROUND; \
+ v0 ^= b; \
+ v2 ^= 0xff; \
+ HSIPROUND; \
+ HSIPROUND; \
+ HSIPROUND; \
+ return (v0 ^ v1) ^ (v2 ^ v3);
+
+u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u64));
+ const u8 left = len & (sizeof(u64) - 1);
+ u64 m;
+ HPREAMBLE(len)
+ for (; data != end; data += sizeof(u64)) {
+ m = le64_to_cpup(data);
+ v3 ^= m;
+ HSIPROUND;
+ v0 ^= m;
+ }
+#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
+ if (left)
+ b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
+ bytemask_from_count(left)));
+#else
+ switch (left) {
+ case 7: b |= ((u64)end[6]) << 48;
+ case 6: b |= ((u64)end[5]) << 40;
+ case 5: b |= ((u64)end[4]) << 32;
+ case 4: b |= le32_to_cpup(data); break;
+ case 3: b |= ((u64)end[2]) << 16;
+ case 2: b |= le16_to_cpup(data); break;
+ case 1: b |= end[0];
+ }
+#endif
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(__hsiphash_aligned);
+
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u64));
+ const u8 left = len & (sizeof(u64) - 1);
+ u64 m;
+ HPREAMBLE(len)
+ for (; data != end; data += sizeof(u64)) {
+ m = get_unaligned_le64(data);
+ v3 ^= m;
+ HSIPROUND;
+ v0 ^= m;
+ }
+#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
+ if (left)
+ b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
+ bytemask_from_count(left)));
+#else
+ switch (left) {
+ case 7: b |= ((u64)end[6]) << 48;
+ case 6: b |= ((u64)end[5]) << 40;
+ case 5: b |= ((u64)end[4]) << 32;
+ case 4: b |= get_unaligned_le32(end); break;
+ case 3: b |= ((u64)end[2]) << 16;
+ case 2: b |= get_unaligned_le16(end); break;
+ case 1: b |= end[0];
+ }
+#endif
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(__hsiphash_unaligned);
+#endif
+
+/**
+ * hsiphash_1u32 - compute 64-bit hsiphash PRF value of a u32
+ * @first: first u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_1u32(const u32 first, const hsiphash_key_t key)
+{
+ HPREAMBLE(4)
+ b |= first;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_1u32);
+
+/**
+ * hsiphash_2u32 - compute 32-bit hsiphash PRF value of 2 u32
+ * @first: first u32
+ * @second: second u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_2u32(const u32 first, const u32 second, const hsiphash_key_t key)
+{
+ u64 combined = (u64)second << 32 | first;
+ HPREAMBLE(8)
+ v3 ^= combined;
+ HSIPROUND;
+ v0 ^= combined;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_2u32);
+
+/**
+ * hsiphash_3u32 - compute 32-bit hsiphash PRF value of 3 u32
+ * @first: first u32
+ * @second: second u32
+ * @third: third u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_3u32(const u32 first, const u32 second, const u32 third,
+ const hsiphash_key_t key)
+{
+ u64 combined = (u64)second << 32 | first;
+ HPREAMBLE(12)
+ v3 ^= combined;
+ HSIPROUND;
+ v0 ^= combined;
+ b |= third;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_3u32);
+
+/**
+ * hsiphash_4u32 - compute 32-bit hsiphash PRF value of 4 u32
+ * @first: first u32
+ * @second: second u32
+ * @third: third u32
+ * @forth: forth u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_4u32(const u32 first, const u32 second, const u32 third,
+ const u32 forth, const hsiphash_key_t key)
+{
+ u64 combined = (u64)second << 32 | first;
+ HPREAMBLE(16)
+ v3 ^= combined;
+ HSIPROUND;
+ v0 ^= combined;
+ combined = (u64)forth << 32 | third;
+ v3 ^= combined;
+ HSIPROUND;
+ v0 ^= combined;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_4u32);
+#else
+#define HSIPROUND \
+ do { \
+ v0 += v1; v1 = rol32(v1, 5); v1 ^= v0; v0 = rol32(v0, 16); \
+ v2 += v3; v3 = rol32(v3, 8); v3 ^= v2; \
+ v0 += v3; v3 = rol32(v3, 7); v3 ^= v0; \
+ v2 += v1; v1 = rol32(v1, 13); v1 ^= v2; v2 = rol32(v2, 16); \
+ } while(0)
+
+#define HPREAMBLE(len) \
+ u32 v0 = 0; \
+ u32 v1 = 0; \
+ u32 v2 = 0x6c796765U; \
+ u32 v3 = 0x74656462U; \
+ u32 b = ((u32)len) << 24; \
+ v3 ^= key[1]; \
+ v2 ^= key[0]; \
+ v1 ^= key[1]; \
+ v0 ^= key[0];
+
+#define HPOSTAMBLE \
+ v3 ^= b; \
+ HSIPROUND; \
+ v0 ^= b; \
+ v2 ^= 0xff; \
+ HSIPROUND; \
+ HSIPROUND; \
+ HSIPROUND; \
+ return v1 ^ v3;
+
+u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u32));
+ const u8 left = len & (sizeof(u32) - 1);
+ u32 m;
+ HPREAMBLE(len)
+ for (; data != end; data += sizeof(u32)) {
+ m = le32_to_cpup(data);
+ v3 ^= m;
+ HSIPROUND;
+ v0 ^= m;
+ }
+ switch (left) {
+ case 3: b |= ((u32)end[2]) << 16;
+ case 2: b |= le16_to_cpup(data); break;
+ case 1: b |= end[0];
+ }
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(__hsiphash_aligned);
+
+#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
+u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key)
+{
+ const u8 *end = data + len - (len % sizeof(u32));
+ const u8 left = len & (sizeof(u32) - 1);
+ u32 m;
+ HPREAMBLE(len)
+ for (; data != end; data += sizeof(u32)) {
+ m = get_unaligned_le32(data);
+ v3 ^= m;
+ HSIPROUND;
+ v0 ^= m;
+ }
+ switch (left) {
+ case 3: b |= ((u32)end[2]) << 16;
+ case 2: b |= get_unaligned_le16(end); break;
+ case 1: b |= end[0];
+ }
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(__hsiphash_unaligned);
+#endif
+
+/**
+ * hsiphash_1u32 - compute 32-bit hsiphash PRF value of a u32
+ * @first: first u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_1u32(const u32 first, const hsiphash_key_t key)
+{
+ HPREAMBLE(4)
+ v3 ^= first;
+ HSIPROUND;
+ v0 ^= first;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_1u32);
+
+/**
+ * hsiphash_2u32 - compute 32-bit hsiphash PRF value of 2 u32
+ * @first: first u32
+ * @second: second u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_2u32(const u32 first, const u32 second, const hsiphash_key_t key)
+{
+ HPREAMBLE(8)
+ v3 ^= first;
+ HSIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ HSIPROUND;
+ v0 ^= second;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_2u32);
+
+/**
+ * hsiphash_3u32 - compute 32-bit hsiphash PRF value of 3 u32
+ * @first: first u32
+ * @second: second u32
+ * @third: third u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_3u32(const u32 first, const u32 second, const u32 third,
+ const hsiphash_key_t key)
+{
+ HPREAMBLE(12)
+ v3 ^= first;
+ HSIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ HSIPROUND;
+ v0 ^= second;
+ v3 ^= third;
+ HSIPROUND;
+ v0 ^= third;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_3u32);
+
+/**
+ * hsiphash_4u32 - compute 32-bit hsiphash PRF value of 4 u32
+ * @first: first u32
+ * @second: second u32
+ * @third: third u32
+ * @forth: forth u32
+ * @key: the hsiphash key
+ */
+u32 hsiphash_4u32(const u32 first, const u32 second, const u32 third,
+ const u32 forth, const hsiphash_key_t key)
+{
+ HPREAMBLE(16)
+ v3 ^= first;
+ HSIPROUND;
+ v0 ^= first;
+ v3 ^= second;
+ HSIPROUND;
+ v0 ^= second;
+ v3 ^= third;
+ HSIPROUND;
+ v0 ^= third;
+ v3 ^= forth;
+ HSIPROUND;
+ v0 ^= forth;
+ HPOSTAMBLE
+}
+EXPORT_SYMBOL(hsiphash_4u32);
+#endif
diff --git a/lib/test_siphash.c b/lib/test_siphash.c
index e0ba2cf8dc67..ac291ec27fb6 100644
--- a/lib/test_siphash.c
+++ b/lib/test_siphash.c
@@ -7,7 +7,9 @@
* SipHash: a fast short-input PRF
* https://131002.net/siphash/
*
- * This implementation is specifically for SipHash2-4.
+ * This implementation is specifically for SipHash2-4 for a secure PRF
+ * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
+ * hashtables.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
@@ -18,10 +20,16 @@
#include <linux/errno.h>
#include <linux/module.h>
-/* Test vectors taken from official reference source available at:
- * https://131002.net/siphash/siphash24.c
+/* Test vectors taken from reference source available at:
+ * https://github.com/veorq/SipHash
*/
-static const u64 test_vectors[64] = {
+
+
+
+static const siphash_key_t test_key_siphash =
+ { 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
+
+static const u64 test_vectors_siphash[64] = {
0x726fdb47dd0e0e31ULL, 0x74f839c593dc67fdULL, 0x0d6c8009d9a94f5aULL,
0x85676696d7fb7e2dULL, 0xcf2794e0277187b7ULL, 0x18765564cd99a68dULL,
0xcbc9466e58fee3ceULL, 0xab0200f58b01d137ULL, 0x93f5f5799a932462ULL,
@@ -45,9 +53,64 @@ static const u64 test_vectors[64] = {
0x6ca4ecb15c5f91e1ULL, 0x9f626da15c9625f3ULL, 0xe51b38608ef25f57ULL,
0x958a324ceb064572ULL
};
-static const siphash_key_t test_key =
+#if BITS_PER_LONG == 64
+static const hsiphash_key_t test_key_hsiphash =
{ 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
+static const u32 test_vectors_hsiphash[64] = {
+ 0x050fc4dcU, 0x7d57ca93U, 0x4dc7d44dU,
+ 0xe7ddf7fbU, 0x88d38328U, 0x49533b67U,
+ 0xc59f22a7U, 0x9bb11140U, 0x8d299a8eU,
+ 0x6c063de4U, 0x92ff097fU, 0xf94dc352U,
+ 0x57b4d9a2U, 0x1229ffa7U, 0xc0f95d34U,
+ 0x2a519956U, 0x7d908b66U, 0x63dbd80cU,
+ 0xb473e63eU, 0x8d297d1cU, 0xa6cce040U,
+ 0x2b45f844U, 0xa320872eU, 0xdae6c123U,
+ 0x67349c8cU, 0x705b0979U, 0xca9913a5U,
+ 0x4ade3b35U, 0xef6cd00dU, 0x4ab1e1f4U,
+ 0x43c5e663U, 0x8c21d1bcU, 0x16a7b60dU,
+ 0x7a8ff9bfU, 0x1f2a753eU, 0xbf186b91U,
+ 0xada26206U, 0xa3c33057U, 0xae3a36a1U,
+ 0x7b108392U, 0x99e41531U, 0x3f1ad944U,
+ 0xc8138825U, 0xc28949a6U, 0xfaf8876bU,
+ 0x9f042196U, 0x68b1d623U, 0x8b5114fdU,
+ 0xdf074c46U, 0x12cc86b3U, 0x0a52098fU,
+ 0x9d292f9aU, 0xa2f41f12U, 0x43a71ed0U,
+ 0x73f0bce6U, 0x70a7e980U, 0x243c6d75U,
+ 0xfdb71513U, 0xa67d8a08U, 0xb7e8f148U,
+ 0xf7a644eeU, 0x0f1837f2U, 0x4b6694e0U,
+ 0xb7bbb3a8U
+};
+#else
+static const hsiphash_key_t test_key_hsiphash =
+ { 0x03020100U, 0x07060504U };
+
+static const u32 test_vectors_hsiphash[64] = {
+ 0x5814c896U, 0xe7e864caU, 0xbc4b0e30U,
+ 0x01539939U, 0x7e059ea6U, 0x88e3d89bU,
+ 0xa0080b65U, 0x9d38d9d6U, 0x577999b1U,
+ 0xc839caedU, 0xe4fa32cfU, 0x959246eeU,
+ 0x6b28096cU, 0x66dd9cd6U, 0x16658a7cU,
+ 0xd0257b04U, 0x8b31d501U, 0x2b1cd04bU,
+ 0x06712339U, 0x522aca67U, 0x911bb605U,
+ 0x90a65f0eU, 0xf826ef7bU, 0x62512debU,
+ 0x57150ad7U, 0x5d473507U, 0x1ec47442U,
+ 0xab64afd3U, 0x0a4100d0U, 0x6d2ce652U,
+ 0x2331b6a3U, 0x08d8791aU, 0xbc6dda8dU,
+ 0xe0f6c934U, 0xb0652033U, 0x9b9851ccU,
+ 0x7c46fb7fU, 0x732ba8cbU, 0xf142997aU,
+ 0xfcc9aa1bU, 0x05327eb2U, 0xe110131cU,
+ 0xf9e5e7c0U, 0xa7d708a6U, 0x11795ab1U,
+ 0x65671619U, 0x9f5fff91U, 0xd89c5267U,
+ 0x007783ebU, 0x95766243U, 0xab639262U,
+ 0x9c7e1390U, 0xc368dda6U, 0x38ddc455U,
+ 0xfa13d379U, 0x979ea4e8U, 0x53ecd77eU,
+ 0x2ee80657U, 0x33dbb66aU, 0xae3f0577U,
+ 0x88b4c4ccU, 0x3e7f480bU, 0x74c1ebf8U,
+ 0x87178304U
+};
+#endif
+
static int __init siphash_test_init(void)
{
u8 in[64] __aligned(SIPHASH_ALIGNMENT);
@@ -58,49 +121,75 @@ static int __init siphash_test_init(void)
for (i = 0; i < 64; ++i) {
in[i] = i;
in_unaligned[i + 1] = i;
- if (siphash(in, i, test_key) != test_vectors[i]) {
- pr_info("self-test aligned %u: FAIL\n", i + 1);
+ if (siphash(in, i, test_key_siphash) != test_vectors_siphash[i]) {
+ pr_info("siphash self-test aligned %u: FAIL\n", i + 1);
+ ret = -EINVAL;
+ }
+ if (siphash(in_unaligned + 1, i, test_key_siphash) != test_vectors_siphash[i]) {
+ pr_info("siphash self-test unaligned %u: FAIL\n", i + 1);
ret = -EINVAL;
}
- if (siphash(in_unaligned + 1, i, test_key) != test_vectors[i]) {
- pr_info("self-test unaligned %u: FAIL\n", i + 1);
+ if (hsiphash(in, i, test_key_hsiphash) != test_vectors_hsiphash[i]) {
+ pr_info("hsiphash self-test aligned %u: FAIL\n", i + 1);
+ ret = -EINVAL;
+ }
+ if (hsiphash(in_unaligned + 1, i, test_key_hsiphash) != test_vectors_hsiphash[i]) {
+ pr_info("hsiphash self-test unaligned %u: FAIL\n", i + 1);
ret = -EINVAL;
}
}
- if (siphash_1u64(0x0706050403020100ULL, test_key) != test_vectors[8]) {
- pr_info("self-test 1u64: FAIL\n");
+ if (siphash_1u64(0x0706050403020100ULL, test_key_siphash) != test_vectors_siphash[8]) {
+ pr_info("siphash self-test 1u64: FAIL\n");
ret = -EINVAL;
}
- if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key) != test_vectors[16]) {
- pr_info("self-test 2u64: FAIL\n");
+ if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key_siphash) != test_vectors_siphash[16]) {
+ pr_info("siphash self-test 2u64: FAIL\n");
ret = -EINVAL;
}
if (siphash_3u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
- 0x1716151413121110ULL, test_key) != test_vectors[24]) {
- pr_info("self-test 3u64: FAIL\n");
+ 0x1716151413121110ULL, test_key_siphash) != test_vectors_siphash[24]) {
+ pr_info("siphash self-test 3u64: FAIL\n");
ret = -EINVAL;
}
if (siphash_4u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
- 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key) != test_vectors[32]) {
- pr_info("self-test 4u64: FAIL\n");
+ 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key_siphash) != test_vectors_siphash[32]) {
+ pr_info("siphash self-test 4u64: FAIL\n");
ret = -EINVAL;
}
- if (siphash_1u32(0x03020100U, test_key) != test_vectors[4]) {
- pr_info("self-test 1u32: FAIL\n");
+ if (siphash_1u32(0x03020100U, test_key_siphash) != test_vectors_siphash[4]) {
+ pr_info("siphash self-test 1u32: FAIL\n");
ret = -EINVAL;
}
- if (siphash_2u32(0x03020100U, 0x07060504U, test_key) != test_vectors[8]) {
- pr_info("self-test 2u32: FAIL\n");
+ if (siphash_2u32(0x03020100U, 0x07060504U, test_key_siphash) != test_vectors_siphash[8]) {
+ pr_info("siphash self-test 2u32: FAIL\n");
ret = -EINVAL;
}
if (siphash_3u32(0x03020100U, 0x07060504U,
- 0x0b0a0908U, test_key) != test_vectors[12]) {
- pr_info("self-test 3u32: FAIL\n");
+ 0x0b0a0908U, test_key_siphash) != test_vectors_siphash[12]) {
+ pr_info("siphash self-test 3u32: FAIL\n");
ret = -EINVAL;
}
if (siphash_4u32(0x03020100U, 0x07060504U,
- 0x0b0a0908U, 0x0f0e0d0cU, test_key) != test_vectors[16]) {
- pr_info("self-test 4u32: FAIL\n");
+ 0x0b0a0908U, 0x0f0e0d0cU, test_key_siphash) != test_vectors_siphash[16]) {
+ pr_info("siphash self-test 4u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (hsiphash_1u32(0x03020100U, test_key_hsiphash) != test_vectors_hsiphash[4]) {
+ pr_info("hsiphash self-test 1u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (hsiphash_2u32(0x03020100U, 0x07060504U, test_key_hsiphash) != test_vectors_hsiphash[8]) {
+ pr_info("hsiphash self-test 2u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (hsiphash_3u32(0x03020100U, 0x07060504U,
+ 0x0b0a0908U, test_key_hsiphash) != test_vectors_hsiphash[12]) {
+ pr_info("hsiphash self-test 3u32: FAIL\n");
+ ret = -EINVAL;
+ }
+ if (hsiphash_4u32(0x03020100U, 0x07060504U,
+ 0x0b0a0908U, 0x0f0e0d0cU, test_key_hsiphash) != test_vectors_hsiphash[16]) {
+ pr_info("hsiphash self-test 4u32: FAIL\n");
ret = -EINVAL;
}
if (!ret)
--
2.11.0
^ permalink raw reply related
* [PATCH net-next 4/4] syncookies: use SipHash in place of SHA1
From: Jason A. Donenfeld @ 2017-01-06 20:10 UTC (permalink / raw)
To: davem, netdev, linux-kernel; +Cc: Jason A. Donenfeld, Eric Dumazet
In-Reply-To: <20170106201055.13765-1-Jason@zx2c4.com>
SHA1 is slower and less secure than SipHash, and so replacing syncookie
generation with SipHash makes natural sense. Some BSDs have been doing
this for several years in fact.
The speedup should be similar -- and even more impressive -- to the
speedup from the sequence number fix in this series.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: David Miller <davem@davemloft.net>
---
net/ipv4/syncookies.c | 20 ++++----------------
net/ipv6/syncookies.c | 37 ++++++++++++++++---------------------
2 files changed, 20 insertions(+), 37 deletions(-)
diff --git a/net/ipv4/syncookies.c b/net/ipv4/syncookies.c
index 3e88467d70ee..03bb068f8888 100644
--- a/net/ipv4/syncookies.c
+++ b/net/ipv4/syncookies.c
@@ -13,13 +13,13 @@
#include <linux/tcp.h>
#include <linux/slab.h>
#include <linux/random.h>
-#include <linux/cryptohash.h>
+#include <linux/siphash.h>
#include <linux/kernel.h>
#include <linux/export.h>
#include <net/tcp.h>
#include <net/route.h>
-static u32 syncookie_secret[2][16-4+SHA_DIGEST_WORDS] __read_mostly;
+static siphash_key_t syncookie_secret[2] __read_mostly;
#define COOKIEBITS 24 /* Upper bits store count */
#define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1)
@@ -48,24 +48,12 @@ static u32 syncookie_secret[2][16-4+SHA_DIGEST_WORDS] __read_mostly;
#define TSBITS 6
#define TSMASK (((__u32)1 << TSBITS) - 1)
-static DEFINE_PER_CPU(__u32 [16 + 5 + SHA_WORKSPACE_WORDS], ipv4_cookie_scratch);
-
static u32 cookie_hash(__be32 saddr, __be32 daddr, __be16 sport, __be16 dport,
u32 count, int c)
{
- __u32 *tmp;
-
net_get_random_once(syncookie_secret, sizeof(syncookie_secret));
-
- tmp = this_cpu_ptr(ipv4_cookie_scratch);
- memcpy(tmp + 4, syncookie_secret[c], sizeof(syncookie_secret[c]));
- tmp[0] = (__force u32)saddr;
- tmp[1] = (__force u32)daddr;
- tmp[2] = ((__force u32)sport << 16) + (__force u32)dport;
- tmp[3] = count;
- sha_transform(tmp + 16, (__u8 *)tmp, tmp + 16 + 5);
-
- return tmp[17];
+ return siphash_4u32(saddr, daddr, (u32)sport << 16 | dport, count,
+ syncookie_secret[c]);
}
diff --git a/net/ipv6/syncookies.c b/net/ipv6/syncookies.c
index a4d49760bf43..be51fc0d99ad 100644
--- a/net/ipv6/syncookies.c
+++ b/net/ipv6/syncookies.c
@@ -16,7 +16,7 @@
#include <linux/tcp.h>
#include <linux/random.h>
-#include <linux/cryptohash.h>
+#include <linux/siphash.h>
#include <linux/kernel.h>
#include <net/ipv6.h>
#include <net/tcp.h>
@@ -24,7 +24,7 @@
#define COOKIEBITS 24 /* Upper bits store count */
#define COOKIEMASK (((__u32)1 << COOKIEBITS) - 1)
-static u32 syncookie6_secret[2][16-4+SHA_DIGEST_WORDS] __read_mostly;
+static siphash_key_t syncookie6_secret[2] __read_mostly;
/* RFC 2460, Section 8.3:
* [ipv6 tcp] MSS must be computed as the maximum packet size minus 60 [..]
@@ -41,30 +41,25 @@ static __u16 const msstab[] = {
9000 - 60,
};
-static DEFINE_PER_CPU(__u32 [16 + 5 + SHA_WORKSPACE_WORDS], ipv6_cookie_scratch);
-
static u32 cookie_hash(const struct in6_addr *saddr, const struct in6_addr *daddr,
__be16 sport, __be16 dport, u32 count, int c)
{
- __u32 *tmp;
+ const struct {
+ struct in6_addr saddr;
+ struct in6_addr daddr;
+ u32 count;
+ u16 sport;
+ u16 dport;
+ } __aligned(SIPHASH_ALIGNMENT) combined = {
+ .saddr = *saddr,
+ .daddr = *daddr,
+ .count = count,
+ .sport = sport,
+ .dport = dport
+ };
net_get_random_once(syncookie6_secret, sizeof(syncookie6_secret));
-
- tmp = this_cpu_ptr(ipv6_cookie_scratch);
-
- /*
- * we have 320 bits of information to hash, copy in the remaining
- * 192 bits required for sha_transform, from the syncookie6_secret
- * and overwrite the digest with the secret
- */
- memcpy(tmp + 10, syncookie6_secret[c], 44);
- memcpy(tmp, saddr, 16);
- memcpy(tmp + 4, daddr, 16);
- tmp[8] = ((__force u32)sport << 16) + (__force u32)dport;
- tmp[9] = count;
- sha_transform(tmp + 16, (__u8 *)tmp, tmp + 16 + 5);
-
- return tmp[17];
+ return siphash(&combined, offsetofend(typeof(combined), dport), syncookie6_secret[c]);
}
static __u32 secure_tcp_syn_cookie(const struct in6_addr *saddr,
--
2.11.0
^ permalink raw reply related
* Re: [PATCH net] net: Fix inconsistent rtnl_lock usage on dev_get_stats().
From: Eric Dumazet @ 2017-01-06 20:13 UTC (permalink / raw)
To: David Miller; +Cc: michael.chan, netdev
In-Reply-To: <20170106.130134.50153126758574257.davem@davemloft.net>
On Fri, 2017-01-06 at 13:01 -0500, David Miller wrote:
> From: Eric Dumazet <eric.dumazet@gmail.com>
> Date: Fri, 06 Jan 2017 09:32:56 -0800
>
> > This makes no sense to me.
> >
> > RTNL is absolutely not needed to get device stats.
> >
> > We try to not add RTNL, especially when not required.
> >
> > Sure, RTNETLINK dumps currently hold RTNL, but we had various attempts
> > in the past to get rid of this behavior.
> >
> > If a device driver expects RTNL being locked, it is clearly a bug that
> > needs a fix anyway.
>
> This is extremely problematic when the driver has to synchronize some
> piece of state between the get stats method and open/close. It is
> exactly the case we are trying to solve in tg3, and lots of drivers
> end up hitting the same exact issue.
>
> If open/close can happen asynchronously to get stats, it is very hard
> to make dynamically allocated data structures or DMA buffers usable
> from the stats call.
Yes, I had some issues lately with mlx4. netdevices are protected by
RCU, adding proper RCU logic for the stats is doable.
>
> Drivers in this situation will just add a mutex specifically for this
> situation if we don't consistently apply RTNL locking here.
Well, there are cases where RTNL is quite contended, but supervisions
like to get /proc/net/devices or various sysfs attributes
(netstat_show() can be called very very often
for /sys/class/net/*/statistics/*) in a reasonable amount of time.
I fear that such a change will add drifts, when devices are constantly
added/removed.
^ permalink raw reply
* Re: [PATCH net-next] liquidio VF: fix incorrect struct being used
From: David Miller @ 2017-01-06 20:31 UTC (permalink / raw)
To: felix.manlunas; +Cc: netdev, raghu.vatsavayi, derek.chickles, satananda.burla
In-Reply-To: <20170104193155.GA19084@felix.cavium.com>
From: Felix Manlunas <felix.manlunas@cavium.com>
Date: Wed, 4 Jan 2017 11:31:55 -0800
> From: Prasad Kanneganti <prasad.kanneganti@cavium.com>
>
> The VF driver is using the wrong struct when sending commands to the NIC
> firmware, sometimes causing adverse effects in the firmware. The right
> struct is the one that the PF is using, so make the VF use that as well.
>
> Signed-off-by: Prasad Kanneganti <prasad.kanneganti@cavium.com>
> Signed-off-by: Felix Manlunas <felix.manlunas@cavium.com>
> Signed-off-by: Derek Chickles <derek.chickles@cavium.com>
> Signed-off-by: Satanand Burla <satananda.burla@cavium.com>
Applied.
^ permalink raw reply
* Re: [PATCH net-next 1/4] siphash: add cryptographically secure PRF
From: Jean-Philippe Aumasson @ 2017-01-06 20:32 UTC (permalink / raw)
To: Jason A. Donenfeld
Cc: David Miller, Netdev, LKML, Linus Torvalds, Eric Biggers,
David Laight, Eric Dumazet
In-Reply-To: <20170106201055.13765-2-Jason@zx2c4.com>
>
>
> On Fri, Jan 6, 2017 at 9:11 PM Jason A. Donenfeld <Jason@zx2c4.com> wrote:
>
> SipHash is a 64-bit keyed hash function that is actually a
> cryptographically secure PRF, like HMAC. Except SipHash is super fast,
> and is meant to be used as a hashtable keyed lookup function, or as a
> general PRF for short input use cases, such as sequence numbers or RNG
> chaining.
>
> For the first usage:
>
> There are a variety of attacks known as "hashtable poisoning" in which an
> attacker forms some data such that the hash of that data will be the
> same, and then preceeds to fill up all entries of a hashbucket. This is
> a realistic and well-known denial-of-service vector. Currently
> hashtables use jhash, which is fast but not secure, and some kind of
> rotating key scheme (or none at all, which isn't good). SipHash is meant
> as a replacement for jhash in these cases.
>
> There are a modicum of places in the kernel that are vulnerable to
> hashtable poisoning attacks, either via userspace vectors or network
> vectors, and there's not a reliable mechanism inside the kernel at the
> moment to fix it. The first step toward fixing these issues is actually
> getting a secure primitive into the kernel for developers to use. Then
> we can, bit by bit, port things over to it as deemed appropriate.
>
> While SipHash is extremely fast for a cryptographically secure function,
> it is likely a bit slower than the insecure jhash, and so replacements
> will be evaluated on a case-by-case basis based on whether or not the
> difference in speed is negligible and whether or not the current jhash usage
> poses a real security risk.
>
> For the second usage:
>
> A few places in the kernel are using MD5 or SHA1 for creating secure
> sequence numbers, syn cookies, port numbers, or fast random numbers.
> SipHash is a faster and more fitting, and more secure replacement for MD5
> in those situations. Replacing MD5 and SHA1 with SipHash for these uses is
> obvious and straight-forward, and so is submitted along with this patch
> series. There shouldn't be much of a debate over its efficacy.
>
> Dozens of languages are already using this internally for their hash
> tables and PRFs. Some of the BSDs already use this in their kernels.
> SipHash is a widely known high-speed solution to a widely known set of
> problems, and it's time we catch-up.
>
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Eric Biggers <ebiggers3@gmail.com>
> Cc: David Laight <David.Laight@aculab.com>
> Cc: Eric Dumazet <eric.dumazet@gmail.com>
> ---
> Documentation/siphash.txt | 79 ++++++++++++++++
> MAINTAINERS | 7 ++
> include/linux/siphash.h | 79 ++++++++++++++++
> lib/Kconfig.debug | 6 +-
> lib/Makefile | 5 +-
> lib/siphash.c | 232 ++++++++++++++++++++++++++++++++++++++++++++++
> lib/test_siphash.c | 119 ++++++++++++++++++++++++
> 7 files changed, 522 insertions(+), 5 deletions(-)
> create mode 100644 Documentation/siphash.txt
> create mode 100644 include/linux/siphash.h
> create mode 100644 lib/siphash.c
> create mode 100644 lib/test_siphash.c
>
> diff --git a/Documentation/siphash.txt b/Documentation/siphash.txt
> new file mode 100644
> index 000000000000..39ff7f0438e7
> --- /dev/null
> +++ b/Documentation/siphash.txt
> @@ -0,0 +1,79 @@
> + SipHash - a short input PRF
> +-----------------------------------------------
> +Written by Jason A. Donenfeld <jason@zx2c4.com>
> +
> +SipHash is a cryptographically secure PRF -- a keyed hash function -- that
> +performs very well for short inputs, hence the name. It was designed by
> +cryptographers Daniel J. Bernstein and Jean-Philippe Aumasson. It is intended
> +as a replacement for some uses of: `jhash`, `md5_transform`, `sha_transform`,
> +and so forth.
> +
> +SipHash takes a secret key filled with randomly generated numbers and either
> +an input buffer or several input integers. It spits out an integer that is
> +indistinguishable from random. You may then use that integer as part of secure
> +sequence numbers, secure cookies, or mask it off for use in a hash table.
> +
> +1. Generating a key
> +
> +Keys should always be generated from a cryptographically secure source of
> +random numbers, either using get_random_bytes or get_random_once:
> +
> +siphash_key_t key;
> +get_random_bytes(key, sizeof(key));
> +
> +If you're not deriving your key from here, you're doing it wrong.
> +
> +2. Using the functions
> +
> +There are two variants of the function, one that takes a list of integers, and
> +one that takes a buffer:
> +
> +u64 siphash(const void *data, size_t len, siphash_key_t key);
> +
> +And:
> +
> +u64 siphash_1u64(u64, siphash_key_t key);
> +u64 siphash_2u64(u64, u64, siphash_key_t key);
> +u64 siphash_3u64(u64, u64, u64, siphash_key_t key);
> +u64 siphash_4u64(u64, u64, u64, u64, siphash_key_t key);
> +u64 siphash_1u32(u32, siphash_key_t key);
> +u64 siphash_2u32(u32, u32, siphash_key_t key);
> +u64 siphash_3u32(u32, u32, u32, siphash_key_t key);
> +u64 siphash_4u32(u32, u32, u32, u32, siphash_key_t key);
> +
> +If you pass the generic siphash function something of a constant length, it
> +will constant fold at compile-time and automatically choose one of the
> +optimized functions.
> +
> +3. Hashtable key function usage:
> +
> +struct some_hashtable {
> + DECLARE_HASHTABLE(hashtable, 8);
> + siphash_key_t key;
> +};
> +
> +void init_hashtable(struct some_hashtable *table)
> +{
> + get_random_bytes(table->key, sizeof(table->key));
> +}
> +
> +static inline hlist_head *some_hashtable_bucket(struct some_hashtable *table, struct interesting_input *input)
> +{
> + return &table->hashtable[siphash(input, sizeof(*input), table->key) & (HASH_SIZE(table->hashtable) - 1)];
> +}
> +
> +You may then iterate like usual over the returned hash bucket.
> +
> +4. Security
> +
> +SipHash has a very high security margin, with its 128-bit key. So long as the
> +key is kept secret, it is impossible for an attacker to guess the outputs of
> +the function, even if being able to observe many outputs, since 2^128 outputs
> +is significant.
> +
> +Linux implements the "2-4" variant of SipHash.
> +
> +5. Resources
> +
> +Read the SipHash paper if you're interested in learning more:
> +https://131002.net/siphash/siphash.pdf
> diff --git a/MAINTAINERS b/MAINTAINERS
> index cfff2c9e3d94..e1384ae37344 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -11291,6 +11291,13 @@ F: arch/arm/mach-s3c24xx/mach-bast.c
> F: arch/arm/mach-s3c24xx/bast-ide.c
> F: arch/arm/mach-s3c24xx/bast-irq.c
>
> +SIPHASH PRF ROUTINES
> +M: Jason A. Donenfeld <Jason@zx2c4.com>
> +S: Maintained
> +F: lib/siphash.c
> +F: lib/test_siphash.c
> +F: include/linux/siphash.h
> +
> TI DAVINCI MACHINE SUPPORT
> M: Sekhar Nori <nsekhar@ti.com>
> M: Kevin Hilman <khilman@kernel.org>
> diff --git a/include/linux/siphash.h b/include/linux/siphash.h
> new file mode 100644
> index 000000000000..7aa666eb00d9
> --- /dev/null
> +++ b/include/linux/siphash.h
> @@ -0,0 +1,79 @@
> +/* Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
> + *
> + * This file is provided under a dual BSD/GPLv2 license.
> + *
> + * SipHash: a fast short-input PRF
> + * https://131002.net/siphash/
> + *
> + * This implementation is specifically for SipHash2-4.
> + */
> +
> +#ifndef _LINUX_SIPHASH_H
> +#define _LINUX_SIPHASH_H
> +
> +#include <linux/types.h>
> +#include <linux/kernel.h>
> +
> +#define SIPHASH_ALIGNMENT __alignof__(u64)
> +typedef u64 siphash_key_t[2];
> +
> +u64 __siphash_aligned(const void *data, size_t len, const siphash_key_t key);
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +u64 __siphash_unaligned(const void *data, size_t len, const siphash_key_t key);
> +#endif
> +
> +u64 siphash_1u64(const u64 a, const siphash_key_t key);
> +u64 siphash_2u64(const u64 a, const u64 b, const siphash_key_t key);
> +u64 siphash_3u64(const u64 a, const u64 b, const u64 c,
> + const siphash_key_t key);
> +u64 siphash_4u64(const u64 a, const u64 b, const u64 c, const u64 d,
> + const siphash_key_t key);
> +u64 siphash_1u32(const u32 a, const siphash_key_t key);
> +u64 siphash_3u32(const u32 a, const u32 b, const u32 c, const siphash_key_t key);
> +
> +static inline u64 siphash_2u32(const u32 a, const u32 b, const siphash_key_t key)
> +{
> + return siphash_1u64((u64)b << 32 | a, key);
> +}
> +static inline u64 siphash_4u32(const u32 a, const u32 b, const u32 c, const u32 d,
> + const siphash_key_t key)
> +{
> + return siphash_2u64((u64)b << 32 | a, (u64)d << 32 | c, key);
> +}
> +
> +
> +static inline u64 ___siphash_aligned(const __le64 *data, size_t len, const siphash_key_t key)
> +{
> + if (__builtin_constant_p(len) && len == 4)
> + return siphash_1u32(le32_to_cpu(data[0]), key);
> + if (__builtin_constant_p(len) && len == 8)
> + return siphash_1u64(le64_to_cpu(data[0]), key);
> + if (__builtin_constant_p(len) && len == 16)
> + return siphash_2u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
> + key);
> + if (__builtin_constant_p(len) && len == 24)
> + return siphash_3u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
> + le64_to_cpu(data[2]), key);
> + if (__builtin_constant_p(len) && len == 32)
> + return siphash_4u64(le64_to_cpu(data[0]), le64_to_cpu(data[1]),
> + le64_to_cpu(data[2]), le64_to_cpu(data[3]),
> + key);
> + return __siphash_aligned(data, len, key);
> +}
> +
> +/**
> + * siphash - compute 64-bit siphash PRF value
> + * @data: buffer to hash
> + * @size: size of @data
> + * @key: the siphash key
> + */
> +static inline u64 siphash(const void *data, size_t len, const siphash_key_t key)
> +{
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> + if (!IS_ALIGNED((unsigned long)data, SIPHASH_ALIGNMENT))
> + return __siphash_unaligned(data, len, key);
> +#endif
> + return ___siphash_aligned(data, len, key);
> +}
> +
> +#endif /* _LINUX_SIPHASH_H */
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index b06848a104e6..3d2515a770c3 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -1819,9 +1819,9 @@ config TEST_HASH
> tristate "Perform selftest on hash functions"
> default n
> help
> - Enable this option to test the kernel's integer (<linux/hash,h>)
> - and string (<linux/stringhash.h>) hash functions on boot
> - (or module load).
> + Enable this option to test the kernel's integer (<linux/hash.h>),
> + string (<linux/stringhash.h>), and siphash (<linux/siphash.h>)
> + hash functions on boot (or module load).
>
> This is intended to help people writing architecture-specific
> optimized versions. If unsure, say N.
> diff --git a/lib/Makefile b/lib/Makefile
> index bc4073a8cd08..7b3008d58600 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -22,7 +22,8 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> sha1.o chacha20.o md5.o irq_regs.o argv_split.o \
> flex_proportions.o ratelimit.o show_mem.o \
> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> - earlycpio.o seq_buf.o nmi_backtrace.o nodemask.o win_minmax.o
> + earlycpio.o seq_buf.o siphash.o \
> + nmi_backtrace.o nodemask.o win_minmax.o
>
> lib-$(CONFIG_MMU) += ioremap.o
> lib-$(CONFIG_SMP) += cpumask.o
> @@ -44,7 +45,7 @@ obj-$(CONFIG_TEST_HEXDUMP) += test_hexdump.o
> obj-y += kstrtox.o
> obj-$(CONFIG_TEST_BPF) += test_bpf.o
> obj-$(CONFIG_TEST_FIRMWARE) += test_firmware.o
> -obj-$(CONFIG_TEST_HASH) += test_hash.o
> +obj-$(CONFIG_TEST_HASH) += test_hash.o test_siphash.o
> obj-$(CONFIG_TEST_KASAN) += test_kasan.o
> obj-$(CONFIG_TEST_KSTRTOX) += test-kstrtox.o
> obj-$(CONFIG_TEST_LKM) += test_module.o
> diff --git a/lib/siphash.c b/lib/siphash.c
> new file mode 100644
> index 000000000000..ff2151313667
> --- /dev/null
> +++ b/lib/siphash.c
> @@ -0,0 +1,232 @@
> +/* Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
> + *
> + * This file is provided under a dual BSD/GPLv2 license.
> + *
> + * SipHash: a fast short-input PRF
> + * https://131002.net/siphash/
> + *
> + * This implementation is specifically for SipHash2-4.
> + */
> +
> +#include <linux/siphash.h>
> +#include <asm/unaligned.h>
> +
> +#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
> +#include <linux/dcache.h>
> +#include <asm/word-at-a-time.h>
> +#endif
> +
> +#define SIPROUND \
> + do { \
> + v0 += v1; v1 = rol64(v1, 13); v1 ^= v0; v0 = rol64(v0, 32); \
> + v2 += v3; v3 = rol64(v3, 16); v3 ^= v2; \
> + v0 += v3; v3 = rol64(v3, 21); v3 ^= v0; \
> + v2 += v1; v1 = rol64(v1, 17); v1 ^= v2; v2 = rol64(v2, 32); \
> + } while(0)
> +
> +#define PREAMBLE(len) \
> + u64 v0 = 0x736f6d6570736575ULL; \
> + u64 v1 = 0x646f72616e646f6dULL; \
> + u64 v2 = 0x6c7967656e657261ULL; \
> + u64 v3 = 0x7465646279746573ULL; \
> + u64 b = ((u64)len) << 56; \
> + v3 ^= key[1]; \
> + v2 ^= key[0]; \
> + v1 ^= key[1]; \
> + v0 ^= key[0];
> +
> +#define POSTAMBLE \
> + v3 ^= b; \
> + SIPROUND; \
> + SIPROUND; \
> + v0 ^= b; \
> + v2 ^= 0xff; \
> + SIPROUND; \
> + SIPROUND; \
> + SIPROUND; \
> + SIPROUND; \
> + return (v0 ^ v1) ^ (v2 ^ v3);
> +
> +u64 __siphash_aligned(const void *data, size_t len, const siphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u64));
> + const u8 left = len & (sizeof(u64) - 1);
> + u64 m;
> + PREAMBLE(len)
> + for (; data != end; data += sizeof(u64)) {
> + m = le64_to_cpup(data);
> + v3 ^= m;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= m;
> + }
> +#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
> + if (left)
> + b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
> + bytemask_from_count(left)));
> +#else
> + switch (left) {
> + case 7: b |= ((u64)end[6]) << 48;
> + case 6: b |= ((u64)end[5]) << 40;
> + case 5: b |= ((u64)end[4]) << 32;
> + case 4: b |= le32_to_cpup(data); break;
> + case 3: b |= ((u64)end[2]) << 16;
> + case 2: b |= le16_to_cpup(data); break;
> + case 1: b |= end[0];
> + }
> +#endif
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(__siphash_aligned);
> +
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +u64 __siphash_unaligned(const void *data, size_t len, const siphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u64));
> + const u8 left = len & (sizeof(u64) - 1);
> + u64 m;
> + PREAMBLE(len)
> + for (; data != end; data += sizeof(u64)) {
> + m = get_unaligned_le64(data);
> + v3 ^= m;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= m;
> + }
> +#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
> + if (left)
> + b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
> + bytemask_from_count(left)));
> +#else
> + switch (left) {
> + case 7: b |= ((u64)end[6]) << 48;
> + case 6: b |= ((u64)end[5]) << 40;
> + case 5: b |= ((u64)end[4]) << 32;
> + case 4: b |= get_unaligned_le32(end); break;
> + case 3: b |= ((u64)end[2]) << 16;
> + case 2: b |= get_unaligned_le16(end); break;
> + case 1: b |= end[0];
> + }
> +#endif
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(__siphash_unaligned);
> +#endif
> +
> +/**
> + * siphash_1u64 - compute 64-bit siphash PRF value of a u64
> + * @first: first u64
> + * @key: the siphash key
> + */
> +u64 siphash_1u64(const u64 first, const siphash_key_t key)
> +{
> + PREAMBLE(8)
> + v3 ^= first;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= first;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_1u64);
> +
> +/**
> + * siphash_2u64 - compute 64-bit siphash PRF value of 2 u64
> + * @first: first u64
> + * @second: second u64
> + * @key: the siphash key
> + */
> +u64 siphash_2u64(const u64 first, const u64 second, const siphash_key_t key)
> +{
> + PREAMBLE(16)
> + v3 ^= first;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= second;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_2u64);
> +
> +/**
> + * siphash_3u64 - compute 64-bit siphash PRF value of 3 u64
> + * @first: first u64
> + * @second: second u64
> + * @third: third u64
> + * @key: the siphash key
> + */
> +u64 siphash_3u64(const u64 first, const u64 second, const u64 third,
> + const siphash_key_t key)
> +{
> + PREAMBLE(24)
> + v3 ^= first;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= second;
> + v3 ^= third;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= third;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_3u64);
> +
> +/**
> + * siphash_4u64 - compute 64-bit siphash PRF value of 4 u64
> + * @first: first u64
> + * @second: second u64
> + * @third: third u64
> + * @forth: forth u64
> + * @key: the siphash key
> + */
> +u64 siphash_4u64(const u64 first, const u64 second, const u64 third,
> + const u64 forth, const siphash_key_t key)
> +{
> + PREAMBLE(32)
> + v3 ^= first;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= second;
> + v3 ^= third;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= third;
> + v3 ^= forth;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= forth;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_4u64);
> +
> +u64 siphash_1u32(const u32 first, const siphash_key_t key)
> +{
> + PREAMBLE(4)
> + b |= first;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_1u32);
> +
> +u64 siphash_3u32(const u32 first, const u32 second, const u32 third,
> + const siphash_key_t key)
> +{
> + u64 combined = (u64)second << 32 | first;
> + PREAMBLE(12)
> + v3 ^= combined;
> + SIPROUND;
> + SIPROUND;
> + v0 ^= combined;
> + b |= third;
> + POSTAMBLE
> +}
> +EXPORT_SYMBOL(siphash_3u32);
> diff --git a/lib/test_siphash.c b/lib/test_siphash.c
> new file mode 100644
> index 000000000000..e0ba2cf8dc67
> --- /dev/null
> +++ b/lib/test_siphash.c
> @@ -0,0 +1,119 @@
> +/* Test cases for siphash.c
> + *
> + * Copyright (C) 2016 Jason A. Donenfeld <Jason@zx2c4.com>. All Rights Reserved.
> + *
> + * This file is provided under a dual BSD/GPLv2 license.
> + *
> + * SipHash: a fast short-input PRF
> + * https://131002.net/siphash/
> + *
> + * This implementation is specifically for SipHash2-4.
> + */
> +
> +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> +
> +#include <linux/siphash.h>
> +#include <linux/kernel.h>
> +#include <linux/string.h>
> +#include <linux/errno.h>
> +#include <linux/module.h>
> +
> +/* Test vectors taken from official reference source available at:
> + * https://131002.net/siphash/siphash24.c
> + */
> +static const u64 test_vectors[64] = {
> + 0x726fdb47dd0e0e31ULL, 0x74f839c593dc67fdULL, 0x0d6c8009d9a94f5aULL,
> + 0x85676696d7fb7e2dULL, 0xcf2794e0277187b7ULL, 0x18765564cd99a68dULL,
> + 0xcbc9466e58fee3ceULL, 0xab0200f58b01d137ULL, 0x93f5f5799a932462ULL,
> + 0x9e0082df0ba9e4b0ULL, 0x7a5dbbc594ddb9f3ULL, 0xf4b32f46226bada7ULL,
> + 0x751e8fbc860ee5fbULL, 0x14ea5627c0843d90ULL, 0xf723ca908e7af2eeULL,
> + 0xa129ca6149be45e5ULL, 0x3f2acc7f57c29bdbULL, 0x699ae9f52cbe4794ULL,
> + 0x4bc1b3f0968dd39cULL, 0xbb6dc91da77961bdULL, 0xbed65cf21aa2ee98ULL,
> + 0xd0f2cbb02e3b67c7ULL, 0x93536795e3a33e88ULL, 0xa80c038ccd5ccec8ULL,
> + 0xb8ad50c6f649af94ULL, 0xbce192de8a85b8eaULL, 0x17d835b85bbb15f3ULL,
> + 0x2f2e6163076bcfadULL, 0xde4daaaca71dc9a5ULL, 0xa6a2506687956571ULL,
> + 0xad87a3535c49ef28ULL, 0x32d892fad841c342ULL, 0x7127512f72f27cceULL,
> + 0xa7f32346f95978e3ULL, 0x12e0b01abb051238ULL, 0x15e034d40fa197aeULL,
> + 0x314dffbe0815a3b4ULL, 0x027990f029623981ULL, 0xcadcd4e59ef40c4dULL,
> + 0x9abfd8766a33735cULL, 0x0e3ea96b5304a7d0ULL, 0xad0c42d6fc585992ULL,
> + 0x187306c89bc215a9ULL, 0xd4a60abcf3792b95ULL, 0xf935451de4f21df2ULL,
> + 0xa9538f0419755787ULL, 0xdb9acddff56ca510ULL, 0xd06c98cd5c0975ebULL,
> + 0xe612a3cb9ecba951ULL, 0xc766e62cfcadaf96ULL, 0xee64435a9752fe72ULL,
> + 0xa192d576b245165aULL, 0x0a8787bf8ecb74b2ULL, 0x81b3e73d20b49b6fULL,
> + 0x7fa8220ba3b2eceaULL, 0x245731c13ca42499ULL, 0xb78dbfaf3a8d83bdULL,
> + 0xea1ad565322a1a0bULL, 0x60e61c23a3795013ULL, 0x6606d7e446282b93ULL,
> + 0x6ca4ecb15c5f91e1ULL, 0x9f626da15c9625f3ULL, 0xe51b38608ef25f57ULL,
> + 0x958a324ceb064572ULL
> +};
> +static const siphash_key_t test_key =
> + { 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
> +
> +static int __init siphash_test_init(void)
> +{
> + u8 in[64] __aligned(SIPHASH_ALIGNMENT);
> + u8 in_unaligned[65];
> + u8 i;
> + int ret = 0;
> +
> + for (i = 0; i < 64; ++i) {
> + in[i] = i;
> + in_unaligned[i + 1] = i;
> + if (siphash(in, i, test_key) != test_vectors[i]) {
> + pr_info("self-test aligned %u: FAIL\n", i + 1);
> + ret = -EINVAL;
> + }
> + if (siphash(in_unaligned + 1, i, test_key) != test_vectors[i]) {
> + pr_info("self-test unaligned %u: FAIL\n", i + 1);
> + ret = -EINVAL;
> + }
> + }
> + if (siphash_1u64(0x0706050403020100ULL, test_key) != test_vectors[8]) {
> + pr_info("self-test 1u64: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key) != test_vectors[16]) {
> + pr_info("self-test 2u64: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_3u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
> + 0x1716151413121110ULL, test_key) != test_vectors[24]) {
> + pr_info("self-test 3u64: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_4u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
> + 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key) != test_vectors[32]) {
> + pr_info("self-test 4u64: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_1u32(0x03020100U, test_key) != test_vectors[4]) {
> + pr_info("self-test 1u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_2u32(0x03020100U, 0x07060504U, test_key) != test_vectors[8]) {
> + pr_info("self-test 2u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_3u32(0x03020100U, 0x07060504U,
> + 0x0b0a0908U, test_key) != test_vectors[12]) {
> + pr_info("self-test 3u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (siphash_4u32(0x03020100U, 0x07060504U,
> + 0x0b0a0908U, 0x0f0e0d0cU, test_key) != test_vectors[16]) {
> + pr_info("self-test 4u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (!ret)
> + pr_info("self-tests: pass\n");
> + return ret;
> +}
> +
> +static void __exit siphash_test_exit(void)
> +{
> +}
> +
> +module_init(siphash_test_init);
> +module_exit(siphash_test_exit);
> +
> +MODULE_AUTHOR("Jason A. Donenfeld <Jason@zx2c4.com>");
> +MODULE_LICENSE("Dual BSD/GPL");
> --
> 2.11.0
Looks good to me (co-designer of SipHash).
Reviewed-by: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
^ permalink raw reply
* Re: [PATCH net-next 2/4] siphash: implement HalfSipHash1-3 for hash tables
From: Jean-Philippe Aumasson @ 2017-01-06 20:33 UTC (permalink / raw)
To: Jason A. Donenfeld; +Cc: David Miller, Netdev, LKML
In-Reply-To: <20170106201055.13765-3-Jason@zx2c4.com>
On Fri, Jan 6, 2017 at 9:10 PM, Jason A. Donenfeld <Jason@zx2c4.com> wrote:
> HalfSipHash, or hsiphash, is a shortened version of SipHash, which
> generates 32-bit outputs using a weaker 64-bit key. It has *much* lower
> security margins, and shouldn't be used for anything too sensitive, but
> it could be used as a hashtable key function replacement, if the output
> is never exposed, and if the security requirement is not too high.
>
> The goal is to make this something that performance-critical jhash users
> would be willing to use.
>
> On 64-bit machines, HalfSipHash1-3 is slower than SipHash1-3, so we alias
> SipHash1-3 to HalfSipHash1-3 on those systems.
>
> 64-bit x86_64:
> [ 0.509409] test_siphash: SipHash2-4 cycles: 4049181
> [ 0.510650] test_siphash: SipHash1-3 cycles: 2512884
> [ 0.512205] test_siphash: HalfSipHash1-3 cycles: 3429920
> [ 0.512904] test_siphash: JenkinsHash cycles: 978267
> So, we map hsiphash() -> SipHash1-3
>
> 32-bit x86:
> [ 0.509868] test_siphash: SipHash2-4 cycles: 14812892
> [ 0.513601] test_siphash: SipHash1-3 cycles: 9510710
> [ 0.515263] test_siphash: HalfSipHash1-3 cycles: 3856157
> [ 0.515952] test_siphash: JenkinsHash cycles: 1148567
> So, we map hsiphash() -> HalfSipHash1-3
>
> hsiphash() is roughly 3 times slower than jhash(), but comes with a
> considerable security improvement.
>
> Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
> Cc: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
> ---
> Documentation/siphash.txt | 75 +++++++++++
> include/linux/siphash.h | 56 +++++++-
> lib/siphash.c | 318 +++++++++++++++++++++++++++++++++++++++++++++-
> lib/test_siphash.c | 139 ++++++++++++++++----
> 4 files changed, 561 insertions(+), 27 deletions(-)
>
> diff --git a/Documentation/siphash.txt b/Documentation/siphash.txt
> index 39ff7f0438e7..f93c1d7104c4 100644
> --- a/Documentation/siphash.txt
> +++ b/Documentation/siphash.txt
> @@ -77,3 +77,78 @@ Linux implements the "2-4" variant of SipHash.
>
> Read the SipHash paper if you're interested in learning more:
> https://131002.net/siphash/siphash.pdf
> +
> +
> +~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
> +
> +HalfSipHash - SipHash's insecure younger cousin
> +-----------------------------------------------
> +Written by Jason A. Donenfeld <jason@zx2c4.com>
> +
> +On the off-chance that SipHash is not fast enough for your needs, you might be
> +able to justify using HalfSipHash, a terrifying but potentially useful
> +possibility. HalfSipHash cuts SipHash's rounds down from "2-4" to "1-3" and,
> +even scarier, uses an easily brute-forcable 64-bit key (with a 32-bit output)
> +instead of SipHash's 128-bit key. However, this may appeal to some
> +high-performance `jhash` users.
> +
> +Danger!
> +
> +Do not ever use HalfSipHash except for as a hashtable key function, and only
> +then when you can be absolutely certain that the outputs will never be
> +transmitted out of the kernel. This is only remotely useful over `jhash` as a
> +means of mitigating hashtable flooding denial of service attacks.
> +
> +1. Generating a key
> +
> +Keys should always be generated from a cryptographically secure source of
> +random numbers, either using get_random_bytes or get_random_once:
> +
> +hsiphash_key_t key;
> +get_random_bytes(key, sizeof(key));
> +
> +If you're not deriving your key from here, you're doing it wrong.
> +
> +2. Using the functions
> +
> +There are two variants of the function, one that takes a list of integers, and
> +one that takes a buffer:
> +
> +u32 hsiphash(const void *data, size_t len, siphash_key_t key);
> +
> +And:
> +
> +u32 hsiphash_1u32(u32, hsiphash_key_t key);
> +u32 hsiphash_2u32(u32, u32, hsiphash_key_t key);
> +u32 hsiphash_3u32(u32, u32, u32, hsiphash_key_t key);
> +u32 hsiphash_4u32(u32, u32, u32, u32, hsiphash_key_t key);
> +
> +If you pass the generic hsiphash function something of a constant length, it
> +will constant fold at compile-time and automatically choose one of the
> +optimized functions.
> +
> +3. Hashtable key function usage:
> +
> +struct some_hashtable {
> + DECLARE_HASHTABLE(hashtable, 8);
> + hsiphash_key_t key;
> +};
> +
> +void init_hashtable(struct some_hashtable *table)
> +{
> + get_random_bytes(table->key, sizeof(table->key));
> +}
> +
> +static inline hlist_head *some_hashtable_bucket(struct some_hashtable *table, struct interesting_input *input)
> +{
> + return &table->hashtable[hsiphash(input, sizeof(*input), table->key) & (HASH_SIZE(table->hashtable) - 1)];
> +}
> +
> +You may then iterate like usual over the returned hash bucket.
> +
> +4. Performance
> +
> +HalfSipHash is roughly 3 times slower than JenkinsHash. For many replacements,
> +this will not be a problem, as the hashtable lookup isn't the bottleneck. And
> +in general, this is probably a good sacrifice to make for the security and DoS
> +resistance of HalfSipHash.
> diff --git a/include/linux/siphash.h b/include/linux/siphash.h
> index 7aa666eb00d9..efab44c654f3 100644
> --- a/include/linux/siphash.h
> +++ b/include/linux/siphash.h
> @@ -5,7 +5,9 @@
> * SipHash: a fast short-input PRF
> * https://131002.net/siphash/
> *
> - * This implementation is specifically for SipHash2-4.
> + * This implementation is specifically for SipHash2-4 for a secure PRF
> + * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
> + * hashtables.
> */
>
> #ifndef _LINUX_SIPHASH_H
> @@ -76,4 +78,56 @@ static inline u64 siphash(const void *data, size_t len, const siphash_key_t key)
> return ___siphash_aligned(data, len, key);
> }
>
> +#if BITS_PER_LONG == 64
> +typedef siphash_key_t hsiphash_key_t;
> +#define HSIPHASH_ALIGNMENT SIPHASH_ALIGNMENT
> +#else
> +typedef u32 hsiphash_key_t[2];
> +#define HSIPHASH_ALIGNMENT __alignof__(u32)
> +#endif
> +
> +u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key);
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key);
> +#endif
> +
> +u32 hsiphash_1u32(const u32 a, const hsiphash_key_t key);
> +u32 hsiphash_2u32(const u32 a, const u32 b, const hsiphash_key_t key);
> +u32 hsiphash_3u32(const u32 a, const u32 b, const u32 c,
> + const hsiphash_key_t key);
> +u32 hsiphash_4u32(const u32 a, const u32 b, const u32 c, const u32 d,
> + const hsiphash_key_t key);
> +
> +static inline u32 ___hsiphash_aligned(const __le32 *data, size_t len, const hsiphash_key_t key)
> +{
> + if (__builtin_constant_p(len) && len == 4)
> + return hsiphash_1u32(le32_to_cpu(data[0]), key);
> + if (__builtin_constant_p(len) && len == 8)
> + return hsiphash_2u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
> + key);
> + if (__builtin_constant_p(len) && len == 12)
> + return hsiphash_3u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
> + le32_to_cpu(data[2]), key);
> + if (__builtin_constant_p(len) && len == 16)
> + return hsiphash_4u32(le32_to_cpu(data[0]), le32_to_cpu(data[1]),
> + le32_to_cpu(data[2]), le32_to_cpu(data[3]),
> + key);
> + return __hsiphash_aligned(data, len, key);
> +}
> +
> +/**
> + * hsiphash - compute 32-bit hsiphash PRF value
> + * @data: buffer to hash
> + * @size: size of @data
> + * @key: the hsiphash key
> + */
> +static inline u32 hsiphash(const void *data, size_t len, const hsiphash_key_t key)
> +{
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> + if (!IS_ALIGNED((unsigned long)data, HSIPHASH_ALIGNMENT))
> + return __hsiphash_unaligned(data, len, key);
> +#endif
> + return ___hsiphash_aligned(data, len, key);
> +}
> +
> #endif /* _LINUX_SIPHASH_H */
> diff --git a/lib/siphash.c b/lib/siphash.c
> index ff2151313667..e2481226d96c 100644
> --- a/lib/siphash.c
> +++ b/lib/siphash.c
> @@ -5,7 +5,9 @@
> * SipHash: a fast short-input PRF
> * https://131002.net/siphash/
> *
> - * This implementation is specifically for SipHash2-4.
> + * This implementation is specifically for SipHash2-4 for a secure PRF
> + * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
> + * hashtables.
> */
>
> #include <linux/siphash.h>
> @@ -230,3 +232,317 @@ u64 siphash_3u32(const u32 first, const u32 second, const u32 third,
> POSTAMBLE
> }
> EXPORT_SYMBOL(siphash_3u32);
> +
> +#if BITS_PER_LONG == 64
> +/* Note that this HalfSipHash1-3 implementation on 64-bit
> + * isn't actually HalfSipHash1-3 but rather SipHash1-3. */
> +
> +#define HSIPROUND SIPROUND
> +#define HPREAMBLE(len) PREAMBLE(len)
> +#define HPOSTAMBLE \
> + v3 ^= b; \
> + HSIPROUND; \
> + v0 ^= b; \
> + v2 ^= 0xff; \
> + HSIPROUND; \
> + HSIPROUND; \
> + HSIPROUND; \
> + return (v0 ^ v1) ^ (v2 ^ v3);
> +
> +u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u64));
> + const u8 left = len & (sizeof(u64) - 1);
> + u64 m;
> + HPREAMBLE(len)
> + for (; data != end; data += sizeof(u64)) {
> + m = le64_to_cpup(data);
> + v3 ^= m;
> + HSIPROUND;
> + v0 ^= m;
> + }
> +#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
> + if (left)
> + b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
> + bytemask_from_count(left)));
> +#else
> + switch (left) {
> + case 7: b |= ((u64)end[6]) << 48;
> + case 6: b |= ((u64)end[5]) << 40;
> + case 5: b |= ((u64)end[4]) << 32;
> + case 4: b |= le32_to_cpup(data); break;
> + case 3: b |= ((u64)end[2]) << 16;
> + case 2: b |= le16_to_cpup(data); break;
> + case 1: b |= end[0];
> + }
> +#endif
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(__hsiphash_aligned);
> +
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u64));
> + const u8 left = len & (sizeof(u64) - 1);
> + u64 m;
> + HPREAMBLE(len)
> + for (; data != end; data += sizeof(u64)) {
> + m = get_unaligned_le64(data);
> + v3 ^= m;
> + HSIPROUND;
> + v0 ^= m;
> + }
> +#if defined(CONFIG_DCACHE_WORD_ACCESS) && BITS_PER_LONG == 64
> + if (left)
> + b |= le64_to_cpu((__force __le64)(load_unaligned_zeropad(data) &
> + bytemask_from_count(left)));
> +#else
> + switch (left) {
> + case 7: b |= ((u64)end[6]) << 48;
> + case 6: b |= ((u64)end[5]) << 40;
> + case 5: b |= ((u64)end[4]) << 32;
> + case 4: b |= get_unaligned_le32(end); break;
> + case 3: b |= ((u64)end[2]) << 16;
> + case 2: b |= get_unaligned_le16(end); break;
> + case 1: b |= end[0];
> + }
> +#endif
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(__hsiphash_unaligned);
> +#endif
> +
> +/**
> + * hsiphash_1u32 - compute 64-bit hsiphash PRF value of a u32
> + * @first: first u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_1u32(const u32 first, const hsiphash_key_t key)
> +{
> + HPREAMBLE(4)
> + b |= first;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_1u32);
> +
> +/**
> + * hsiphash_2u32 - compute 32-bit hsiphash PRF value of 2 u32
> + * @first: first u32
> + * @second: second u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_2u32(const u32 first, const u32 second, const hsiphash_key_t key)
> +{
> + u64 combined = (u64)second << 32 | first;
> + HPREAMBLE(8)
> + v3 ^= combined;
> + HSIPROUND;
> + v0 ^= combined;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_2u32);
> +
> +/**
> + * hsiphash_3u32 - compute 32-bit hsiphash PRF value of 3 u32
> + * @first: first u32
> + * @second: second u32
> + * @third: third u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_3u32(const u32 first, const u32 second, const u32 third,
> + const hsiphash_key_t key)
> +{
> + u64 combined = (u64)second << 32 | first;
> + HPREAMBLE(12)
> + v3 ^= combined;
> + HSIPROUND;
> + v0 ^= combined;
> + b |= third;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_3u32);
> +
> +/**
> + * hsiphash_4u32 - compute 32-bit hsiphash PRF value of 4 u32
> + * @first: first u32
> + * @second: second u32
> + * @third: third u32
> + * @forth: forth u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_4u32(const u32 first, const u32 second, const u32 third,
> + const u32 forth, const hsiphash_key_t key)
> +{
> + u64 combined = (u64)second << 32 | first;
> + HPREAMBLE(16)
> + v3 ^= combined;
> + HSIPROUND;
> + v0 ^= combined;
> + combined = (u64)forth << 32 | third;
> + v3 ^= combined;
> + HSIPROUND;
> + v0 ^= combined;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_4u32);
> +#else
> +#define HSIPROUND \
> + do { \
> + v0 += v1; v1 = rol32(v1, 5); v1 ^= v0; v0 = rol32(v0, 16); \
> + v2 += v3; v3 = rol32(v3, 8); v3 ^= v2; \
> + v0 += v3; v3 = rol32(v3, 7); v3 ^= v0; \
> + v2 += v1; v1 = rol32(v1, 13); v1 ^= v2; v2 = rol32(v2, 16); \
> + } while(0)
> +
> +#define HPREAMBLE(len) \
> + u32 v0 = 0; \
> + u32 v1 = 0; \
> + u32 v2 = 0x6c796765U; \
> + u32 v3 = 0x74656462U; \
> + u32 b = ((u32)len) << 24; \
> + v3 ^= key[1]; \
> + v2 ^= key[0]; \
> + v1 ^= key[1]; \
> + v0 ^= key[0];
> +
> +#define HPOSTAMBLE \
> + v3 ^= b; \
> + HSIPROUND; \
> + v0 ^= b; \
> + v2 ^= 0xff; \
> + HSIPROUND; \
> + HSIPROUND; \
> + HSIPROUND; \
> + return v1 ^ v3;
> +
> +u32 __hsiphash_aligned(const void *data, size_t len, const hsiphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u32));
> + const u8 left = len & (sizeof(u32) - 1);
> + u32 m;
> + HPREAMBLE(len)
> + for (; data != end; data += sizeof(u32)) {
> + m = le32_to_cpup(data);
> + v3 ^= m;
> + HSIPROUND;
> + v0 ^= m;
> + }
> + switch (left) {
> + case 3: b |= ((u32)end[2]) << 16;
> + case 2: b |= le16_to_cpup(data); break;
> + case 1: b |= end[0];
> + }
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(__hsiphash_aligned);
> +
> +#ifndef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> +u32 __hsiphash_unaligned(const void *data, size_t len, const hsiphash_key_t key)
> +{
> + const u8 *end = data + len - (len % sizeof(u32));
> + const u8 left = len & (sizeof(u32) - 1);
> + u32 m;
> + HPREAMBLE(len)
> + for (; data != end; data += sizeof(u32)) {
> + m = get_unaligned_le32(data);
> + v3 ^= m;
> + HSIPROUND;
> + v0 ^= m;
> + }
> + switch (left) {
> + case 3: b |= ((u32)end[2]) << 16;
> + case 2: b |= get_unaligned_le16(end); break;
> + case 1: b |= end[0];
> + }
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(__hsiphash_unaligned);
> +#endif
> +
> +/**
> + * hsiphash_1u32 - compute 32-bit hsiphash PRF value of a u32
> + * @first: first u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_1u32(const u32 first, const hsiphash_key_t key)
> +{
> + HPREAMBLE(4)
> + v3 ^= first;
> + HSIPROUND;
> + v0 ^= first;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_1u32);
> +
> +/**
> + * hsiphash_2u32 - compute 32-bit hsiphash PRF value of 2 u32
> + * @first: first u32
> + * @second: second u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_2u32(const u32 first, const u32 second, const hsiphash_key_t key)
> +{
> + HPREAMBLE(8)
> + v3 ^= first;
> + HSIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + HSIPROUND;
> + v0 ^= second;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_2u32);
> +
> +/**
> + * hsiphash_3u32 - compute 32-bit hsiphash PRF value of 3 u32
> + * @first: first u32
> + * @second: second u32
> + * @third: third u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_3u32(const u32 first, const u32 second, const u32 third,
> + const hsiphash_key_t key)
> +{
> + HPREAMBLE(12)
> + v3 ^= first;
> + HSIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + HSIPROUND;
> + v0 ^= second;
> + v3 ^= third;
> + HSIPROUND;
> + v0 ^= third;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_3u32);
> +
> +/**
> + * hsiphash_4u32 - compute 32-bit hsiphash PRF value of 4 u32
> + * @first: first u32
> + * @second: second u32
> + * @third: third u32
> + * @forth: forth u32
> + * @key: the hsiphash key
> + */
> +u32 hsiphash_4u32(const u32 first, const u32 second, const u32 third,
> + const u32 forth, const hsiphash_key_t key)
> +{
> + HPREAMBLE(16)
> + v3 ^= first;
> + HSIPROUND;
> + v0 ^= first;
> + v3 ^= second;
> + HSIPROUND;
> + v0 ^= second;
> + v3 ^= third;
> + HSIPROUND;
> + v0 ^= third;
> + v3 ^= forth;
> + HSIPROUND;
> + v0 ^= forth;
> + HPOSTAMBLE
> +}
> +EXPORT_SYMBOL(hsiphash_4u32);
> +#endif
> diff --git a/lib/test_siphash.c b/lib/test_siphash.c
> index e0ba2cf8dc67..ac291ec27fb6 100644
> --- a/lib/test_siphash.c
> +++ b/lib/test_siphash.c
> @@ -7,7 +7,9 @@
> * SipHash: a fast short-input PRF
> * https://131002.net/siphash/
> *
> - * This implementation is specifically for SipHash2-4.
> + * This implementation is specifically for SipHash2-4 for a secure PRF
> + * and HalfSipHash1-3/SipHash1-3 for an insecure PRF only suitable for
> + * hashtables.
> */
>
> #define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
> @@ -18,10 +20,16 @@
> #include <linux/errno.h>
> #include <linux/module.h>
>
> -/* Test vectors taken from official reference source available at:
> - * https://131002.net/siphash/siphash24.c
> +/* Test vectors taken from reference source available at:
> + * https://github.com/veorq/SipHash
> */
> -static const u64 test_vectors[64] = {
> +
> +
> +
> +static const siphash_key_t test_key_siphash =
> + { 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
> +
> +static const u64 test_vectors_siphash[64] = {
> 0x726fdb47dd0e0e31ULL, 0x74f839c593dc67fdULL, 0x0d6c8009d9a94f5aULL,
> 0x85676696d7fb7e2dULL, 0xcf2794e0277187b7ULL, 0x18765564cd99a68dULL,
> 0xcbc9466e58fee3ceULL, 0xab0200f58b01d137ULL, 0x93f5f5799a932462ULL,
> @@ -45,9 +53,64 @@ static const u64 test_vectors[64] = {
> 0x6ca4ecb15c5f91e1ULL, 0x9f626da15c9625f3ULL, 0xe51b38608ef25f57ULL,
> 0x958a324ceb064572ULL
> };
> -static const siphash_key_t test_key =
> +#if BITS_PER_LONG == 64
> +static const hsiphash_key_t test_key_hsiphash =
> { 0x0706050403020100ULL , 0x0f0e0d0c0b0a0908ULL };
>
> +static const u32 test_vectors_hsiphash[64] = {
> + 0x050fc4dcU, 0x7d57ca93U, 0x4dc7d44dU,
> + 0xe7ddf7fbU, 0x88d38328U, 0x49533b67U,
> + 0xc59f22a7U, 0x9bb11140U, 0x8d299a8eU,
> + 0x6c063de4U, 0x92ff097fU, 0xf94dc352U,
> + 0x57b4d9a2U, 0x1229ffa7U, 0xc0f95d34U,
> + 0x2a519956U, 0x7d908b66U, 0x63dbd80cU,
> + 0xb473e63eU, 0x8d297d1cU, 0xa6cce040U,
> + 0x2b45f844U, 0xa320872eU, 0xdae6c123U,
> + 0x67349c8cU, 0x705b0979U, 0xca9913a5U,
> + 0x4ade3b35U, 0xef6cd00dU, 0x4ab1e1f4U,
> + 0x43c5e663U, 0x8c21d1bcU, 0x16a7b60dU,
> + 0x7a8ff9bfU, 0x1f2a753eU, 0xbf186b91U,
> + 0xada26206U, 0xa3c33057U, 0xae3a36a1U,
> + 0x7b108392U, 0x99e41531U, 0x3f1ad944U,
> + 0xc8138825U, 0xc28949a6U, 0xfaf8876bU,
> + 0x9f042196U, 0x68b1d623U, 0x8b5114fdU,
> + 0xdf074c46U, 0x12cc86b3U, 0x0a52098fU,
> + 0x9d292f9aU, 0xa2f41f12U, 0x43a71ed0U,
> + 0x73f0bce6U, 0x70a7e980U, 0x243c6d75U,
> + 0xfdb71513U, 0xa67d8a08U, 0xb7e8f148U,
> + 0xf7a644eeU, 0x0f1837f2U, 0x4b6694e0U,
> + 0xb7bbb3a8U
> +};
> +#else
> +static const hsiphash_key_t test_key_hsiphash =
> + { 0x03020100U, 0x07060504U };
> +
> +static const u32 test_vectors_hsiphash[64] = {
> + 0x5814c896U, 0xe7e864caU, 0xbc4b0e30U,
> + 0x01539939U, 0x7e059ea6U, 0x88e3d89bU,
> + 0xa0080b65U, 0x9d38d9d6U, 0x577999b1U,
> + 0xc839caedU, 0xe4fa32cfU, 0x959246eeU,
> + 0x6b28096cU, 0x66dd9cd6U, 0x16658a7cU,
> + 0xd0257b04U, 0x8b31d501U, 0x2b1cd04bU,
> + 0x06712339U, 0x522aca67U, 0x911bb605U,
> + 0x90a65f0eU, 0xf826ef7bU, 0x62512debU,
> + 0x57150ad7U, 0x5d473507U, 0x1ec47442U,
> + 0xab64afd3U, 0x0a4100d0U, 0x6d2ce652U,
> + 0x2331b6a3U, 0x08d8791aU, 0xbc6dda8dU,
> + 0xe0f6c934U, 0xb0652033U, 0x9b9851ccU,
> + 0x7c46fb7fU, 0x732ba8cbU, 0xf142997aU,
> + 0xfcc9aa1bU, 0x05327eb2U, 0xe110131cU,
> + 0xf9e5e7c0U, 0xa7d708a6U, 0x11795ab1U,
> + 0x65671619U, 0x9f5fff91U, 0xd89c5267U,
> + 0x007783ebU, 0x95766243U, 0xab639262U,
> + 0x9c7e1390U, 0xc368dda6U, 0x38ddc455U,
> + 0xfa13d379U, 0x979ea4e8U, 0x53ecd77eU,
> + 0x2ee80657U, 0x33dbb66aU, 0xae3f0577U,
> + 0x88b4c4ccU, 0x3e7f480bU, 0x74c1ebf8U,
> + 0x87178304U
> +};
> +#endif
> +
> static int __init siphash_test_init(void)
> {
> u8 in[64] __aligned(SIPHASH_ALIGNMENT);
> @@ -58,49 +121,75 @@ static int __init siphash_test_init(void)
> for (i = 0; i < 64; ++i) {
> in[i] = i;
> in_unaligned[i + 1] = i;
> - if (siphash(in, i, test_key) != test_vectors[i]) {
> - pr_info("self-test aligned %u: FAIL\n", i + 1);
> + if (siphash(in, i, test_key_siphash) != test_vectors_siphash[i]) {
> + pr_info("siphash self-test aligned %u: FAIL\n", i + 1);
> + ret = -EINVAL;
> + }
> + if (siphash(in_unaligned + 1, i, test_key_siphash) != test_vectors_siphash[i]) {
> + pr_info("siphash self-test unaligned %u: FAIL\n", i + 1);
> ret = -EINVAL;
> }
> - if (siphash(in_unaligned + 1, i, test_key) != test_vectors[i]) {
> - pr_info("self-test unaligned %u: FAIL\n", i + 1);
> + if (hsiphash(in, i, test_key_hsiphash) != test_vectors_hsiphash[i]) {
> + pr_info("hsiphash self-test aligned %u: FAIL\n", i + 1);
> + ret = -EINVAL;
> + }
> + if (hsiphash(in_unaligned + 1, i, test_key_hsiphash) != test_vectors_hsiphash[i]) {
> + pr_info("hsiphash self-test unaligned %u: FAIL\n", i + 1);
> ret = -EINVAL;
> }
> }
> - if (siphash_1u64(0x0706050403020100ULL, test_key) != test_vectors[8]) {
> - pr_info("self-test 1u64: FAIL\n");
> + if (siphash_1u64(0x0706050403020100ULL, test_key_siphash) != test_vectors_siphash[8]) {
> + pr_info("siphash self-test 1u64: FAIL\n");
> ret = -EINVAL;
> }
> - if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key) != test_vectors[16]) {
> - pr_info("self-test 2u64: FAIL\n");
> + if (siphash_2u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL, test_key_siphash) != test_vectors_siphash[16]) {
> + pr_info("siphash self-test 2u64: FAIL\n");
> ret = -EINVAL;
> }
> if (siphash_3u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
> - 0x1716151413121110ULL, test_key) != test_vectors[24]) {
> - pr_info("self-test 3u64: FAIL\n");
> + 0x1716151413121110ULL, test_key_siphash) != test_vectors_siphash[24]) {
> + pr_info("siphash self-test 3u64: FAIL\n");
> ret = -EINVAL;
> }
> if (siphash_4u64(0x0706050403020100ULL, 0x0f0e0d0c0b0a0908ULL,
> - 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key) != test_vectors[32]) {
> - pr_info("self-test 4u64: FAIL\n");
> + 0x1716151413121110ULL, 0x1f1e1d1c1b1a1918ULL, test_key_siphash) != test_vectors_siphash[32]) {
> + pr_info("siphash self-test 4u64: FAIL\n");
> ret = -EINVAL;
> }
> - if (siphash_1u32(0x03020100U, test_key) != test_vectors[4]) {
> - pr_info("self-test 1u32: FAIL\n");
> + if (siphash_1u32(0x03020100U, test_key_siphash) != test_vectors_siphash[4]) {
> + pr_info("siphash self-test 1u32: FAIL\n");
> ret = -EINVAL;
> }
> - if (siphash_2u32(0x03020100U, 0x07060504U, test_key) != test_vectors[8]) {
> - pr_info("self-test 2u32: FAIL\n");
> + if (siphash_2u32(0x03020100U, 0x07060504U, test_key_siphash) != test_vectors_siphash[8]) {
> + pr_info("siphash self-test 2u32: FAIL\n");
> ret = -EINVAL;
> }
> if (siphash_3u32(0x03020100U, 0x07060504U,
> - 0x0b0a0908U, test_key) != test_vectors[12]) {
> - pr_info("self-test 3u32: FAIL\n");
> + 0x0b0a0908U, test_key_siphash) != test_vectors_siphash[12]) {
> + pr_info("siphash self-test 3u32: FAIL\n");
> ret = -EINVAL;
> }
> if (siphash_4u32(0x03020100U, 0x07060504U,
> - 0x0b0a0908U, 0x0f0e0d0cU, test_key) != test_vectors[16]) {
> - pr_info("self-test 4u32: FAIL\n");
> + 0x0b0a0908U, 0x0f0e0d0cU, test_key_siphash) != test_vectors_siphash[16]) {
> + pr_info("siphash self-test 4u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (hsiphash_1u32(0x03020100U, test_key_hsiphash) != test_vectors_hsiphash[4]) {
> + pr_info("hsiphash self-test 1u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (hsiphash_2u32(0x03020100U, 0x07060504U, test_key_hsiphash) != test_vectors_hsiphash[8]) {
> + pr_info("hsiphash self-test 2u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (hsiphash_3u32(0x03020100U, 0x07060504U,
> + 0x0b0a0908U, test_key_hsiphash) != test_vectors_hsiphash[12]) {
> + pr_info("hsiphash self-test 3u32: FAIL\n");
> + ret = -EINVAL;
> + }
> + if (hsiphash_4u32(0x03020100U, 0x07060504U,
> + 0x0b0a0908U, 0x0f0e0d0cU, test_key_hsiphash) != test_vectors_hsiphash[16]) {
> + pr_info("hsiphash self-test 4u32: FAIL\n");
> ret = -EINVAL;
> }
> if (!ret)
> --
> 2.11.0
>
Looks good to me (co-designer of SipHash).
Reviewed-by: Jean-Philippe Aumasson <jeanphilippe.aumasson@gmail.com>
^ permalink raw reply
* Re: [PATCH net] amd-xgbe: Fix IRQ processing when running in single IRQ mode
From: David Miller @ 2017-01-06 20:34 UTC (permalink / raw)
To: thomas.lendacky; +Cc: netdev
In-Reply-To: <20170104210716.8401.99717.stgit@tlendack-t1.amdoffice.net>
From: Tom Lendacky <thomas.lendacky@amd.com>
Date: Wed, 4 Jan 2017 15:07:16 -0600
> When running in single IRQ mode, the additional IRQ routines were being
> skipped because only the XGMAC interrupt status was being checked.
> Update the code so that the additional IRQ routines are checked whenever
> an interrupt is received.
>
> Signed-off-by: Tom Lendacky <thomas.lendacky@amd.com>
Applied.
^ permalink raw reply
* Re: [PATCH] sh_eth: fix EESIPR values for SH77{34|63}
From: David Miller @ 2017-01-06 20:35 UTC (permalink / raw)
To: sergei.shtylyov; +Cc: netdev, linux-renesas-soc
In-Reply-To: <1819012.0ROIJmx7mp@wasted.cogentembedded.com>
From: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Date: Wed, 04 Jan 2017 22:18:24 +0300
> As the SH77{34|63} manuals are freely available, I've checked the EESIPR
> values written against the manuals, and they appeared to set the reserved
> bits 11-15 (which should be 0 on write). Fix those EESIPR values.
>
> Fixes: 380af9e390ec ("net: sh_eth: CPU dependency code collect to "struct sh_eth_cpu_data"")
> Fixes: f5d12767c8fd ("sh_eth: get SH77{34|63} support out of #ifdef")
> Signed-off-by: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Applied.
^ permalink raw reply
* Re: [PATCH] sh_eth: R8A7740 supports packet shecksumming
From: David Miller @ 2017-01-06 20:35 UTC (permalink / raw)
To: sergei.shtylyov; +Cc: netdev, linux-renesas-soc
In-Reply-To: <1871204.P0zXJRoIfd@wasted.cogentembedded.com>
From: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Date: Thu, 05 Jan 2017 00:29:32 +0300
> The R8A7740 GEther controller supports the packet checksum offloading
> but the 'hw_crc' (bad name, I'll fix it) flag isn't set in the R8A7740
> data, thus CSMR isn't cleared...
>
> Fixes: 73a0d907301e ("net: sh_eth: add support R8A7740")
> Signed-off-by: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Applied.
^ permalink raw reply
* Re: [PATCH] sh_eth: R8A7740 supports packet shecksumming
From: David Miller @ 2017-01-06 20:35 UTC (permalink / raw)
To: sergei.shtylyov; +Cc: netdev, linux-renesas-soc
In-Reply-To: <9109092d-f781-1177-12f1-de9edda07f69@cogentembedded.com>
From: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Date: Thu, 5 Jan 2017 12:33:21 +0300
> Oops, typo in the subject, "shecksumming". David, should I resend?
I accidently pushed this out without fixing the typo, sorry about
that but that's going to be how it is I'm afraid :-)
^ permalink raw reply
* [net-next v1 0/8] netcp: enhancements and minor fixes
From: Murali Karicheri @ 2017-01-06 20:37 UTC (permalink / raw)
To: davem, arnd, netdev, linux-kernel, mugunthanvnm, linux-omap,
grygorii.strashko
This series is for net-next. This propagates enhancements and minor
bug fixes from internal version of the driver to keep the upstream
in sync. Please review and apply if this looks good.
Tested on all of K2HK/E/L boards with nfs rootfs.
Test logs below
K2HK-EVM: http://pastebin.ubuntu.com/23754106/
k2L-EVM: http://pastebin.ubuntu.com/23754143/
K2E-EVM: http://pastebin.ubuntu.com/23754159/
History:
v1 - dropped 1/10 amd 2/10 of v0 based on comments from Rob as
it needs more work before submission
v0 - Initial version
Michael Scherban (1):
net: netcp: store network statistics in 64 bits
Murali Karicheri (7):
net: netcp: extract eflag from desc for rx_hook handling
net: netcp: remove the redundant memmov()
net: netcp: ethss: get phy-handle only if link interface is MAC-to-PHY
net: netcp: use hw capability to remove FCS word from rx packets
net: netcp: ale: update to support unknown vlan controls for NU switch
net: netcp: ale: use ale_status to size the ale table
net: netcp: ale: add proper ale entry mask bits for netcp switch ALE
drivers/net/ethernet/ti/cpsw_ale.c | 180 +++++++++++++++++++++++++++++-----
drivers/net/ethernet/ti/cpsw_ale.h | 17 +++-
drivers/net/ethernet/ti/netcp.h | 21 ++++
drivers/net/ethernet/ti/netcp_core.c | 102 +++++++++++++++----
drivers/net/ethernet/ti/netcp_ethss.c | 23 +++--
include/linux/soc/ti/knav_dma.h | 2 +
6 files changed, 292 insertions(+), 53 deletions(-)
--
1.9.1
^ permalink raw reply
* [net-next v1 2/8] net: netcp: remove the redundant memmov()
From: Murali Karicheri @ 2017-01-06 20:37 UTC (permalink / raw)
To: davem, arnd, netdev, linux-kernel, mugunthanvnm, linux-omap,
grygorii.strashko
In-Reply-To: <1483735066-14632-1-git-send-email-m-karicheri2@ti.com>
The psdata is populated with command data by netcp modules
to the tail of the buffer and set_words() copy the same
to the front of the psdata. So remove the redundant memmov
function call.
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
---
drivers/net/ethernet/ti/netcp_core.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/ti/netcp_core.c b/drivers/net/ethernet/ti/netcp_core.c
index a136c56..286fd8d 100644
--- a/drivers/net/ethernet/ti/netcp_core.c
+++ b/drivers/net/ethernet/ti/netcp_core.c
@@ -1226,9 +1226,9 @@ static int netcp_tx_submit_skb(struct netcp_intf *netcp,
/* psdata points to both native-endian and device-endian data */
__le32 *psdata = (void __force *)p_info.psdata;
- memmove(p_info.psdata, p_info.psdata + p_info.psdata_len,
- p_info.psdata_len);
- set_words(p_info.psdata, p_info.psdata_len, psdata);
+ set_words((u32 *)psdata +
+ (KNAV_DMA_NUM_PS_WORDS - p_info.psdata_len),
+ p_info.psdata_len, psdata);
tmp |= (p_info.psdata_len & KNAV_DMA_DESC_PSLEN_MASK) <<
KNAV_DMA_DESC_PSLEN_SHIFT;
}
--
1.9.1
^ permalink raw reply related
* [net-next v1 8/8] net: netcp: ale: add proper ale entry mask bits for netcp switch ALE
From: Murali Karicheri @ 2017-01-06 20:37 UTC (permalink / raw)
To: davem, arnd, netdev, linux-kernel, mugunthanvnm, linux-omap,
grygorii.strashko
In-Reply-To: <1483735066-14632-1-git-send-email-m-karicheri2@ti.com>
For NetCP NU Switch ALE, some of the mask bits are different than
defaults used in the driver. Add a new macro DEFINE_ALE_FIELD1 that use
a configurable mask bits and use it in the driver. These bits are set to
correct values by using the new variables added to cpsw_ale structure
and re-used in the macros. The parameter nu_switch_ale is configured by
the caller driver to indicate the ALE is for that switch and is used in
the ALE driver to do customization as needed.
Signed-off-by: Murali Karicheri <m-karicheri2@ti.com>
Signed-off-by: Sekhar Nori <nsekhar@ti.com>
---
drivers/net/ethernet/ti/cpsw_ale.c | 99 ++++++++++++++++++++++++++++++--------
drivers/net/ethernet/ti/cpsw_ale.h | 4 ++
2 files changed, 84 insertions(+), 19 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw_ale.c b/drivers/net/ethernet/ti/cpsw_ale.c
index 62a18d6..ddd43e0 100644
--- a/drivers/net/ethernet/ti/cpsw_ale.c
+++ b/drivers/net/ethernet/ti/cpsw_ale.c
@@ -29,6 +29,7 @@
#define ALE_VERSION_MAJOR(rev, mask) (((rev) >> 8) & (mask))
#define ALE_VERSION_MINOR(rev) (rev & 0xff)
+#define ALE_VERSION_1R3 0x0103
#define ALE_VERSION_1R4 0x0104
/* ALE Registers */
@@ -46,6 +47,7 @@
#define ALE_UNKNOWNVLAN_UNREG_MCAST_FLOOD 0x94
#define ALE_UNKNOWNVLAN_REG_MCAST_FLOOD 0x98
#define ALE_UNKNOWNVLAN_FORCE_UNTAG_EGRESS 0x9C
+#define ALE_VLAN_MASK_MUX(reg) (0xc0 + (0x4 * (reg)))
#define ALE_TABLE_WRITE BIT(31)
@@ -96,20 +98,34 @@ static inline void cpsw_ale_set_field(u32 *ale_entry, u32 start, u32 bits,
cpsw_ale_set_field(ale_entry, start, bits, value); \
}
+#define DEFINE_ALE_FIELD1(name, start) \
+static inline int cpsw_ale_get_##name(u32 *ale_entry, u32 bits) \
+{ \
+ return cpsw_ale_get_field(ale_entry, start, bits); \
+} \
+static inline void cpsw_ale_set_##name(u32 *ale_entry, u32 value, \
+ u32 bits) \
+{ \
+ cpsw_ale_set_field(ale_entry, start, bits, value); \
+}
+
DEFINE_ALE_FIELD(entry_type, 60, 2)
DEFINE_ALE_FIELD(vlan_id, 48, 12)
DEFINE_ALE_FIELD(mcast_state, 62, 2)
-DEFINE_ALE_FIELD(port_mask, 66, 3)
+DEFINE_ALE_FIELD1(port_mask, 66)
DEFINE_ALE_FIELD(super, 65, 1)
DEFINE_ALE_FIELD(ucast_type, 62, 2)
-DEFINE_ALE_FIELD(port_num, 66, 2)
+DEFINE_ALE_FIELD1(port_num, 66)
DEFINE_ALE_FIELD(blocked, 65, 1)
DEFINE_ALE_FIELD(secure, 64, 1)
-DEFINE_ALE_FIELD(vlan_untag_force, 24, 3)
-DEFINE_ALE_FIELD(vlan_reg_mcast, 16, 3)
-DEFINE_ALE_FIELD(vlan_unreg_mcast, 8, 3)
-DEFINE_ALE_FIELD(vlan_member_list, 0, 3)
+DEFINE_ALE_FIELD1(vlan_untag_force, 24)
+DEFINE_ALE_FIELD1(vlan_reg_mcast, 16)
+DEFINE_ALE_FIELD1(vlan_unreg_mcast, 8)
+DEFINE_ALE_FIELD1(vlan_member_list, 0)
DEFINE_ALE_FIELD(mcast, 40, 1)
+/* ALE NetCP nu switch specific */
+DEFINE_ALE_FIELD(vlan_unreg_mcast_idx, 20, 3)
+DEFINE_ALE_FIELD(vlan_reg_mcast_idx, 44, 3)
/* The MAC address field in the ALE entry cannot be macroized as above */
static inline void cpsw_ale_get_addr(u32 *ale_entry, u8 *addr)
@@ -235,14 +251,16 @@ static void cpsw_ale_flush_mcast(struct cpsw_ale *ale, u32 *ale_entry,
{
int mask;
- mask = cpsw_ale_get_port_mask(ale_entry);
+ mask = cpsw_ale_get_port_mask(ale_entry,
+ ale->port_mask_bits);
if ((mask & port_mask) == 0)
return; /* ports dont intersect, not interested */
mask &= ~port_mask;
/* free if only remaining port is host port */
if (mask)
- cpsw_ale_set_port_mask(ale_entry, mask);
+ cpsw_ale_set_port_mask(ale_entry, mask,
+ ale->port_mask_bits);
else
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_FREE);
}
@@ -303,7 +321,7 @@ int cpsw_ale_add_ucast(struct cpsw_ale *ale, u8 *addr, int port,
cpsw_ale_set_ucast_type(ale_entry, ALE_UCAST_PERSISTANT);
cpsw_ale_set_secure(ale_entry, (flags & ALE_SECURE) ? 1 : 0);
cpsw_ale_set_blocked(ale_entry, (flags & ALE_BLOCKED) ? 1 : 0);
- cpsw_ale_set_port_num(ale_entry, port);
+ cpsw_ale_set_port_num(ale_entry, port, ale->port_num_bits);
idx = cpsw_ale_match_addr(ale, addr, (flags & ALE_VLAN) ? vid : 0);
if (idx < 0)
@@ -350,9 +368,11 @@ int cpsw_ale_add_mcast(struct cpsw_ale *ale, u8 *addr, int port_mask,
cpsw_ale_set_super(ale_entry, (flags & ALE_BLOCKED) ? 1 : 0);
cpsw_ale_set_mcast_state(ale_entry, mcast_state);
- mask = cpsw_ale_get_port_mask(ale_entry);
+ mask = cpsw_ale_get_port_mask(ale_entry,
+ ale->port_mask_bits);
port_mask |= mask;
- cpsw_ale_set_port_mask(ale_entry, port_mask);
+ cpsw_ale_set_port_mask(ale_entry, port_mask,
+ ale->port_mask_bits);
if (idx < 0)
idx = cpsw_ale_match_free(ale);
@@ -379,7 +399,8 @@ int cpsw_ale_del_mcast(struct cpsw_ale *ale, u8 *addr, int port_mask,
cpsw_ale_read(ale, idx, ale_entry);
if (port_mask)
- cpsw_ale_set_port_mask(ale_entry, port_mask);
+ cpsw_ale_set_port_mask(ale_entry, port_mask,
+ ale->port_mask_bits);
else
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_FREE);
@@ -388,6 +409,21 @@ int cpsw_ale_del_mcast(struct cpsw_ale *ale, u8 *addr, int port_mask,
}
EXPORT_SYMBOL_GPL(cpsw_ale_del_mcast);
+/* ALE NetCP NU switch specific vlan functions */
+static void cpsw_ale_set_vlan_mcast(struct cpsw_ale *ale, u32 *ale_entry,
+ int reg_mcast, int unreg_mcast)
+{
+ int idx;
+
+ /* Set VLAN registered multicast flood mask */
+ idx = cpsw_ale_get_vlan_reg_mcast_idx(ale_entry);
+ writel(reg_mcast, ale->params.ale_regs + ALE_VLAN_MASK_MUX(idx));
+
+ /* Set VLAN unregistered multicast flood mask */
+ idx = cpsw_ale_get_vlan_unreg_mcast_idx(ale_entry);
+ writel(unreg_mcast, ale->params.ale_regs + ALE_VLAN_MASK_MUX(idx));
+}
+
int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
int reg_mcast, int unreg_mcast)
{
@@ -401,10 +437,16 @@ int cpsw_ale_add_vlan(struct cpsw_ale *ale, u16 vid, int port, int untag,
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_VLAN);
cpsw_ale_set_vlan_id(ale_entry, vid);
- cpsw_ale_set_vlan_untag_force(ale_entry, untag);
- cpsw_ale_set_vlan_reg_mcast(ale_entry, reg_mcast);
- cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_mcast);
- cpsw_ale_set_vlan_member_list(ale_entry, port);
+ cpsw_ale_set_vlan_untag_force(ale_entry, untag, ale->vlan_field_bits);
+ if (!ale->params.nu_switch_ale) {
+ cpsw_ale_set_vlan_reg_mcast(ale_entry, reg_mcast,
+ ale->vlan_field_bits);
+ cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_mcast,
+ ale->vlan_field_bits);
+ } else {
+ cpsw_ale_set_vlan_mcast(ale, ale_entry, reg_mcast, unreg_mcast);
+ }
+ cpsw_ale_set_vlan_member_list(ale_entry, port, ale->vlan_field_bits);
if (idx < 0)
idx = cpsw_ale_match_free(ale);
@@ -430,7 +472,8 @@ int cpsw_ale_del_vlan(struct cpsw_ale *ale, u16 vid, int port_mask)
cpsw_ale_read(ale, idx, ale_entry);
if (port_mask)
- cpsw_ale_set_vlan_member_list(ale_entry, port_mask);
+ cpsw_ale_set_vlan_member_list(ale_entry, port_mask,
+ ale->vlan_field_bits);
else
cpsw_ale_set_entry_type(ale_entry, ALE_TYPE_FREE);
@@ -458,12 +501,15 @@ void cpsw_ale_set_allmulti(struct cpsw_ale *ale, int allmulti)
if (type != ALE_TYPE_VLAN)
continue;
- unreg_mcast = cpsw_ale_get_vlan_unreg_mcast(ale_entry);
+ unreg_mcast =
+ cpsw_ale_get_vlan_unreg_mcast(ale_entry,
+ ale->vlan_field_bits);
if (allmulti)
unreg_mcast |= 1;
else
unreg_mcast &= ~1;
- cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_mcast);
+ cpsw_ale_set_vlan_unreg_mcast(ale_entry, unreg_mcast,
+ ale->vlan_field_bits);
cpsw_ale_write(ale, idx, ale_entry);
}
}
@@ -769,6 +815,14 @@ void cpsw_ale_start(struct cpsw_ale *ale)
dev_info(ale->params.dev,
"ALE Table size %ld\n", ale->params.ale_entries);
+ /* set default bits for existing h/w */
+ ale->port_mask_bits = 3;
+ ale->port_num_bits = 2;
+ ale->vlan_field_bits = 3;
+
+ /* Set defaults override for ALE on NetCP NU switch and for version
+ * 1R3
+ */
if (ale->params.nu_switch_ale) {
/* Separate registers for unknown vlan configuration.
* Also there are N bits, where N is number of ale
@@ -793,6 +847,13 @@ void cpsw_ale_start(struct cpsw_ale *ale)
ale_controls[ALE_PORT_UNTAGGED_EGRESS].shift = 0;
ale_controls[ALE_PORT_UNTAGGED_EGRESS].offset =
ALE_UNKNOWNVLAN_FORCE_UNTAG_EGRESS;
+ ale->port_mask_bits = ale->params.ale_ports;
+ ale->port_num_bits = ale->params.ale_ports - 1;
+ ale->vlan_field_bits = ale->params.ale_ports;
+ } else if (ale->version == ALE_VERSION_1R3) {
+ ale->port_mask_bits = ale->params.ale_ports;
+ ale->port_num_bits = 3;
+ ale->vlan_field_bits = ale->params.ale_ports;
}
cpsw_ale_control_set(ale, 0, ALE_ENABLE, 1);
diff --git a/drivers/net/ethernet/ti/cpsw_ale.h b/drivers/net/ethernet/ti/cpsw_ale.h
index b1c7954..25d24e8 100644
--- a/drivers/net/ethernet/ti/cpsw_ale.h
+++ b/drivers/net/ethernet/ti/cpsw_ale.h
@@ -39,6 +39,10 @@ struct cpsw_ale {
unsigned long ageout;
int allmulti;
u32 version;
+ /* These bits are different on NetCP NU Switch ALE */
+ u32 port_mask_bits;
+ u32 port_num_bits;
+ u32 vlan_field_bits;
};
enum cpsw_ale_control {
--
1.9.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox