* Re: [PATCH net-next 0/8] tcp: default RACK loss recovery
From: hiren panchasara @ 2018-05-21 18:39 UTC (permalink / raw)
To: David Miller; +Cc: ycheng, netdev, edumazet, ncardwell, soheil, priyarjha
In-Reply-To: <20180517.154529.1714253947519146893.davem@davemloft.net>
[-- Attachment #1: Type: text/plain, Size: 1830 bytes --]
On 05/17/18 at 03:45P, David Miller wrote:
> From: Yuchung Cheng <ycheng@google.com>
> Date: Wed, 16 May 2018 16:40:09 -0700
>
> > This patch set implements the features correspond to the
> > draft-ietf-tcpm-rack-03 version of the RACK draft.
> > https://datatracker.ietf.org/meeting/101/materials/slides-101-tcpm-update-on-tcp-rack-00
> >
> > 1. SACK: implement equivalent DUPACK threshold heuristic in RACK to
> > replace existing RFC6675 recovery (tcp_mark_head_lost).
> >
> > 2. Non-SACK: simplify RFC6582 NewReno implementation
> >
> > 3. RTO: apply RACK's time-based approach to avoid spuriouly
> > marking very recently sent packets lost.
> >
> > 4. with (1)(2)(3), make RACK the exclusive fast recovery mechanism to
> > mark losses based on time on S/ACK. Tail loss probe and F-RTO remain
> > enabled by default as complementary mechanisms to send probes in
> > CA_Open and CA_Loss states. The probes would solicit S/ACKs to trigger
> > RACK time-based loss detection.
> >
> > All Google web and internal servers have been running RACK-only mode
> > (4) for a while now. a/b experiments indicate RACK/TLP on average
> > reduces recovery latency by 10% compared to RFC6675. RFC6675
> > is default-off now but can be enabled by disabling RACK (sysctl
> > net.ipv4.tcp_recovery=0) for unseen issues.
>
> Series applied.
>
> These patches, the design of the ordering of changes in the patch series,
> and the commit messages themselves were more than a pleasure to read.
>
> Really, this patch series is a great model for others who want to
> improve the quality and reviewability of their submissions.
Second that as a maintainer of a non-linux stack. Thanks a ton for such
clear and crisp commit-log messages along with the actual changes.
Cheers,
Hiren
[-- Attachment #2: Type: application/pgp-signature, Size: 603 bytes --]
^ permalink raw reply
* [PATCH net-next 0/5] TI Ethernet driver warnings fixes
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
Hi all,
This patch series attempts to fix properly the warnings observed with turning
on COMPILE_TEST and TI Ethernet drivers on 64-bit hosts.
Since I don't have any of this hardware, please review carefully for possible
breakage!
Thank you!
Florian Fainelli (5):
ti: ethernet: cpdma: Use correct format for genpool_*
net: ethernet: ti: cpts: Fix timestamp print
net: ethernet: ti: cpsw: Fix cpsw_add_ch_strings() printk format
net: ethernet: davinci_emac: Fix printing of base address
ti: ethernet: davinci: Fix cast to int warnings
drivers/net/ethernet/ti/cpsw.c | 4 ++--
drivers/net/ethernet/ti/cpts.c | 3 ++-
drivers/net/ethernet/ti/davinci_cpdma.c | 10 +++++-----
drivers/net/ethernet/ti/davinci_emac.c | 4 ++--
4 files changed, 11 insertions(+), 10 deletions(-)
--
2.14.1
^ permalink raw reply
* [PATCH net-next 1/5] ti: ethernet: cpdma: Use correct format for genpool_*
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
Now that we can compile davinci_cpdma.c on 64-bit hosts, we can see that
the format used for printing a size_t type is incorrect, use %zd
accordingly.
Fixes: aeec3021043b ("net: ethernet: ti: cpdma: remove used_desc counter")
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 31ae04117f0a..9ec49213de5e 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -191,7 +191,7 @@ static void cpdma_desc_pool_destroy(struct cpdma_ctlr *ctlr)
return;
WARN(gen_pool_size(pool->gen_pool) != gen_pool_avail(pool->gen_pool),
- "cpdma_desc_pool size %d != avail %d",
+ "cpdma_desc_pool size %zd != avail %zd",
gen_pool_size(pool->gen_pool),
gen_pool_avail(pool->gen_pool));
if (pool->cpumap)
--
2.14.1
^ permalink raw reply related
* [PATCH net-next 2/5] net: ethernet: ti: cpts: Fix timestamp print
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
On 64-bit hosts we will get the following warning:
drivers/net/ethernet/ti/cpts.c: In function 'cpts_overflow_check':
drivers/net/ethernet/ti/cpts.c:297:11: warning: format '%lld' expects
argument of type 'long long int', but argument 3 has type
'__kernel_time_t {aka long int}' [-Wformat=]
pr_debug("cpts overflow check at %lld.%09lu\n", ts.tv_sec,
ts.tv_nsec);
Fix this by using an appropriate casting that works on all bit sizes.
Fixes: a5c79c26e168 ("ptp: cpts: convert to the 64 bit get/set time methods.")
Fixes: 87c0e764d43a ("cpts: introduce time stamping code and a PTP hardware clock.")
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/ti/cpts.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/ti/cpts.c b/drivers/net/ethernet/ti/cpts.c
index 7842f094f2ef..6f63c8729afc 100644
--- a/drivers/net/ethernet/ti/cpts.c
+++ b/drivers/net/ethernet/ti/cpts.c
@@ -294,7 +294,8 @@ static long cpts_overflow_check(struct ptp_clock_info *ptp)
delay = CPTS_SKB_TX_WORK_TIMEOUT;
spin_unlock_irqrestore(&cpts->lock, flags);
- pr_debug("cpts overflow check at %lld.%09lu\n", ts.tv_sec, ts.tv_nsec);
+ pr_debug("cpts overflow check at %lld.%09ld\n",
+ (long long)ts.tv_sec, ts.tv_nsec);
return (long)delay;
}
--
2.14.1
^ permalink raw reply related
* [PATCH net-next 3/5] net: ethernet: ti: cpsw: Fix cpsw_add_ch_strings() printk format
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
When building on a 64-bit host we will get the following warning:
drivers/net/ethernet/ti/cpsw.c: In function 'cpsw_add_ch_strings':
drivers/net/ethernet/ti/cpsw.c:1284:19: warning: format '%d' expects
argument of type 'int', but argument 5 has type 'long unsigned int'
[-Wformat=]
"%s DMA chan %d: %s", rx_dir ? "Rx" : "Tx",
~^
%ld
Fix this by using an %ld format and casting to long.
Fixes: e05107e6b747 ("net: ethernet: ti: cpsw: add multi queue support")
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/ti/cpsw.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index a7285dddfd29..643cd2d9dfb6 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -1281,8 +1281,8 @@ static void cpsw_add_ch_strings(u8 **p, int ch_num, int rx_dir)
for (i = 0; i < ch_stats_len; i++) {
line = i % CPSW_STATS_CH_LEN;
snprintf(*p, ETH_GSTRING_LEN,
- "%s DMA chan %d: %s", rx_dir ? "Rx" : "Tx",
- i / CPSW_STATS_CH_LEN,
+ "%s DMA chan %ld: %s", rx_dir ? "Rx" : "Tx",
+ (long)(i / CPSW_STATS_CH_LEN),
cpsw_gstrings_ch_stats[line].stat_string);
*p += ETH_GSTRING_LEN;
}
--
2.14.1
^ permalink raw reply related
* [PATCH net-next 5/5] ti: ethernet: davinci: Fix cast to int warnings
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
Now that we can compile test this driver on 64-bit hosts, we get some
warnings about how a pointer/address is written/read to/from a register
(sw_token). Fix this by doing the appropriate conversions, we cannot
possibly have the driver work on 64-bit hosts the way the tokens are
managed though, since the registers being written to a 32-bit only.
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/ti/davinci_cpdma.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_cpdma.c b/drivers/net/ethernet/ti/davinci_cpdma.c
index 9ec49213de5e..cdbddf16dd29 100644
--- a/drivers/net/ethernet/ti/davinci_cpdma.c
+++ b/drivers/net/ethernet/ti/davinci_cpdma.c
@@ -1080,7 +1080,7 @@ int cpdma_chan_submit(struct cpdma_chan *chan, void *token, void *data,
writel_relaxed(buffer, &desc->hw_buffer);
writel_relaxed(len, &desc->hw_len);
writel_relaxed(mode | len, &desc->hw_mode);
- writel_relaxed(token, &desc->sw_token);
+ writel_relaxed((uintptr_t)token, &desc->sw_token);
writel_relaxed(buffer, &desc->sw_buffer);
writel_relaxed(len, &desc->sw_len);
desc_read(desc, sw_len);
@@ -1121,15 +1121,15 @@ static void __cpdma_chan_free(struct cpdma_chan *chan,
struct cpdma_desc_pool *pool = ctlr->pool;
dma_addr_t buff_dma;
int origlen;
- void *token;
+ uintptr_t token;
- token = (void *)desc_read(desc, sw_token);
+ token = desc_read(desc, sw_token);
buff_dma = desc_read(desc, sw_buffer);
origlen = desc_read(desc, sw_len);
dma_unmap_single(ctlr->dev, buff_dma, origlen, chan->dir);
cpdma_desc_free(pool, desc, 1);
- (*chan->handler)(token, outlen, status);
+ (*chan->handler)((void *)token, outlen, status);
}
static int __cpdma_chan_process(struct cpdma_chan *chan)
--
2.14.1
^ permalink raw reply related
* [PATCH net-next 4/5] net: ethernet: davinci_emac: Fix printing of base address
From: Florian Fainelli @ 2018-05-21 18:45 UTC (permalink / raw)
To: netdev
Cc: Florian Fainelli, Grygorii Strashko, David S. Miller,
Ivan Khoronzhuk, Andrew Morton, Keerthy, Bhumika Goyal,
Hernán Gonzalez, Richard Cochran, Lukas Wunner, Rob Herring,
open list:TI ETHERNET SWITCH DRIVER (CPSW), open list
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
Use %pa which is the correct formatter to print a physical address,
instead of %p which is just a pointer.
Fixes: a6286ee630f6 ("net: Add TI DaVinci EMAC driver")
Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
---
drivers/net/ethernet/ti/davinci_emac.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/ti/davinci_emac.c b/drivers/net/ethernet/ti/davinci_emac.c
index abceea802ea1..be0fec17d95d 100644
--- a/drivers/net/ethernet/ti/davinci_emac.c
+++ b/drivers/net/ethernet/ti/davinci_emac.c
@@ -1930,8 +1930,8 @@ static int davinci_emac_probe(struct platform_device *pdev)
if (netif_msg_probe(priv)) {
dev_notice(&pdev->dev, "DaVinci EMAC Probe found device "
- "(regs: %p, irq: %d)\n",
- (void *)priv->emac_base_phys, ndev->irq);
+ "(regs: %pa, irq: %d)\n",
+ &priv->emac_base_phys, ndev->irq);
}
pm_runtime_put(&pdev->dev);
--
2.14.1
^ permalink raw reply related
* Re: [RFC feedback] AF_XDP and non-Intel hardware
From: Björn Töpel @ 2018-05-21 18:55 UTC (permalink / raw)
To: Mykyta Iziumtsev
Cc: Netdev, Björn Töpel, Karlsson, Magnus, Zhang, Qi Z,
Francois Ozog, Ilias Apalodimas, Brian Brooks,
Jesper Dangaard Brouer, andy, michael.chan, luke
In-Reply-To: <CAPvmOuHhoLDB+214_1BptHtUtyU0of=4_99cSXNyv8T-WYjvvg@mail.gmail.com>
2018-05-21 14:34 GMT+02:00 Mykyta Iziumtsev <mykyta.iziumtsev@linaro.org>:
> Hi Björn and Magnus,
>
> (This thread is a follow up to private dialogue. The intention is to
> let community know that AF_XDP can be enhanced further to make it
> compatible with wider range of NIC vendors).
>
Mykyta, thanks for doing the write-up and sending it to the netdev
list! The timing could not be better -- we need to settle on an uapi
that works for all vendors prior enabling it in the kernel.
> There are two NIC variations which don't fit well with current AF_XDP proposal.
>
> The first variation is represented by some NXP and Cavium NICs. AF_XDP
> expects NIC to put incoming frames into slots defined by UMEM area.
> Here slot size is set in XDP_UMEM_REG xdp_umem.reg.frame_size and
> slots available to NIC are communicated to the kernel via UMEM fill
> queue. While Intel NICs support only one slot size, NXP and Cavium
> support multiple slot sizes to optimize memory usage (e.g. { 128, 512,
> 2048 }, please refer to [1] for rationale). On frame reception the NIC
> pulls a slot from best fit pool based on frame size.
>
> The second variation is represented by e.g. Chelsio T5/T6 and Netcope
> NICs. As shown above, AF_XDP expects NIC to put incoming frames at
> predefined addresses. This is not the case here as the NIC is in
> charge of placing frames in memory based on it's own algorithm. For
> example, Chelsio T5/T6 is expecting to get whole pages from the driver
> and puts incoming frames on the page in a 'tape recorder' fashion.
> Assuming 'base' is page address and flen[N] is an array of frame
> lengths, the frame placement in memory will look like that:
> base + 0
> base + frame_len[0]
> base + frame_len[0] + frame_len[1]
> base + frame_len[0] + frame_len[1] + frame_len[2]
> ...
>
> To better support these two NIC variations I suggest to abandon 'frame
> size' structuring in UMEM and stick with 'pages' instead. The NIC
> kernel driver is then responsible for splitting provided pages into
> slots expected by underlying HW (or not splitting at all in case of
> Chelsio/Netcope).
>
Let's call the first variation "multi-pool" and the second
"type-writer" for simplicity. The type-writer model is very
interesting, and makes a lot of sense when the PCIe bus is a
constraint.
> On XDP_UMEM_REG the application needs to specify page_size. Then the
> application can pass empty pages to the kernel driver using UMEM
> 'fill' queue by specifying page offset within the UMEM area. xdp_desc
> format needs to be changed as well: frame location will be defined by
> offset from the beginning of UMEM area instead of frame index. As
> payload headroom can vary with AF_XDP we'll need to specify it in
> xdp_desc as well. Beside that it could be worth to consider changing
> payload length to u16 as 64k+ frames aren't very common in networking.
> The resulting xdp_desc would look like that:
>
> struct xdp_desc {
> __u64 offset;
> __u16 headroom;
> __u16 len;
> __u8 flags;
> __u8 padding[3];
> };
>
Let's expand a bit here; Instead of passing indicies to fixed sized
frames to the fill ring, we now pass a memory region. For simplicity,
let's say a page. The fill ring descriptor requires offset and
len. The offset is a global offset from an UMEM perspective, and len
is the size of the region.
On the Rx ring the descriptor, as you wrote, must be changed as well
to your suggestion above. Note, that headroom is still needed, since
XDP can change the size of a packet, so the fixed headroom stated in
UMEM registration is not sufficient.
This model is obviously more flexible, but then also a bit more
complex. E.g. a fixed-frame NIC (like ixgbe), what should the
behaviour be? Should the fill ring entry be used only for *one* frame,
or chopped up to multiple receive frames? Should it be configurable?
Driver specific?
Also, validating the entries in the fill queue require more work
(compared to the current index variant). Currently, we only skip
invalid indicies. What should we do when say, you pass a memory window
that is too narrow (say 128B) but correct from a UMEM
perspective. Going this path, we need to add pretty hard constraints
so we don't end up it too complex code -- because then it'll be too
slow.
> In current proposal you have a notion of 'frame ownership': 'owned by
> kernel' or 'owned by application'. The ownership is transferred by
> means of enqueueing frame index in UMEM 'fill' queue (from application
> to kernel) or in UMEM 'tx completion' queue (from kernel to
> application). If you decide to adopt 'page' approach this notion needs
> to be changed a bit. This is because in case of packet forwarding one
> and the same page can be used for RX (parts of it enqueued in HW 'free
> lists') and TX (forwarding of previously RXed packets).
>
> I propose to define 'ownership' as a right to manipulate the
> partitioning of the page into frames. Whenever application passes a
> page to the kernel via UMEM 'fill' queue -- the ownership is
> transferred to the kernel. The application can't allocate packets on
> this page until kernel is done with it, but it can change payload of
> RXed packets before forwarding them. The kernel can pass ownership
> back by means of 'end-of-page' in xdp_desc.flags.
>
I like the end-of-page mechanism.
> The pages are definitely supposed to be recycled sooner or later. Even
> if it's not part of kernel API and the housekeeping implementation
> resided completely in application I still would like to propose
> possible (hopefully, cost efficient) solution to that. The recycling
> could be achieved by keeping refcount on pages and recycling the page
> only when it's owned by application and refcount reaches 0.
>
> Whenever application transfers page ownership to the kernel the
> refcount shall be initialized to 0. With each incoming RX xdp_desc the
> corresponding page needs to be identified (xdp_desc.offset >>
> PAGE_SHIFT) and refcount incremented. When the packet gets freed the
> refcount shall be decremented. If packet is forwarded in TX xdp_desc
> -- the refcount gets decremented only on TX completion (again,
> tx_completion.desc >> PAGE_SHIFT). For packets originating from the
> application itself the payload buffers needs to be allocated from
> empty page owned by the application and refcount needs to be
> incremented as well.
>
Strictly speaking, we're not enforcing correctness in the current
solution. If the userspace application passes index 1 mulitple times
to the fill ring, and at the same time send index 1, things will
break. So, with the existing solution the userspace application
*still* needs to track the frames. With this new model, the
tracking/refcounting will be more complex. That might be a concern.
For the multi-pool NICs I think we can still just have one UMEM, and
let the driver decide where in which pool to place this certain chunk
of memory. Thoughts?
Now, how do we go forward? I think this is very important, and I will
hack a copy-mode version for this new API. I'm a bit worried that the
complexity/configuration space will grow and impact performance, but
let's see.
To prove that the API works for the NICs you mentioned, we need an
actual zero-copy implementation for them. Do you think Linaro could
work on a zero-copy variant for any of the NICs above?
Again thanks for bringing this up!
Björn
> [1] https://en.wikipedia.org/wiki/Internet_Mix
>
> With best regards,
> Mykyta
^ permalink raw reply
* Re: [PATCH 0/3] bpf: add boot parameters for sysctl knobs
From: Alexei Starovoitov @ 2018-05-21 18:58 UTC (permalink / raw)
To: Eugene Syromiatnikov
Cc: netdev, linux-kernel, linux-doc, Kees Cook, Kai-Heng Feng,
Daniel Borkmann, Alexei Starovoitov, Jonathan Corbet, Jiri Olsa,
Jesper Dangaard Brouer
In-Reply-To: <20180521122923.GA15717@asgard.redhat.com>
On Mon, May 21, 2018 at 02:29:30PM +0200, Eugene Syromiatnikov wrote:
> Hello.
>
> This patch set adds ability to set default values for
> kernel.unprivileged_bpf_disable, net.core.bpf_jit_harden,
> net.core.bpf_jit_kallsyms sysctl knobs as well as option to override
> them via a boot-time kernel parameter.
Commits log not only should explain 'what' is being done by the patch,
but 'why' as well.
^ permalink raw reply
* Re: [RFC PATCH ghak32 V2 13/13] debug audit: read container ID of a process
From: Steve Grubb @ 2018-05-21 19:16 UTC (permalink / raw)
To: linux-audit
Cc: Richard Guy Briggs, cgroups, containers, linux-api, linux-fsdevel,
LKML, netdev, ebiederm, luto, jlayton, carlos, dhowells, viro,
simo, eparis, serge
In-Reply-To: <1081821010c124fe4e35984ec3dac1654453bb7c.1521179281.git.rgb@redhat.com>
On Friday, March 16, 2018 5:00:40 AM EDT Richard Guy Briggs wrote:
> Add support for reading the container ID from the proc filesystem.
I think this could be useful in general. Please consider this to be part of
the full patch set and not something merely used to debug the patches.
-Steve
> This is a read from the proc entry of the form /proc/PID/containerid
> where PID is the process ID of the task whose container ID is sought.
>
> The read expects up to a u64 value (unset: 18446744073709551615).
>
> Signed-off-by: Richard Guy Briggs <rgb@redhat.com>
> ---
> fs/proc/base.c | 20 ++++++++++++++++++--
> 1 file changed, 18 insertions(+), 2 deletions(-)
>
> diff --git a/fs/proc/base.c b/fs/proc/base.c
> index 6ce4fbe..f66d1e2 100644
> --- a/fs/proc/base.c
> +++ b/fs/proc/base.c
> @@ -1300,6 +1300,21 @@ static ssize_t proc_sessionid_read(struct file *
> file, char __user * buf, .llseek = generic_file_llseek,
> };
>
> +static ssize_t proc_containerid_read(struct file *file, char __user *buf,
> + size_t count, loff_t *ppos)
> +{
> + struct inode *inode = file_inode(file);
> + struct task_struct *task = get_proc_task(inode);
> + ssize_t length;
> + char tmpbuf[TMPBUFLEN*2];
> +
> + if (!task)
> + return -ESRCH;
> + length = scnprintf(tmpbuf, TMPBUFLEN*2, "%llu",
> audit_get_containerid(task)); + put_task_struct(task);
> + return simple_read_from_buffer(buf, count, ppos, tmpbuf, length);
> +}
> +
> static ssize_t proc_containerid_write(struct file *file, const char __user
> *buf, size_t count, loff_t *ppos)
> {
> @@ -1330,6 +1345,7 @@ static ssize_t proc_containerid_write(struct file
> *file, const char __user *buf, }
>
> static const struct file_operations proc_containerid_operations = {
> + .read = proc_containerid_read,
> .write = proc_containerid_write,
> .llseek = generic_file_llseek,
> };
> @@ -2996,7 +3012,7 @@ static int proc_pid_patch_state(struct seq_file *m,
> struct pid_namespace *ns, #ifdef CONFIG_AUDITSYSCALL
> REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations),
> REG("sessionid", S_IRUGO, proc_sessionid_operations),
> - REG("containerid", S_IWUSR, proc_containerid_operations),
> + REG("containerid", S_IWUSR|S_IRUSR, proc_containerid_operations),
> #endif
> #ifdef CONFIG_FAULT_INJECTION
> REG("make-it-fail", S_IRUGO|S_IWUSR, proc_fault_inject_operations),
> @@ -3391,7 +3407,7 @@ static int proc_tid_comm_permission(struct inode
> *inode, int mask) #ifdef CONFIG_AUDITSYSCALL
> REG("loginuid", S_IWUSR|S_IRUGO, proc_loginuid_operations),
> REG("sessionid", S_IRUGO, proc_sessionid_operations),
> - REG("containerid", S_IWUSR, proc_containerid_operations),
> + REG("containerid", S_IWUSR|S_IRUSR, proc_containerid_operations),
> #endif
> #ifdef CONFIG_FAULT_INJECTION
> REG("make-it-fail", S_IRUGO|S_IWUSR, proc_fault_inject_operations),
^ permalink raw reply
* Re: [RFC PATCH ghak32 V2 03/13] audit: log container info of syscalls
From: Steve Grubb @ 2018-05-21 19:19 UTC (permalink / raw)
To: Richard Guy Briggs
Cc: cgroups, containers, linux-api, Linux-Audit Mailing List,
linux-fsdevel, LKML, netdev, ebiederm, luto, jlayton, carlos,
dhowells, viro, simo, eparis, serge
In-Reply-To: <20180517214102.qhg4gofwrbsn2eru@madcap2.tricolour.ca>
On Thursday, May 17, 2018 5:41:02 PM EDT Richard Guy Briggs wrote:
> On 2018-05-17 17:09, Steve Grubb wrote:
> > On Fri, 16 Mar 2018 05:00:30 -0400
> >
> > Richard Guy Briggs <rgb@redhat.com> wrote:
> > > Create a new audit record AUDIT_CONTAINER_INFO to document the
> > > container ID of a process if it is present.
> >
> > As mentioned in a previous email, I think AUDIT_CONTAINER is more
> > suitable for the container record. One more comment below...
> >
> > > Called from audit_log_exit(), syscalls are covered.
> > >
> > > A sample raw event:
> > > type=SYSCALL msg=audit(1519924845.499:257): arch=c000003e syscall=257
> > > success=yes exit=3 a0=ffffff9c a1=56374e1cef30 a2=241 a3=1b6 items=2
> > > ppid=606 pid=635 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0
> > > sgid=0 fsgid=0 tty=pts0 ses=3 comm="bash" exe="/usr/bin/bash"
> > > subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
> > > key="tmpcontainerid" type=CWD msg=audit(1519924845.499:257):
> > > cwd="/root" type=PATH msg=audit(1519924845.499:257): item=0
> > > name="/tmp/" inode=13863 dev=00:27 mode=041777 ouid=0 ogid=0
> > > rdev=00:00 obj=system_u:object_r:tmp_t:s0 nametype= PARENT
> > > cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
> > > type=PATH msg=audit(1519924845.499:257): item=1
> > > name="/tmp/tmpcontainerid" inode=17729 dev=00:27 mode=0100644 ouid=0
> > > ogid=0 rdev=00:00 obj=unconfined_u:object_r:user_tmp_t:s0
> > > nametype=CREATE cap_fp=0000000000000000 cap_fi=0000000000000000
> > > cap_fe=0 cap_fver=0 type=PROCTITLE msg=audit(1519924845.499:257):
> > > proctitle=62617368002D6300736C65657020313B206563686F2074657374203E202F7
> > > 46D702F746D70636F6E7461696E65726964 type=CONTAINER_INFO
> > > msg=audit(1519924845.499:257): op=task
> > > contid=123458
> > >
> > > See: https://github.com/linux-audit/audit-kernel/issues/32
> > > Signed-off-by: Richard Guy Briggs <rgb@redhat.com>
> > > ---
> > >
> > > include/linux/audit.h | 5 +++++
> > > include/uapi/linux/audit.h | 1 +
> > > kernel/audit.c | 20 ++++++++++++++++++++
> > > kernel/auditsc.c | 2 ++
> > > 4 files changed, 28 insertions(+)
> > >
> > > diff --git a/include/linux/audit.h b/include/linux/audit.h
> > > index fe4ba3f..3acbe9d 100644
> > > --- a/include/linux/audit.h
> > > +++ b/include/linux/audit.h
> > > @@ -154,6 +154,8 @@ extern void
> > > audit_log_link_denied(const char *operation, extern int
> > > audit_log_task_context(struct audit_buffer *ab); extern void
> > > audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk);
> > > +extern int audit_log_container_info(struct task_struct *tsk,
> > > + struct audit_context *context);
> > >
> > > extern int audit_update_lsm_rules(void);
> > >
> > > @@ -205,6 +207,9 @@ static inline int audit_log_task_context(struct
> > > audit_buffer *ab) static inline void audit_log_task_info(struct
> > > audit_buffer *ab, struct task_struct *tsk)
> > >
> > > { }
> > >
> > > +static inline int audit_log_container_info(struct task_struct *tsk,
> > > + struct audit_context
> > > *context); +{ }
> > >
> > > #define audit_enabled 0
> > > #endif /* CONFIG_AUDIT */
> > >
> > > diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h
> > > index 921a71f..e83ccbd 100644
> > > --- a/include/uapi/linux/audit.h
> > > +++ b/include/uapi/linux/audit.h
> > > @@ -115,6 +115,7 @@
> > >
> > > #define AUDIT_REPLACE 1329 /* Replace auditd
> > >
> > > if this packet unanswerd */ #define AUDIT_KERN_MODULE
> > > 1330 /* Kernel Module events */ #define
> > > AUDIT_FANOTIFY 1331 /* Fanotify access decision
> > > */ +#define AUDIT_CONTAINER_INFO 1332 /* Container ID
> > > information */ #define AUDIT_AVC 1400 /* SE
> > > Linux avc denial or grant */ #define AUDIT_SELINUX_ERR
> > > 1401 /* Internal SE Linux Errors */ diff --git
> > > a/kernel/audit.c b/kernel/audit.c index 3f2f143..a12f21f 100644
> > > --- a/kernel/audit.c
> > > +++ b/kernel/audit.c
> > > @@ -2049,6 +2049,26 @@ void audit_log_session_info(struct
> > > audit_buffer *ab) audit_log_format(ab, " auid=%u ses=%u", auid,
> > > sessionid); }
> > >
> > > +/*
> > > + * audit_log_container_info - report container info
> > > + * @tsk: task to be recorded
> > > + * @context: task or local context for record
> > > + */
> > > +int audit_log_container_info(struct task_struct *tsk, struct
> > > audit_context *context) +{
> > > + struct audit_buffer *ab;
> > > +
> > > + if (!audit_containerid_set(tsk))
> > > + return 0;
> > > + /* Generate AUDIT_CONTAINER_INFO with container ID */
> > > + ab = audit_log_start(context, GFP_KERNEL,
> > > AUDIT_CONTAINER_INFO);
> > > + if (!ab)
> > > + return -ENOMEM;
> > > + audit_log_format(ab, "contid=%llu",
> > > audit_get_containerid(tsk));
> > > + audit_log_end(ab);
> > > + return 0;
> > > +}
> > > +
> > >
> > > void audit_log_key(struct audit_buffer *ab, char *key)
> > > {
> > >
> > > audit_log_format(ab, " key=");
> > >
> > > diff --git a/kernel/auditsc.c b/kernel/auditsc.c
> > > index a6b0a52..65be110 100644
> > > --- a/kernel/auditsc.c
> > > +++ b/kernel/auditsc.c
> > > @@ -1453,6 +1453,8 @@ static void audit_log_exit(struct audit_context
> > > *context, struct task_struct *ts
> > >
> > > audit_log_proctitle(tsk, context);
> > >
> > > + audit_log_container_info(tsk, context);
> >
> > Would there be any problem moving audit_log_container_info before
> > audit_log_proctitle? There are some assumptions that proctitle is the
> > last record in some situations.
>
> I see no problem doing that.
Actually...just leave it as is. I have to fix things for simple events and
they do not have a proctitle record. So, leave it as you intended and I'll
work around this on my end.
Thanks,
-Steve
> > > /* Send end of event record to help user space know we are
> > >
> > > finished */ ab = audit_log_start(context, GFP_KERNEL, AUDIT_EOE);
> > >
> > > if (ab)
>
> - RGB
>
> --
> Richard Guy Briggs <rgb@redhat.com>
> Sr. S/W Engineer, Kernel Security, Base Operating Systems
> Remote, Ottawa, Red Hat Canada
> IRC: rgb, SunRaycer
> Voice: +1.647.777.2635, Internal: (81) 32635
^ permalink raw reply
* Re: [RFC PATCH ghak32 V2 13/13] debug audit: read container ID of a process
From: Eric W. Biederman @ 2018-05-21 19:19 UTC (permalink / raw)
To: Steve Grubb
Cc: simo-H+wXaHxf7aLQT0dZR+AlfA, jlayton-H+wXaHxf7aLQT0dZR+AlfA,
linux-api-u79uwXL29TY76Z2rM5mHXA,
containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, LKML,
eparis-FjpueFixGhCM4zKIHC2jIg, dhowells-H+wXaHxf7aLQT0dZR+AlfA,
carlos-H+wXaHxf7aLQT0dZR+AlfA, linux-audit-H+wXaHxf7aLQT0dZR+AlfA,
viro-RmSDqhL/yNMiFSDQTTA3OLVCufUGDwFn,
luto-DgEjT+Ai2ygdnm+yROfE0A, netdev-u79uwXL29TY76Z2rM5mHXA,
linux-fsdevel-u79uwXL29TY76Z2rM5mHXA,
cgroups-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <3001737.MkQ41rgtZF@x2>
Steve Grubb <sgrubb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
> On Friday, March 16, 2018 5:00:40 AM EDT Richard Guy Briggs wrote:
>> Add support for reading the container ID from the proc filesystem.
>
> I think this could be useful in general. Please consider this to be part of
> the full patch set and not something merely used to debug the patches.
Only with an audit specific name.
As it is:
Nacked-by: "Eric W. Biederman" <ebiederm-aS9lmoZGLiVWk0Htik3J/w@public.gmane.org>
The truth is the containerid name really stinks and is quite confusing
and does not imply that the label applies only to audit. And little
things like this make me extremely uncofortable with it.
Eric
^ permalink raw reply
* [PATCH net 1/1] qed: Fix mask for physical address in ILT entry
From: Shahed Shaikh @ 2018-05-21 19:31 UTC (permalink / raw)
To: davem; +Cc: netdev, Ariel.Elior, Dept-EngEverestLinuxL2
ILT entry requires 12 bit right shifted physical address.
Existing mask for ILT entry of physical address i.e.
ILT_ENTRY_PHY_ADDR_MASK is not sufficient to handle 64bit
address because upper 8 bits of 64 bit address were getting
masked which resulted in completer abort error on
PCIe bus due to invalid address.
Fix that mask to handle 64bit physical address.
Fixes: fe56b9e6a8d9 ("qed: Add module with basic common support")
Signed-off-by: Shahed Shaikh <shahed.shaikh@cavium.com>
Signed-off-by: Ariel Elior <ariel.elior@cavium.com>
---
drivers/net/ethernet/qlogic/qed/qed_cxt.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ethernet/qlogic/qed/qed_cxt.c b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
index 00f41c1..820b226 100644
--- a/drivers/net/ethernet/qlogic/qed/qed_cxt.c
+++ b/drivers/net/ethernet/qlogic/qed/qed_cxt.c
@@ -77,7 +77,7 @@
#define ILT_CFG_REG(cli, reg) PSWRQ2_REG_ ## cli ## _ ## reg ## _RT_OFFSET
/* ILT entry structure */
-#define ILT_ENTRY_PHY_ADDR_MASK 0x000FFFFFFFFFFFULL
+#define ILT_ENTRY_PHY_ADDR_MASK (~0ULL >> 12)
#define ILT_ENTRY_PHY_ADDR_SHIFT 0
#define ILT_ENTRY_VALID_MASK 0x1ULL
#define ILT_ENTRY_VALID_SHIFT 52
--
1.7.1
^ permalink raw reply related
* Re: [PATCH bpf v2 6/6] bpf: fix JITed dump for multi-function programs via syscall
From: Sandipan Das @ 2018-05-21 19:42 UTC (permalink / raw)
To: Daniel Borkmann
Cc: ast, netdev, linuxppc-dev, naveen.n.rao, mpe, jakub.kicinski
In-Reply-To: <654d4424-ad99-2b5e-dcbf-9e352aec75d6@iogearbox.net>
Hi Daniel,
On 05/18/2018 09:21 PM, Daniel Borkmann wrote:
> On 05/18/2018 02:50 PM, Sandipan Das wrote:
>> Currently, for multi-function programs, we cannot get the JITed
>> instructions using the bpf system call's BPF_OBJ_GET_INFO_BY_FD
>> command. Because of this, userspace tools such as bpftool fail
>> to identify a multi-function program as being JITed or not.
>>
>> With the JIT enabled and the test program running, this can be
>> verified as follows:
>>
>> # cat /proc/sys/net/core/bpf_jit_enable
>> 1
>>
>> Before applying this patch:
>>
>> # bpftool prog list
>> 1: kprobe name foo tag b811aab41a39ad3d gpl
>> loaded_at 2018-05-16T11:43:38+0530 uid 0
>> xlated 216B not jited memlock 65536B
>> ...
>>
>> # bpftool prog dump jited id 1
>> no instructions returned
>>
>> After applying this patch:
>>
>> # bpftool prog list
>> 1: kprobe name foo tag b811aab41a39ad3d gpl
>> loaded_at 2018-05-16T12:13:01+0530 uid 0
>> xlated 216B jited 308B memlock 65536B
>> ...
>
> That's really nice! One comment inline below:
>
>> # bpftool prog dump jited id 1
>> 0: nop
>> 4: nop
>> 8: mflr r0
>> c: std r0,16(r1)
>> 10: stdu r1,-112(r1)
>> 14: std r31,104(r1)
>> 18: addi r31,r1,48
>> 1c: li r3,10
>> ...
>>
>> Signed-off-by: Sandipan Das <sandipan@linux.vnet.ibm.com>
>> ---
>> kernel/bpf/syscall.c | 38 ++++++++++++++++++++++++++++++++------
>> 1 file changed, 32 insertions(+), 6 deletions(-)
>>
>> diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c
>> index 54a72fafe57c..2430d159078c 100644
>> --- a/kernel/bpf/syscall.c
>> +++ b/kernel/bpf/syscall.c
>> @@ -1896,7 +1896,7 @@ static int bpf_prog_get_info_by_fd(struct bpf_prog *prog,
>> struct bpf_prog_info info = {};
>> u32 info_len = attr->info.info_len;
>> char __user *uinsns;
>> - u32 ulen;
>> + u32 ulen, i;
>> int err;
>>
>> err = check_uarg_tail_zero(uinfo, sizeof(info), info_len);
>> @@ -1922,7 +1922,6 @@ static int bpf_prog_get_info_by_fd(struct bpf_prog *prog,
>> ulen = min_t(u32, info.nr_map_ids, ulen);
>> if (ulen) {
>> u32 __user *user_map_ids = u64_to_user_ptr(info.map_ids);
>> - u32 i;
>>
>> for (i = 0; i < ulen; i++)
>> if (put_user(prog->aux->used_maps[i]->id,
>> @@ -1970,13 +1969,41 @@ static int bpf_prog_get_info_by_fd(struct bpf_prog *prog,
>> * for offload.
>> */
>> ulen = info.jited_prog_len;
>> - info.jited_prog_len = prog->jited_len;
>> + if (prog->aux->func_cnt) {
>> + info.jited_prog_len = 0;
>> + for (i = 0; i < prog->aux->func_cnt; i++)
>> + info.jited_prog_len += prog->aux->func[i]->jited_len;
>> + } else {
>> + info.jited_prog_len = prog->jited_len;
>> + }
>> +
>> if (info.jited_prog_len && ulen) {
>> if (bpf_dump_raw_ok()) {
>> uinsns = u64_to_user_ptr(info.jited_prog_insns);
>> ulen = min_t(u32, info.jited_prog_len, ulen);
>> - if (copy_to_user(uinsns, prog->bpf_func, ulen))
>> - return -EFAULT;
>> +
>> + /* for multi-function programs, copy the JITed
>> + * instructions for all the functions
>> + */
>> + if (prog->aux->func_cnt) {
>> + u32 len, free;
>> + u8 *img;
>> +
>> + free = ulen;
>> + for (i = 0; i < prog->aux->func_cnt; i++) {
>> + len = prog->aux->func[i]->jited_len;
>> + img = (u8 *) prog->aux->func[i]->bpf_func;
>> + if (len > free)
>> + break;
>> + if (copy_to_user(uinsns, img, len))
>> + return -EFAULT;
>> + uinsns += len;
>> + free -= len;
>
> Is there any way we can introduce a delimiter between the different
> images such that they could be more easily correlated with the call
> from the main (or other sub-)program instead of having one contiguous
> dump blob?
>
Can we have another member in bpf_prog_info that points to a list of the lengths of the
JITed images for each subprogram? We can use this information to split up the dump.
- Sandipan
>> + }
>> + } else {
>> + if (copy_to_user(uinsns, prog->bpf_func, ulen))
>> + return -EFAULT;
>> + }
>> } else {
>> info.jited_prog_insns = 0;
>> }
>> @@ -1987,7 +2014,6 @@ static int bpf_prog_get_info_by_fd(struct bpf_prog *prog,
>> if (info.nr_jited_ksyms && ulen) {
>> u64 __user *user_jited_ksyms = u64_to_user_ptr(info.jited_ksyms);
>> ulong ksym_addr;
>> - u32 i;
>>
>> /* copy the address of the kernel symbol corresponding to
>> * each function
>>
>
>
^ permalink raw reply
* Re: [PATCH bpf-next 1/7] bpf: Expose check_uarg_tail_zero()
From: Yonghong Song @ 2018-05-21 20:00 UTC (permalink / raw)
To: Martin KaFai Lau, netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
In-Reply-To: <20180519001650.4043980-2-kafai@fb.com>
On 5/18/18 5:16 PM, Martin KaFai Lau wrote:
> This patch exposes check_uarg_tail_zero() which will
> be reused by a later BTF patch. Its name is changed to
> bpf_check_uarg_tail_zero().
>
> Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Yonghong Song <yhs@fb.com>
^ permalink raw reply
* [PATCH net-next v2] net: sched: don't disable bh when accessing action idr
From: Vlad Buslov @ 2018-05-21 20:03 UTC (permalink / raw)
To: davem; +Cc: netdev, jhs, xiyou.wangcong, jiri, linux-kernel, Vlad Buslov
In-Reply-To: <20180519.230258.1374885458106197707.davem@davemloft.net>
Initial net_device implementation used ingress_lock spinlock to synchronize
ingress path of device. This lock was used in both process and bh context.
In some code paths action map lock was obtained while holding ingress_lock.
Commit e1e992e52faa ("[NET_SCHED] protect action config/dump from irqs")
modified actions to always disable bh, while using action map lock, in
order to prevent deadlock on ingress_lock in softirq. This lock was removed
from net_device, so disabling bh, while accessing action map, is no longer
necessary.
Replace all action idr spinlock usage with regular calls that do not
disable bh.
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
---
net/sched/act_api.c | 20 ++++++++++----------
1 file changed, 10 insertions(+), 10 deletions(-)
diff --git a/net/sched/act_api.c b/net/sched/act_api.c
index 72251241665a..3f4cf930f809 100644
--- a/net/sched/act_api.c
+++ b/net/sched/act_api.c
@@ -77,9 +77,9 @@ static void free_tcf(struct tc_action *p)
static void tcf_idr_remove(struct tcf_idrinfo *idrinfo, struct tc_action *p)
{
- spin_lock_bh(&idrinfo->lock);
+ spin_lock(&idrinfo->lock);
idr_remove(&idrinfo->action_idr, p->tcfa_index);
- spin_unlock_bh(&idrinfo->lock);
+ spin_unlock(&idrinfo->lock);
gen_kill_estimator(&p->tcfa_rate_est);
free_tcf(p);
}
@@ -156,7 +156,7 @@ static int tcf_dump_walker(struct tcf_idrinfo *idrinfo, struct sk_buff *skb,
struct tc_action *p;
unsigned long id = 1;
- spin_lock_bh(&idrinfo->lock);
+ spin_lock(&idrinfo->lock);
s_i = cb->args[0];
@@ -191,7 +191,7 @@ static int tcf_dump_walker(struct tcf_idrinfo *idrinfo, struct sk_buff *skb,
if (index >= 0)
cb->args[0] = index + 1;
- spin_unlock_bh(&idrinfo->lock);
+ spin_unlock(&idrinfo->lock);
if (n_i) {
if (act_flags & TCA_FLAG_LARGE_DUMP_ON)
cb->args[1] = n_i;
@@ -261,9 +261,9 @@ static struct tc_action *tcf_idr_lookup(u32 index, struct tcf_idrinfo *idrinfo)
{
struct tc_action *p = NULL;
- spin_lock_bh(&idrinfo->lock);
+ spin_lock(&idrinfo->lock);
p = idr_find(&idrinfo->action_idr, index);
- spin_unlock_bh(&idrinfo->lock);
+ spin_unlock(&idrinfo->lock);
return p;
}
@@ -323,7 +323,7 @@ int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,
}
spin_lock_init(&p->tcfa_lock);
idr_preload(GFP_KERNEL);
- spin_lock_bh(&idrinfo->lock);
+ spin_lock(&idrinfo->lock);
/* user doesn't specify an index */
if (!index) {
index = 1;
@@ -331,7 +331,7 @@ int tcf_idr_create(struct tc_action_net *tn, u32 index, struct nlattr *est,
} else {
err = idr_alloc_u32(idr, NULL, &index, index, GFP_ATOMIC);
}
- spin_unlock_bh(&idrinfo->lock);
+ spin_unlock(&idrinfo->lock);
idr_preload_end();
if (err)
goto err3;
@@ -369,9 +369,9 @@ void tcf_idr_insert(struct tc_action_net *tn, struct tc_action *a)
{
struct tcf_idrinfo *idrinfo = tn->idrinfo;
- spin_lock_bh(&idrinfo->lock);
+ spin_lock(&idrinfo->lock);
idr_replace(&idrinfo->action_idr, a, a->tcfa_index);
- spin_unlock_bh(&idrinfo->lock);
+ spin_unlock(&idrinfo->lock);
}
EXPORT_SYMBOL(tcf_idr_insert);
--
2.7.5
^ permalink raw reply related
* Re: [RFC PATCH ghak32 V2 13/13] debug audit: read container ID of a process
From: Paul Moore @ 2018-05-21 20:06 UTC (permalink / raw)
To: Eric W. Biederman, Steve Grubb, Richard Guy Briggs
Cc: simo, jlayton, linux-api, containers, LKML, Eric Paris, dhowells,
carlos, linux-audit, viro, luto, netdev, linux-fsdevel, cgroups,
serge
In-Reply-To: <87muwshl4z.fsf@xmission.com>
On Mon, May 21, 2018 at 3:19 PM, Eric W. Biederman
<ebiederm@xmission.com> wrote:
> Steve Grubb <sgrubb@redhat.com> writes:
>
>> On Friday, March 16, 2018 5:00:40 AM EDT Richard Guy Briggs wrote:
>>> Add support for reading the container ID from the proc filesystem.
>>
>> I think this could be useful in general. Please consider this to be part of
>> the full patch set and not something merely used to debug the patches.
>
> Only with an audit specific name.
>
> As it is:
>
> Nacked-by: "Eric W. Biederman" <ebiederm@xmission.com>
>
> The truth is the containerid name really stinks and is quite confusing
> and does not imply that the label applies only to audit. And little
> things like this make me extremely uncofortable with it.
It also makes the audit container ID (notice how I *always* call it
the *audit* container ID? that is not an accident) available for
userspace applications to abuse. Perhaps in the future we can look at
ways to make this more available to applications, but this patch is
not the answer.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: [PATCH 07/33] iwlwifi: mvm: use match_string() helper
From: Luca Coelho @ 2018-05-21 20:12 UTC (permalink / raw)
To: Yisheng Xie, linux-kernel
Cc: Kalle Valo, Intel Linux Wireless, Johannes Berg,
Emmanuel Grumbach, linux-wireless, netdev
In-Reply-To: <1526903890-35761-8-git-send-email-xieyisheng1@huawei.com>
On Mon, 2018-05-21 at 19:57 +0800, Yisheng Xie wrote:
> match_string() returns the index of an array for a matching string,
> which can be used intead of open coded variant.
>
> Cc: Kalle Valo <kvalo@codeaurora.org>
> Cc: Intel Linux Wireless <linuxwifi@intel.com>
> Cc: Johannes Berg <johannes.berg@intel.com>
> Cc: Emmanuel Grumbach <emmanuel.grumbach@intel.com>
> Cc: linux-wireless@vger.kernel.org
> Cc: netdev@vger.kernel.org
> Signed-off-by: Yisheng Xie <xieyisheng1@huawei.com>
> ---
> drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c | 11 +++--------
> 1 file changed, 3 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
> b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
> index 0e6401c..e8249a6 100644
> --- a/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
> +++ b/drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c
> @@ -671,14 +671,9 @@ static ssize_t iwl_dbgfs_bt_cmd_read(struct file
> *file, char __user *user_buf,
> };
> int ret, bt_force_ant_mode;
>
> - for (bt_force_ant_mode = 0;
> - bt_force_ant_mode < ARRAY_SIZE(modes_str);
> - bt_force_ant_mode++) {
> - if (!strcmp(buf, modes_str[bt_force_ant_mode]))
> - break;
> - }
> -
> - if (bt_force_ant_mode >= ARRAY_SIZE(modes_str))
> + bt_force_ant_mode = match_string(modes_str,
> + ARRAY_SIZE(modes_str),
> buf);
> + if (bt_force_ant_mode < 0)
> return -EINVAL;
>
> ret = 0;
Looks fine, I'll push this to our internal tree for review and take a
closer look at what the match_string() function does exactly.
Thanks for the patch.
--
Cheers,
Luca.
^ permalink raw reply
* Re: [PATCH bpf-next 2/7] bpf: btf: Change how section is supported in btf_header
From: Yonghong Song @ 2018-05-21 20:15 UTC (permalink / raw)
To: Martin KaFai Lau, netdev; +Cc: Alexei Starovoitov, Daniel Borkmann, kernel-team
In-Reply-To: <20180519001650.4043980-3-kafai@fb.com>
On 5/18/18 5:16 PM, Martin KaFai Lau wrote:
> There are currently unused section descriptions in the btf_header. Those
> sections are here to support future BTF use cases. For example, the
> func section (func_off) is to support function signature (e.g. the BPF
> prog function signature).
>
> Instead of spelling out all potential sections up-front in the btf_header.
> This patch makes changes to btf_header such that extending it (e.g. adding
> a section) is possible later. The unused ones can be removed for now and
> they can be added back later.
>
> This patch:
> 1. adds a hdr_len to the btf_header. It will allow adding
> sections (and other info like parent_label and parent_name)
> later. The check is similar to the existing bpf_attr.
> If a user passes in a longer hdr_len, the kernel
> ensures the extra tailing bytes are 0.
>
> 2. allows the section order in the BTF object to be
> different from its sec_off order in btf_header.
>
> 3. each sec_off is followed by a sec_len. It must not have gap or
> overlapping among sections.
>
> The string section is ensured to be at the end due to the 4 bytes
> alignment requirement of the type section.
>
> The above changes will allow enough flexibility to
> add new sections (and other info) to the btf_header later.
>
> This patch also removes an unnecessary !err check
> at the end of btf_parse().
>
> Signed-off-by: Martin KaFai Lau <kafai@fb.com>
> ---
> include/uapi/linux/btf.h | 8 +-
> kernel/bpf/btf.c | 207 +++++++++++++++++++++++++++++++++++------------
> 2 files changed, 158 insertions(+), 57 deletions(-)
>
> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> index bcb56ee47014..4fa479741a02 100644
> --- a/include/uapi/linux/btf.h
> +++ b/include/uapi/linux/btf.h
> @@ -12,15 +12,11 @@ struct btf_header {
> __u16 magic;
> __u8 version;
> __u8 flags;
> -
> - __u32 parent_label;
> - __u32 parent_name;
> + __u32 hdr_len;
>
> /* All offsets are in bytes relative to the end of this header */
> - __u32 label_off; /* offset of label section */
> - __u32 object_off; /* offset of data object section*/
> - __u32 func_off; /* offset of function section */
> __u32 type_off; /* offset of type section */
> + __u32 type_len; /* length of type section */
> __u32 str_off; /* offset of string section */
> __u32 str_len; /* length of string section */
> };
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index ded10ab47b8a..536e5981ad8c 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -12,6 +12,7 @@
> #include <linux/uaccess.h>
> #include <linux/kernel.h>
> #include <linux/idr.h>
> +#include <linux/sort.h>
> #include <linux/bpf_verifier.h>
> #include <linux/btf.h>
>
> @@ -184,15 +185,13 @@ static DEFINE_IDR(btf_idr);
> static DEFINE_SPINLOCK(btf_idr_lock);
>
> struct btf {
> - union {
> - struct btf_header *hdr;
> - void *data;
> - };
> + void *data;
> struct btf_type **types;
> u32 *resolved_ids;
> u32 *resolved_sizes;
> const char *strings;
> void *nohdr_data;
> + struct btf_header hdr;
> u32 nr_types;
> u32 types_size;
> u32 data_size;
> @@ -227,6 +226,12 @@ enum resolve_mode {
> };
>
> #define MAX_RESOLVE_DEPTH 32
> +#define NR_SECS 2
Not sure whether it is necessary to define NR_SECS 2 here or not.
See below.
> +
> +struct btf_sec_info {
> + u32 off;
> + u32 len;
> +};
>
> struct btf_verifier_env {
> struct btf *btf;
> @@ -418,14 +423,14 @@ static const struct btf_kind_operations *btf_type_ops(const struct btf_type *t)
> static bool btf_name_offset_valid(const struct btf *btf, u32 offset)
> {
> return !BTF_STR_TBL_ELF_ID(offset) &&
> - BTF_STR_OFFSET(offset) < btf->hdr->str_len;
> + BTF_STR_OFFSET(offset) < btf->hdr.str_len;
> }
>
> static const char *btf_name_by_offset(const struct btf *btf, u32 offset)
> {
> if (!BTF_STR_OFFSET(offset))
> return "(anon)";
> - else if (BTF_STR_OFFSET(offset) < btf->hdr->str_len)
> + else if (BTF_STR_OFFSET(offset) < btf->hdr.str_len)
> return &btf->strings[BTF_STR_OFFSET(offset)];
> else
> return "(invalid-name-offset)";
> @@ -536,7 +541,8 @@ static void btf_verifier_log_member(struct btf_verifier_env *env,
> __btf_verifier_log(log, "\n");
> }
>
> -static void btf_verifier_log_hdr(struct btf_verifier_env *env)
> +static void btf_verifier_log_hdr(struct btf_verifier_env *env,
> + u32 btf_data_size)
> {
> struct bpf_verifier_log *log = &env->log;
> const struct btf *btf = env->btf;
> @@ -545,19 +551,16 @@ static void btf_verifier_log_hdr(struct btf_verifier_env *env)
> if (!bpf_verifier_log_needed(log))
> return;
>
> - hdr = btf->hdr;
> + hdr = &btf->hdr;
> __btf_verifier_log(log, "magic: 0x%x\n", hdr->magic);
> __btf_verifier_log(log, "version: %u\n", hdr->version);
> __btf_verifier_log(log, "flags: 0x%x\n", hdr->flags);
> - __btf_verifier_log(log, "parent_label: %u\n", hdr->parent_label);
> - __btf_verifier_log(log, "parent_name: %u\n", hdr->parent_name);
> - __btf_verifier_log(log, "label_off: %u\n", hdr->label_off);
> - __btf_verifier_log(log, "object_off: %u\n", hdr->object_off);
> - __btf_verifier_log(log, "func_off: %u\n", hdr->func_off);
> + __btf_verifier_log(log, "hdr_len: %u\n", hdr->hdr_len);
> __btf_verifier_log(log, "type_off: %u\n", hdr->type_off);
> + __btf_verifier_log(log, "type_len: %u\n", hdr->type_len);
> __btf_verifier_log(log, "str_off: %u\n", hdr->str_off);
> __btf_verifier_log(log, "str_len: %u\n", hdr->str_len);
> - __btf_verifier_log(log, "btf_total_size: %u\n", btf->data_size);
> + __btf_verifier_log(log, "btf_total_size: %u\n", btf_data_size);
> }
>
> static int btf_add_type(struct btf_verifier_env *env, struct btf_type *t)
> @@ -1754,9 +1757,9 @@ static int btf_check_all_metas(struct btf_verifier_env *env)
> struct btf_header *hdr;
> void *cur, *end;
>
> - hdr = btf->hdr;
> + hdr = &btf->hdr;
> cur = btf->nohdr_data + hdr->type_off;
> - end = btf->nohdr_data + hdr->str_off;
> + end = btf->nohdr_data + hdr->type_len;
>
> env->log_type_id = 1;
> while (cur < end) {
> @@ -1866,8 +1869,20 @@ static int btf_check_all_types(struct btf_verifier_env *env)
>
> static int btf_parse_type_sec(struct btf_verifier_env *env)
> {
> + const struct btf_header *hdr = &env->btf->hdr;
> int err;
>
> + /* Type section must align to 4 bytes */
> + if (hdr->type_off & (sizeof(u32) - 1)) {
> + btf_verifier_log(env, "Unaligned type_off");
> + return -EINVAL;
> + }
> +
> + if (!hdr->type_len) {
> + btf_verifier_log(env, "No type found");
> + return -EINVAL;
> + }
> +
> err = btf_check_all_metas(env);
> if (err)
> return err;
> @@ -1881,10 +1896,15 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
> struct btf *btf = env->btf;
> const char *start, *end;
>
> - hdr = btf->hdr;
> + hdr = &btf->hdr;
> start = btf->nohdr_data + hdr->str_off;
> end = start + hdr->str_len;
>
> + if (end != btf->data + btf->data_size) {
> + btf_verifier_log(env, "String section is not at the end");
> + return -EINVAL;
> + }
> +
> if (!hdr->str_len || hdr->str_len - 1 > BTF_MAX_NAME_OFFSET ||
> start[0] || end[-1]) {
> btf_verifier_log(env, "Invalid string section");
> @@ -1896,20 +1916,119 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
> return 0;
> }
>
> -static int btf_parse_hdr(struct btf_verifier_env *env)
> +static const size_t btf_sec_info_offset[] = {
> + offsetof(struct btf_header, type_off),
> + offsetof(struct btf_header, str_off),
> +};
Maybe we can define NR_SECS is
ARRAY_SIZE(btf_sec_info_offset)/sizeof(size_t)?
> +
> +static int btf_sec_info_cmp(const void *a, const void *b)
> {
> + const struct btf_sec_info *x = a;
> + const struct btf_sec_info *y = b;
> +
> + return (int)(x->off - y->off) ? : (int)(x->len - y->len);
> +}
> +
> +static int btf_check_sec_info(struct btf_verifier_env *env,
> + u32 btf_data_size)
> +{
> + struct btf_sec_info secs[NR_SECS];
> + u32 total, expected_total, i;
> const struct btf_header *hdr;
> - struct btf *btf = env->btf;
> - u32 meta_left;
> + const struct btf *btf;
> +
> + BUILD_BUG_ON(ARRAY_SIZE(btf_sec_info_offset) != NR_SECS);
If we define NR_SECS depending on the size of btf_sec_info_offset, this
BUILD_BUG_ON is not needed.
> +
> + btf = env->btf;
> + hdr = &btf->hdr;
> +
> + /* Populate the secs from hdr */
> + for (i = 0; i < NR_SECS; i++)
> + secs[i] = *(struct btf_sec_info *)((void *)hdr +
> + btf_sec_info_offset[i]);
> +
> + sort(secs, NR_SECS, sizeof(struct btf_sec_info),
> + btf_sec_info_cmp, NULL);
> +
> + /* Check for gaps and overlap among sections */
> + total = 0;
> + expected_total = btf_data_size - hdr->hdr_len;
> + for (i = 0; i < NR_SECS; i++) {
> + if (expected_total < secs[i].off) {
> + btf_verifier_log(env, "Invalid section offset");
> + return -EINVAL;
> + }
> + if (total < secs[i].off) {
> + /* gap */
> + btf_verifier_log(env, "Unsupported section found");
> + return -EINVAL;
> + }
> + if (total > secs[i].off) {
> + btf_verifier_log(env, "Section overlap found");
> + return -EINVAL;
> + }
> + if (expected_total - total < secs[i].len) {
> + btf_verifier_log(env,
> + "Total section length too long");
> + return -EINVAL;
> + }
> + total += secs[i].len;
> + }
> +
> + /* There is data other than hdr and known sections */
> + if (expected_total != total) {
> + btf_verifier_log(env, "Unsupported section found");
> + return -EINVAL;
> + }
Does this patch try to address compatibility issue? That is, the old btf
format with 2 sections should still work with future kernel with more
than 2 sections? The above comparision seems suggesting that newer
kernel will not support non-compatible number of sections?
> +
> + return 0;
> +}
> +
> +static int btf_parse_hdr(struct btf_verifier_env *env, void __user *btf_data,
> + u32 btf_data_size)
> +{
> + const struct btf_header *hdr;
> + u32 hdr_len, hdr_copy;
> + struct btf_min_header {
> + u16 magic;
> + u8 version;
> + u8 flags;
> + u32 hdr_len;
> + } __user *min_hdr;
This is the partial structure with the first four fields
for struct btf_header.
It is okay, but I feel a little bit duplication here.
Looks like only sizeof(*min_hdr) and min_hdr->hdr_len is used.
Maybe we can just cast bpf_data to bpf_header and
sizeof(*min_hdr) being offsetof(bpf_data, type_off), and
min_hdr->hdr_len being bpf_data->hdr_len.
Do this work?
> + struct btf *btf;
> + int err;
> +
> + btf = env->btf;
> + min_hdr = btf_data;
> +
> + if (btf_data_size < sizeof(*min_hdr)) {
> + btf_verifier_log(env, "hdr_len not found");
> + return -EINVAL;
> + }
>
> - if (btf->data_size < sizeof(*hdr)) {
> + if (get_user(hdr_len, &min_hdr->hdr_len))
> + return -EFAULT;
> +
> + if (btf_data_size < hdr_len) {
> btf_verifier_log(env, "btf_header not found");
> return -EINVAL;
> }
>
> - btf_verifier_log_hdr(env);
> + err = bpf_check_uarg_tail_zero(btf_data, sizeof(btf->hdr), hdr_len);
> + if (err) {
> + if (err == -E2BIG)
> + btf_verifier_log(env, "Unsupported btf_header");
> + return err;
> + }
> +
> + hdr_copy = min_t(u32, hdr_len, sizeof(btf->hdr));
> + if (copy_from_user(&btf->hdr, btf_data, hdr_copy))
> + return -EFAULT;
> +
> + hdr = &btf->hdr;
> +
> + btf_verifier_log_hdr(env, btf_data_size);
>
> - hdr = btf->hdr;
> if (hdr->magic != BTF_MAGIC) {
> btf_verifier_log(env, "Invalid magic");
> return -EINVAL;
> @@ -1925,26 +2044,14 @@ static int btf_parse_hdr(struct btf_verifier_env *env)
> return -ENOTSUPP;
> }
>
> - meta_left = btf->data_size - sizeof(*hdr);
> - if (!meta_left) {
> + if (btf_data_size == hdr->hdr_len) {
> btf_verifier_log(env, "No data");
> return -EINVAL;
> }
>
> - if (meta_left < hdr->type_off || hdr->str_off <= hdr->type_off ||
> - /* Type section must align to 4 bytes */
> - hdr->type_off & (sizeof(u32) - 1)) {
> - btf_verifier_log(env, "Invalid type_off");
> - return -EINVAL;
> - }
> -
> - if (meta_left < hdr->str_off ||
> - meta_left - hdr->str_off < hdr->str_len) {
> - btf_verifier_log(env, "Invalid str_off or str_len");
> - return -EINVAL;
> - }
> -
> - btf->nohdr_data = btf->hdr + 1;
> + err = btf_check_sec_info(env, btf_data_size);
> + if (err)
> + return err;
>
> return 0;
> }
> @@ -1987,6 +2094,11 @@ static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
> err = -ENOMEM;
> goto errout;
> }
> + env->btf = btf;
> +
> + err = btf_parse_hdr(env, btf_data, btf_data_size);
> + if (err)
> + goto errout;
>
> data = kvmalloc(btf_data_size, GFP_KERNEL | __GFP_NOWARN);
> if (!data) {
> @@ -1996,18 +2108,13 @@ static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
>
> btf->data = data;
> btf->data_size = btf_data_size;
> + btf->nohdr_data = btf->data + btf->hdr.hdr_len;
>
> if (copy_from_user(data, btf_data, btf_data_size)) {
> err = -EFAULT;
> goto errout;
> }
>
> - env->btf = btf;
> -
> - err = btf_parse_hdr(env);
> - if (err)
> - goto errout;
> -
> err = btf_parse_str_sec(env);
> if (err)
> goto errout;
> @@ -2016,16 +2123,14 @@ static struct btf *btf_parse(void __user *btf_data, u32 btf_data_size,
> if (err)
> goto errout;
>
> - if (!err && log->level && bpf_verifier_log_full(log)) {
> + if (log->level && bpf_verifier_log_full(log)) {
> err = -ENOSPC;
> goto errout;
> }
>
> - if (!err) {
> - btf_verifier_env_free(env);
> - refcount_set(&btf->refcnt, 1);
> - return btf;
> - }
> + btf_verifier_env_free(env);
> + refcount_set(&btf->refcnt, 1);
> + return btf;
>
> errout:
> btf_verifier_env_free(env);
>
^ permalink raw reply
* Re: [PATCH net-next 0/5] TI Ethernet driver warnings fixes
From: David Miller @ 2018-05-21 20:18 UTC (permalink / raw)
To: f.fainelli
Cc: netdev, grygorii.strashko, ivan.khoronzhuk, akpm, j-keerthy,
bhumirks, hernan, richardcochran, lukas, robh, linux-omap,
linux-kernel
In-Reply-To: <20180521184555.21555-1-f.fainelli@gmail.com>
From: Florian Fainelli <f.fainelli@gmail.com>
Date: Mon, 21 May 2018 11:45:50 -0700
> Hi all,
>
> This patch series attempts to fix properly the warnings observed with turning
> on COMPILE_TEST and TI Ethernet drivers on 64-bit hosts.
>
> Since I don't have any of this hardware, please review carefully for possible
> breakage!
These seem extremely safe to me, and since I have a vested interest in seeing
these warning disappear from my tree....
Series applied, thanks :-)
^ permalink raw reply
* [PATCH net-next v14 3/7] sch_cake: Add optional ACK filter
From: Toke Høiland-Jørgensen @ 2018-05-21 20:35 UTC (permalink / raw)
To: netdev, cake; +Cc: Yuchung Cheng, Neal Cardwell
In-Reply-To: <152693459693.32668.4272129427997495747.stgit@alrua-kau>
The ACK filter is an optional feature of CAKE which is designed to improve
performance on links with very asymmetrical rate limits. On such links
(which are unfortunately quite prevalent, especially for DSL and cable
subscribers), the downstream throughput can be limited by the number of
ACKs capable of being transmitted in the *upstream* direction.
Filtering ACKs can, in general, have adverse effects on TCP performance
because it interferes with ACK clocking (especially in slow start), and it
reduces the flow's resiliency to ACKs being dropped further along the path.
To alleviate these drawbacks, the ACK filter in CAKE tries its best to
always keep enough ACKs queued to ensure forward progress in the TCP flow
being filtered. It does this by only filtering redundant ACKs. In its
default 'conservative' mode, the filter will always keep at least two
redundant ACKs in the queue, while in 'aggressive' mode, it will filter
down to a single ACK.
The ACK filter works by inspecting the per-flow queue on every packet
enqueue. Starting at the head of the queue, the filter looks for another
eligible packet to drop (so the ACK being dropped is always closer to the
head of the queue than the packet being enqueued). An ACK is eligible only
if it ACKs *fewer* bytes than the new packet being enqueued, including any
SACK options. This prevents duplicate ACKs from being filtered, to avoid
interfering with retransmission logic. In addition, we check TCP header
options and only drop those that are known to not interfere with sender
state. In particular, packets with unknown option codes are never dropped.
In aggressive mode, an eligible packet is always dropped, while in
conservative mode, at least two ACKs are kept in the queue. Only pure ACKs
(with no data segments) are considered eligible for dropping, but when an
ACK with data segments is enqueued, this can cause another pure ACK to
become eligible for dropping.
The approach described above ensures that this ACK filter avoids most of
the drawbacks of a naive filtering mechanism that only keeps flow state but
does not inspect the queue. This is the rationale for including the ACK
filter in CAKE itself rather than as separate module (as the TC filter, for
instance).
Our performance evaluation has shown that on a 30/1 Mbps link with a
bidirectional traffic test (RRUL), turning on the ACK filter on the
upstream link improves downstream throughput by ~20% (both modes) and
upstream throughput by ~12% in conservative mode and ~40% in aggressive
mode, at the cost of ~5ms of inter-flow latency due to the increased
congestion.
In *really* pathological cases, the effect can be a lot more; for instance,
the ACK filter increases the achievable downstream throughput on a link
with 100 Kbps in the upstream direction by an order of magnitude (from ~2.5
Mbps to ~25 Mbps).
Finally, even though we consider the ACK filter to be safer than most, we
do not recommend turning it on everywhere: on more symmetrical link
bandwidths the effect is negligible at best.
Cc: Yuchung Cheng <ycheng@google.com>
Cc: Neal Cardwell <ncardwell@google.com>
Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
---
net/sched/sch_cake.c | 428 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 426 insertions(+), 2 deletions(-)
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 10e208e4255d..92623160d43e 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -757,6 +757,407 @@ static void flow_queue_add(struct cake_flow *flow, struct sk_buff *skb)
skb->next = NULL;
}
+static struct iphdr *cake_get_iphdr(const struct sk_buff *skb,
+ struct ipv6hdr *buf)
+{
+ unsigned int offset = skb_network_offset(skb);
+ struct iphdr *iph;
+
+ iph = skb_header_pointer(skb, offset, sizeof(struct iphdr), buf);
+
+ if (!iph)
+ return NULL;
+
+ if (iph->version == 4 && iph->protocol == IPPROTO_IPV6)
+ return skb_header_pointer(skb, offset + iph->ihl * 4,
+ sizeof(struct ipv6hdr), buf);
+
+ else if (iph->version == 4)
+ return iph;
+
+ else if (iph->version == 6)
+ return skb_header_pointer(skb, offset, sizeof(struct ipv6hdr),
+ buf);
+
+ return NULL;
+}
+
+static struct tcphdr *cake_get_tcphdr(const struct sk_buff *skb,
+ void *buf, unsigned int bufsize)
+{
+ unsigned int offset = skb_network_offset(skb);
+ const struct ipv6hdr *ipv6h;
+ const struct tcphdr *tcph;
+ const struct iphdr *iph;
+ struct ipv6hdr _ipv6h;
+ struct tcphdr _tcph;
+
+ ipv6h = skb_header_pointer(skb, offset, sizeof(_ipv6h), &_ipv6h);
+
+ if (!ipv6h)
+ return NULL;
+
+ if (ipv6h->version == 4) {
+ iph = (struct iphdr *)ipv6h;
+ offset += iph->ihl * 4;
+
+ /* special-case 6in4 tunnelling, as that is a common way to get
+ * v6 connectivity in the home
+ */
+ if (iph->protocol == IPPROTO_IPV6) {
+ ipv6h = skb_header_pointer(skb, offset,
+ sizeof(_ipv6h), &_ipv6h);
+
+ if (!ipv6h || ipv6h->nexthdr != IPPROTO_TCP)
+ return NULL;
+
+ offset += sizeof(struct ipv6hdr);
+
+ } else if (iph->protocol != IPPROTO_TCP) {
+ return NULL;
+ }
+
+ } else if (ipv6h->version == 6) {
+ if (ipv6h->nexthdr != IPPROTO_TCP)
+ return NULL;
+
+ offset += sizeof(struct ipv6hdr);
+ } else {
+ return NULL;
+ }
+
+ tcph = skb_header_pointer(skb, offset, sizeof(_tcph), &_tcph);
+ if (!tcph)
+ return NULL;
+
+ return skb_header_pointer(skb, offset,
+ min(__tcp_hdrlen(tcph), bufsize), buf);
+}
+
+static const void *cake_get_tcpopt(const struct tcphdr *tcph,
+ int code, int *oplen)
+{
+ /* inspired by tcp_parse_options in tcp_input.c */
+ int length = __tcp_hdrlen(tcph) - sizeof(struct tcphdr);
+ const u8 *ptr = (const u8 *)(tcph + 1);
+
+ while (length > 0) {
+ int opcode = *ptr++;
+ int opsize;
+
+ if (opcode == TCPOPT_EOL)
+ break;
+ if (opcode == TCPOPT_NOP) {
+ length--;
+ continue;
+ }
+ opsize = *ptr++;
+ if (opsize < 2 || opsize > length)
+ break;
+
+ if (opcode == code) {
+ *oplen = opsize;
+ return ptr;
+ }
+
+ ptr += opsize - 2;
+ length -= opsize;
+ }
+
+ return NULL;
+}
+
+/* Compare two SACK sequences. A sequence is considered greater if it SACKs more
+ * bytes than the other. In the case where both sequences ACKs bytes that the
+ * other doesn't, A is considered greater. DSACKs in A also makes A be
+ * considered greater.
+ *
+ * @return -1, 0 or 1 as normal compare functions
+ */
+static int cake_tcph_sack_compare(const struct tcphdr *tcph_a,
+ const struct tcphdr *tcph_b)
+{
+ const struct tcp_sack_block_wire *sack_a, *sack_b;
+ u32 ack_seq_a = ntohl(tcph_a->ack_seq);
+ u32 bytes_a = 0, bytes_b = 0;
+ int oplen_a, oplen_b;
+ bool first = true;
+
+ sack_a = cake_get_tcpopt(tcph_a, TCPOPT_SACK, &oplen_a);
+ sack_b = cake_get_tcpopt(tcph_b, TCPOPT_SACK, &oplen_b);
+
+ /* pointers point to option contents */
+ oplen_a -= TCPOLEN_SACK_BASE;
+ oplen_b -= TCPOLEN_SACK_BASE;
+
+ if (sack_a && oplen_a >= sizeof(*sack_a) &&
+ (!sack_b || oplen_b < sizeof(*sack_b)))
+ return -1;
+ else if (sack_b && oplen_b >= sizeof(*sack_b) &&
+ (!sack_a || oplen_a < sizeof(*sack_a)))
+ return 1;
+ else if ((!sack_a || oplen_a < sizeof(*sack_a)) &&
+ (!sack_b || oplen_b < sizeof(*sack_b)))
+ return 0;
+
+ while (oplen_a >= sizeof(*sack_a)) {
+ const struct tcp_sack_block_wire *sack_tmp = sack_b;
+ u32 start_a = get_unaligned_be32(&sack_a->start_seq);
+ u32 end_a = get_unaligned_be32(&sack_a->end_seq);
+ int oplen_tmp = oplen_b;
+ bool found = false;
+
+ /* DSACK; always considered greater to prevent dropping */
+ if (before(start_a, ack_seq_a))
+ return -1;
+
+ bytes_a += end_a - start_a;
+
+ while (oplen_tmp >= sizeof(*sack_tmp)) {
+ u32 start_b = get_unaligned_be32(&sack_tmp->start_seq);
+ u32 end_b = get_unaligned_be32(&sack_tmp->end_seq);
+
+ /* first time through we count the total size */
+ if (first)
+ bytes_b += end_b - start_b;
+
+ if (!after(start_b, start_a) && !before(end_b, end_a)) {
+ found = true;
+ if (!first)
+ break;
+ }
+ oplen_tmp -= sizeof(*sack_tmp);
+ sack_tmp++;
+ }
+
+ if (!found)
+ return -1;
+
+ oplen_a -= sizeof(*sack_a);
+ sack_a++;
+ first = false;
+ }
+
+ /* If we made it this far, all ranges SACKed by A are covered by B, so
+ * either the SACKs are equal, or B SACKs more bytes.
+ */
+ return bytes_b > bytes_a ? 1 : 0;
+}
+
+static void cake_tcph_get_tstamp(const struct tcphdr *tcph,
+ u32 *tsval, u32 *tsecr)
+{
+ const u8 *ptr;
+ int opsize;
+
+ ptr = cake_get_tcpopt(tcph, TCPOPT_TIMESTAMP, &opsize);
+
+ if (ptr && opsize == TCPOLEN_TIMESTAMP) {
+ *tsval = get_unaligned_be32(ptr);
+ *tsecr = get_unaligned_be32(ptr + 4);
+ }
+}
+
+static bool cake_tcph_may_drop(const struct tcphdr *tcph,
+ u32 tstamp_new, u32 tsecr_new)
+{
+ /* inspired by tcp_parse_options in tcp_input.c */
+ int length = __tcp_hdrlen(tcph) - sizeof(struct tcphdr);
+ const u8 *ptr = (const u8 *)(tcph + 1);
+ u32 tstamp, tsecr;
+
+ /* 3 reserved flags must be unset to avoid future breakage
+ * ECE/CWR/NS can be safely ignored
+ * ACK must be set
+ * All other flags URG/PSH/RST/SYN/FIN must be unset
+ * 0x0FFF0000 = all TCP flags (confirm ACK=1, others zero)
+ * 0x01C00000 = NS/CWR/ECE (safe to ignore)
+ * 0x0E3F0000 = 0x0FFF0000 & ~0x01C00000
+ */
+ if (((tcp_flag_word(tcph) &
+ cpu_to_be32(0x0E3F0000)) != TCP_FLAG_ACK))
+ return false;
+
+ while (length > 0) {
+ int opcode = *ptr++;
+ int opsize;
+
+ if (opcode == TCPOPT_EOL)
+ break;
+ if (opcode == TCPOPT_NOP) {
+ length--;
+ continue;
+ }
+ opsize = *ptr++;
+ if (opsize < 2 || opsize > length)
+ break;
+
+ switch (opcode) {
+ case TCPOPT_MD5SIG: /* doesn't influence state */
+ break;
+
+ case TCPOPT_SACK: /* stricter checking performed later */
+ if (opsize % 8 != 2)
+ return false;
+ break;
+
+ case TCPOPT_TIMESTAMP:
+ /* only drop timestamps lower than new */
+ if (opsize != TCPOLEN_TIMESTAMP)
+ return false;
+ tstamp = get_unaligned_be32(ptr);
+ tsecr = get_unaligned_be32(ptr + 4);
+ if (after(tstamp, tstamp_new) ||
+ after(tsecr, tsecr_new))
+ return false;
+ break;
+
+ case TCPOPT_MSS: /* these should only be set on SYN */
+ case TCPOPT_WINDOW:
+ case TCPOPT_SACK_PERM:
+ case TCPOPT_FASTOPEN:
+ case TCPOPT_EXP:
+ default: /* don't drop if any unknown options are present */
+ return false;
+ }
+
+ ptr += opsize - 2;
+ length -= opsize;
+ }
+
+ return true;
+}
+
+static struct sk_buff *cake_ack_filter(struct cake_sched_data *q,
+ struct cake_flow *flow)
+{
+ bool aggressive = q->ack_filter == CAKE_ACK_AGGRESSIVE;
+ struct sk_buff *elig_ack = NULL, *elig_ack_prev = NULL;
+ struct sk_buff *skb_check, *skb_prev = NULL;
+ const struct ipv6hdr *ipv6h, *ipv6h_check;
+ unsigned char _tcph[64], _tcph_check[64];
+ const struct tcphdr *tcph, *tcph_check;
+ const struct iphdr *iph, *iph_check;
+ struct ipv6hdr _iph, _iph_check;
+ const struct sk_buff *skb;
+ int seglen, num_found = 0;
+ u32 tstamp = 0, tsecr = 0;
+ int sack_comp;
+
+ /* no other possible ACKs to filter */
+ if (flow->head == flow->tail)
+ return NULL;
+
+ skb = flow->tail;
+ tcph = cake_get_tcphdr(skb, _tcph, sizeof(_tcph));
+ iph = cake_get_iphdr(skb, &_iph);
+ if (!tcph)
+ return NULL;
+
+ cake_tcph_get_tstamp(tcph, &tstamp, &tsecr);
+
+ /* the 'triggering' packet need only have the ACK flag set.
+ * also check that SYN is not set, as there won't be any previous ACKs.
+ */
+ if ((tcp_flag_word(tcph) &
+ (TCP_FLAG_ACK | TCP_FLAG_SYN)) != TCP_FLAG_ACK)
+ return NULL;
+
+ /* the 'triggering' ACK is at the tail of the queue, we have already
+ * returned if it is the only packet in the flow. loop through the rest
+ * of the queue looking for pure ACKs with the same 5-tuple as the
+ * triggering one.
+ */
+ for (skb_check = flow->head;
+ skb_check && skb_check != skb;
+ skb_prev = skb_check, skb_check = skb_check->next) {
+ iph_check = cake_get_iphdr(skb_check, &_iph_check);
+ tcph_check = cake_get_tcphdr(skb_check, &_tcph_check,
+ sizeof(_tcph_check));
+
+ /* only TCP packets with matching 5-tuple are eligible, and only
+ * drop safe headers
+ */
+ if (!tcph_check || iph->version != iph_check->version ||
+ tcph_check->source != tcph->source ||
+ tcph_check->dest != tcph->dest ||
+ !cake_tcph_may_drop(tcph_check, tstamp, tsecr))
+ continue;
+
+ if (iph_check->version == 4) {
+ if (iph_check->saddr != iph->saddr ||
+ iph_check->daddr != iph->daddr)
+ continue;
+
+ seglen = ntohs(iph_check->tot_len) -
+ (4 * iph_check->ihl);
+ } else if (iph_check->version == 6) {
+ ipv6h = (struct ipv6hdr *)iph;
+ ipv6h_check = (struct ipv6hdr *)iph_check;
+
+ if (ipv6_addr_cmp(&ipv6h_check->saddr, &ipv6h->saddr) ||
+ ipv6_addr_cmp(&ipv6h_check->daddr, &ipv6h->daddr))
+ continue;
+
+ seglen = ntohs(ipv6h_check->payload_len);
+ } else {
+ WARN_ON(1); /* shouldn't happen */
+ continue;
+ }
+
+ /* Don't drop ACKs with segment data, and don't drop ACKs higher
+ * cumulative ACK counter than triggering packet. Check ACK
+ * seqno here to avoid parsing SACK options of packets we are
+ * going to exclude anyway.
+ */
+ if ((seglen - __tcp_hdrlen(tcph_check)) != 0 ||
+ after(ntohl(tcph_check->ack_seq), ntohl(tcph->ack_seq)))
+ continue;
+
+ /* Check SACK options. The triggering packet must SACK more data
+ * than the ACK under consideration, or SACK the same range but
+ * have a larger cumulative ACK counter. The latter is a
+ * pathological case, but is contained in the following check
+ * anyway, just to be safe.
+ */
+ sack_comp = cake_tcph_sack_compare(tcph_check, tcph);
+
+ if (sack_comp < 0 ||
+ (ntohl(tcph_check->ack_seq) == ntohl(tcph->ack_seq) &&
+ sack_comp == 0))
+ continue;
+
+ /* At this point we have found an eligible pure ACK to drop; if
+ * we are in aggressive mode, we are done. Otherwise, keep
+ * searching unless this is the second eligible ACK we
+ * found.
+ *
+ * Since we want to drop ACK closest to the head of the queue,
+ * save the first eligible ACK we find, even if we need to loop
+ * again.
+ */
+ if (!elig_ack) {
+ elig_ack = skb_check;
+ elig_ack_prev = skb_prev;
+ }
+
+ if (num_found++ > 0 || aggressive)
+ goto found;
+ }
+
+ return NULL;
+
+found:
+ if (elig_ack_prev)
+ elig_ack_prev->next = elig_ack->next;
+ else
+ flow->head = elig_ack->next;
+
+ elig_ack->next = NULL;
+
+ return elig_ack;
+}
+
static u64 cake_ewma(u64 avg, u64 sample, u32 shift)
{
avg -= avg >> shift;
@@ -934,6 +1335,7 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch,
struct cake_sched_data *q = qdisc_priv(sch);
int len = qdisc_pkt_len(skb);
int uninitialized_var(ret);
+ struct sk_buff *ack = NULL;
ktime_t now = ktime_get();
struct cake_tin_data *b;
struct cake_flow *flow;
@@ -980,8 +1382,24 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch,
cobalt_set_enqueue_time(skb, now);
flow_queue_add(flow, skb);
- sch->q.qlen++;
- q->buffer_used += skb->truesize;
+ if (q->ack_filter)
+ ack = cake_ack_filter(q, flow);
+
+ if (ack) {
+ b->ack_drops++;
+ sch->qstats.drops++;
+ b->bytes += qdisc_pkt_len(ack);
+ len -= qdisc_pkt_len(ack);
+ q->buffer_used += skb->truesize - ack->truesize;
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ cake_advance_shaper(q, b, ack, now, true);
+
+ qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(ack));
+ consume_skb(ack);
+ } else {
+ sch->q.qlen++;
+ q->buffer_used += skb->truesize;
+ }
/* stats */
b->packets++;
@@ -1511,6 +1929,9 @@ static int cake_change(struct Qdisc *sch, struct nlattr *opt,
q->rate_flags &= ~CAKE_FLAG_INGRESS;
}
+ if (tb[TCA_CAKE_ACK_FILTER])
+ q->ack_filter = nla_get_u32(tb[TCA_CAKE_ACK_FILTER]);
+
if (tb[TCA_CAKE_MEMORY])
q->buffer_config_limit = nla_get_u32(tb[TCA_CAKE_MEMORY]);
@@ -1642,6 +2063,9 @@ static int cake_dump(struct Qdisc *sch, struct sk_buff *skb)
!!(q->rate_flags & CAKE_FLAG_INGRESS)))
goto nla_put_failure;
+ if (nla_put_u32(skb, TCA_CAKE_ACK_FILTER, q->ack_filter))
+ goto nla_put_failure;
+
return nla_nest_end(skb, opts);
nla_put_failure:
^ permalink raw reply related
* [PATCH net-next v14 4/7] sch_cake: Add NAT awareness to packet classifier
From: Toke Høiland-Jørgensen @ 2018-05-21 20:35 UTC (permalink / raw)
To: netdev, cake
In-Reply-To: <152693459693.32668.4272129427997495747.stgit@alrua-kau>
When CAKE is deployed on a gateway that also performs NAT (which is a
common deployment mode), the host fairness mechanism cannot distinguish
internal hosts from each other, and so fails to work correctly.
To fix this, we add an optional NAT awareness mode, which will query the
kernel conntrack mechanism to obtain the pre-NAT addresses for each packet
and use that in the flow and host hashing.
When the shaper is enabled and the host is already performing NAT, the cost
of this lookup is negligible. However, in unlimited mode with no NAT being
performed, there is a significant CPU cost at higher bandwidths. For this
reason, the feature is turned off by default.
Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
---
net/sched/sch_cake.c | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 92623160d43e..04364993ce19 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -71,6 +71,12 @@
#include <net/tcp.h>
#include <net/flow_dissector.h>
+#if IS_REACHABLE(CONFIG_NF_CONNTRACK)
+#include <net/netfilter/nf_conntrack_core.h>
+#include <net/netfilter/nf_conntrack_zones.h>
+#include <net/netfilter/nf_conntrack.h>
+#endif
+
#define CAKE_SET_WAYS (8)
#define CAKE_MAX_TINS (8)
#define CAKE_QUEUES (1024)
@@ -516,6 +522,60 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
return drop;
}
+#if IS_REACHABLE(CONFIG_NF_CONNTRACK)
+
+static void cake_update_flowkeys(struct flow_keys *keys,
+ const struct sk_buff *skb)
+{
+ const struct nf_conntrack_tuple *tuple;
+ enum ip_conntrack_info ctinfo;
+ struct nf_conn *ct;
+ bool rev = false;
+
+ if (tc_skb_protocol(skb) != htons(ETH_P_IP))
+ return;
+
+ ct = nf_ct_get(skb, &ctinfo);
+ if (ct) {
+ tuple = nf_ct_tuple(ct, CTINFO2DIR(ctinfo));
+ } else {
+ const struct nf_conntrack_tuple_hash *hash;
+ struct nf_conntrack_tuple srctuple;
+
+ if (!nf_ct_get_tuplepr(skb, skb_network_offset(skb),
+ NFPROTO_IPV4, dev_net(skb->dev),
+ &srctuple))
+ return;
+
+ hash = nf_conntrack_find_get(dev_net(skb->dev),
+ &nf_ct_zone_dflt,
+ &srctuple);
+ if (!hash)
+ return;
+
+ rev = true;
+ ct = nf_ct_tuplehash_to_ctrack(hash);
+ tuple = nf_ct_tuple(ct, !hash->tuple.dst.dir);
+ }
+
+ keys->addrs.v4addrs.src = rev ? tuple->dst.u3.ip : tuple->src.u3.ip;
+ keys->addrs.v4addrs.dst = rev ? tuple->src.u3.ip : tuple->dst.u3.ip;
+
+ if (keys->ports.ports) {
+ keys->ports.src = rev ? tuple->dst.u.all : tuple->src.u.all;
+ keys->ports.dst = rev ? tuple->src.u.all : tuple->dst.u.all;
+ }
+ if (rev)
+ nf_ct_put(ct);
+}
+#else
+static void cake_update_flowkeys(struct flow_keys *keys,
+ const struct sk_buff *skb)
+{
+ /* There is nothing we can do here without CONNTRACK */
+}
+#endif
+
/* Cake has several subtle multiple bit settings. In these cases you
* would be matching triple isolate mode as well.
*/
@@ -543,6 +603,9 @@ static u32 cake_hash(struct cake_tin_data *q, const struct sk_buff *skb,
skb_flow_dissect_flow_keys(skb, &keys,
FLOW_DISSECTOR_F_STOP_AT_FLOW_LABEL);
+ if (flow_mode & CAKE_FLOW_NAT_FLAG)
+ cake_update_flowkeys(&keys, skb);
+
/* flow_hash_from_keys() sorts the addresses by value, so we have
* to preserve their order in a separate data structure to treat
* src and dst host addresses as independently selectable.
@@ -1894,6 +1957,18 @@ static int cake_change(struct Qdisc *sch, struct nlattr *opt,
if (err < 0)
return err;
+ if (tb[TCA_CAKE_NAT]) {
+#if IS_REACHABLE(CONFIG_NF_CONNTRACK)
+ q->flow_mode &= ~CAKE_FLOW_NAT_FLAG;
+ q->flow_mode |= CAKE_FLOW_NAT_FLAG *
+ !!nla_get_u32(tb[TCA_CAKE_NAT]);
+#else
+ NL_SET_ERR_MSG_ATTR(extack, "No conntrack support in kernel",
+ tb[TCA_CAKE_NAT]);
+ return -EOPNOTSUPP;
+#endif
+ }
+
if (tb[TCA_CAKE_BASE_RATE64])
q->rate_bps = nla_get_u64(tb[TCA_CAKE_BASE_RATE64]);
@@ -2066,6 +2141,10 @@ static int cake_dump(struct Qdisc *sch, struct sk_buff *skb)
if (nla_put_u32(skb, TCA_CAKE_ACK_FILTER, q->ack_filter))
goto nla_put_failure;
+ if (nla_put_u32(skb, TCA_CAKE_NAT,
+ !!(q->flow_mode & CAKE_FLOW_NAT_FLAG)))
+ goto nla_put_failure;
+
return nla_nest_end(skb, opts);
nla_put_failure:
^ permalink raw reply related
* [PATCH net-next v14 2/7] sch_cake: Add ingress mode
From: Toke Høiland-Jørgensen @ 2018-05-21 20:35 UTC (permalink / raw)
To: netdev, cake
In-Reply-To: <152693459693.32668.4272129427997495747.stgit@alrua-kau>
The ingress mode is meant to be enabled when CAKE runs downlink of the
actual bottleneck (such as on an IFB device). The mode changes the shaper
to also account dropped packets to the shaped rate, as these have already
traversed the bottleneck.
Enabling ingress mode will also tune the AQM to always keep at least two
packets queued *for each flow*. This is done by scaling the minimum queue
occupancy level that will disable the AQM by the number of active bulk
flows. The rationale for this is that retransmits are more expensive in
ingress mode, since dropped packets have to traverse the bottleneck again
when they are retransmitted; thus, being more lenient and keeping a minimum
number of packets queued will improve throughput in cases where the number
of active flows are so large that they saturate the bottleneck even at
their minimum window size.
This commit also adds a separate switch to enable ingress mode rate
autoscaling. If enabled, the autoscaling code will observe the actual
traffic rate and adjust the shaper rate to match it. This can help avoid
latency increases in the case where the actual bottleneck rate decreases
below the shaped rate. The scaling filters out spikes by an EWMA filter.
Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
---
net/sched/sch_cake.c | 85 ++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 81 insertions(+), 4 deletions(-)
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 7ea4aa261cec..10e208e4255d 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -435,7 +435,8 @@ static bool cobalt_queue_empty(struct cobalt_vars *vars,
static bool cobalt_should_drop(struct cobalt_vars *vars,
struct cobalt_params *p,
ktime_t now,
- struct sk_buff *skb)
+ struct sk_buff *skb,
+ u32 bulk_flows)
{
bool next_due, over_target, drop = false;
ktime_t schedule;
@@ -459,6 +460,7 @@ static bool cobalt_should_drop(struct cobalt_vars *vars,
sojourn = ktime_to_ns(ktime_sub(now, cobalt_get_enqueue_time(skb)));
schedule = ktime_sub(now, vars->drop_next);
over_target = sojourn > p->target &&
+ sojourn > p->mtu_time * bulk_flows * 2 &&
sojourn > p->mtu_time * 4;
next_due = vars->count && ktime_to_ns(schedule) >= 0;
@@ -913,6 +915,9 @@ static unsigned int cake_drop(struct Qdisc *sch, struct sk_buff **to_free)
b->tin_dropped++;
sch->qstats.drops++;
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ cake_advance_shaper(q, b, skb, now, true);
+
__qdisc_drop(skb, to_free);
sch->q.qlen--;
@@ -990,8 +995,46 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch,
cake_heapify_up(q, b->overflow_idx[idx]);
/* incoming bandwidth capacity estimate */
- q->avg_window_bytes = 0;
- q->last_packet_time = now;
+ if (q->rate_flags & CAKE_FLAG_AUTORATE_INGRESS) {
+ u64 packet_interval = \
+ ktime_to_ns(ktime_sub(now, q->last_packet_time));
+
+ if (packet_interval > NSEC_PER_SEC)
+ packet_interval = NSEC_PER_SEC;
+
+ /* filter out short-term bursts, eg. wifi aggregation */
+ q->avg_packet_interval = \
+ cake_ewma(q->avg_packet_interval,
+ packet_interval,
+ (packet_interval > q->avg_packet_interval ?
+ 2 : 8));
+
+ q->last_packet_time = now;
+
+ if (packet_interval > q->avg_packet_interval) {
+ u64 window_interval = \
+ ktime_to_ns(ktime_sub(now,
+ q->avg_window_begin));
+ u64 b = q->avg_window_bytes * (u64)NSEC_PER_SEC;
+
+ do_div(b, window_interval);
+ q->avg_peak_bandwidth =
+ cake_ewma(q->avg_peak_bandwidth, b,
+ b > q->avg_peak_bandwidth ? 2 : 8);
+ q->avg_window_bytes = 0;
+ q->avg_window_begin = now;
+
+ if (ktime_after(now,
+ ktime_add_ms(q->last_reconfig_time,
+ 250))) {
+ q->rate_bps = (q->avg_peak_bandwidth * 15) >> 4;
+ cake_reconfigure(sch);
+ }
+ }
+ } else {
+ q->avg_window_bytes = 0;
+ q->last_packet_time = now;
+ }
/* flowchain */
if (!flow->set || flow->set == CAKE_SET_DECAYING) {
@@ -1251,15 +1294,27 @@ static struct sk_buff *cake_dequeue(struct Qdisc *sch)
}
/* Last packet in queue may be marked, shouldn't be dropped */
- if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb) ||
+ if (!cobalt_should_drop(&flow->cvars, &b->cparams, now, skb,
+ (b->bulk_flow_count *
+ !!(q->rate_flags &
+ CAKE_FLAG_INGRESS))) ||
!flow->head)
break;
+ /* drop this packet, get another one */
+ if (q->rate_flags & CAKE_FLAG_INGRESS) {
+ len = cake_advance_shaper(q, b, skb,
+ now, true);
+ flow->deficit -= len;
+ b->tin_deficit -= len;
+ }
flow->dropped++;
b->tin_dropped++;
qdisc_tree_reduce_backlog(sch, 1, qdisc_pkt_len(skb));
qdisc_qstats_drop(sch);
kfree_skb(skb);
+ if (q->rate_flags & CAKE_FLAG_INGRESS)
+ goto retry;
}
b->tin_ecn_mark += !!flow->cvars.ecn_marked;
@@ -1442,6 +1497,20 @@ static int cake_change(struct Qdisc *sch, struct nlattr *opt,
q->target = 1;
}
+ if (tb[TCA_CAKE_AUTORATE]) {
+ if (!!nla_get_u32(tb[TCA_CAKE_AUTORATE]))
+ q->rate_flags |= CAKE_FLAG_AUTORATE_INGRESS;
+ else
+ q->rate_flags &= ~CAKE_FLAG_AUTORATE_INGRESS;
+ }
+
+ if (tb[TCA_CAKE_INGRESS]) {
+ if (!!nla_get_u32(tb[TCA_CAKE_INGRESS]))
+ q->rate_flags |= CAKE_FLAG_INGRESS;
+ else
+ q->rate_flags &= ~CAKE_FLAG_INGRESS;
+ }
+
if (tb[TCA_CAKE_MEMORY])
q->buffer_config_limit = nla_get_u32(tb[TCA_CAKE_MEMORY]);
@@ -1565,6 +1634,14 @@ static int cake_dump(struct Qdisc *sch, struct sk_buff *skb)
if (nla_put_u32(skb, TCA_CAKE_MEMORY, q->buffer_config_limit))
goto nla_put_failure;
+ if (nla_put_u32(skb, TCA_CAKE_AUTORATE,
+ !!(q->rate_flags & CAKE_FLAG_AUTORATE_INGRESS)))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_INGRESS,
+ !!(q->rate_flags & CAKE_FLAG_INGRESS)))
+ goto nla_put_failure;
+
return nla_nest_end(skb, opts);
nla_put_failure:
^ permalink raw reply related
* [PATCH net-next v14 0/7] sched: Add Common Applications Kept Enhanced (cake) qdisc
From: Toke Høiland-Jørgensen @ 2018-05-21 20:35 UTC (permalink / raw)
To: netdev, cake
Cc: Georgios Amanakis, Pete Heist, Yuchung Cheng, Neal Cardwell,
Dave Taht
This patch series adds the CAKE qdisc, and has been split up to ease
review.
I have attempted to split out each configurable feature into its own patch.
The first commit adds the base shaper and packet scheduler, while
subsequent commits add the optional features. The full userspace API and
most data structures are included in this commit, but options not
understood in the base version will be ignored.
The result of applying the entire series is identical to the out of tree
version that have seen extensive testing in previous deployments, most
notably as an out of tree patch to OpenWrt. However, note that I have only
compile tested the individual patches; so the whole series should be
considered as a unit.
---
Changelog
v14:
- Handle seqno wraps and DSACKs in ACK filter
v13:
- Avoid ktime_t to scalar compares
- Add class dumping and basic stats
- Fail with ENOTSUPP when requesting NAT mode and conntrack is not
available.
- Parse all TCP options in ACK filter and make sure to only drop safe
ones. Also handle SACK ranges properly.
v12:
- Get rid of custom time typedefs. Use ktime_t for time and u64 for
duration instead.
v11:
- Fix overhead compensation calculation for GSO packets
- Change configured rate to be u64 (I ran out of bits before I ran out
of CPU when testing the effects of the above)
v10:
- Christmas tree gardening (fix variable declarations to be in reverse
line length order)
v9:
- Remove duplicated checks around kvfree() and just call it
unconditionally.
- Don't pass __GFP_NOWARN when allocating memory
- Move options in cake_dump() that are related to optional features to
later patches implementing the features.
- Support attaching filters to the qdisc and use the classification
result to select flow queue.
- Support overriding diffserv priority tin from skb->priority
v8:
- Remove inline keyword from function definitions
- Simplify ACK filter; remove the complex state handling to make the
logic easier to follow. This will potentially be a bit less efficient,
but I have not been able to measure a difference.
v7:
- Split up patch into a series to ease review.
- Constify the ACK filter.
v6:
- Fix 6in4 encapsulation checks in ACK filter code
- Checkpatch fixes
v5:
- Refactor ACK filter code and hopefully fix the safety issues
properly this time.
v4:
- Only split GSO packets if shaping at speeds <= 1Gbps
- Fix overhead calculation code to also work for GSO packets
- Don't re-implement kvzalloc()
- Remove local header include from out-of-tree build (fixes kbuild-bot
complaint).
- Several fixes to the ACK filter:
- Check pskb_may_pull() before deref of transport headers.
- Don't run ACK filter logic on split GSO packets
- Fix TCP sequence number compare to deal with wraparounds
v3:
- Use IS_REACHABLE() macro to fix compilation when sch_cake is
built-in and conntrack is a module.
- Switch the stats output to use nested netlink attributes instead
of a versioned struct.
- Remove GPL boilerplate.
- Fix array initialisation style.
v2:
- Fix kbuild test bot complaint
- Clean up the netlink ABI
- Fix checkpatch complaints
- A few tweaks to the behaviour of cake based on testing carried out
while writing the paper.
---
Toke Høiland-Jørgensen (7):
sched: Add Common Applications Kept Enhanced (cake) qdisc
sch_cake: Add ingress mode
sch_cake: Add optional ACK filter
sch_cake: Add NAT awareness to packet classifier
sch_cake: Add DiffServ handling
sch_cake: Add overhead compensation support to the rate shaper
sch_cake: Conditionally split GSO segments
include/uapi/linux/pkt_sched.h | 113 ++
net/sched/Kconfig | 11
net/sched/Makefile | 1
net/sched/sch_cake.c | 2995 ++++++++++++++++++++++++++++++++++++++++
4 files changed, 3120 insertions(+)
create mode 100644 net/sched/sch_cake.c
^ permalink raw reply
* [PATCH net-next v14 6/7] sch_cake: Add overhead compensation support to the rate shaper
From: Toke Høiland-Jørgensen @ 2018-05-21 20:35 UTC (permalink / raw)
To: netdev, cake
In-Reply-To: <152693459693.32668.4272129427997495747.stgit@alrua-kau>
This commit adds configurable overhead compensation support to the rate
shaper. With this feature, userspace can configure the actual bottleneck
link overhead and encapsulation mode used, which will be used by the shaper
to calculate the precise duration of each packet on the wire.
This feature is needed because CAKE is often deployed one or two hops
upstream of the actual bottleneck (which can be, e.g., inside a DSL or
cable modem). In this case, the link layer characteristics and overhead
reported by the kernel does not match the actual bottleneck. Being able to
set the actual values in use makes it possible to configure the shaper rate
much closer to the actual bottleneck rate (our experience shows it is
possible to get with 0.1% of the actual physical bottleneck rate), thus
keeping latency low without sacrificing bandwidth.
The overhead compensation has three tunables: A fixed per-packet overhead
size (which, if set, will be accounted from the IP packet header), a
minimum packet size (MPU) and a framing mode supporting either ATM or PTM
framing. We include a set of common keywords in TC to help users configure
the right parameters. If no overhead value is set, the value reported by
the kernel is used.
Signed-off-by: Toke Høiland-Jørgensen <toke@toke.dk>
---
net/sched/sch_cake.c | 124 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 123 insertions(+), 1 deletion(-)
diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c
index 687fa9a38a0d..21785dc31acc 100644
--- a/net/sched/sch_cake.c
+++ b/net/sched/sch_cake.c
@@ -272,6 +272,7 @@ enum {
struct cobalt_skb_cb {
ktime_t enqueue_time;
+ u32 adjusted_len;
};
static u64 us_to_ns(u64 us)
@@ -1290,6 +1291,88 @@ static u64 cake_ewma(u64 avg, u64 sample, u32 shift)
return avg;
}
+static u32 cake_calc_overhead(struct cake_sched_data *q, u32 len, u32 off)
+{
+ if (q->rate_flags & CAKE_FLAG_OVERHEAD)
+ len -= off;
+
+ if (q->max_netlen < len)
+ q->max_netlen = len;
+ if (q->min_netlen > len)
+ q->min_netlen = len;
+
+ len += q->rate_overhead;
+
+ if (len < q->rate_mpu)
+ len = q->rate_mpu;
+
+ if (q->atm_mode == CAKE_ATM_ATM) {
+ len += 47;
+ len /= 48;
+ len *= 53;
+ } else if (q->atm_mode == CAKE_ATM_PTM) {
+ /* Add one byte per 64 bytes or part thereof.
+ * This is conservative and easier to calculate than the
+ * precise value.
+ */
+ len += (len + 63) / 64;
+ }
+
+ if (q->max_adjlen < len)
+ q->max_adjlen = len;
+ if (q->min_adjlen > len)
+ q->min_adjlen = len;
+
+ return len;
+}
+
+static u32 cake_overhead(struct cake_sched_data *q, const struct sk_buff *skb)
+{
+ const struct skb_shared_info *shinfo = skb_shinfo(skb);
+ unsigned int hdr_len, last_len = 0;
+ u32 off = skb_network_offset(skb);
+ u32 len = qdisc_pkt_len(skb);
+ u16 segs = 1;
+
+ q->avg_netoff = cake_ewma(q->avg_netoff, off << 16, 8);
+
+ if (!shinfo->gso_size)
+ return cake_calc_overhead(q, len, off);
+
+ /* borrowed from qdisc_pkt_len_init() */
+ hdr_len = skb_transport_header(skb) - skb_mac_header(skb);
+
+ /* + transport layer */
+ if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 |
+ SKB_GSO_TCPV6))) {
+ const struct tcphdr *th;
+ struct tcphdr _tcphdr;
+
+ th = skb_header_pointer(skb, skb_transport_offset(skb),
+ sizeof(_tcphdr), &_tcphdr);
+ if (likely(th))
+ hdr_len += __tcp_hdrlen(th);
+ } else {
+ struct udphdr _udphdr;
+
+ if (skb_header_pointer(skb, skb_transport_offset(skb),
+ sizeof(_udphdr), &_udphdr))
+ hdr_len += sizeof(struct udphdr);
+ }
+
+ if (unlikely(shinfo->gso_type & SKB_GSO_DODGY))
+ segs = DIV_ROUND_UP(skb->len - hdr_len,
+ shinfo->gso_size);
+ else
+ segs = shinfo->gso_segs;
+
+ len = shinfo->gso_size + hdr_len;
+ last_len = skb->len - shinfo->gso_size * (segs - 1);
+
+ return (cake_calc_overhead(q, len, off) * (segs - 1) +
+ cake_calc_overhead(q, last_len, off));
+}
+
static void cake_heap_swap(struct cake_sched_data *q, u16 i, u16 j)
{
struct cake_heap_entry ii = q->overflow_heap[i];
@@ -1367,7 +1450,7 @@ static int cake_advance_shaper(struct cake_sched_data *q,
struct sk_buff *skb,
ktime_t now, bool drop)
{
- u32 len = qdisc_pkt_len(skb);
+ u32 len = get_cobalt_cb(skb)->adjusted_len;
/* charge packet bandwidth to this tin
* and to the global shaper.
@@ -1564,6 +1647,7 @@ static s32 cake_enqueue(struct sk_buff *skb, struct Qdisc *sch,
b->max_skblen = len;
cobalt_set_enqueue_time(skb, now);
+ get_cobalt_cb(skb)->adjusted_len = cake_overhead(q, skb);
flow_queue_add(flow, skb);
if (q->ack_filter)
@@ -2364,6 +2448,31 @@ static int cake_change(struct Qdisc *sch, struct nlattr *opt,
q->flow_mode = (nla_get_u32(tb[TCA_CAKE_FLOW_MODE]) &
CAKE_FLOW_MASK);
+ if (tb[TCA_CAKE_ATM])
+ q->atm_mode = nla_get_u32(tb[TCA_CAKE_ATM]);
+
+ if (tb[TCA_CAKE_OVERHEAD]) {
+ q->rate_overhead = nla_get_s32(tb[TCA_CAKE_OVERHEAD]);
+ q->rate_flags |= CAKE_FLAG_OVERHEAD;
+
+ q->max_netlen = 0;
+ q->max_adjlen = 0;
+ q->min_netlen = ~0;
+ q->min_adjlen = ~0;
+ }
+
+ if (tb[TCA_CAKE_RAW]) {
+ q->rate_flags &= ~CAKE_FLAG_OVERHEAD;
+
+ q->max_netlen = 0;
+ q->max_adjlen = 0;
+ q->min_netlen = ~0;
+ q->min_adjlen = ~0;
+ }
+
+ if (tb[TCA_CAKE_MPU])
+ q->rate_mpu = nla_get_u32(tb[TCA_CAKE_MPU]);
+
if (tb[TCA_CAKE_RTT]) {
q->interval = nla_get_u32(tb[TCA_CAKE_RTT]);
@@ -2540,6 +2649,19 @@ static int cake_dump(struct Qdisc *sch, struct sk_buff *skb)
!!(q->rate_flags & CAKE_FLAG_WASH)))
goto nla_put_failure;
+ if (nla_put_u32(skb, TCA_CAKE_OVERHEAD, q->rate_overhead))
+ goto nla_put_failure;
+
+ if (!(q->rate_flags & CAKE_FLAG_OVERHEAD))
+ if (nla_put_u32(skb, TCA_CAKE_RAW, 0))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_ATM, q->atm_mode))
+ goto nla_put_failure;
+
+ if (nla_put_u32(skb, TCA_CAKE_MPU, q->rate_mpu))
+ goto nla_put_failure;
+
return nla_nest_end(skb, opts);
nla_put_failure:
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox