* Re: [PATCH] isci: fix typo in deg_dbg message
From: Bart Van Assche @ 2016-11-14 22:23 UTC (permalink / raw)
To: Colin King, Intel SCU Linux support, Artur Paszkiewicz,
James E . J . Bottomley, Martin K . Petersen, linux-scsi
Cc: linux-kernel
In-Reply-To: <20161112183026.9626-1-colin.king@canonical.com>
On 11/12/2016 10:30 AM, Colin King wrote:
> Trivial fix to typo "repsonse" to "response" in dev_dbg message.
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
^ permalink raw reply
* Re: [PATCH] iscsi: fix spelling mistakes in dev_warn messages
From: Bart Van Assche @ 2016-11-14 22:24 UTC (permalink / raw)
To: Colin King, Intel SCU Linux support, Artur Paszkiewicz,
James E . J . Bottomley, Martin K . Petersen, linux-scsi
Cc: linux-kernel
In-Reply-To: <20161112164950.5605-1-colin.king@canonical.com>
On 11/12/2016 08:49 AM, Colin King wrote:
> Trivial fix to spelling mistake "suspeneded" to "suspended" in
> dev_warn messages
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
^ permalink raw reply
* Re: [PATCH v6 9/9] tpm: cleanup of printk error messages
From: Jarkko Sakkinen @ 2016-11-14 22:24 UTC (permalink / raw)
To: Nayna Jain
Cc: linux-kernel-u79uwXL29TY76Z2rM5mHXA,
linux-security-module-u79uwXL29TY76Z2rM5mHXA,
tpmdd-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f
In-Reply-To: <1479117656-12403-10-git-send-email-nayna-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
On Mon, Nov 14, 2016 at 05:00:56AM -0500, Nayna Jain wrote:
> This patch removes the unnecessary error messages on failing to
> allocate memory and replaces pr_err/printk with dev_dbg/dev_info
> as applicable.
>
> Suggested-by: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
> Signed-off-by: Nayna Jain <nayna-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Reviewed-by: Jarkko Sakkinen <jarkko.sakkinen-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
/Jarkko
> ---
> drivers/char/tpm/tpm_acpi.c | 16 ++++------------
> drivers/char/tpm/tpm_of.c | 29 +++++++++--------------------
> 2 files changed, 13 insertions(+), 32 deletions(-)
>
> diff --git a/drivers/char/tpm/tpm_acpi.c b/drivers/char/tpm/tpm_acpi.c
> index fa30c969..ddbaef2 100644
> --- a/drivers/char/tpm/tpm_acpi.c
> +++ b/drivers/char/tpm/tpm_acpi.c
> @@ -60,11 +60,8 @@ int read_log_acpi(struct tpm_chip *chip)
> status = acpi_get_table(ACPI_SIG_TCPA, 1,
> (struct acpi_table_header **)&buff);
>
> - if (ACPI_FAILURE(status)) {
> - printk(KERN_ERR "%s: ERROR - Could not get TCPA table\n",
> - __func__);
> + if (ACPI_FAILURE(status))
> return -EIO;
> - }
>
> switch(buff->platform_class) {
> case BIOS_SERVER:
> @@ -78,25 +75,20 @@ int read_log_acpi(struct tpm_chip *chip)
> break;
> }
> if (!len) {
> - printk(KERN_ERR "%s: ERROR - TCPA log area empty\n", __func__);
> + dev_warn(&chip->dev, "%s: TCPA log area empty\n", __func__);
> return -EIO;
> }
>
> /* malloc EventLog space */
> log->bios_event_log = kmalloc(len, GFP_KERNEL);
> - if (!log->bios_event_log) {
> - printk("%s: ERROR - Not enough Memory for BIOS measurements\n",
> - __func__);
> + if (!log->bios_event_log)
> return -ENOMEM;
> - }
>
> log->bios_event_log_end = log->bios_event_log + len;
>
> virt = acpi_os_map_iomem(start, len);
> - if (!virt) {
> - printk("%s: ERROR - Unable to map memory\n", __func__);
> + if (!virt)
> goto err;
> - }
>
> memcpy_fromio(log->bios_event_log, virt, len);
>
> diff --git a/drivers/char/tpm/tpm_of.c b/drivers/char/tpm/tpm_of.c
> index 22b8f81..3af829f 100644
> --- a/drivers/char/tpm/tpm_of.c
> +++ b/drivers/char/tpm/tpm_of.c
> @@ -31,40 +31,29 @@ int read_log_of(struct tpm_chip *chip)
> log = &chip->log;
> if (chip->dev.parent->of_node)
> np = chip->dev.parent->of_node;
> - if (!np) {
> - pr_err("%s: ERROR - IBMVTPM not supported\n", __func__);
> + if (!np)
> return -ENODEV;
> - }
>
> sizep = of_get_property(np, "linux,sml-size", NULL);
> - if (sizep == NULL) {
> - pr_err("%s: ERROR - SML size not found\n", __func__);
> - goto cleanup_eio;
> - }
> + if (sizep == NULL)
> + return -EIO;
> +
> if (*sizep == 0) {
> - pr_err("%s: ERROR - event log area empty\n", __func__);
> - goto cleanup_eio;
> + dev_warn(&chip->dev, "%s: Event log area empty\n", __func__);
> + return -EIO;
> }
>
> basep = of_get_property(np, "linux,sml-base", NULL);
> - if (basep == NULL) {
> - pr_err("%s: ERROR - SML not found\n", __func__);
> - goto cleanup_eio;
> - }
> + if (basep == NULL)
> + return -EIO;
>
> log->bios_event_log = kmalloc(*sizep, GFP_KERNEL);
> - if (!log->bios_event_log) {
> - pr_err("%s: ERROR - Not enough memory for BIOS measurements\n",
> - __func__);
> + if (!log->bios_event_log)
> return -ENOMEM;
> - }
>
> log->bios_event_log_end = log->bios_event_log + *sizep;
>
> memcpy(log->bios_event_log, __va(*basep), *sizep);
>
> return 0;
> -
> -cleanup_eio:
> - return -EIO;
> }
> --
> 2.5.0
>
------------------------------------------------------------------------------
^ permalink raw reply
* Re: [PATCH] isci: fix typo in deg_dbg message
From: Bart Van Assche @ 2016-11-14 22:23 UTC (permalink / raw)
To: Colin King, Intel SCU Linux support, Artur Paszkiewicz,
James E . J . Bottomley, Martin K . Petersen, linux-scsi
Cc: linux-kernel
In-Reply-To: <20161112183026.9626-1-colin.king@canonical.com>
On 11/12/2016 10:30 AM, Colin King wrote:
> Trivial fix to typo "repsonse" to "response" in dev_dbg message.
Reviewed-by: Bart Van Assche <bart.vanassche@sandisk.com>
^ permalink raw reply
* Re: [PATCH v4 2/3] PCI: qcom: add support to msm8996 PCIE controller
From: Bjorn Helgaas @ 2016-11-14 22:23 UTC (permalink / raw)
To: Srinivas Kandagatla
Cc: svarbanov, linux-pci, bhelgaas, robh+dt, linux-arm-msm,
devicetree
In-Reply-To: <1479122155-13393-3-git-send-email-srinivas.kandagatla@linaro.org>
On Mon, Nov 14, 2016 at 11:15:54AM +0000, Srinivas Kandagatla wrote:
> This patch adds support to msm8996/apq8096 pcie, MSM8996 supports
> Gen 1/2, One lane, 3 pcie root-complex with support to MSI and
> legacy interrupts and it conforms to PCI Express Base 2.1 specification.
>
> This patch adds post_init callback to qcom_pcie_ops, as this is pcie
> pipe clocks are only setup after the phy is powered on.
> It also adds ltssm_enable callback as it is very much different to other
> supported SOCs in the driver.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Will need ack from Stanimir before I can apply it.
> ---
> .../devicetree/bindings/pci/qcom,pcie.txt | 67 +++++++-
> drivers/pci/host/pcie-qcom.c | 177 ++++++++++++++++++++-
> 2 files changed, 238 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/pci/qcom,pcie.txt b/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> index 4059a6f..141d8c3 100644
> --- a/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> +++ b/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> @@ -7,6 +7,7 @@
> - "qcom,pcie-ipq8064" for ipq8064
> - "qcom,pcie-apq8064" for apq8064
> - "qcom,pcie-apq8084" for apq8084
> + - "qcom,pcie-msm8996" for msm8996 or apq8096
>
> - reg:
> Usage: required
> @@ -92,6 +93,17 @@
> - "aux" Auxiliary (AUX) clock
> - "bus_master" Master AXI clock
> - "bus_slave" Slave AXI clock
> +
> +- clock-names:
> + Usage: required for msm8996/apq8096
> + Value type: <stringlist>
> + Definition: Should contain the following entries
> + - "pipe" Pipe Clock driving internal logic.
> + - "aux" Auxiliary (AUX) clock.
> + - "cfg" Configuration clk.
> + - "bus_master" Master AXI clock.
> + - "bus_slave" Slave AXI clock.
> +
> - resets:
> Usage: required
> Value type: <prop-encoded-array>
> @@ -115,7 +127,7 @@
> - "core" Core reset
>
> - power-domains:
> - Usage: required for apq8084
> + Usage: required for apq8084 and msm8996/apq8096
> Value type: <prop-encoded-array>
> Definition: A phandle and power domain specifier pair to the
> power domain which is responsible for collapsing
> @@ -231,3 +243,56 @@
> pinctrl-0 = <&pcie0_pins_default>;
> pinctrl-names = "default";
> };
> +
> +* Example for apq8096:
> +
> + pcie@608000{
> + compatible = "qcom,pcie-msm8996", "snps,dw-pcie";
> + power-domains = <&gcc PCIE1_GDSC>;
> + bus-range = <0x00 0xff>;
> + num-lanes = <1>;
> +
> + reg = <0x00608000 0x2000>,
> + <0x0d000000 0xf1d>,
> + <0x0d000f20 0xa8>,
> + <0x0d100000 0x100000>;
> +
> + reg-names = "parf", "dbi", "elbi", "config";
> +
> + phys = <&pcie_phy 1>;
> + phy-names = "pciephy";
> +
> + #address-cells = <3>;
> + #size-cells = <2>;
> + ranges = <0x01000000 0x0 0x0d200000 0x0d200000 0x0 0x100000>,
> + <0x02000000 0x0 0x0d300000 0x0d300000 0x0 0xd00000>;
> +
> + interrupts = <GIC_SPI 413 IRQ_TYPE_NONE>;
> + interrupt-names = "msi";
> + #interrupt-cells = <1>;
> + interrupt-map-mask = <0 0 0 0x7>;
> + interrupt-map = <0 0 0 1 &intc 0 272 IRQ_TYPE_LEVEL_HIGH>, /* int_a */
> + <0 0 0 2 &intc 0 273 IRQ_TYPE_LEVEL_HIGH>, /* int_b */
> + <0 0 0 3 &intc 0 274 IRQ_TYPE_LEVEL_HIGH>, /* int_c */
> + <0 0 0 4 &intc 0 275 IRQ_TYPE_LEVEL_HIGH>; /* int_d */
> +
> + pinctrl-names = "default", "sleep";
> + pinctrl-0 = <&pcie1_clkreq_default &pcie1_perst_default &pcie1_wake_default>;
> + pinctrl-1 = <&pcie1_clkreq_sleep &pcie1_perst_default &pcie1_wake_sleep>;
> +
> + vdda-1p8-supply = <&pm8994_l12>;
> + vdda-supply = <&pm8994_l28>;
> + linux,pci-domain = <1>;
> +
> + clocks = <&gcc GCC_PCIE_1_PIPE_CLK>,
> + <&gcc GCC_PCIE_1_AUX_CLK>,
> + <&gcc GCC_PCIE_1_CFG_AHB_CLK>,
> + <&gcc GCC_PCIE_1_MSTR_AXI_CLK>,
> + <&gcc GCC_PCIE_1_SLV_AXI_CLK>;
> +
> + clock-names = "pipe",
> + "aux",
> + "cfg",
> + "bus_master",
> + "bus_slave";
> + };
> diff --git a/drivers/pci/host/pcie-qcom.c b/drivers/pci/host/pcie-qcom.c
> index 3593640..03ba6b1 100644
> --- a/drivers/pci/host/pcie-qcom.c
> +++ b/drivers/pci/host/pcie-qcom.c
> @@ -36,11 +36,19 @@
>
> #include "pcie-designware.h"
>
> +#define PCIE20_PARF_DBI_BASE_ADDR 0x168
> +
> +#define PCIE20_PARF_SYS_CTRL 0x00
> #define PCIE20_PARF_PHY_CTRL 0x40
> #define PCIE20_PARF_PHY_REFCLK 0x4C
> #define PCIE20_PARF_DBI_BASE_ADDR 0x168
> #define PCIE20_PARF_SLV_ADDR_SPACE_SIZE 0x16c
> +#define PCIE20_PARF_MHI_CLOCK_RESET_CTRL 0x174
> #define PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT 0x178
> +#define MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT 0x1A8
> +#define PCIE20_PARF_LTSSM 0x1B0
> +#define PCIE20_PARF_SID_OFFSET 0x234
> +#define PCIE20_PARF_BDF_TRANSLATE_CFG 0x24C
>
> #define PCIE20_ELBI_SYS_CTRL 0x04
> #define PCIE20_ELBI_SYS_CTRL_LT_ENABLE BIT(0)
> @@ -72,9 +80,18 @@ struct qcom_pcie_resources_v1 {
> struct regulator *vdda;
> };
>
> +struct qcom_pcie_resources_v2 {
> + struct clk *aux_clk;
> + struct clk *master_clk;
> + struct clk *slave_clk;
> + struct clk *cfg_clk;
> + struct clk *pipe_clk;
> +};
> +
> union qcom_pcie_resources {
> struct qcom_pcie_resources_v0 v0;
> struct qcom_pcie_resources_v1 v1;
> + struct qcom_pcie_resources_v2 v2;
> };
>
> struct qcom_pcie;
> @@ -82,7 +99,9 @@ struct qcom_pcie;
> struct qcom_pcie_ops {
> int (*get_resources)(struct qcom_pcie *pcie);
> int (*init)(struct qcom_pcie *pcie);
> + int (*post_init)(struct qcom_pcie *pcie);
> void (*deinit)(struct qcom_pcie *pcie);
> + void (*ltssm_enable)(struct qcom_pcie *pcie);
> };
>
> struct qcom_pcie {
> @@ -116,17 +135,33 @@ static irqreturn_t qcom_pcie_msi_irq_handler(int irq, void *arg)
> return dw_handle_msi_irq(pp);
> }
>
> -static int qcom_pcie_establish_link(struct qcom_pcie *pcie)
> +static void qcom_pcie_v0_v1_ltssm_enable(struct qcom_pcie *pcie)
> {
> u32 val;
> -
> - if (dw_pcie_link_up(&pcie->pp))
> - return 0;
> -
> /* enable link training */
> val = readl(pcie->elbi + PCIE20_ELBI_SYS_CTRL);
> val |= PCIE20_ELBI_SYS_CTRL_LT_ENABLE;
> writel(val, pcie->elbi + PCIE20_ELBI_SYS_CTRL);
> +}
> +
> +static void qcom_pcie_v2_ltssm_enable(struct qcom_pcie *pcie)
> +{
> + u32 val;
> + /* enable link training */
> + val = readl(pcie->parf + PCIE20_PARF_LTSSM);
> + val |= BIT(8);
> + writel(val, pcie->parf + PCIE20_PARF_LTSSM);
> +}
> +
> +static int qcom_pcie_establish_link(struct qcom_pcie *pcie)
> +{
> +
> + if (dw_pcie_link_up(&pcie->pp))
> + return 0;
> +
> + /* Enable Link Training state machine */
> + if (pcie->ops->ltssm_enable)
> + pcie->ops->ltssm_enable(pcie);
>
> return dw_pcie_wait_for_link(&pcie->pp);
> }
> @@ -421,6 +456,113 @@ static int qcom_pcie_init_v1(struct qcom_pcie *pcie)
> return ret;
> }
>
> +static int qcom_pcie_get_resources_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> +
> + res->aux_clk = devm_clk_get(dev, "aux");
> + if (IS_ERR(res->aux_clk))
> + return PTR_ERR(res->aux_clk);
> +
> + res->cfg_clk = devm_clk_get(dev, "cfg");
> + if (IS_ERR(res->cfg_clk))
> + return PTR_ERR(res->cfg_clk);
> +
> + res->master_clk = devm_clk_get(dev, "bus_master");
> + if (IS_ERR(res->master_clk))
> + return PTR_ERR(res->master_clk);
> +
> + res->slave_clk = devm_clk_get(dev, "bus_slave");
> + if (IS_ERR(res->slave_clk))
> + return PTR_ERR(res->slave_clk);
> +
> + res->pipe_clk = devm_clk_get(dev, "pipe");
> + if (IS_ERR(res->pipe_clk))
> + return PTR_ERR(res->pipe_clk);
> +
> + return 0;
> +}
> +
> +static int qcom_pcie_init_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> + u32 val;
> + int ret;
> +
> + ret = clk_prepare_enable(res->aux_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable aux clock\n");
> + return ret;
> + }
> +
> + ret = clk_prepare_enable(res->cfg_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable cfg clock\n");
> + goto err_cfg_clk;
> + }
> +
> + ret = clk_prepare_enable(res->master_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable master clock\n");
> + goto err_master_clk;
> + }
> +
> + ret = clk_prepare_enable(res->slave_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable slave clock\n");
> + goto err_slave_clk;
> + }
> +
> + /* enable PCIe clocks and resets */
> + val = readl(pcie->parf + PCIE20_PARF_PHY_CTRL);
> + val &= ~BIT(0);
> + writel(val, pcie->parf + PCIE20_PARF_PHY_CTRL);
> +
> + /* change DBI base address */
> + writel(0, pcie->parf + PCIE20_PARF_DBI_BASE_ADDR);
> +
> + /* MAC PHY_POWERDOWN MUX DISABLE */
> + val = readl(pcie->parf + PCIE20_PARF_SYS_CTRL);
> + val &= ~BIT(29);
> + writel(val, pcie->parf + PCIE20_PARF_SYS_CTRL);
> +
> + val = readl(pcie->parf + PCIE20_PARF_MHI_CLOCK_RESET_CTRL);
> + val |= BIT(4);
> + writel(val, pcie->parf + PCIE20_PARF_MHI_CLOCK_RESET_CTRL);
> +
> + val = readl(pcie->parf + MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT);
> + val |= BIT(31);
> + writel(val, pcie->parf + MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT);
> +
> + return 0;
> +
> +err_slave_clk:
> + clk_disable_unprepare(res->master_clk);
> +err_master_clk:
> + clk_disable_unprepare(res->cfg_clk);
> +err_cfg_clk:
> + clk_disable_unprepare(res->aux_clk);
> +
> + return ret;
> +}
> +
> +static int qcom_pcie_post_init_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> + int ret;
> +
> + ret = clk_prepare_enable(res->pipe_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable pipe clock\n");
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> static int qcom_pcie_link_up(struct pcie_port *pp)
> {
> struct qcom_pcie *pcie = to_qcom_pcie(pp);
> @@ -429,6 +571,17 @@ static int qcom_pcie_link_up(struct pcie_port *pp)
> return !!(val & PCI_EXP_LNKSTA_DLLLA);
> }
>
> +static void qcom_pcie_deinit_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> +
> + clk_disable_unprepare(res->pipe_clk);
> + clk_disable_unprepare(res->slave_clk);
> + clk_disable_unprepare(res->master_clk);
> + clk_disable_unprepare(res->cfg_clk);
> + clk_disable_unprepare(res->aux_clk);
> +}
> +
> static void qcom_pcie_host_init(struct pcie_port *pp)
> {
> struct qcom_pcie *pcie = to_qcom_pcie(pp);
> @@ -444,6 +597,9 @@ static void qcom_pcie_host_init(struct pcie_port *pp)
> if (ret)
> goto err_deinit;
>
> + if (pcie->ops->post_init)
> + pcie->ops->post_init(pcie);
> +
> dw_pcie_setup_rc(pp);
>
> if (IS_ENABLED(CONFIG_PCI_MSI))
> @@ -487,12 +643,22 @@ static const struct qcom_pcie_ops ops_v0 = {
> .get_resources = qcom_pcie_get_resources_v0,
> .init = qcom_pcie_init_v0,
> .deinit = qcom_pcie_deinit_v0,
> + .ltssm_enable = qcom_pcie_v0_v1_ltssm_enable,
> };
>
> static const struct qcom_pcie_ops ops_v1 = {
> .get_resources = qcom_pcie_get_resources_v1,
> .init = qcom_pcie_init_v1,
> .deinit = qcom_pcie_deinit_v1,
> + .ltssm_enable = qcom_pcie_v0_v1_ltssm_enable,
> +};
> +
> +static const struct qcom_pcie_ops ops_v2 = {
> + .get_resources = qcom_pcie_get_resources_v2,
> + .init = qcom_pcie_init_v2,
> + .post_init = qcom_pcie_post_init_v2,
> + .deinit = qcom_pcie_deinit_v2,
> + .ltssm_enable = qcom_pcie_v2_ltssm_enable,
> };
>
> static int qcom_pcie_probe(struct platform_device *pdev)
> @@ -572,6 +738,7 @@ static const struct of_device_id qcom_pcie_match[] = {
> { .compatible = "qcom,pcie-ipq8064", .data = &ops_v0 },
> { .compatible = "qcom,pcie-apq8064", .data = &ops_v0 },
> { .compatible = "qcom,pcie-apq8084", .data = &ops_v1 },
> + { .compatible = "qcom,pcie-msm8996", .data = &ops_v2 },
> { }
> };
>
> --
> 2.10.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH v4 2/3] PCI: qcom: add support to msm8996 PCIE controller
From: Bjorn Helgaas @ 2016-11-14 22:23 UTC (permalink / raw)
To: Srinivas Kandagatla
Cc: svarbanov-NEYub+7Iv8PQT0dZR+AlfA,
linux-pci-u79uwXL29TY76Z2rM5mHXA, bhelgaas-hpIqsD4AKlfQT0dZR+AlfA,
robh+dt-DgEjT+Ai2ygdnm+yROfE0A,
linux-arm-msm-u79uwXL29TY76Z2rM5mHXA,
devicetree-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <1479122155-13393-3-git-send-email-srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
On Mon, Nov 14, 2016 at 11:15:54AM +0000, Srinivas Kandagatla wrote:
> This patch adds support to msm8996/apq8096 pcie, MSM8996 supports
> Gen 1/2, One lane, 3 pcie root-complex with support to MSI and
> legacy interrupts and it conforms to PCI Express Base 2.1 specification.
>
> This patch adds post_init callback to qcom_pcie_ops, as this is pcie
> pipe clocks are only setup after the phy is powered on.
> It also adds ltssm_enable callback as it is very much different to other
> supported SOCs in the driver.
>
> Signed-off-by: Srinivas Kandagatla <srinivas.kandagatla-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org>
Will need ack from Stanimir before I can apply it.
> ---
> .../devicetree/bindings/pci/qcom,pcie.txt | 67 +++++++-
> drivers/pci/host/pcie-qcom.c | 177 ++++++++++++++++++++-
> 2 files changed, 238 insertions(+), 6 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/pci/qcom,pcie.txt b/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> index 4059a6f..141d8c3 100644
> --- a/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> +++ b/Documentation/devicetree/bindings/pci/qcom,pcie.txt
> @@ -7,6 +7,7 @@
> - "qcom,pcie-ipq8064" for ipq8064
> - "qcom,pcie-apq8064" for apq8064
> - "qcom,pcie-apq8084" for apq8084
> + - "qcom,pcie-msm8996" for msm8996 or apq8096
>
> - reg:
> Usage: required
> @@ -92,6 +93,17 @@
> - "aux" Auxiliary (AUX) clock
> - "bus_master" Master AXI clock
> - "bus_slave" Slave AXI clock
> +
> +- clock-names:
> + Usage: required for msm8996/apq8096
> + Value type: <stringlist>
> + Definition: Should contain the following entries
> + - "pipe" Pipe Clock driving internal logic.
> + - "aux" Auxiliary (AUX) clock.
> + - "cfg" Configuration clk.
> + - "bus_master" Master AXI clock.
> + - "bus_slave" Slave AXI clock.
> +
> - resets:
> Usage: required
> Value type: <prop-encoded-array>
> @@ -115,7 +127,7 @@
> - "core" Core reset
>
> - power-domains:
> - Usage: required for apq8084
> + Usage: required for apq8084 and msm8996/apq8096
> Value type: <prop-encoded-array>
> Definition: A phandle and power domain specifier pair to the
> power domain which is responsible for collapsing
> @@ -231,3 +243,56 @@
> pinctrl-0 = <&pcie0_pins_default>;
> pinctrl-names = "default";
> };
> +
> +* Example for apq8096:
> +
> + pcie@608000{
> + compatible = "qcom,pcie-msm8996", "snps,dw-pcie";
> + power-domains = <&gcc PCIE1_GDSC>;
> + bus-range = <0x00 0xff>;
> + num-lanes = <1>;
> +
> + reg = <0x00608000 0x2000>,
> + <0x0d000000 0xf1d>,
> + <0x0d000f20 0xa8>,
> + <0x0d100000 0x100000>;
> +
> + reg-names = "parf", "dbi", "elbi", "config";
> +
> + phys = <&pcie_phy 1>;
> + phy-names = "pciephy";
> +
> + #address-cells = <3>;
> + #size-cells = <2>;
> + ranges = <0x01000000 0x0 0x0d200000 0x0d200000 0x0 0x100000>,
> + <0x02000000 0x0 0x0d300000 0x0d300000 0x0 0xd00000>;
> +
> + interrupts = <GIC_SPI 413 IRQ_TYPE_NONE>;
> + interrupt-names = "msi";
> + #interrupt-cells = <1>;
> + interrupt-map-mask = <0 0 0 0x7>;
> + interrupt-map = <0 0 0 1 &intc 0 272 IRQ_TYPE_LEVEL_HIGH>, /* int_a */
> + <0 0 0 2 &intc 0 273 IRQ_TYPE_LEVEL_HIGH>, /* int_b */
> + <0 0 0 3 &intc 0 274 IRQ_TYPE_LEVEL_HIGH>, /* int_c */
> + <0 0 0 4 &intc 0 275 IRQ_TYPE_LEVEL_HIGH>; /* int_d */
> +
> + pinctrl-names = "default", "sleep";
> + pinctrl-0 = <&pcie1_clkreq_default &pcie1_perst_default &pcie1_wake_default>;
> + pinctrl-1 = <&pcie1_clkreq_sleep &pcie1_perst_default &pcie1_wake_sleep>;
> +
> + vdda-1p8-supply = <&pm8994_l12>;
> + vdda-supply = <&pm8994_l28>;
> + linux,pci-domain = <1>;
> +
> + clocks = <&gcc GCC_PCIE_1_PIPE_CLK>,
> + <&gcc GCC_PCIE_1_AUX_CLK>,
> + <&gcc GCC_PCIE_1_CFG_AHB_CLK>,
> + <&gcc GCC_PCIE_1_MSTR_AXI_CLK>,
> + <&gcc GCC_PCIE_1_SLV_AXI_CLK>;
> +
> + clock-names = "pipe",
> + "aux",
> + "cfg",
> + "bus_master",
> + "bus_slave";
> + };
> diff --git a/drivers/pci/host/pcie-qcom.c b/drivers/pci/host/pcie-qcom.c
> index 3593640..03ba6b1 100644
> --- a/drivers/pci/host/pcie-qcom.c
> +++ b/drivers/pci/host/pcie-qcom.c
> @@ -36,11 +36,19 @@
>
> #include "pcie-designware.h"
>
> +#define PCIE20_PARF_DBI_BASE_ADDR 0x168
> +
> +#define PCIE20_PARF_SYS_CTRL 0x00
> #define PCIE20_PARF_PHY_CTRL 0x40
> #define PCIE20_PARF_PHY_REFCLK 0x4C
> #define PCIE20_PARF_DBI_BASE_ADDR 0x168
> #define PCIE20_PARF_SLV_ADDR_SPACE_SIZE 0x16c
> +#define PCIE20_PARF_MHI_CLOCK_RESET_CTRL 0x174
> #define PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT 0x178
> +#define MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT 0x1A8
> +#define PCIE20_PARF_LTSSM 0x1B0
> +#define PCIE20_PARF_SID_OFFSET 0x234
> +#define PCIE20_PARF_BDF_TRANSLATE_CFG 0x24C
>
> #define PCIE20_ELBI_SYS_CTRL 0x04
> #define PCIE20_ELBI_SYS_CTRL_LT_ENABLE BIT(0)
> @@ -72,9 +80,18 @@ struct qcom_pcie_resources_v1 {
> struct regulator *vdda;
> };
>
> +struct qcom_pcie_resources_v2 {
> + struct clk *aux_clk;
> + struct clk *master_clk;
> + struct clk *slave_clk;
> + struct clk *cfg_clk;
> + struct clk *pipe_clk;
> +};
> +
> union qcom_pcie_resources {
> struct qcom_pcie_resources_v0 v0;
> struct qcom_pcie_resources_v1 v1;
> + struct qcom_pcie_resources_v2 v2;
> };
>
> struct qcom_pcie;
> @@ -82,7 +99,9 @@ struct qcom_pcie;
> struct qcom_pcie_ops {
> int (*get_resources)(struct qcom_pcie *pcie);
> int (*init)(struct qcom_pcie *pcie);
> + int (*post_init)(struct qcom_pcie *pcie);
> void (*deinit)(struct qcom_pcie *pcie);
> + void (*ltssm_enable)(struct qcom_pcie *pcie);
> };
>
> struct qcom_pcie {
> @@ -116,17 +135,33 @@ static irqreturn_t qcom_pcie_msi_irq_handler(int irq, void *arg)
> return dw_handle_msi_irq(pp);
> }
>
> -static int qcom_pcie_establish_link(struct qcom_pcie *pcie)
> +static void qcom_pcie_v0_v1_ltssm_enable(struct qcom_pcie *pcie)
> {
> u32 val;
> -
> - if (dw_pcie_link_up(&pcie->pp))
> - return 0;
> -
> /* enable link training */
> val = readl(pcie->elbi + PCIE20_ELBI_SYS_CTRL);
> val |= PCIE20_ELBI_SYS_CTRL_LT_ENABLE;
> writel(val, pcie->elbi + PCIE20_ELBI_SYS_CTRL);
> +}
> +
> +static void qcom_pcie_v2_ltssm_enable(struct qcom_pcie *pcie)
> +{
> + u32 val;
> + /* enable link training */
> + val = readl(pcie->parf + PCIE20_PARF_LTSSM);
> + val |= BIT(8);
> + writel(val, pcie->parf + PCIE20_PARF_LTSSM);
> +}
> +
> +static int qcom_pcie_establish_link(struct qcom_pcie *pcie)
> +{
> +
> + if (dw_pcie_link_up(&pcie->pp))
> + return 0;
> +
> + /* Enable Link Training state machine */
> + if (pcie->ops->ltssm_enable)
> + pcie->ops->ltssm_enable(pcie);
>
> return dw_pcie_wait_for_link(&pcie->pp);
> }
> @@ -421,6 +456,113 @@ static int qcom_pcie_init_v1(struct qcom_pcie *pcie)
> return ret;
> }
>
> +static int qcom_pcie_get_resources_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> +
> + res->aux_clk = devm_clk_get(dev, "aux");
> + if (IS_ERR(res->aux_clk))
> + return PTR_ERR(res->aux_clk);
> +
> + res->cfg_clk = devm_clk_get(dev, "cfg");
> + if (IS_ERR(res->cfg_clk))
> + return PTR_ERR(res->cfg_clk);
> +
> + res->master_clk = devm_clk_get(dev, "bus_master");
> + if (IS_ERR(res->master_clk))
> + return PTR_ERR(res->master_clk);
> +
> + res->slave_clk = devm_clk_get(dev, "bus_slave");
> + if (IS_ERR(res->slave_clk))
> + return PTR_ERR(res->slave_clk);
> +
> + res->pipe_clk = devm_clk_get(dev, "pipe");
> + if (IS_ERR(res->pipe_clk))
> + return PTR_ERR(res->pipe_clk);
> +
> + return 0;
> +}
> +
> +static int qcom_pcie_init_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> + u32 val;
> + int ret;
> +
> + ret = clk_prepare_enable(res->aux_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable aux clock\n");
> + return ret;
> + }
> +
> + ret = clk_prepare_enable(res->cfg_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable cfg clock\n");
> + goto err_cfg_clk;
> + }
> +
> + ret = clk_prepare_enable(res->master_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable master clock\n");
> + goto err_master_clk;
> + }
> +
> + ret = clk_prepare_enable(res->slave_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable slave clock\n");
> + goto err_slave_clk;
> + }
> +
> + /* enable PCIe clocks and resets */
> + val = readl(pcie->parf + PCIE20_PARF_PHY_CTRL);
> + val &= ~BIT(0);
> + writel(val, pcie->parf + PCIE20_PARF_PHY_CTRL);
> +
> + /* change DBI base address */
> + writel(0, pcie->parf + PCIE20_PARF_DBI_BASE_ADDR);
> +
> + /* MAC PHY_POWERDOWN MUX DISABLE */
> + val = readl(pcie->parf + PCIE20_PARF_SYS_CTRL);
> + val &= ~BIT(29);
> + writel(val, pcie->parf + PCIE20_PARF_SYS_CTRL);
> +
> + val = readl(pcie->parf + PCIE20_PARF_MHI_CLOCK_RESET_CTRL);
> + val |= BIT(4);
> + writel(val, pcie->parf + PCIE20_PARF_MHI_CLOCK_RESET_CTRL);
> +
> + val = readl(pcie->parf + MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT);
> + val |= BIT(31);
> + writel(val, pcie->parf + MSM8996_PCIE20_PARF_AXI_MSTR_WR_ADDR_HALT);
> +
> + return 0;
> +
> +err_slave_clk:
> + clk_disable_unprepare(res->master_clk);
> +err_master_clk:
> + clk_disable_unprepare(res->cfg_clk);
> +err_cfg_clk:
> + clk_disable_unprepare(res->aux_clk);
> +
> + return ret;
> +}
> +
> +static int qcom_pcie_post_init_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> + struct device *dev = pcie->pp.dev;
> + int ret;
> +
> + ret = clk_prepare_enable(res->pipe_clk);
> + if (ret) {
> + dev_err(dev, "cannot prepare/enable pipe clock\n");
> + return ret;
> + }
> +
> + return 0;
> +}
> +
> static int qcom_pcie_link_up(struct pcie_port *pp)
> {
> struct qcom_pcie *pcie = to_qcom_pcie(pp);
> @@ -429,6 +571,17 @@ static int qcom_pcie_link_up(struct pcie_port *pp)
> return !!(val & PCI_EXP_LNKSTA_DLLLA);
> }
>
> +static void qcom_pcie_deinit_v2(struct qcom_pcie *pcie)
> +{
> + struct qcom_pcie_resources_v2 *res = &pcie->res.v2;
> +
> + clk_disable_unprepare(res->pipe_clk);
> + clk_disable_unprepare(res->slave_clk);
> + clk_disable_unprepare(res->master_clk);
> + clk_disable_unprepare(res->cfg_clk);
> + clk_disable_unprepare(res->aux_clk);
> +}
> +
> static void qcom_pcie_host_init(struct pcie_port *pp)
> {
> struct qcom_pcie *pcie = to_qcom_pcie(pp);
> @@ -444,6 +597,9 @@ static void qcom_pcie_host_init(struct pcie_port *pp)
> if (ret)
> goto err_deinit;
>
> + if (pcie->ops->post_init)
> + pcie->ops->post_init(pcie);
> +
> dw_pcie_setup_rc(pp);
>
> if (IS_ENABLED(CONFIG_PCI_MSI))
> @@ -487,12 +643,22 @@ static const struct qcom_pcie_ops ops_v0 = {
> .get_resources = qcom_pcie_get_resources_v0,
> .init = qcom_pcie_init_v0,
> .deinit = qcom_pcie_deinit_v0,
> + .ltssm_enable = qcom_pcie_v0_v1_ltssm_enable,
> };
>
> static const struct qcom_pcie_ops ops_v1 = {
> .get_resources = qcom_pcie_get_resources_v1,
> .init = qcom_pcie_init_v1,
> .deinit = qcom_pcie_deinit_v1,
> + .ltssm_enable = qcom_pcie_v0_v1_ltssm_enable,
> +};
> +
> +static const struct qcom_pcie_ops ops_v2 = {
> + .get_resources = qcom_pcie_get_resources_v2,
> + .init = qcom_pcie_init_v2,
> + .post_init = qcom_pcie_post_init_v2,
> + .deinit = qcom_pcie_deinit_v2,
> + .ltssm_enable = qcom_pcie_v2_ltssm_enable,
> };
>
> static int qcom_pcie_probe(struct platform_device *pdev)
> @@ -572,6 +738,7 @@ static const struct of_device_id qcom_pcie_match[] = {
> { .compatible = "qcom,pcie-ipq8064", .data = &ops_v0 },
> { .compatible = "qcom,pcie-apq8064", .data = &ops_v0 },
> { .compatible = "qcom,pcie-apq8084", .data = &ops_v1 },
> + { .compatible = "qcom,pcie-msm8996", .data = &ops_v2 },
> { }
> };
>
> --
> 2.10.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe devicetree" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* [PATCH V4 15/15] blk-throttle: add latency target support
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
One hard problem adding .high limit is to detect idle cgroup. If one
cgroup doesn't dispatch enough IO against its high limit, we must have a
mechanism to determine if other cgroups dispatch more IO. We added the
think time detection mechanism before, but it doesn't work for all
workloads. Here we add a latency based approach.
We calculate the average request size and average latency of a cgroup.
Then we can calculate the target latency for the cgroup with the average
request size and the equation. In queue LIMIT_HIGH state, if a cgroup
doesn't dispatch enough IO against high limit but its average latency is
lower than its target latency, we treat the cgroup idle. In this case
other cgroups can dispatch more IO, eg, across their high limit.
Similarly in queue LIMIT_MAX state, if a cgroup doesn't dispatch enough
IO but its average latency is higher than its target latency, we treat
the cgroup busy. In this case, we should throttle other cgroups to make
the first cgroup's latency lower.
If cgroup's average request size is big (currently sets to 128k), we
always treat the cgroup busy (the think time check is still effective
though).
Currently this latency target check is only for SSD as we can't
calcualte the latency target for hard disk. And this is only for cgroup
leaf node so far.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 58 ++++++++++++++++++++++++++++++++++++++++++++---
include/linux/blk_types.h | 1 +
2 files changed, 56 insertions(+), 3 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index ac4d9ea..d07f332 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -156,6 +156,12 @@ struct throtl_grp {
u64 last_finish_time;
u64 checked_last_finish_time;
u64 avg_ttime;
+
+ unsigned int bio_batch;
+ u64 total_latency;
+ u64 avg_latency;
+ u64 total_size;
+ u64 avg_size;
};
/* We measure latency for request size from 4k to 4k * ( 1 << 4) */
@@ -1734,12 +1740,30 @@ static unsigned long tg_last_high_overflow_time(struct throtl_grp *tg)
return ret;
}
+static u64 throtl_target_latency(struct throtl_data *td,
+ struct throtl_grp *tg)
+{
+ if (td->line_slope == 0 || tg->latency_target == 0)
+ return 0;
+
+ /* latency_target + f(avg_size) - f(4k) */
+ return td->line_slope * ((tg->avg_size >> 10) - 4) +
+ tg->latency_target;
+}
+
static bool throtl_tg_is_idle(struct throtl_grp *tg)
{
- /* cgroup is idle if average think time is more than threshold */
- return ktime_get_ns() - tg->last_finish_time >
+ /*
+ * cgroup is idle if:
+ * 1. average think time is higher than threshold
+ * 2. average request size is small and average latency is higher
+ * than target
+ */
+ return (ktime_get_ns() - tg->last_finish_time >
4 * tg->td->idle_ttime_threshold ||
- tg->avg_ttime > tg->td->idle_ttime_threshold;
+ tg->avg_ttime > tg->td->idle_ttime_threshold) ||
+ (tg->avg_latency && tg->avg_size && tg->avg_size <= 128 * 1024 &&
+ tg->avg_latency < throtl_target_latency(tg->td, tg));
}
static bool throtl_upgrade_check_one(struct throtl_grp *tg)
@@ -2123,6 +2147,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
bio_associate_current(bio);
bio->bi_cg_private = q;
bio->bi_cg_size = bio_sectors(bio);
+ bio->bi_cg_enter_time = ktime_get_ns();
blk_throtl_update_ttime(tg);
@@ -2264,6 +2289,33 @@ void blk_throtl_bio_endio(struct bio *bio)
}
}
+ if (bio->bi_cg_enter_time && finish_time > bio->bi_cg_enter_time &&
+ tg->latency_target) {
+ lat = finish_time - bio->bi_cg_enter_time;
+ tg->total_latency += lat;
+ tg->total_size += bio->bi_cg_size << 9;
+ tg->bio_batch++;
+ }
+
+ if (tg->bio_batch >= 8) {
+ int batch = tg->bio_batch;
+ u64 size = tg->total_size;
+
+ lat = tg->total_latency;
+
+ tg->bio_batch = 0;
+ tg->total_latency = 0;
+ tg->total_size = 0;
+
+ if (batch) {
+ do_div(lat, batch);
+ tg->avg_latency = (tg->avg_latency * 7 +
+ lat) >> 3;
+ do_div(size, batch);
+ tg->avg_size = (tg->avg_size * 7 + size) >> 3;
+ }
+ }
+
end:
rcu_read_unlock();
}
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 45bb437..fe87a20 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -61,6 +61,7 @@ struct bio {
struct cgroup_subsys_state *bi_css;
void *bi_cg_private;
u64 bi_cg_issue_time;
+ u64 bi_cg_enter_time;
sector_t bi_cg_size;
#endif
union {
--
2.9.3
^ permalink raw reply related
* [PATCH V4 13/15] blk-throttle: add a mechanism to estimate IO latency
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
We try to set a latency target for each cgroup. The problem is latency
highly depends on request size, users can't configure the target for
every request size. The idea is users configure latency target for 4k
IO, we estimate the target latency for other request size IO.
To do this, we sample some data, eg, average latency for request size
4k, 8k, 16k, 32k, 64k. We then use an equation f(x) = a * x + b to fit
the data (x is request size in KB, f(x) is the latency). Then we can use
the equation to estimate IO target latency for any request.
To increase the chance of sampling, we actually collect data for any IO
size less than 64k, then calcualte an average latency/size. This is ok
for line fit because the equation should work for average request
size/latency too.
But we shouldn't sample data at any time. If disk is congested, the
calculated data will not represent the disk's capability. Hence we only
do the sampling when block throttling is in the HIGH limit, with
assumption disk isn't congested in such state. If the assumption isn't
true, eg, high limit is too high, calculated latency target will be
higher.
How does the equation fit to actual data? I collected data from 4
different SSDs (one SATA, 3 NVMe). The error range is quite small. The
big difference between measured latency and calculated latency generally
comes from 4k IO. The biggest one has around 30% difference, which isn't
terrible as we don't need accurate latency target. We don't know if line
fit works for other SSDs though. For big request size latency, the error
range seems big. But this mechanism is to determine if we should
throttle IO (eg, if cgroup is idle). If cgroups average request size is
big, we can simply treat it as busy, hence we don't need the mechanism.
Hard disk is completely different. Latency depends on spindle seek
instead of request size. So this latency target feature is for SSD only.
The patch uses below algorithm to calculate the equation:
https://en.wikipedia.org/wiki/Simple_linear_regression
TODO: the latency sampling is better moving to request layer
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 191 +++++++++++++++++++++++++++++++++++++++++++++-
include/linux/blk_types.h | 2 +
2 files changed, 190 insertions(+), 3 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 01b494d..a05d351 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -156,6 +156,20 @@ struct throtl_grp {
u64 avg_ttime;
};
+/* We measure latency for request size from 4k to 4k * ( 1 << 4) */
+#define LATENCY_BUCKET_SIZE 5
+
+struct latency_bucket {
+ u64 total_latency;
+ u64 total_size;
+ int samples;
+};
+
+struct avg_latency_bucket {
+ u64 latency;
+ u64 size;
+};
+
struct throtl_data
{
/* service tree for active throtl groups */
@@ -179,6 +193,12 @@ struct throtl_data
unsigned int scale;
u64 idle_ttime_threshold;
+
+ struct latency_bucket tmp_buckets[LATENCY_BUCKET_SIZE];
+ struct avg_latency_bucket avg_buckets[LATENCY_BUCKET_SIZE];
+ struct latency_bucket __percpu *latency_buckets;
+ s64 line_slope;
+ unsigned long last_calculate_time;
};
static void throtl_pending_timer_fn(unsigned long arg);
@@ -288,6 +308,19 @@ static unsigned int tg_iops_limit(struct throtl_grp *tg, int rw)
return ret;
}
+static int request_bucket_index(sector_t sectors)
+{
+ int i;
+
+ for (i = LATENCY_BUCKET_SIZE - 1; i >= 0; i--) {
+ if (sectors > (1 << (i + 3)))
+ break;
+ }
+ if (i == LATENCY_BUCKET_SIZE - 1)
+ return -1;
+ return i + 1;
+}
+
/**
* throtl_log - log debug message via blktrace
* @sq: the service_queue being reported
@@ -1877,6 +1910,120 @@ static void blk_throtl_update_ttime(struct throtl_grp *tg)
tg->checked_last_finish_time = last_finish_time;
}
+static void throtl_calculate_line_slope(struct throtl_data *td)
+{
+ struct avg_latency_bucket avg_latency[LATENCY_BUCKET_SIZE];
+ s64 sumX;
+ s64 sumY;
+ s64 sumXY;
+ s64 sumX2;
+ s64 xMean;
+ s64 yMean;
+ s64 denominator;
+ s64 slope;
+ int i, cpu;
+ int valid_lat;
+ u64 last_latency = 0;
+
+ if (!blk_queue_nonrot(td->queue))
+ return;
+ if (time_before(jiffies, td->last_calculate_time + HZ))
+ return;
+ td->last_calculate_time = jiffies;
+
+ memset(avg_latency, 0, sizeof(avg_latency));
+ for (i = 0; i < LATENCY_BUCKET_SIZE; i++) {
+ struct latency_bucket *tmp = &td->tmp_buckets[i];
+
+ for_each_possible_cpu(cpu) {
+ struct latency_bucket *bucket;
+
+ bucket = per_cpu_ptr(td->latency_buckets, cpu);
+ tmp->total_latency += bucket[i].total_latency;
+ tmp->total_size += bucket[i].total_size;
+ tmp->samples += bucket[i].samples;
+ bucket[i].total_latency = 0;
+ bucket[i].total_size = 0;
+ bucket[i].samples = 0;
+ }
+
+ if (tmp->samples >= 32) {
+ u64 latency = tmp->total_latency;
+ u64 size = tmp->total_size;
+ int samples = tmp->samples;
+
+ tmp->total_latency = 0;
+ tmp->total_size = 0;
+ tmp->samples = 0;
+ do_div(size, samples);
+ if (size == 0 || size > (1 << (i + 12)))
+ continue;
+ avg_latency[i].size = size;
+ do_div(latency, samples);
+ if (latency == 0)
+ continue;
+ avg_latency[i].latency = latency;
+ }
+ }
+
+ valid_lat = 0;
+ for (i = 0; i < LATENCY_BUCKET_SIZE; i++) {
+ if (!td->avg_buckets[i].latency && !avg_latency[i].latency)
+ continue;
+ valid_lat++;
+ if (!td->avg_buckets[i].latency) {
+ td->avg_buckets[i].latency = avg_latency[i].latency;
+ td->avg_buckets[i].size = avg_latency[i].size;
+ continue;
+ }
+ if (!avg_latency[i].latency)
+ continue;
+ /* make it smooth */
+ td->avg_buckets[i].latency = (td->avg_buckets[i].latency * 7 +
+ avg_latency[i].latency) >> 3;
+ td->avg_buckets[i].size = (td->avg_buckets[i].size * 7 +
+ avg_latency[i].size) >> 3;
+ /* filter out abnormal latency */
+ if (td->avg_buckets[i].latency <= last_latency) {
+ td->avg_buckets[i].latency = 0;
+ valid_lat--;
+ } else
+ last_latency = td->avg_buckets[i].latency;
+ }
+
+ if (valid_lat < 2)
+ return;
+
+ sumX = 0;
+ sumY = 0;
+ sumXY = 0;
+ sumX2 = 0;
+ for (i = 0; i < LATENCY_BUCKET_SIZE; i++) {
+ u64 x, y;
+
+ if (td->avg_buckets[i].latency == 0)
+ continue;
+
+ x = td->avg_buckets[i].size >> 10;
+ y = td->avg_buckets[i].latency;
+ sumX += x;
+ sumY += y;
+
+ sumXY += x * y;
+ sumX2 += x * x;
+ }
+
+ xMean = sumX;
+ do_div(xMean, valid_lat);
+ yMean = sumY;
+ do_div(yMean, valid_lat);
+ denominator = sumX2 - sumX * xMean;
+
+ slope = sumXY - sumX * yMean;
+ do_div(slope, denominator);
+ td->line_slope = slope;
+}
+
bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
struct bio *bio)
{
@@ -1901,11 +2048,14 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
spin_lock_irq(q->queue_lock);
+ throtl_calculate_line_slope(tg->td);
+
if (unlikely(blk_queue_bypass(q)))
goto out_unlock;
bio_associate_current(bio);
bio->bi_cg_private = q;
+ bio->bi_cg_size = bio_sectors(bio);
blk_throtl_update_ttime(tg);
@@ -1992,8 +2142,11 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
* don't want bios to leave with the flag set. Clear the flag if
* being issued.
*/
- if (!throttled)
+ if (!throttled) {
+ if (blk_queue_nonrot(q))
+ bio->bi_cg_issue_time = ktime_get_ns();
bio->bi_opf &= ~REQ_THROTTLED;
+ }
return throttled;
}
@@ -2003,6 +2156,9 @@ void blk_throtl_bio_endio(struct bio *bio)
struct blkcg_gq *blkg;
struct throtl_grp *tg;
struct request_queue *q;
+ struct throtl_data *td;
+ u64 finish_time;
+ u64 lat;
q = bio->bi_cg_private;
if (!q)
@@ -2019,7 +2175,27 @@ void blk_throtl_bio_endio(struct bio *bio)
tg = blkg_to_tg(blkg ?: q->root_blkg);
- tg->last_finish_time = ktime_get_ns();
+ finish_time = ktime_get_ns();
+ tg->last_finish_time = finish_time;
+
+ td = tg->td;
+
+ if (bio->bi_cg_issue_time && finish_time > bio->bi_cg_issue_time) {
+ int index;
+
+ lat = finish_time - bio->bi_cg_issue_time;
+ index = request_bucket_index(bio->bi_cg_size);
+ if (index >= 0 && bio_op(bio) == REQ_OP_READ &&
+ td->limit_index == LIMIT_HIGH) {
+ struct latency_bucket *latency;
+
+ latency = get_cpu_ptr(td->latency_buckets);
+ latency[index].total_latency += lat;
+ latency[index].total_size += bio->bi_cg_size << 9;
+ latency[index].samples++;
+ put_cpu_ptr(td->latency_buckets);
+ }
+ }
end:
rcu_read_unlock();
@@ -2097,6 +2273,12 @@ int blk_throtl_init(struct request_queue *q)
td = kzalloc_node(sizeof(*td), GFP_KERNEL, q->node);
if (!td)
return -ENOMEM;
+ td->latency_buckets = __alloc_percpu(sizeof(struct latency_bucket) *
+ LATENCY_BUCKET_SIZE, __alignof__(u64));
+ if (!td->latency_buckets) {
+ kfree(td);
+ return -ENOMEM;
+ }
INIT_WORK(&td->dispatch_work, blk_throtl_dispatch_work_fn);
throtl_service_queue_init(&td->service_queue);
@@ -2113,8 +2295,10 @@ int blk_throtl_init(struct request_queue *q)
td->idle_ttime_threshold = -1;
/* activate policy */
ret = blkcg_activate_policy(q, &blkcg_policy_throtl);
- if (ret)
+ if (ret) {
+ free_percpu(td->latency_buckets);
kfree(td);
+ }
return ret;
}
@@ -2123,6 +2307,7 @@ void blk_throtl_exit(struct request_queue *q)
BUG_ON(!q->td);
throtl_shutdown_wq(q);
blkcg_deactivate_policy(q, &blkcg_policy_throtl);
+ free_percpu(q->td->latency_buckets);
kfree(q->td);
}
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index ff8dd24..45bb437 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -60,6 +60,8 @@ struct bio {
struct io_context *bi_ioc;
struct cgroup_subsys_state *bi_css;
void *bi_cg_private;
+ u64 bi_cg_issue_time;
+ sector_t bi_cg_size;
#endif
union {
#if defined(CONFIG_BLK_DEV_INTEGRITY)
--
2.9.3
^ permalink raw reply related
* [PATCH V4 09/15] blk-throttle: make bandwidth change smooth
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
When cgroups all reach high limit, cgroups can dispatch more IO. This
could make some cgroups dispatch more IO but others not, and even some
cgroups could dispatch less IO than their high limit. For example, cg1
high limit 10MB/s, cg2 limit 80MB/s, assume disk maximum bandwidth is
120M/s for the workload. Their bps could something like this:
cg1/cg2 bps: T1: 10/80 -> T2: 60/60 -> T3: 10/80
At T1, all cgroups reach high limit, so they can dispatch more IO later.
Then cg1 dispatch more IO and cg2 has no room to dispatch enough IO. At
T2, cg2 only dispatches 60M/s. Since We detect cg2 dispatches less IO
than its high limit 80M/s, we downgrade the queue from LIMIT_MAX to
LIMIT_HIGH, then all cgroups are throttled to their high limit (T3). cg2
will have bandwidth below its high limit at most time.
The big problem here is we don't know the maximum bandwidth of the
workload, so we can't make smart decision to avoid the situation. This
patch makes cgroup bandwidth change smooth. After disk upgrades from
LIMIT_HIGH to LIMIT_MAX, we don't allow cgroups use all bandwidth upto
their max limit immediately. Their bandwidth limit will be increased
gradually to avoid above situation. So above example will became
something like:
cg1/cg2 bps: 10/80 -> 15/105 -> 20/100 -> 25/95 -> 30/90 -> 35/85 -> 40/80
-> 45/75 -> 10/80
In this way cgroups bandwidth will be above their limit in majority
time, this still doesn't fully utilize disk bandwidth, but that's
something we pay for sharing.
Note this doesn't completely avoid cgroup running under its high limit.
The best way to guarantee cgroup doesn't run under its limit is to set
max limit. For example, if we set cg1 max limit to 40, cg2 will never
run under its high limit.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 42 ++++++++++++++++++++++++++++++++++++++----
1 file changed, 38 insertions(+), 4 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 32cc6ec..45a28c4 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -170,6 +170,8 @@ struct throtl_data
unsigned long high_upgrade_time;
unsigned long high_downgrade_time;
+
+ unsigned int scale;
};
static void throtl_pending_timer_fn(unsigned long arg);
@@ -224,12 +226,27 @@ static struct throtl_data *sq_to_td(struct throtl_service_queue *sq)
static uint64_t tg_bps_limit(struct throtl_grp *tg, int rw)
{
struct blkcg_gq *blkg = tg_to_blkg(tg);
+ struct throtl_data *td;
uint64_t ret;
if (cgroup_subsys_on_dfl(io_cgrp_subsys) && !blkg->parent)
return -1;
- ret = tg->bps[rw][tg->td->limit_index];
- if (ret == -1 && tg->td->limit_index == LIMIT_HIGH)
+ td = tg->td;
+ ret = tg->bps[rw][td->limit_index];
+ if (td->limit_index == LIMIT_MAX && tg->bps[rw][LIMIT_HIGH] != -1) {
+ uint64_t increase;
+
+ if (td->scale < 4096 && time_after_eq(jiffies,
+ td->high_upgrade_time + td->scale * td->throtl_slice)) {
+ unsigned int time = jiffies - td->high_upgrade_time;
+
+ td->scale = time / td->throtl_slice;
+ }
+ increase = (tg->bps[rw][LIMIT_HIGH] >> 1) * td->scale;
+ ret = min(tg->bps[rw][LIMIT_MAX],
+ tg->bps[rw][LIMIT_HIGH] + increase);
+ }
+ if (ret == -1 && td->limit_index == LIMIT_HIGH)
return tg->bps[rw][LIMIT_MAX];
return ret;
@@ -238,12 +255,28 @@ static uint64_t tg_bps_limit(struct throtl_grp *tg, int rw)
static unsigned int tg_iops_limit(struct throtl_grp *tg, int rw)
{
struct blkcg_gq *blkg = tg_to_blkg(tg);
+ struct throtl_data *td;
unsigned int ret;
if (cgroup_subsys_on_dfl(io_cgrp_subsys) && !blkg->parent)
return -1;
- ret = tg->iops[rw][tg->td->limit_index];
- if (ret == -1 && tg->td->limit_index == LIMIT_HIGH)
+ td = tg->td;
+ ret = tg->iops[rw][td->limit_index];
+ if (td->limit_index == LIMIT_MAX && tg->iops[rw][LIMIT_HIGH] != -1) {
+ uint64_t increase;
+
+ if (td->scale < 4096 && time_after_eq(jiffies,
+ td->high_upgrade_time + td->scale * td->throtl_slice)) {
+ unsigned int time = jiffies - td->high_upgrade_time;
+
+ td->scale = time / td->throtl_slice;
+ }
+
+ increase = (tg->iops[rw][LIMIT_HIGH] >> 1) * td->scale;
+ ret = min(tg->iops[rw][LIMIT_MAX],
+ tg->iops[rw][LIMIT_HIGH] + (unsigned int)increase);
+ }
+ if (ret == -1 && td->limit_index == LIMIT_HIGH)
return tg->iops[rw][LIMIT_MAX];
return ret;
}
@@ -1676,6 +1709,7 @@ static void throtl_upgrade_state(struct throtl_data *td)
td->limit_index = LIMIT_MAX;
td->high_upgrade_time = jiffies;
+ td->scale = 0;
rcu_read_lock();
blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
struct throtl_grp *tg = blkg_to_tg(blkg);
--
2.9.3
^ permalink raw reply related
* [PATCH V4 08/15] blk-throttle: detect completed idle cgroup
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
cgroup could be assigned a limit, but doesn't dispatch enough IO, eg the
cgroup is idle. When this happens, the cgroup doesn't hit its limit, so
we can't move the state machine to higher level and all cgroups will be
throttled to thier lower limit, so we waste bandwidth. Detecting idle
cgroup is hard. This patch handles a simple case, a cgroup doesn't
dispatch any IO. We ignore such cgroup's limit, so other cgroups can use
the bandwidth.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index e85b2b6..32cc6ec 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -144,6 +144,8 @@ struct throtl_grp {
unsigned long last_check_time;
+ unsigned long last_dispatch_time[2];
+
/* When did we start a new slice */
unsigned long slice_start[2];
unsigned long slice_end[2];
@@ -438,11 +440,14 @@ static void tg_update_has_rules(struct throtl_grp *tg)
static void throtl_pd_online(struct blkg_policy_data *pd)
{
+ struct throtl_grp *tg = pd_to_tg(pd);
/*
* We don't want new groups to escape the limits of its ancestors.
* Update has_rules[] after a new group is brought online.
*/
- tg_update_has_rules(pd_to_tg(pd));
+ tg_update_has_rules(tg);
+ tg->last_dispatch_time[READ] = jiffies;
+ tg->last_dispatch_time[WRITE] = jiffies;
}
static void blk_throtl_update_valid_limit(struct throtl_data *td)
@@ -1611,6 +1616,12 @@ static bool throtl_upgrade_check_one(struct throtl_grp *tg)
if (write_limit && sq->nr_queued[WRITE] &&
(!read_limit || sq->nr_queued[READ]))
return true;
+
+ if (time_after_eq(jiffies,
+ tg->last_dispatch_time[READ] + tg->td->throtl_slice) &&
+ time_after_eq(jiffies,
+ tg->last_dispatch_time[WRITE] + tg->td->throtl_slice))
+ return true;
return false;
}
@@ -1691,6 +1702,11 @@ static bool throtl_downgrade_check_one(struct throtl_grp *tg)
struct throtl_data *td = tg->td;
unsigned long now = jiffies;
+ if (time_after_eq(now, tg->last_dispatch_time[READ] +
+ td->throtl_slice) &&
+ time_after_eq(now, tg->last_dispatch_time[WRITE] +
+ td->throtl_slice))
+ return false;
/*
* If cgroup is below high limit, consider downgrade and throttle other
* cgroups
@@ -1811,6 +1827,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
again:
while (true) {
+ tg->last_dispatch_time[rw] = jiffies;
if (tg->last_high_overflow_time[rw] == 0)
tg->last_high_overflow_time[rw] = jiffies;
throtl_downgrade_check(tg);
--
2.9.3
^ permalink raw reply related
* [PATCH V4 14/15] blk-throttle: add interface for per-cgroup target latency
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
Add interface for per-cgroup target latency. This latency is for 4k
request.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 67 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index a05d351..ac4d9ea 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -147,6 +147,8 @@ struct throtl_grp {
unsigned long last_check_time;
int upgrade_check_batch;
+
+ u64 latency_target;
/* When did we start a new slice */
unsigned long slice_start[2];
unsigned long slice_end[2];
@@ -463,6 +465,7 @@ static struct blkg_policy_data *throtl_pd_alloc(gfp_t gfp, int node)
tg->iops[rw][index] = -1;
}
}
+ /* target latency default 0, eg, always not meet */
return &tg->pd;
}
@@ -1572,6 +1575,64 @@ static ssize_t tg_set_limit(struct kernfs_open_file *of,
return ret ?: nbytes;
}
+static u64 tg_prfill_latency_target(struct seq_file *sf,
+ struct blkg_policy_data *pd, int off)
+{
+ struct throtl_grp *tg = pd_to_tg(pd);
+ const char *dname = blkg_dev_name(pd->blkg);
+
+ if (!dname)
+ return 0;
+ if (tg->latency_target == 0)
+ return 0;
+
+ seq_printf(sf, "%s 4k_lat=%llu\n", dname, tg->latency_target);
+ return 0;
+}
+
+static int tg_print_latency_target(struct seq_file *sf, void *v)
+{
+ blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)),
+ tg_prfill_latency_target, &blkcg_policy_throtl,
+ seq_cft(sf)->private, false);
+ return 0;
+}
+
+static ssize_t tg_set_latency_target(struct kernfs_open_file *of,
+ char *buf, size_t nbytes, loff_t off)
+{
+ struct blkcg *blkcg = css_to_blkcg(of_css(of));
+ struct blkg_conf_ctx ctx;
+ struct throtl_grp *tg;
+ int ret = -EINVAL;
+ char tok[27];
+ char *p;
+ u64 val;
+
+ ret = blkg_conf_prep(blkcg, &blkcg_policy_throtl, buf, &ctx);
+ if (ret)
+ return ret;
+
+ tg = blkg_to_tg(ctx.blkg);
+
+ if (sscanf(ctx.body, "%26s", tok) != 1)
+ goto out_finish;
+
+ p = tok;
+ strsep(&p, "=");
+ if (!p || kstrtou64(p, 10, &val))
+ goto out_finish;
+
+ if (strcmp(tok, "4k_lat"))
+ goto out_finish;
+
+ tg->latency_target = val;
+ ret = 0;
+out_finish:
+ blkg_conf_finish(&ctx);
+ return ret ?: nbytes;
+}
+
static struct cftype throtl_files[] = {
{
.name = "high",
@@ -1587,6 +1648,12 @@ static struct cftype throtl_files[] = {
.write = tg_set_limit,
.private = LIMIT_MAX,
},
+ {
+ .name = "latency_target",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = tg_print_latency_target,
+ .write = tg_set_latency_target,
+ },
{ } /* terminate */
};
--
2.9.3
^ permalink raw reply related
* [PATCH V4 12/15] blk-throttle: ignore idle cgroup limit
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
Last patch introduces a way to detect idle cgroup. We use it to make
upgrade/downgrade decision. And the new algorithm can detect completely
idle cgroup too, so we can delete the corresponding code.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 39 +++++++++++++++++++++++++--------------
1 file changed, 25 insertions(+), 14 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index e403e88..01b494d 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -146,8 +146,7 @@ struct throtl_grp {
unsigned long last_check_time;
- unsigned long last_dispatch_time[2];
-
+ int upgrade_check_batch;
/* When did we start a new slice */
unsigned long slice_start[2];
unsigned long slice_end[2];
@@ -487,8 +486,6 @@ static void throtl_pd_online(struct blkg_policy_data *pd)
* Update has_rules[] after a new group is brought online.
*/
tg_update_has_rules(tg);
- tg->last_dispatch_time[READ] = jiffies;
- tg->last_dispatch_time[WRITE] = jiffies;
}
static void blk_throtl_update_valid_limit(struct throtl_data *td)
@@ -1667,9 +1664,8 @@ static bool throtl_upgrade_check_one(struct throtl_grp *tg)
return true;
if (time_after_eq(jiffies,
- tg->last_dispatch_time[READ] + tg->td->throtl_slice) &&
- time_after_eq(jiffies,
- tg->last_dispatch_time[WRITE] + tg->td->throtl_slice))
+ tg_last_high_overflow_time(tg) + tg->td->throtl_slice) &&
+ throtl_tg_is_idle(tg))
return true;
return false;
}
@@ -1718,6 +1714,24 @@ static bool throtl_can_upgrade(struct throtl_data *td,
return true;
}
+static void throtl_upgrade_check(struct throtl_grp *tg)
+{
+ if (tg->td->limit_index != LIMIT_HIGH)
+ return;
+
+ if (!time_after_eq(jiffies,
+ __tg_last_high_overflow_time(tg) + tg->td->throtl_slice))
+ return;
+
+ tg->upgrade_check_batch++;
+ if (tg->upgrade_check_batch < 16)
+ return;
+ tg->upgrade_check_batch = 0;
+
+ if (throtl_can_upgrade(tg->td, NULL))
+ throtl_upgrade_state(tg->td);
+}
+
static void throtl_upgrade_state(struct throtl_data *td)
{
struct cgroup_subsys_state *pos_css;
@@ -1752,18 +1766,15 @@ static bool throtl_downgrade_check_one(struct throtl_grp *tg)
struct throtl_data *td = tg->td;
unsigned long now = jiffies;
- if (time_after_eq(now, tg->last_dispatch_time[READ] +
- td->throtl_slice) &&
- time_after_eq(now, tg->last_dispatch_time[WRITE] +
- td->throtl_slice))
- return false;
/*
* If cgroup is below high limit, consider downgrade and throttle other
* cgroups
*/
if (time_after_eq(now, td->high_upgrade_time + td->throtl_slice) &&
time_after_eq(now, tg_last_high_overflow_time(tg) +
- td->throtl_slice))
+ td->throtl_slice) &&
+ (!throtl_tg_is_idle(tg) ||
+ !list_empty(&tg_to_blkg(tg)->blkcg->css.children)))
return true;
return false;
}
@@ -1902,10 +1913,10 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
again:
while (true) {
- tg->last_dispatch_time[rw] = jiffies;
if (tg->last_high_overflow_time[rw] == 0)
tg->last_high_overflow_time[rw] = jiffies;
throtl_downgrade_check(tg);
+ throtl_upgrade_check(tg);
/* throtl is FIFO - if bios are already queued, should queue */
if (sq->nr_queued[rw])
break;
--
2.9.3
^ permalink raw reply related
* [PATCH V4 11/15] blk-throttle: add interface to configure think time threshold
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
Add interface to configure the threshold
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-sysfs.c | 7 +++++++
block/blk-throttle.c | 25 +++++++++++++++++++++++++
block/blk.h | 4 ++++
3 files changed, 36 insertions(+)
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 3e284e4..f15aeed 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -532,6 +532,12 @@ static struct queue_sysfs_entry throtl_slice_entry = {
.show = blk_throtl_slice_show,
.store = blk_throtl_slice_store,
};
+
+static struct queue_sysfs_entry throtl_idle_threshold_entry = {
+ .attr = {.name = "throttling_idle_threshold", .mode = S_IRUGO | S_IWUSR },
+ .show = blk_throtl_idle_threshold_show,
+ .store = blk_throtl_idle_threshold_store,
+};
#endif
static struct attribute *default_attrs[] = {
@@ -563,6 +569,7 @@ static struct attribute *default_attrs[] = {
&queue_dax_entry.attr,
#ifdef CONFIG_BLK_DEV_THROTTLING
&throtl_slice_entry.attr,
+ &throtl_idle_threshold_entry.attr,
#endif
NULL,
};
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index cb5fd85..e403e88 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -2139,6 +2139,31 @@ ssize_t blk_throtl_slice_store(struct request_queue *q,
return count;
}
+ssize_t blk_throtl_idle_threshold_show(struct request_queue *q, char *page)
+{
+ u64 threshold = q->td->idle_ttime_threshold;
+
+ if (!q->td)
+ return -EINVAL;
+ do_div(threshold, 1000);
+ return sprintf(page, "%lluus\n", threshold);
+}
+
+ssize_t blk_throtl_idle_threshold_store(struct request_queue *q,
+ const char *page, size_t count)
+{
+ unsigned long v;
+
+ if (!q->td)
+ return -EINVAL;
+ if (kstrtoul(page, 10, &v))
+ return -EINVAL;
+ if (v == 0)
+ return -EINVAL;
+ q->td->idle_ttime_threshold = v * 1000;
+ return count;
+}
+
static int __init throtl_init(void)
{
kthrotld_workqueue = alloc_workqueue("kthrotld", WQ_MEM_RECLAIM, 0);
diff --git a/block/blk.h b/block/blk.h
index b433f35..2ebde12 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -292,6 +292,10 @@ extern void blk_throtl_exit(struct request_queue *q);
extern ssize_t blk_throtl_slice_show(struct request_queue *q, char *page);
extern ssize_t blk_throtl_slice_store(struct request_queue *q,
const char *page, size_t count);
+extern ssize_t blk_throtl_idle_threshold_show(struct request_queue *q,
+ char *page);
+extern ssize_t blk_throtl_idle_threshold_store(struct request_queue *q,
+ const char *page, size_t count);
extern void blk_throtl_bio_endio(struct bio *bio);
#else /* CONFIG_BLK_DEV_THROTTLING */
static inline void blk_throtl_drain(struct request_queue *q) { }
--
2.9.3
^ permalink raw reply related
* Re: [PATCH 4/4] selinux: Convert isec->lock into a spinlock
From: Paul Moore @ 2016-11-14 22:22 UTC (permalink / raw)
To: Andreas Gruenbacher; +Cc: Stephen Smalley, Eric Paris, selinux
In-Reply-To: <1478812710-17190-5-git-send-email-agruenba@redhat.com>
On Thu, Nov 10, 2016 at 4:18 PM, Andreas Gruenbacher
<agruenba@redhat.com> wrote:
> Convert isec->lock from a mutex into a spinlock. Instead of holding the
> lock while sleeping in inode_doinit_with_dentry, set isec->initialized
> to LABEL_PENDING and release the lock. Then, when the sid has been
> determined, re-acquire the lock. If isec->initialized is still set to
> LABEL_PENDING, set isec->sid; otherwise, the sid has been set by another
> task (LABEL_INITIALIZED) or invalidated (LABEL_INVALID) in the meantime.
>
> This fixes a deadlock on gfs2 where
>
> * one task is in inode_doinit_with_dentry -> gfs2_getxattr, holds
> isec->lock, and tries to acquire the inode's glock, and
>
> * another task is in do_xmote -> inode_go_inval ->
> selinux_inode_invalidate_secctx, holds the inode's glock, and tries
> to acquire isec->lock.
>
> Signed-off-by: Andreas Gruenbacher <agruenba@redhat.com>
> ---
> security/selinux/hooks.c | 108 ++++++++++++++++++++++++--------------
> security/selinux/include/objsec.h | 5 +-
> 2 files changed, 72 insertions(+), 41 deletions(-)
We shouldn't need the spinlocks on the socket_post_create() and the
socket_accept() hooks as the callers should still have exclusive
access to the socket/inode at that point.
I didn't check all the callers of the inode_init_security(), but it
looks like the same idea applies.
> diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c
> index cf5067e..4af31f1 100644
> --- a/security/selinux/hooks.c
> +++ b/security/selinux/hooks.c
> @@ -231,7 +231,7 @@ static int inode_alloc_security(struct inode *inode)
> if (!isec)
> return -ENOMEM;
>
> - mutex_init(&isec->lock);
> + spin_lock_init(&isec->lock);
> INIT_LIST_HEAD(&isec->list);
> isec->inode = inode;
> isec->sid = SECINITSID_UNLABELED;
> @@ -1381,7 +1381,8 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> {
> struct superblock_security_struct *sbsec = NULL;
> struct inode_security_struct *isec = inode->i_security;
> - u32 sid;
> + u32 task_sid, sid = 0;
> + u16 sclass;
> struct dentry *dentry;
> #define INITCONTEXTLEN 255
> char *context = NULL;
> @@ -1391,7 +1392,7 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> if (isec->initialized == LABEL_INITIALIZED)
> return 0;
>
> - mutex_lock(&isec->lock);
> + spin_lock(&isec->lock);
> if (isec->initialized == LABEL_INITIALIZED)
> goto out_unlock;
>
> @@ -1410,12 +1411,18 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> goto out_unlock;
> }
>
> + sclass = isec->sclass;
> + task_sid = isec->task_sid;
> + sid = isec->sid;
> + isec->initialized = LABEL_PENDING;
> + spin_unlock(&isec->lock);
> +
> switch (sbsec->behavior) {
> case SECURITY_FS_USE_NATIVE:
> break;
> case SECURITY_FS_USE_XATTR:
> if (!(inode->i_opflags & IOP_XATTR)) {
> - isec->sid = sbsec->def_sid;
> + sid = sbsec->def_sid;
> break;
> }
> /* Need a dentry, since the xattr API requires one.
> @@ -1437,7 +1444,7 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> * inode_doinit with a dentry, before these inodes could
> * be used again by userspace.
> */
> - goto out_unlock;
> + goto out;
> }
>
> len = INITCONTEXTLEN;
> @@ -1445,7 +1452,7 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> if (!context) {
> rc = -ENOMEM;
> dput(dentry);
> - goto out_unlock;
> + goto out;
> }
> context[len] = '\0';
> rc = __vfs_getxattr(dentry, inode, XATTR_NAME_SELINUX, context, len);
> @@ -1456,14 +1463,14 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> rc = __vfs_getxattr(dentry, inode, XATTR_NAME_SELINUX, NULL, 0);
> if (rc < 0) {
> dput(dentry);
> - goto out_unlock;
> + goto out;
> }
> len = rc;
> context = kmalloc(len+1, GFP_NOFS);
> if (!context) {
> rc = -ENOMEM;
> dput(dentry);
> - goto out_unlock;
> + goto out;
> }
> context[len] = '\0';
> rc = __vfs_getxattr(dentry, inode, XATTR_NAME_SELINUX, context, len);
> @@ -1475,7 +1482,7 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> "%d for dev=%s ino=%ld\n", __func__,
> -rc, inode->i_sb->s_id, inode->i_ino);
> kfree(context);
> - goto out_unlock;
> + goto out;
> }
> /* Map ENODATA to the default file SID */
> sid = sbsec->def_sid;
> @@ -1505,28 +1512,25 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> }
> }
> kfree(context);
> - isec->sid = sid;
> break;
> case SECURITY_FS_USE_TASK:
> - isec->sid = isec->task_sid;
> + sid = task_sid;
> break;
> case SECURITY_FS_USE_TRANS:
> /* Default to the fs SID. */
> - isec->sid = sbsec->sid;
> + sid = sbsec->sid;
>
> /* Try to obtain a transition SID. */
> - rc = security_transition_sid(isec->task_sid, sbsec->sid,
> - isec->sclass, NULL, &sid);
> + rc = security_transition_sid(task_sid, sid, sclass, NULL, &sid);
> if (rc)
> - goto out_unlock;
> - isec->sid = sid;
> + goto out;
> break;
> case SECURITY_FS_USE_MNTPOINT:
> - isec->sid = sbsec->mntpoint_sid;
> + sid = sbsec->mntpoint_sid;
> break;
> default:
> /* Default to the fs superblock SID. */
> - isec->sid = sbsec->sid;
> + sid = sbsec->sid;
>
> if ((sbsec->flags & SE_SBGENFS) && !S_ISLNK(inode->i_mode)) {
> /* We must have a dentry to determine the label on
> @@ -1549,21 +1553,29 @@ static int inode_doinit_with_dentry(struct inode *inode, struct dentry *opt_dent
> * could be used again by userspace.
> */
> if (!dentry)
> - goto out_unlock;
> - rc = selinux_genfs_get_sid(dentry, isec->sclass,
> - sbsec->flags, &sid);
> + goto out;
> + rc = selinux_genfs_get_sid(dentry, sclass, sbsec->flags, &sid);
> dput(dentry);
> if (rc)
> - goto out_unlock;
> - isec->sid = sid;
> + goto out;
> }
> break;
> }
>
> - isec->initialized = LABEL_INITIALIZED;
> +out:
> + spin_lock(&isec->lock);
> + if (isec->initialized == LABEL_PENDING) {
> + if (!sid || rc) {
> + isec->initialized = LABEL_INVALID;
> + goto out_unlock;
> + }
> +
> + isec->initialized = LABEL_INITIALIZED;
> + isec->sid = sid;
> + }
>
> out_unlock:
> - mutex_unlock(&isec->lock);
> + spin_unlock(&isec->lock);
> return rc;
> }
>
> @@ -2889,9 +2901,11 @@ static int selinux_inode_init_security(struct inode *inode, struct inode *dir,
> /* Possibly defer initialization to selinux_complete_init. */
> if (sbsec->flags & SE_SBINITIALIZED) {
> struct inode_security_struct *isec = inode->i_security;
> + spin_lock(&isec->lock);
> isec->sclass = inode_mode_to_security_class(inode->i_mode);
> isec->sid = newsid;
> isec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&isec->lock);
> }
>
> if (!ss_initialized || !(sbsec->flags & SBLABEL_MNT))
> @@ -3194,9 +3208,11 @@ static void selinux_inode_post_setxattr(struct dentry *dentry, const char *name,
> }
>
> isec = backing_inode_security(dentry);
> + spin_lock(&isec->lock);
> isec->sclass = inode_mode_to_security_class(inode->i_mode);
> isec->sid = newsid;
> isec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&isec->lock);
>
> return;
> }
> @@ -3289,9 +3305,11 @@ static int selinux_inode_setsecurity(struct inode *inode, const char *name,
> if (rc)
> return rc;
>
> + spin_lock(&isec->lock);
> isec->sclass = inode_mode_to_security_class(inode->i_mode);
> isec->sid = newsid;
> isec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&isec->lock);
> return 0;
> }
>
> @@ -3952,9 +3970,11 @@ static void selinux_task_to_inode(struct task_struct *p,
> struct inode_security_struct *isec = inode->i_security;
> u32 sid = task_sid(p);
>
> + spin_lock(&isec->lock);
> isec->sclass = inode_mode_to_security_class(inode->i_mode);
> isec->sid = sid;
> isec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&isec->lock);
> }
>
> /* Returns error only if unable to parse addresses */
> @@ -4273,24 +4293,26 @@ static int selinux_socket_post_create(struct socket *sock, int family,
> const struct task_security_struct *tsec = current_security();
> struct inode_security_struct *isec = inode_security_novalidate(SOCK_INODE(sock));
> struct sk_security_struct *sksec;
> + u16 sclass = socket_type_to_security_class(family, type, protocol);
> + u32 sid = SECINITSID_KERNEL;
> int err = 0;
>
> - isec->sclass = socket_type_to_security_class(family, type, protocol);
> -
> - if (kern)
> - isec->sid = SECINITSID_KERNEL;
> - else {
> - err = socket_sockcreate_sid(tsec, isec->sclass, &(isec->sid));
> + if (!kern) {
> + err = socket_sockcreate_sid(tsec, sclass, &sid);
> if (err)
> return err;
> }
>
> + spin_lock(&isec->lock);
> + isec->sclass = sclass;
> + isec->sid = sid;
> isec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&isec->lock);
>
> if (sock->sk) {
> sksec = sock->sk->sk_security;
> - sksec->sid = isec->sid;
> - sksec->sclass = isec->sclass;
> + sksec->sclass = sclass;
> + sksec->sid = sid;
> err = selinux_netlbl_socket_post_create(sock->sk, family);
> }
>
> @@ -4466,17 +4488,25 @@ static int selinux_socket_accept(struct socket *sock, struct socket *newsock)
> int err;
> struct inode_security_struct *isec;
> struct inode_security_struct *newisec;
> + u16 sclass;
> + u32 sid;
>
> err = sock_has_perm(current, sock->sk, SOCKET__ACCEPT);
> if (err)
> return err;
>
> - newisec = inode_security_novalidate(SOCK_INODE(newsock));
> -
> isec = inode_security_novalidate(SOCK_INODE(sock));
> - newisec->sclass = isec->sclass;
> - newisec->sid = isec->sid;
> + spin_lock(&isec->lock);
> + sclass = isec->sclass;
> + sid = isec->sid;
> + spin_unlock(&isec->lock);
> +
> + newisec = inode_security_novalidate(SOCK_INODE(newsock));
> + spin_lock(&newisec->lock);
> + newisec->sclass = sclass;
> + newisec->sid = sid;
> newisec->initialized = LABEL_INITIALIZED;
> + spin_unlock(&newisec->lock);
>
> return 0;
> }
> @@ -5978,9 +6008,9 @@ static void selinux_inode_invalidate_secctx(struct inode *inode)
> {
> struct inode_security_struct *isec = inode->i_security;
>
> - mutex_lock(&isec->lock);
> + spin_lock(&isec->lock);
> isec->initialized = LABEL_INVALID;
> - mutex_unlock(&isec->lock);
> + spin_unlock(&isec->lock);
> }
>
> /*
> diff --git a/security/selinux/include/objsec.h b/security/selinux/include/objsec.h
> index c21e135..7e770e9 100644
> --- a/security/selinux/include/objsec.h
> +++ b/security/selinux/include/objsec.h
> @@ -39,7 +39,8 @@ struct task_security_struct {
>
> enum label_initialized {
> LABEL_INVALID, /* invalid or not initialized */
> - LABEL_INITIALIZED /* initialized */
> + LABEL_INITIALIZED, /* initialized */
> + LABEL_PENDING
> };
>
> struct inode_security_struct {
> @@ -52,7 +53,7 @@ struct inode_security_struct {
> u32 sid; /* SID of this object */
> u16 sclass; /* security class of this object */
> unsigned char initialized; /* initialization flag */
> - struct mutex lock;
> + struct spinlock lock;
> };
>
> struct file_security_struct {
> --
> 2.7.4
>
--
paul moore
www.paul-moore.com
^ permalink raw reply
* [PATCH V4 10/15] blk-throttle: add a simple idle detection
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
A cgroup gets assigned a high limit, but the cgroup could never dispatch
enough IO to cross the high limit. In such case, the queue state machine
will remain in LIMIT_HIGH state and all other cgroups will be throttled
according to high limit. This is unfair for other cgroups. We should
treat the cgroup idle and upgrade the state machine to higher state.
We also have a downgrade logic. If the state machine upgrades because of
cgroup idle (real idle), the state machine will downgrade soon as the
cgroup is below its high limit. This isn't what we want. A more
complicated case is cgroup isn't idle when queue is in LIMIT_HIGH. But
when queue gets upgraded to higher state, other cgroups could dispatch
more IO and this cgroup can't dispatch enough IO, so the cgroup is below
its high limit and looks like idle (fake idle). In this case, the queue
should downgrade soon. The key to determine if we should do downgrade is
to detect if cgroup is truely idle.
Unfortunately it's very hard to determine if a cgroup is real idle. This
patch uses the 'think time check' idea from CFQ for the purpose. Please
note, the idea doesn't work for all workloads. For example, a workload
with io depth 8 has disk utilization 100%, hence think time is 0, eg,
not idle. But the workload can run higher bandwidth with io depth 16.
Compared to io depth 16, the io depth 8 workload is idle. We use the
idea to roughly determine if a cgroup is idle.
We treat a cgroup idle if its think time is above a threshold (by
default 50us for SSD and 1ms for HD). The idea is think time above the
threshold will start to harm performance. HD is much slower so a longer
think time is ok.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/bio.c | 2 ++
block/blk-throttle.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++-
block/blk.h | 2 ++
include/linux/blk_types.h | 1 +
4 files changed, 76 insertions(+), 1 deletion(-)
diff --git a/block/bio.c b/block/bio.c
index db85c57..7baa86d 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -30,6 +30,7 @@
#include <linux/cgroup.h>
#include <trace/events/block.h>
+#include "blk.h"
/*
* Test patch to inline a certain number of bi_io_vec's inside the bio
@@ -1759,6 +1760,7 @@ void bio_endio(struct bio *bio)
goto again;
}
+ blk_throtl_bio_endio(bio);
if (bio->bi_end_io)
bio->bi_end_io(bio);
}
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 45a28c4..cb5fd85 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -21,6 +21,8 @@ static int throtl_quantum = 32;
/* Throttling is performed over 100ms slice and after that slice is renewed */
#define DFL_THROTL_SLICE (HZ / 10)
#define MAX_THROTL_SLICE (HZ / 5)
+#define DFL_IDLE_THRESHOLD_SSD (50 * 1000) /* 50 us */
+#define DFL_IDLE_THRESHOLD_HD (1000 * 1000) /* 1 ms */
static struct blkcg_policy blkcg_policy_throtl;
@@ -149,6 +151,10 @@ struct throtl_grp {
/* When did we start a new slice */
unsigned long slice_start[2];
unsigned long slice_end[2];
+
+ u64 last_finish_time;
+ u64 checked_last_finish_time;
+ u64 avg_ttime;
};
struct throtl_data
@@ -172,6 +178,8 @@ struct throtl_data
unsigned long high_downgrade_time;
unsigned int scale;
+
+ u64 idle_ttime_threshold;
};
static void throtl_pending_timer_fn(unsigned long arg);
@@ -1629,6 +1637,14 @@ static unsigned long tg_last_high_overflow_time(struct throtl_grp *tg)
return ret;
}
+static bool throtl_tg_is_idle(struct throtl_grp *tg)
+{
+ /* cgroup is idle if average think time is more than threshold */
+ return ktime_get_ns() - tg->last_finish_time >
+ 4 * tg->td->idle_ttime_threshold ||
+ tg->avg_ttime > tg->td->idle_ttime_threshold;
+}
+
static bool throtl_upgrade_check_one(struct throtl_grp *tg)
{
struct throtl_service_queue *sq = &tg->service_queue;
@@ -1837,6 +1853,19 @@ static void throtl_downgrade_check(struct throtl_grp *tg)
tg->last_io_disp[WRITE] = 0;
}
+static void blk_throtl_update_ttime(struct throtl_grp *tg)
+{
+ u64 now = ktime_get_ns();
+ u64 last_finish_time = tg->last_finish_time;
+
+ if (now <= last_finish_time || last_finish_time == 0 ||
+ last_finish_time == tg->checked_last_finish_time)
+ return;
+
+ tg->avg_ttime = (tg->avg_ttime * 7 + now - last_finish_time) >> 3;
+ tg->checked_last_finish_time = last_finish_time;
+}
+
bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
struct bio *bio)
{
@@ -1848,6 +1877,13 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
WARN_ON_ONCE(!rcu_read_lock_held());
+ if (tg->td->idle_ttime_threshold == -1) {
+ if (blk_queue_nonrot(q))
+ tg->td->idle_ttime_threshold = DFL_IDLE_THRESHOLD_SSD;
+ else
+ tg->td->idle_ttime_threshold = DFL_IDLE_THRESHOLD_HD;
+ }
+
/* see throtl_charge_bio() */
if ((bio->bi_opf & REQ_THROTTLED) || !tg->has_rules[rw])
goto out;
@@ -1857,6 +1893,11 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
if (unlikely(blk_queue_bypass(q)))
goto out_unlock;
+ bio_associate_current(bio);
+ bio->bi_cg_private = q;
+
+ blk_throtl_update_ttime(tg);
+
sq = &tg->service_queue;
again:
@@ -1917,7 +1958,6 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
tg->last_high_overflow_time[rw] = jiffies;
- bio_associate_current(bio);
tg->td->nr_queued[rw]++;
throtl_add_bio_tg(bio, qn, tg);
throttled = true;
@@ -1946,6 +1986,34 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
return throttled;
}
+void blk_throtl_bio_endio(struct bio *bio)
+{
+ struct blkcg *blkcg;
+ struct blkcg_gq *blkg;
+ struct throtl_grp *tg;
+ struct request_queue *q;
+
+ q = bio->bi_cg_private;
+ if (!q)
+ return;
+ bio->bi_cg_private = NULL;
+
+ rcu_read_lock();
+ blkcg = bio_blkcg(bio);
+ if (!blkcg)
+ goto end;
+ blkg = blkg_lookup(blkcg, q);
+ if (!blkg)
+ goto end;
+
+ tg = blkg_to_tg(blkg ?: q->root_blkg);
+
+ tg->last_finish_time = ktime_get_ns();
+
+end:
+ rcu_read_unlock();
+}
+
/*
* Dispatch all bios from all children tg's queued on @parent_sq. On
* return, @parent_sq is guaranteed to not have any active children tg's
@@ -2030,6 +2098,8 @@ int blk_throtl_init(struct request_queue *q)
td->limit_index = LIMIT_MAX;
td->high_upgrade_time = jiffies;
td->high_downgrade_time = jiffies;
+
+ td->idle_ttime_threshold = -1;
/* activate policy */
ret = blkcg_activate_policy(q, &blkcg_policy_throtl);
if (ret)
diff --git a/block/blk.h b/block/blk.h
index 39c14dd..b433f35 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -292,10 +292,12 @@ extern void blk_throtl_exit(struct request_queue *q);
extern ssize_t blk_throtl_slice_show(struct request_queue *q, char *page);
extern ssize_t blk_throtl_slice_store(struct request_queue *q,
const char *page, size_t count);
+extern void blk_throtl_bio_endio(struct bio *bio);
#else /* CONFIG_BLK_DEV_THROTTLING */
static inline void blk_throtl_drain(struct request_queue *q) { }
static inline int blk_throtl_init(struct request_queue *q) { return 0; }
static inline void blk_throtl_exit(struct request_queue *q) { }
+static inline void blk_throtl_bio_endio(struct bio *bio) { }
#endif /* CONFIG_BLK_DEV_THROTTLING */
#endif /* BLK_INTERNAL_H */
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index cd395ec..ff8dd24 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -59,6 +59,7 @@ struct bio {
*/
struct io_context *bi_ioc;
struct cgroup_subsys_state *bi_css;
+ void *bi_cg_private;
#endif
union {
#if defined(CONFIG_BLK_DEV_INTEGRITY)
--
2.9.3
^ permalink raw reply related
* [PATCH V4 05/15] blk-throttle: add downgrade logic
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
When queue state machine is in LIMIT_MAX state, but a cgroup is below
its high limit for some time, the queue should be downgraded to lower
state as one cgroup's high limit isn't met.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 188 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 188 insertions(+)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 34a75e5..d177252 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -136,6 +136,13 @@ struct throtl_grp {
/* Number of bio's dispatched in current slice */
unsigned int io_disp[2];
+ unsigned long last_high_overflow_time[2];
+
+ uint64_t last_bytes_disp[2];
+ unsigned int last_io_disp[2];
+
+ unsigned long last_check_time;
+
/* When did we start a new slice */
unsigned long slice_start[2];
unsigned long slice_end[2];
@@ -155,6 +162,9 @@ struct throtl_data
struct work_struct dispatch_work;
unsigned int limit_index;
bool limit_valid[LIMIT_CNT];
+
+ unsigned long high_upgrade_time;
+ unsigned long high_downgrade_time;
};
static void throtl_pending_timer_fn(unsigned long arg);
@@ -896,6 +906,8 @@ static void throtl_charge_bio(struct throtl_grp *tg, struct bio *bio)
/* Charge the bio to the group */
tg->bytes_disp[rw] += bio->bi_iter.bi_size;
tg->io_disp[rw]++;
+ tg->last_bytes_disp[rw] += bio->bi_iter.bi_size;
+ tg->last_io_disp[rw]++;
/*
* REQ_THROTTLED is used to prevent the same bio to be throttled
@@ -1510,6 +1522,65 @@ static struct blkcg_policy blkcg_policy_throtl = {
.pd_free_fn = throtl_pd_free,
};
+static unsigned long __tg_last_high_overflow_time(struct throtl_grp *tg)
+{
+ unsigned long rtime = -1, wtime = -1;
+
+ if (tg->bps[READ][LIMIT_HIGH] != -1 ||
+ tg->iops[READ][LIMIT_HIGH] != -1 ||
+ tg->bps[READ][LIMIT_MAX] != -1 ||
+ tg->iops[READ][LIMIT_MAX] != -1)
+ rtime = tg->last_high_overflow_time[READ];
+ if (tg->bps[WRITE][LIMIT_HIGH] != -1 ||
+ tg->iops[WRITE][LIMIT_HIGH] != -1 ||
+ tg->bps[WRITE][LIMIT_MAX] != -1 ||
+ tg->iops[WRITE][LIMIT_MAX] != -1)
+ wtime = tg->last_high_overflow_time[WRITE];
+ return min(rtime, wtime) == -1 ? 0 : min(rtime, wtime);
+}
+
+static unsigned long tg_last_high_overflow_time(struct throtl_grp *tg)
+{
+ struct throtl_service_queue *parent_sq;
+ struct throtl_grp *parent = tg;
+ unsigned long ret = __tg_last_high_overflow_time(tg);
+
+ while (true) {
+ parent_sq = parent->service_queue.parent_sq;
+ parent = sq_to_tg(parent_sq);
+ if (!parent)
+ break;
+ if (((parent->bps[READ][LIMIT_HIGH] != -1 &&
+ parent->bps[READ][LIMIT_HIGH] >=
+ tg->bps[READ][LIMIT_HIGH]) ||
+ (parent->bps[READ][LIMIT_HIGH] == -1 &&
+ parent->bps[READ][LIMIT_MAX] >=
+ tg->bps[READ][LIMIT_HIGH])) &&
+ ((parent->bps[WRITE][LIMIT_HIGH] != -1 &&
+ parent->bps[WRITE][LIMIT_HIGH] >=
+ tg->bps[WRITE][LIMIT_HIGH]) ||
+ (parent->bps[WRITE][LIMIT_HIGH] == -1 &&
+ parent->bps[WRITE][LIMIT_MAX] >=
+ tg->bps[WRITE][LIMIT_HIGH])) &&
+ ((parent->iops[READ][LIMIT_HIGH] != -1 &&
+ parent->iops[READ][LIMIT_HIGH] >=
+ tg->iops[READ][LIMIT_HIGH]) ||
+ (parent->iops[READ][LIMIT_HIGH] == -1 &&
+ parent->iops[READ][LIMIT_MAX] >=
+ tg->iops[READ][LIMIT_HIGH])) &&
+ ((parent->iops[WRITE][LIMIT_HIGH] != -1 &&
+ parent->iops[WRITE][LIMIT_HIGH] >=
+ tg->iops[WRITE][LIMIT_HIGH]) ||
+ (parent->iops[WRITE][LIMIT_HIGH] == -1 &&
+ parent->iops[WRITE][LIMIT_MAX] >=
+ tg->iops[WRITE][LIMIT_HIGH])))
+ break;
+ if (time_after(__tg_last_high_overflow_time(parent), ret))
+ ret = __tg_last_high_overflow_time(parent);
+ }
+ return ret;
+}
+
static bool throtl_upgrade_check_one(struct throtl_grp *tg)
{
struct throtl_service_queue *sq = &tg->service_queue;
@@ -1557,6 +1628,9 @@ static bool throtl_can_upgrade(struct throtl_data *td,
if (td->limit_index != LIMIT_HIGH)
return false;
+ if (time_before(jiffies, td->high_downgrade_time + throtl_slice))
+ return false;
+
rcu_read_lock();
blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
struct throtl_grp *tg = blkg_to_tg(blkg);
@@ -1580,6 +1654,7 @@ static void throtl_upgrade_state(struct throtl_data *td)
struct blkcg_gq *blkg;
td->limit_index = LIMIT_MAX;
+ td->high_upgrade_time = jiffies;
rcu_read_lock();
blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
struct throtl_grp *tg = blkg_to_tg(blkg);
@@ -1595,6 +1670,111 @@ static void throtl_upgrade_state(struct throtl_data *td)
queue_work(kthrotld_workqueue, &td->dispatch_work);
}
+static void throtl_downgrade_state(struct throtl_data *td, int new)
+{
+ td->limit_index = new;
+ td->high_downgrade_time = jiffies;
+}
+
+static bool throtl_downgrade_check_one(struct throtl_grp *tg)
+{
+ struct throtl_data *td = tg->td;
+ unsigned long now = jiffies;
+
+ /*
+ * If cgroup is below high limit, consider downgrade and throttle other
+ * cgroups
+ */
+ if (time_after_eq(now, td->high_upgrade_time + throtl_slice) &&
+ time_after_eq(now, tg_last_high_overflow_time(tg) + throtl_slice))
+ return true;
+ return false;
+}
+
+static bool throtl_downgrade_check_hierarchy(struct throtl_grp *tg)
+{
+ if (!throtl_downgrade_check_one(tg))
+ return false;
+ while (true) {
+ if (!tg || (cgroup_subsys_on_dfl(io_cgrp_subsys) &&
+ !tg_to_blkg(tg)->parent))
+ break;
+
+ if (!throtl_downgrade_check_one(tg))
+ return false;
+ tg = sq_to_tg(tg->service_queue.parent_sq);
+ }
+ return true;
+}
+
+static void throtl_downgrade_check(struct throtl_grp *tg)
+{
+ uint64_t bps;
+ unsigned int iops;
+ unsigned long elapsed_time;
+ unsigned long now = jiffies;
+
+ if (tg->td->limit_index != LIMIT_MAX ||
+ !tg->td->limit_valid[LIMIT_HIGH])
+ return;
+ if (!list_empty(&tg_to_blkg(tg)->blkcg->css.children))
+ return;
+ if (time_after(tg->last_check_time + throtl_slice, now))
+ return;
+
+ if (time_before(now, tg_last_high_overflow_time(tg) + throtl_slice))
+ return;
+
+ elapsed_time = now - tg->last_check_time;
+ tg->last_check_time = now;
+
+ if (tg->bps[READ][LIMIT_HIGH] != -1 ||
+ tg->bps[READ][LIMIT_MAX] != -1) {
+ bps = tg->last_bytes_disp[READ] * HZ;
+ do_div(bps, elapsed_time);
+ if (bps >= tg->bps[READ][LIMIT_HIGH] ||
+ bps >= tg->bps[READ][LIMIT_MAX])
+ tg->last_high_overflow_time[READ] = now;
+ }
+
+ if (tg->bps[WRITE][LIMIT_HIGH] != -1 ||
+ tg->bps[WRITE][LIMIT_MAX] != -1) {
+ bps = tg->last_bytes_disp[WRITE] * HZ;
+ do_div(bps, elapsed_time);
+ if (bps >= tg->bps[WRITE][LIMIT_HIGH] ||
+ bps >= tg->bps[WRITE][LIMIT_MAX])
+ tg->last_high_overflow_time[WRITE] = now;
+ }
+
+ if (tg->iops[READ][LIMIT_HIGH] != -1 ||
+ tg->iops[READ][LIMIT_MAX] != -1) {
+ iops = tg->last_io_disp[READ] * HZ / elapsed_time;
+ if (iops >= tg->iops[READ][LIMIT_HIGH] ||
+ iops >= tg->iops[READ][LIMIT_MAX])
+ tg->last_high_overflow_time[READ] = now;
+ }
+
+ if (tg->iops[WRITE][LIMIT_HIGH] != -1 ||
+ tg->iops[WRITE][LIMIT_MAX] != -1) {
+ iops = tg->last_io_disp[WRITE] * HZ / elapsed_time;
+ if (iops >= tg->iops[WRITE][LIMIT_HIGH] ||
+ iops >= tg->iops[WRITE][LIMIT_MAX])
+ tg->last_high_overflow_time[WRITE] = now;
+ }
+
+ /*
+ * If cgroup is below high limit, consider downgrade and throttle other
+ * cgroups
+ */
+ if (throtl_downgrade_check_hierarchy(tg))
+ throtl_downgrade_state(tg->td, LIMIT_HIGH);
+
+ tg->last_bytes_disp[READ] = 0;
+ tg->last_bytes_disp[WRITE] = 0;
+ tg->last_io_disp[READ] = 0;
+ tg->last_io_disp[WRITE] = 0;
+}
+
bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
struct bio *bio)
{
@@ -1619,12 +1799,16 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
again:
while (true) {
+ if (tg->last_high_overflow_time[rw] == 0)
+ tg->last_high_overflow_time[rw] = jiffies;
+ throtl_downgrade_check(tg);
/* throtl is FIFO - if bios are already queued, should queue */
if (sq->nr_queued[rw])
break;
/* if above limits, break to queue */
if (!tg_may_dispatch(tg, bio, NULL)) {
+ tg->last_high_overflow_time[rw] = jiffies;
if (throtl_can_upgrade(tg->td, tg)) {
throtl_upgrade_state(tg->td);
goto again;
@@ -1668,6 +1852,8 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
tg->io_disp[rw], tg_iops_limit(tg, rw),
sq->nr_queued[READ], sq->nr_queued[WRITE]);
+ tg->last_high_overflow_time[rw] = jiffies;
+
bio_associate_current(bio);
tg->td->nr_queued[rw]++;
throtl_add_bio_tg(bio, qn, tg);
@@ -1778,6 +1964,8 @@ int blk_throtl_init(struct request_queue *q)
td->limit_valid[LIMIT_MAX] = true;
td->limit_index = LIMIT_MAX;
+ td->high_upgrade_time = jiffies;
+ td->high_downgrade_time = jiffies;
/* activate policy */
ret = blkcg_activate_policy(q, &blkcg_policy_throtl);
if (ret)
--
2.9.3
^ permalink raw reply related
* [PATCH V4 04/15] blk-throttle: add upgrade logic for LIMIT_HIGH state
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
When queue is in LIMIT_HIGH state and all cgroups with high limit cross
the bps/iops limitation, we will upgrade queue's state to
LIMIT_MAX
For a cgroup hierarchy, there are two cases. Children has lower high
limit than parent. Parent's high limit is meaningless. If children's
bps/iops cross high limit, we can upgrade queue state. The other case is
children has higher high limit than parent. Children's high limit is
meaningless. As long as parent's bps/iops cross high limit, we can
upgrade queue state.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 104 +++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 100 insertions(+), 4 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index ec53671..34a75e5 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -456,6 +456,7 @@ static void blk_throtl_update_valid_limit(struct throtl_data *td)
td->limit_valid[LIMIT_HIGH] = false;
}
+static void throtl_upgrade_state(struct throtl_data *td);
static void throtl_pd_offline(struct blkg_policy_data *pd)
{
struct throtl_grp *tg = pd_to_tg(pd);
@@ -467,9 +468,8 @@ static void throtl_pd_offline(struct blkg_policy_data *pd)
blk_throtl_update_valid_limit(tg->td);
- if (tg->td->limit_index == LIMIT_HIGH &&
- !tg->td->limit_valid[LIMIT_HIGH])
- tg->td->limit_index = LIMIT_MAX;
+ if (!tg->td->limit_valid[tg->td->limit_index])
+ throtl_upgrade_state(tg->td);
}
static void throtl_pd_free(struct blkg_policy_data *pd)
@@ -1077,6 +1077,8 @@ static int throtl_select_dispatch(struct throtl_service_queue *parent_sq)
return nr_disp;
}
+static bool throtl_can_upgrade(struct throtl_data *td,
+ struct throtl_grp *this_tg);
/**
* throtl_pending_timer_fn - timer function for service_queue->pending_timer
* @arg: the throtl_service_queue being serviced
@@ -1103,6 +1105,9 @@ static void throtl_pending_timer_fn(unsigned long arg)
int ret;
spin_lock_irq(q->queue_lock);
+ if (throtl_can_upgrade(td, NULL))
+ throtl_upgrade_state(td);
+
again:
parent_sq = sq->parent_sq;
dispatched = false;
@@ -1505,6 +1510,91 @@ static struct blkcg_policy blkcg_policy_throtl = {
.pd_free_fn = throtl_pd_free,
};
+static bool throtl_upgrade_check_one(struct throtl_grp *tg)
+{
+ struct throtl_service_queue *sq = &tg->service_queue;
+ bool read_limit, write_limit;
+
+ /* if cgroup reaches high/max limit, it's ok to next limit */
+ read_limit = tg->bps[READ][LIMIT_HIGH] != -1 ||
+ tg->iops[READ][LIMIT_HIGH] != -1 ||
+ tg->bps[READ][LIMIT_MAX] != -1 ||
+ tg->iops[READ][LIMIT_MAX] != -1;
+ write_limit = tg->bps[WRITE][LIMIT_HIGH] != -1 ||
+ tg->iops[WRITE][LIMIT_HIGH] != -1 ||
+ tg->bps[WRITE][LIMIT_MAX] != -1 ||
+ tg->iops[WRITE][LIMIT_MAX] != -1;
+ if (read_limit && sq->nr_queued[READ] &&
+ (!write_limit || sq->nr_queued[WRITE]))
+ return true;
+ if (write_limit && sq->nr_queued[WRITE] &&
+ (!read_limit || sq->nr_queued[READ]))
+ return true;
+ return false;
+}
+
+static bool throtl_upgrade_check_hierarchy(struct throtl_grp *tg)
+{
+ if (throtl_upgrade_check_one(tg))
+ return true;
+ while (true) {
+ if (!tg || (cgroup_subsys_on_dfl(io_cgrp_subsys) &&
+ !tg_to_blkg(tg)->parent))
+ return false;
+ if (throtl_upgrade_check_one(tg))
+ return true;
+ tg = sq_to_tg(tg->service_queue.parent_sq);
+ }
+ return false;
+}
+
+static bool throtl_can_upgrade(struct throtl_data *td,
+ struct throtl_grp *this_tg)
+{
+ struct cgroup_subsys_state *pos_css;
+ struct blkcg_gq *blkg;
+
+ if (td->limit_index != LIMIT_HIGH)
+ return false;
+
+ rcu_read_lock();
+ blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
+ struct throtl_grp *tg = blkg_to_tg(blkg);
+
+ if (tg == this_tg)
+ continue;
+ if (!list_empty(&tg_to_blkg(tg)->blkcg->css.children))
+ continue;
+ if (!throtl_upgrade_check_hierarchy(tg)) {
+ rcu_read_unlock();
+ return false;
+ }
+ }
+ rcu_read_unlock();
+ return true;
+}
+
+static void throtl_upgrade_state(struct throtl_data *td)
+{
+ struct cgroup_subsys_state *pos_css;
+ struct blkcg_gq *blkg;
+
+ td->limit_index = LIMIT_MAX;
+ rcu_read_lock();
+ blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
+ struct throtl_grp *tg = blkg_to_tg(blkg);
+ struct throtl_service_queue *sq = &tg->service_queue;
+
+ tg->disptime = jiffies - 1;
+ throtl_select_dispatch(sq);
+ throtl_schedule_next_dispatch(sq, false);
+ }
+ rcu_read_unlock();
+ throtl_select_dispatch(&td->service_queue);
+ throtl_schedule_next_dispatch(&td->service_queue, false);
+ queue_work(kthrotld_workqueue, &td->dispatch_work);
+}
+
bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
struct bio *bio)
{
@@ -1527,14 +1617,20 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
sq = &tg->service_queue;
+again:
while (true) {
/* throtl is FIFO - if bios are already queued, should queue */
if (sq->nr_queued[rw])
break;
/* if above limits, break to queue */
- if (!tg_may_dispatch(tg, bio, NULL))
+ if (!tg_may_dispatch(tg, bio, NULL)) {
+ if (throtl_can_upgrade(tg->td, tg)) {
+ throtl_upgrade_state(tg->td);
+ goto again;
+ }
break;
+ }
/* within limits, let's charge and dispatch directly */
throtl_charge_bio(tg, bio);
--
2.9.3
^ permalink raw reply related
* [PATCH V4 00/15] blk-throttle: add .high limit
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
Hi,
The background is we don't have an ioscheduler for blk-mq yet, so we can't
prioritize processes/cgroups. This patch set tries to add basic arbitration
between cgroups with blk-throttle. It adds a new limit io.high for
blk-throttle. It's only for cgroup2.
io.max is a hard limit throttling. cgroups with a max limit never dispatch more
IO than their max limit. While io.high is a best effort throttling. cgroups
with high limit can run above their high limit at appropriate time.
Specifically, if all cgroups reach their high limit, all cgroups can run above
their high limit. If any cgroup runs under its high limit, all other cgroups
will run according to their high limit.
An example usage is we have a high prio cgroup with high high limit and a low
prio cgroup with low high limit. If the high prio cgroup isn't running, the low
prio can run above its high limit, so we don't waste the bandwidth. When the
high prio cgroup runs and is below its high limit, low prio cgroup will run
under its high limit. This will protect high prio cgroup to get more resources.
If both cgroups reach their high limit, both can run above their high limit
(eg, fully utilize disk bandwidth). All these can't be done with io.max limit.
The implementation is simple. The disk queue has 2 states LIMIT_HIGH and
LIMIT_MAX. In each disk state, we throttle cgroups according to the limit of
the state. That is io.high limit for LIMIT_HIGH state, io.max limit for
LIMIT_MAX. The disk state can be upgraded/downgraded between
LIMIT_HIGH/LIMIT_MAX according to the rule above. Initially disk state is
LIMIT_MAX. And if no cgroup sets io.high, the disk state will remain in
LIMIT_MAX state. Users with only io.max set will find nothing changed with the
patches.
The first 8 patches implement the basic framework. Add interface, handle
upgrade and downgrade logic. The patch 8 detects a special case a cgroup is
completely idle. In this case, we ignore the cgroup's limit. The patch 9-15
adds more heuristics.
The basic framework has 2 major issues.
1. fluctuation. When the state is upgraded from LIMIT_HIGH to LIMIT_MAX, the
cgroup's bandwidth can change dramatically, sometimes in a way not expected.
For example, one cgroup's bandwidth will drop below its io.high limit very soon
after a upgrade. patch 9 has more details about the issue.
2. idle cgroup. cgroup with a io.high limit doesn't always dispatch enough IO.
In above upgrade rule, the disk will remain in LIMIT_HIGH state and all other
cgroups can't dispatch more IO above their high limit. Hence this is a waste of
disk bandwidth. patch 10 has more details about the issue.
For issue 1, we make cgroup bandwidth increase smoothly after a upgrade. This
will reduce the chance a cgroup's bandwidth drop under its high limit rapidly.
The smoothness means we could waste some bandwidth in the transition though.
But we must pay something for sharing.
The issue 2 is very hard to solve. The patch 10 uses the 'think time check'
idea borrowed from CFQ to detect idle cgroup. It's not perfect, eg, not works
well for high IO depth workloads. But it's the best I tried so far and in
practice works well. This definitively needs more tuning.
The big change in this version comes from patch 13 - 15. We add a latency
target for each cgroup. The goal is to solve issue 2. If a cgroup's average io
latency exceeds its latency target, the cgroup is considered as busy.
Please review, test and consider merge.
Thanks,
Shaohua
V3->V4:
- Add latency target for cgroup
- Fix bugs
V2->V3:
- Rebase
- Fix several bugs
- Make harddisk think time threshold bigger
http://marc.info/?l=linux-kernel&m=147552964708965&w=2
V1->V2:
- Drop io.low interface for simplicity and the interface isn't a must-have to
prioritize cgroups.
- Remove the 'trial' logic, which creates too much fluctuation
- Add a new idle cgroup detection
- Other bug fixes and improvements
http://marc.info/?l=linux-block&m=147395674732335&w=2
V1:
http://marc.info/?l=linux-block&m=146292596425689&w=2
Shaohua Li (15):
blk-throttle: prepare support multiple limits
blk-throttle: add .high interface
blk-throttle: configure bps/iops limit for cgroup in high limit
blk-throttle: add upgrade logic for LIMIT_HIGH state
blk-throttle: add downgrade logic
blk-throttle: make sure expire time isn't too big
blk-throttle: make throtl_slice tunable
blk-throttle: detect completed idle cgroup
blk-throttle: make bandwidth change smooth
blk-throttle: add a simple idle detection
blk-throttle: add interface to configure think time threshold
blk-throttle: ignore idle cgroup limit
blk-throttle: add a mechanism to estimate IO latency
blk-throttle: add interface for per-cgroup target latency
blk-throttle: add latency target support
block/bio.c | 2 +
block/blk-sysfs.c | 18 +
block/blk-throttle.c | 1035 ++++++++++++++++++++++++++++++++++++++++++---
block/blk.h | 9 +
include/linux/blk_types.h | 4 +
5 files changed, 1001 insertions(+), 67 deletions(-)
--
2.9.3
^ permalink raw reply
* [PATCH V4 02/15] blk-throttle: add .high interface
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
Add high limit for cgroup and corresponding cgroup interface.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 132 ++++++++++++++++++++++++++++++++++++++++-----------
1 file changed, 103 insertions(+), 29 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 925aa1ed..a564215 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -84,6 +84,7 @@ enum tg_state_flags {
#define rb_entry_tg(node) rb_entry((node), struct throtl_grp, rb_node)
enum {
+ LIMIT_HIGH,
LIMIT_MAX,
LIMIT_CNT,
};
@@ -414,6 +415,46 @@ static void throtl_pd_online(struct blkg_policy_data *pd)
tg_update_has_rules(pd_to_tg(pd));
}
+static void blk_throtl_update_valid_limit(struct throtl_data *td)
+{
+ struct cgroup_subsys_state *pos_css;
+ struct blkcg_gq *blkg;
+ bool high_valid = false;
+
+ rcu_read_lock();
+ blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) {
+ struct throtl_grp *tg = blkg_to_tg(blkg);
+
+ if (tg->bps[READ][LIMIT_HIGH] != -1 ||
+ tg->bps[WRITE][LIMIT_HIGH] != -1 ||
+ tg->iops[READ][LIMIT_HIGH] != -1 ||
+ tg->iops[WRITE][LIMIT_HIGH] != -1)
+ high_valid = true;
+ }
+ rcu_read_unlock();
+
+ if (high_valid)
+ td->limit_valid[LIMIT_HIGH] = true;
+ else
+ td->limit_valid[LIMIT_HIGH] = false;
+}
+
+static void throtl_pd_offline(struct blkg_policy_data *pd)
+{
+ struct throtl_grp *tg = pd_to_tg(pd);
+
+ tg->bps[READ][LIMIT_HIGH] = -1;
+ tg->bps[WRITE][LIMIT_HIGH] = -1;
+ tg->iops[READ][LIMIT_HIGH] = -1;
+ tg->iops[WRITE][LIMIT_HIGH] = -1;
+
+ blk_throtl_update_valid_limit(tg->td);
+
+ if (tg->td->limit_index == LIMIT_HIGH &&
+ !tg->td->limit_valid[LIMIT_HIGH])
+ tg->td->limit_index = LIMIT_MAX;
+}
+
static void throtl_pd_free(struct blkg_policy_data *pd)
{
struct throtl_grp *tg = pd_to_tg(pd);
@@ -1283,7 +1324,7 @@ static struct cftype throtl_legacy_files[] = {
{ } /* terminate */
};
-static u64 tg_prfill_max(struct seq_file *sf, struct blkg_policy_data *pd,
+static u64 tg_prfill_limit(struct seq_file *sf, struct blkg_policy_data *pd,
int off)
{
struct throtl_grp *tg = pd_to_tg(pd);
@@ -1292,36 +1333,32 @@ static u64 tg_prfill_max(struct seq_file *sf, struct blkg_policy_data *pd,
if (!dname)
return 0;
- if (tg->bps[READ][LIMIT_MAX] == -1 && tg->bps[WRITE][LIMIT_MAX] == -1 &&
- tg->iops[READ][LIMIT_MAX] == -1 && tg->iops[WRITE][LIMIT_MAX] == -1)
+ if (tg->bps[READ][off] == -1 && tg->bps[WRITE][off] == -1 &&
+ tg->iops[READ][off] == -1 && tg->iops[WRITE][off] == -1)
return 0;
- if (tg->bps[READ][LIMIT_MAX] != -1)
- snprintf(bufs[0], sizeof(bufs[0]), "%llu",
- tg->bps[READ][LIMIT_MAX]);
- if (tg->bps[WRITE][LIMIT_MAX] != -1)
- snprintf(bufs[1], sizeof(bufs[1]), "%llu",
- tg->bps[WRITE][LIMIT_MAX]);
- if (tg->iops[READ][LIMIT_MAX] != -1)
- snprintf(bufs[2], sizeof(bufs[2]), "%u",
- tg->iops[READ][LIMIT_MAX]);
- if (tg->iops[WRITE][LIMIT_MAX] != -1)
- snprintf(bufs[3], sizeof(bufs[3]), "%u",
- tg->iops[WRITE][LIMIT_MAX]);
+ if (tg->bps[READ][off] != -1)
+ snprintf(bufs[0], sizeof(bufs[0]), "%llu", tg->bps[READ][off]);
+ if (tg->bps[WRITE][off] != -1)
+ snprintf(bufs[1], sizeof(bufs[1]), "%llu", tg->bps[WRITE][off]);
+ if (tg->iops[READ][off] != -1)
+ snprintf(bufs[2], sizeof(bufs[2]), "%u", tg->iops[READ][off]);
+ if (tg->iops[WRITE][off] != -1)
+ snprintf(bufs[3], sizeof(bufs[3]), "%u", tg->iops[WRITE][off]);
seq_printf(sf, "%s rbps=%s wbps=%s riops=%s wiops=%s\n",
dname, bufs[0], bufs[1], bufs[2], bufs[3]);
return 0;
}
-static int tg_print_max(struct seq_file *sf, void *v)
+static int tg_print_limit(struct seq_file *sf, void *v)
{
- blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), tg_prfill_max,
+ blkcg_print_blkgs(sf, css_to_blkcg(seq_css(sf)), tg_prfill_limit,
&blkcg_policy_throtl, seq_cft(sf)->private, false);
return 0;
}
-static ssize_t tg_set_max(struct kernfs_open_file *of,
+static ssize_t tg_set_limit(struct kernfs_open_file *of,
char *buf, size_t nbytes, loff_t off)
{
struct blkcg *blkcg = css_to_blkcg(of_css(of));
@@ -1329,6 +1366,7 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
struct throtl_grp *tg;
u64 v[4];
int ret;
+ int index = of_cft(of)->private;
ret = blkg_conf_prep(blkcg, &blkcg_policy_throtl, buf, &ctx);
if (ret)
@@ -1336,10 +1374,10 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
tg = blkg_to_tg(ctx.blkg);
- v[0] = tg->bps[READ][LIMIT_MAX];
- v[1] = tg->bps[WRITE][LIMIT_MAX];
- v[2] = tg->iops[READ][LIMIT_MAX];
- v[3] = tg->iops[WRITE][LIMIT_MAX];
+ v[0] = tg->bps[READ][index];
+ v[1] = tg->bps[WRITE][index];
+ v[2] = tg->iops[READ][index];
+ v[3] = tg->iops[WRITE][index];
while (true) {
char tok[27]; /* wiops=18446744073709551616 */
@@ -1376,11 +1414,37 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
goto out_finish;
}
- tg->bps[READ][LIMIT_MAX] = v[0];
- tg->bps[WRITE][LIMIT_MAX] = v[1];
- tg->iops[READ][LIMIT_MAX] = v[2];
- tg->iops[WRITE][LIMIT_MAX] = v[3];
-
+ if (index == LIMIT_MAX) {
+ if ((v[0] < tg->bps[READ][LIMIT_HIGH] &&
+ tg->bps[READ][LIMIT_HIGH] != -1) ||
+ (v[1] < tg->bps[WRITE][LIMIT_HIGH] &&
+ tg->bps[WRITE][LIMIT_HIGH] != -1) ||
+ (v[2] < tg->iops[READ][LIMIT_HIGH] &&
+ tg->iops[READ][LIMIT_HIGH] != -1) ||
+ (v[3] < tg->iops[WRITE][LIMIT_HIGH] &&
+ tg->iops[WRITE][LIMIT_HIGH] != -1)) {
+ ret = -EINVAL;
+ goto out_finish;
+ }
+ } else if (index == LIMIT_HIGH) {
+ if ((v[0] > tg->bps[READ][LIMIT_MAX] && v[0] != -1) ||
+ (v[1] > tg->bps[WRITE][LIMIT_MAX] && v[1] != -1) ||
+ (v[2] > tg->iops[READ][LIMIT_MAX] && v[2] != -1) ||
+ (v[3] > tg->iops[WRITE][LIMIT_MAX] && v[3] != -1)) {
+ ret = -EINVAL;
+ goto out_finish;
+ }
+ }
+ tg->bps[READ][index] = v[0];
+ tg->bps[WRITE][index] = v[1];
+ tg->iops[READ][index] = v[2];
+ tg->iops[WRITE][index] = v[3];
+
+ if (index == LIMIT_HIGH) {
+ blk_throtl_update_valid_limit(tg->td);
+ if (tg->td->limit_valid[LIMIT_HIGH])
+ tg->td->limit_index = LIMIT_HIGH;
+ }
tg_conf_updated(tg);
ret = 0;
out_finish:
@@ -1390,10 +1454,18 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
static struct cftype throtl_files[] = {
{
+ .name = "high",
+ .flags = CFTYPE_NOT_ON_ROOT,
+ .seq_show = tg_print_limit,
+ .write = tg_set_limit,
+ .private = LIMIT_HIGH,
+ },
+ {
.name = "max",
.flags = CFTYPE_NOT_ON_ROOT,
- .seq_show = tg_print_max,
- .write = tg_set_max,
+ .seq_show = tg_print_limit,
+ .write = tg_set_limit,
+ .private = LIMIT_MAX,
},
{ } /* terminate */
};
@@ -1412,6 +1484,7 @@ static struct blkcg_policy blkcg_policy_throtl = {
.pd_alloc_fn = throtl_pd_alloc,
.pd_init_fn = throtl_pd_init,
.pd_online_fn = throtl_pd_online,
+ .pd_offline_fn = throtl_pd_offline,
.pd_free_fn = throtl_pd_free,
};
@@ -1591,6 +1664,7 @@ int blk_throtl_init(struct request_queue *q)
td->queue = q;
td->limit_valid[LIMIT_MAX] = true;
+ td->limit_index = LIMIT_MAX;
/* activate policy */
ret = blkcg_activate_policy(q, &blkcg_policy_throtl);
if (ret)
--
2.9.3
^ permalink raw reply related
* [PATCH V4 07/15] blk-throttle: make throtl_slice tunable
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
throtl_slice is important for blk-throttling. A lot of stuffes depend on
it, for example, throughput measurement. It has 100ms default value,
which is not appropriate for all disks. For example, for SSD we might
use a smaller value to make the throughput smoother. This patch makes it
tunable.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-sysfs.c | 11 ++++++++
block/blk-throttle.c | 77 +++++++++++++++++++++++++++++++++++++---------------
block/blk.h | 3 ++
3 files changed, 69 insertions(+), 22 deletions(-)
diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index 9cc8d7c..3e284e4 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -526,6 +526,14 @@ static struct queue_sysfs_entry queue_dax_entry = {
.show = queue_dax_show,
};
+#ifdef CONFIG_BLK_DEV_THROTTLING
+static struct queue_sysfs_entry throtl_slice_entry = {
+ .attr = {.name = "throttling_slice", .mode = S_IRUGO | S_IWUSR },
+ .show = blk_throtl_slice_show,
+ .store = blk_throtl_slice_store,
+};
+#endif
+
static struct attribute *default_attrs[] = {
&queue_requests_entry.attr,
&queue_ra_entry.attr,
@@ -553,6 +561,9 @@ static struct attribute *default_attrs[] = {
&queue_poll_entry.attr,
&queue_wc_entry.attr,
&queue_dax_entry.attr,
+#ifdef CONFIG_BLK_DEV_THROTTLING
+ &throtl_slice_entry.attr,
+#endif
NULL,
};
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index eff3120..e85b2b6 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -19,7 +19,8 @@ static int throtl_grp_quantum = 8;
static int throtl_quantum = 32;
/* Throttling is performed over 100ms slice and after that slice is renewed */
-static unsigned long throtl_slice = HZ/10; /* 100 ms */
+#define DFL_THROTL_SLICE (HZ / 10)
+#define MAX_THROTL_SLICE (HZ / 5)
static struct blkcg_policy blkcg_policy_throtl;
@@ -158,6 +159,8 @@ struct throtl_data
/* Total Number of queued bios on READ and WRITE lists */
unsigned int nr_queued[2];
+ unsigned int throtl_slice;
+
/* Work for dispatching throttled bios */
struct work_struct dispatch_work;
unsigned int limit_index;
@@ -589,7 +592,7 @@ static void throtl_dequeue_tg(struct throtl_grp *tg)
static void throtl_schedule_pending_timer(struct throtl_service_queue *sq,
unsigned long expires)
{
- unsigned long max_expire = jiffies + 8 * throtl_slice;
+ unsigned long max_expire = jiffies + 8 * sq_to_tg(sq)->td->throtl_slice;
if (time_after(expires, max_expire))
expires = max_expire;
@@ -650,7 +653,7 @@ static inline void throtl_start_new_slice_with_credit(struct throtl_grp *tg,
if (time_after_eq(start, tg->slice_start[rw]))
tg->slice_start[rw] = start;
- tg->slice_end[rw] = jiffies + throtl_slice;
+ tg->slice_end[rw] = jiffies + tg->td->throtl_slice;
throtl_log(&tg->service_queue,
"[%c] new slice with credit start=%lu end=%lu jiffies=%lu",
rw == READ ? 'R' : 'W', tg->slice_start[rw],
@@ -662,7 +665,7 @@ static inline void throtl_start_new_slice(struct throtl_grp *tg, bool rw)
tg->bytes_disp[rw] = 0;
tg->io_disp[rw] = 0;
tg->slice_start[rw] = jiffies;
- tg->slice_end[rw] = jiffies + throtl_slice;
+ tg->slice_end[rw] = jiffies + tg->td->throtl_slice;
throtl_log(&tg->service_queue,
"[%c] new slice start=%lu end=%lu jiffies=%lu",
rw == READ ? 'R' : 'W', tg->slice_start[rw],
@@ -672,13 +675,13 @@ static inline void throtl_start_new_slice(struct throtl_grp *tg, bool rw)
static inline void throtl_set_slice_end(struct throtl_grp *tg, bool rw,
unsigned long jiffy_end)
{
- tg->slice_end[rw] = roundup(jiffy_end, throtl_slice);
+ tg->slice_end[rw] = roundup(jiffy_end, tg->td->throtl_slice);
}
static inline void throtl_extend_slice(struct throtl_grp *tg, bool rw,
unsigned long jiffy_end)
{
- tg->slice_end[rw] = roundup(jiffy_end, throtl_slice);
+ tg->slice_end[rw] = roundup(jiffy_end, tg->td->throtl_slice);
throtl_log(&tg->service_queue,
"[%c] extend slice start=%lu end=%lu jiffies=%lu",
rw == READ ? 'R' : 'W', tg->slice_start[rw],
@@ -718,19 +721,20 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw)
* is bad because it does not allow new slice to start.
*/
- throtl_set_slice_end(tg, rw, jiffies + throtl_slice);
+ throtl_set_slice_end(tg, rw, jiffies + tg->td->throtl_slice);
time_elapsed = jiffies - tg->slice_start[rw];
- nr_slices = time_elapsed / throtl_slice;
+ nr_slices = time_elapsed / tg->td->throtl_slice;
if (!nr_slices)
return;
- tmp = tg_bps_limit(tg, rw) * throtl_slice * nr_slices;
+ tmp = tg_bps_limit(tg, rw) * tg->td->throtl_slice * nr_slices;
do_div(tmp, HZ);
bytes_trim = tmp;
- io_trim = (tg_iops_limit(tg, rw) * throtl_slice * nr_slices) / HZ;
+ io_trim = (tg_iops_limit(tg, rw) * tg->td->throtl_slice * nr_slices) /
+ HZ;
if (!bytes_trim && !io_trim)
return;
@@ -745,7 +749,7 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw)
else
tg->io_disp[rw] = 0;
- tg->slice_start[rw] += nr_slices * throtl_slice;
+ tg->slice_start[rw] += nr_slices * tg->td->throtl_slice;
throtl_log(&tg->service_queue,
"[%c] trim slice nr=%lu bytes=%llu io=%lu start=%lu end=%lu jiffies=%lu",
@@ -765,9 +769,9 @@ static bool tg_with_in_iops_limit(struct throtl_grp *tg, struct bio *bio,
/* Slice has just started. Consider one slice interval */
if (!jiffy_elapsed)
- jiffy_elapsed_rnd = throtl_slice;
+ jiffy_elapsed_rnd = tg->td->throtl_slice;
- jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, throtl_slice);
+ jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, tg->td->throtl_slice);
/*
* jiffy_elapsed_rnd should not be a big value as minimum iops can be
@@ -814,9 +818,9 @@ static bool tg_with_in_bps_limit(struct throtl_grp *tg, struct bio *bio,
/* Slice has just started. Consider one slice interval */
if (!jiffy_elapsed)
- jiffy_elapsed_rnd = throtl_slice;
+ jiffy_elapsed_rnd = tg->td->throtl_slice;
- jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, throtl_slice);
+ jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, tg->td->throtl_slice);
tmp = tg_bps_limit(tg, rw) * jiffy_elapsed_rnd;
do_div(tmp, HZ);
@@ -881,8 +885,10 @@ static bool tg_may_dispatch(struct throtl_grp *tg, struct bio *bio,
if (throtl_slice_used(tg, rw) && !(tg->service_queue.nr_queued[rw]))
throtl_start_new_slice(tg, rw);
else {
- if (time_before(tg->slice_end[rw], jiffies + throtl_slice))
- throtl_extend_slice(tg, rw, jiffies + throtl_slice);
+ if (time_before(tg->slice_end[rw],
+ jiffies + tg->td->throtl_slice))
+ throtl_extend_slice(tg, rw,
+ jiffies + tg->td->throtl_slice);
}
if (tg_with_in_bps_limit(tg, bio, &bps_wait) &&
@@ -1632,7 +1638,7 @@ static bool throtl_can_upgrade(struct throtl_data *td,
if (td->limit_index != LIMIT_HIGH)
return false;
- if (time_before(jiffies, td->high_downgrade_time + throtl_slice))
+ if (time_before(jiffies, td->high_downgrade_time + td->throtl_slice))
return false;
rcu_read_lock();
@@ -1689,8 +1695,9 @@ static bool throtl_downgrade_check_one(struct throtl_grp *tg)
* If cgroup is below high limit, consider downgrade and throttle other
* cgroups
*/
- if (time_after_eq(now, td->high_upgrade_time + throtl_slice) &&
- time_after_eq(now, tg_last_high_overflow_time(tg) + throtl_slice))
+ if (time_after_eq(now, td->high_upgrade_time + td->throtl_slice) &&
+ time_after_eq(now, tg_last_high_overflow_time(tg) +
+ td->throtl_slice))
return true;
return false;
}
@@ -1723,10 +1730,11 @@ static void throtl_downgrade_check(struct throtl_grp *tg)
return;
if (!list_empty(&tg_to_blkg(tg)->blkcg->css.children))
return;
- if (time_after(tg->last_check_time + throtl_slice, now))
+ if (time_after(tg->last_check_time + tg->td->throtl_slice, now))
return;
- if (time_before(now, tg_last_high_overflow_time(tg) + throtl_slice))
+ if (time_before(now, tg_last_high_overflow_time(tg) +
+ tg->td->throtl_slice))
return;
elapsed_time = now - tg->last_check_time;
@@ -1965,6 +1973,7 @@ int blk_throtl_init(struct request_queue *q)
q->td = td;
td->queue = q;
+ td->throtl_slice = DFL_THROTL_SLICE;
td->limit_valid[LIMIT_MAX] = true;
td->limit_index = LIMIT_MAX;
@@ -1985,6 +1994,30 @@ void blk_throtl_exit(struct request_queue *q)
kfree(q->td);
}
+ssize_t blk_throtl_slice_show(struct request_queue *q, char *page)
+{
+ if (!q->td)
+ return -EINVAL;
+ return sprintf(page, "%ums\n", jiffies_to_msecs(q->td->throtl_slice));
+}
+
+ssize_t blk_throtl_slice_store(struct request_queue *q,
+ const char *page, size_t count)
+{
+ unsigned long v;
+ unsigned long t;
+
+ if (!q->td)
+ return -EINVAL;
+ if (kstrtoul(page, 10, &v))
+ return -EINVAL;
+ t = msecs_to_jiffies(v);
+ if (t == 0 || t > MAX_THROTL_SLICE)
+ return -EINVAL;
+ q->td->throtl_slice = t;
+ return count;
+}
+
static int __init throtl_init(void)
{
kthrotld_workqueue = alloc_workqueue("kthrotld", WQ_MEM_RECLAIM, 0);
diff --git a/block/blk.h b/block/blk.h
index 74444c4..39c14dd 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -289,6 +289,9 @@ static inline struct io_context *create_io_context(gfp_t gfp_mask, int node)
extern void blk_throtl_drain(struct request_queue *q);
extern int blk_throtl_init(struct request_queue *q);
extern void blk_throtl_exit(struct request_queue *q);
+extern ssize_t blk_throtl_slice_show(struct request_queue *q, char *page);
+extern ssize_t blk_throtl_slice_store(struct request_queue *q,
+ const char *page, size_t count);
#else /* CONFIG_BLK_DEV_THROTTLING */
static inline void blk_throtl_drain(struct request_queue *q) { }
static inline int blk_throtl_init(struct request_queue *q) { return 0; }
--
2.9.3
^ permalink raw reply related
* [PATCH V4 01/15] blk-throttle: prepare support multiple limits
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
We are going to support high/max limit, each cgroup will have 2 limits
after that. This patch prepares for the multiple limits change.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 110 ++++++++++++++++++++++++++++++++-------------------
1 file changed, 69 insertions(+), 41 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index a3ea826..925aa1ed 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -83,6 +83,11 @@ enum tg_state_flags {
#define rb_entry_tg(node) rb_entry((node), struct throtl_grp, rb_node)
+enum {
+ LIMIT_MAX,
+ LIMIT_CNT,
+};
+
struct throtl_grp {
/* must be the first member */
struct blkg_policy_data pd;
@@ -120,10 +125,10 @@ struct throtl_grp {
bool has_rules[2];
/* bytes per second rate limits */
- uint64_t bps[2];
+ uint64_t bps[2][LIMIT_CNT];
/* IOPS limits */
- unsigned int iops[2];
+ unsigned int iops[2][LIMIT_CNT];
/* Number of bytes disptached in current slice */
uint64_t bytes_disp[2];
@@ -147,6 +152,8 @@ struct throtl_data
/* Work for dispatching throttled bios */
struct work_struct dispatch_work;
+ unsigned int limit_index;
+ bool limit_valid[LIMIT_CNT];
};
static void throtl_pending_timer_fn(unsigned long arg);
@@ -198,6 +205,16 @@ static struct throtl_data *sq_to_td(struct throtl_service_queue *sq)
return container_of(sq, struct throtl_data, service_queue);
}
+static uint64_t tg_bps_limit(struct throtl_grp *tg, int rw)
+{
+ return tg->bps[rw][tg->td->limit_index];
+}
+
+static unsigned int tg_iops_limit(struct throtl_grp *tg, int rw)
+{
+ return tg->iops[rw][tg->td->limit_index];
+}
+
/**
* throtl_log - log debug message via blktrace
* @sq: the service_queue being reported
@@ -320,7 +337,7 @@ static void throtl_service_queue_init(struct throtl_service_queue *sq)
static struct blkg_policy_data *throtl_pd_alloc(gfp_t gfp, int node)
{
struct throtl_grp *tg;
- int rw;
+ int rw, index;
tg = kzalloc_node(sizeof(*tg), gfp, node);
if (!tg)
@@ -334,10 +351,12 @@ static struct blkg_policy_data *throtl_pd_alloc(gfp_t gfp, int node)
}
RB_CLEAR_NODE(&tg->rb_node);
- tg->bps[READ] = -1;
- tg->bps[WRITE] = -1;
- tg->iops[READ] = -1;
- tg->iops[WRITE] = -1;
+ for (rw = READ; rw <= WRITE; rw++) {
+ for (index = 0; index < LIMIT_CNT; index++) {
+ tg->bps[rw][index] = -1;
+ tg->iops[rw][index] = -1;
+ }
+ }
return &tg->pd;
}
@@ -376,11 +395,14 @@ static void throtl_pd_init(struct blkg_policy_data *pd)
static void tg_update_has_rules(struct throtl_grp *tg)
{
struct throtl_grp *parent_tg = sq_to_tg(tg->service_queue.parent_sq);
+ struct throtl_data *td = tg->td;
int rw;
for (rw = READ; rw <= WRITE; rw++)
tg->has_rules[rw] = (parent_tg && parent_tg->has_rules[rw]) ||
- (tg->bps[rw] != -1 || tg->iops[rw] != -1);
+ (td->limit_valid[td->limit_index] &&
+ (tg_bps_limit(tg, rw) != -1 ||
+ tg_iops_limit(tg, rw) != -1));
}
static void throtl_pd_online(struct blkg_policy_data *pd)
@@ -632,11 +654,11 @@ static inline void throtl_trim_slice(struct throtl_grp *tg, bool rw)
if (!nr_slices)
return;
- tmp = tg->bps[rw] * throtl_slice * nr_slices;
+ tmp = tg_bps_limit(tg, rw) * throtl_slice * nr_slices;
do_div(tmp, HZ);
bytes_trim = tmp;
- io_trim = (tg->iops[rw] * throtl_slice * nr_slices)/HZ;
+ io_trim = (tg_iops_limit(tg, rw) * throtl_slice * nr_slices) / HZ;
if (!bytes_trim && !io_trim)
return;
@@ -682,7 +704,7 @@ static bool tg_with_in_iops_limit(struct throtl_grp *tg, struct bio *bio,
* have been trimmed.
*/
- tmp = (u64)tg->iops[rw] * jiffy_elapsed_rnd;
+ tmp = (u64)tg_iops_limit(tg, rw) * jiffy_elapsed_rnd;
do_div(tmp, HZ);
if (tmp > UINT_MAX)
@@ -697,7 +719,7 @@ static bool tg_with_in_iops_limit(struct throtl_grp *tg, struct bio *bio,
}
/* Calc approx time to dispatch */
- jiffy_wait = ((tg->io_disp[rw] + 1) * HZ)/tg->iops[rw] + 1;
+ jiffy_wait = ((tg->io_disp[rw] + 1) * HZ) / tg_iops_limit(tg, rw) + 1;
if (jiffy_wait > jiffy_elapsed)
jiffy_wait = jiffy_wait - jiffy_elapsed;
@@ -724,7 +746,7 @@ static bool tg_with_in_bps_limit(struct throtl_grp *tg, struct bio *bio,
jiffy_elapsed_rnd = roundup(jiffy_elapsed_rnd, throtl_slice);
- tmp = tg->bps[rw] * jiffy_elapsed_rnd;
+ tmp = tg_bps_limit(tg, rw) * jiffy_elapsed_rnd;
do_div(tmp, HZ);
bytes_allowed = tmp;
@@ -736,7 +758,7 @@ static bool tg_with_in_bps_limit(struct throtl_grp *tg, struct bio *bio,
/* Calc approx time to dispatch */
extra_bytes = tg->bytes_disp[rw] + bio->bi_iter.bi_size - bytes_allowed;
- jiffy_wait = div64_u64(extra_bytes * HZ, tg->bps[rw]);
+ jiffy_wait = div64_u64(extra_bytes * HZ, tg_bps_limit(tg, rw));
if (!jiffy_wait)
jiffy_wait = 1;
@@ -771,7 +793,7 @@ static bool tg_may_dispatch(struct throtl_grp *tg, struct bio *bio,
bio != throtl_peek_queued(&tg->service_queue.queued[rw]));
/* If tg->bps = -1, then BW is unlimited */
- if (tg->bps[rw] == -1 && tg->iops[rw] == -1) {
+ if (tg_bps_limit(tg, rw) == -1 && tg_iops_limit(tg, rw) == -1) {
if (wait)
*wait = 0;
return true;
@@ -1148,8 +1170,8 @@ static void tg_conf_updated(struct throtl_grp *tg)
throtl_log(&tg->service_queue,
"limit change rbps=%llu wbps=%llu riops=%u wiops=%u",
- tg->bps[READ], tg->bps[WRITE],
- tg->iops[READ], tg->iops[WRITE]);
+ tg_bps_limit(tg, READ), tg_bps_limit(tg, WRITE),
+ tg_iops_limit(tg, READ), tg_iops_limit(tg, WRITE));
/*
* Update has_rules[] flags for the updated tg's subtree. A tg is
@@ -1226,25 +1248,25 @@ static ssize_t tg_set_conf_uint(struct kernfs_open_file *of,
static struct cftype throtl_legacy_files[] = {
{
.name = "throttle.read_bps_device",
- .private = offsetof(struct throtl_grp, bps[READ]),
+ .private = offsetof(struct throtl_grp, bps[READ][LIMIT_MAX]),
.seq_show = tg_print_conf_u64,
.write = tg_set_conf_u64,
},
{
.name = "throttle.write_bps_device",
- .private = offsetof(struct throtl_grp, bps[WRITE]),
+ .private = offsetof(struct throtl_grp, bps[WRITE][LIMIT_MAX]),
.seq_show = tg_print_conf_u64,
.write = tg_set_conf_u64,
},
{
.name = "throttle.read_iops_device",
- .private = offsetof(struct throtl_grp, iops[READ]),
+ .private = offsetof(struct throtl_grp, iops[READ][LIMIT_MAX]),
.seq_show = tg_print_conf_uint,
.write = tg_set_conf_uint,
},
{
.name = "throttle.write_iops_device",
- .private = offsetof(struct throtl_grp, iops[WRITE]),
+ .private = offsetof(struct throtl_grp, iops[WRITE][LIMIT_MAX]),
.seq_show = tg_print_conf_uint,
.write = tg_set_conf_uint,
},
@@ -1270,18 +1292,22 @@ static u64 tg_prfill_max(struct seq_file *sf, struct blkg_policy_data *pd,
if (!dname)
return 0;
- if (tg->bps[READ] == -1 && tg->bps[WRITE] == -1 &&
- tg->iops[READ] == -1 && tg->iops[WRITE] == -1)
+ if (tg->bps[READ][LIMIT_MAX] == -1 && tg->bps[WRITE][LIMIT_MAX] == -1 &&
+ tg->iops[READ][LIMIT_MAX] == -1 && tg->iops[WRITE][LIMIT_MAX] == -1)
return 0;
- if (tg->bps[READ] != -1)
- snprintf(bufs[0], sizeof(bufs[0]), "%llu", tg->bps[READ]);
- if (tg->bps[WRITE] != -1)
- snprintf(bufs[1], sizeof(bufs[1]), "%llu", tg->bps[WRITE]);
- if (tg->iops[READ] != -1)
- snprintf(bufs[2], sizeof(bufs[2]), "%u", tg->iops[READ]);
- if (tg->iops[WRITE] != -1)
- snprintf(bufs[3], sizeof(bufs[3]), "%u", tg->iops[WRITE]);
+ if (tg->bps[READ][LIMIT_MAX] != -1)
+ snprintf(bufs[0], sizeof(bufs[0]), "%llu",
+ tg->bps[READ][LIMIT_MAX]);
+ if (tg->bps[WRITE][LIMIT_MAX] != -1)
+ snprintf(bufs[1], sizeof(bufs[1]), "%llu",
+ tg->bps[WRITE][LIMIT_MAX]);
+ if (tg->iops[READ][LIMIT_MAX] != -1)
+ snprintf(bufs[2], sizeof(bufs[2]), "%u",
+ tg->iops[READ][LIMIT_MAX]);
+ if (tg->iops[WRITE][LIMIT_MAX] != -1)
+ snprintf(bufs[3], sizeof(bufs[3]), "%u",
+ tg->iops[WRITE][LIMIT_MAX]);
seq_printf(sf, "%s rbps=%s wbps=%s riops=%s wiops=%s\n",
dname, bufs[0], bufs[1], bufs[2], bufs[3]);
@@ -1310,10 +1336,10 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
tg = blkg_to_tg(ctx.blkg);
- v[0] = tg->bps[READ];
- v[1] = tg->bps[WRITE];
- v[2] = tg->iops[READ];
- v[3] = tg->iops[WRITE];
+ v[0] = tg->bps[READ][LIMIT_MAX];
+ v[1] = tg->bps[WRITE][LIMIT_MAX];
+ v[2] = tg->iops[READ][LIMIT_MAX];
+ v[3] = tg->iops[WRITE][LIMIT_MAX];
while (true) {
char tok[27]; /* wiops=18446744073709551616 */
@@ -1350,10 +1376,10 @@ static ssize_t tg_set_max(struct kernfs_open_file *of,
goto out_finish;
}
- tg->bps[READ] = v[0];
- tg->bps[WRITE] = v[1];
- tg->iops[READ] = v[2];
- tg->iops[WRITE] = v[3];
+ tg->bps[READ][LIMIT_MAX] = v[0];
+ tg->bps[WRITE][LIMIT_MAX] = v[1];
+ tg->iops[READ][LIMIT_MAX] = v[2];
+ tg->iops[WRITE][LIMIT_MAX] = v[3];
tg_conf_updated(tg);
ret = 0;
@@ -1451,8 +1477,9 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
/* out-of-limit, queue to @tg */
throtl_log(sq, "[%c] bio. bdisp=%llu sz=%u bps=%llu iodisp=%u iops=%u queued=%d/%d",
rw == READ ? 'R' : 'W',
- tg->bytes_disp[rw], bio->bi_iter.bi_size, tg->bps[rw],
- tg->io_disp[rw], tg->iops[rw],
+ tg->bytes_disp[rw], bio->bi_iter.bi_size,
+ tg_bps_limit(tg, rw),
+ tg->io_disp[rw], tg_iops_limit(tg, rw),
sq->nr_queued[READ], sq->nr_queued[WRITE]);
bio_associate_current(bio);
@@ -1563,6 +1590,7 @@ int blk_throtl_init(struct request_queue *q)
q->td = td;
td->queue = q;
+ td->limit_valid[LIMIT_MAX] = true;
/* activate policy */
ret = blkcg_activate_policy(q, &blkcg_policy_throtl);
if (ret)
--
2.9.3
^ permalink raw reply related
* [PATCH V4 06/15] blk-throttle: make sure expire time isn't too big
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
cgroup could be throttled to a limit but when all cgroups cross high
limit, queue enters a higher state and so the group should be throttled
to a higher limit. It's possible the cgroup is sleeping because of
throttle and other cgroups don't dispatch IO any more. In this case,
nobody can trigger current downgrade/upgrade logic. To fix this issue,
we could either set up a timer to wakeup the cgroup if other cgroups are
idle or make sure this cgroup doesn't sleep too long. Setting up a timer
means we must change the timer very frequently. This patch chooses the
latter. Making cgroup sleep time not too big wouldn't change cgroup
bps/iops, but could make it wakeup more frequently, which isn't a big
issue because throtl_slice * 8 is already quite big.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index d177252..eff3120 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -589,6 +589,10 @@ static void throtl_dequeue_tg(struct throtl_grp *tg)
static void throtl_schedule_pending_timer(struct throtl_service_queue *sq,
unsigned long expires)
{
+ unsigned long max_expire = jiffies + 8 * throtl_slice;
+
+ if (time_after(expires, max_expire))
+ expires = max_expire;
mod_timer(&sq->pending_timer, expires);
throtl_log(sq, "schedule timer. delay=%lu jiffies=%lu",
expires - jiffies, jiffies);
--
2.9.3
^ permalink raw reply related
* [PATCH V4 03/15] blk-throttle: configure bps/iops limit for cgroup in high limit
From: Shaohua Li @ 2016-11-14 22:22 UTC (permalink / raw)
To: linux-block, linux-kernel; +Cc: Kernel-team, axboe, tj, vgoyal
In-Reply-To: <cover.1479161136.git.shli@fb.com>
each queue will have a state machine. Initially queue is in LIMIT_HIGH
state, which means all cgroups will be throttled according to their high
limit. After all cgroups with high limit cross the limit, the queue state
gets upgraded to LIMIT_MAX state.
cgroups without high limit will use max limit for their high limit.
Signed-off-by: Shaohua Li <shli@fb.com>
---
block/blk-throttle.c | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index a564215..ec53671 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -208,12 +208,29 @@ static struct throtl_data *sq_to_td(struct throtl_service_queue *sq)
static uint64_t tg_bps_limit(struct throtl_grp *tg, int rw)
{
- return tg->bps[rw][tg->td->limit_index];
+ struct blkcg_gq *blkg = tg_to_blkg(tg);
+ uint64_t ret;
+
+ if (cgroup_subsys_on_dfl(io_cgrp_subsys) && !blkg->parent)
+ return -1;
+ ret = tg->bps[rw][tg->td->limit_index];
+ if (ret == -1 && tg->td->limit_index == LIMIT_HIGH)
+ return tg->bps[rw][LIMIT_MAX];
+
+ return ret;
}
static unsigned int tg_iops_limit(struct throtl_grp *tg, int rw)
{
- return tg->iops[rw][tg->td->limit_index];
+ struct blkcg_gq *blkg = tg_to_blkg(tg);
+ unsigned int ret;
+
+ if (cgroup_subsys_on_dfl(io_cgrp_subsys) && !blkg->parent)
+ return -1;
+ ret = tg->iops[rw][tg->td->limit_index];
+ if (ret == -1 && tg->td->limit_index == LIMIT_HIGH)
+ return tg->iops[rw][LIMIT_MAX];
+ return ret;
}
/**
--
2.9.3
^ permalink raw reply related
* Re: [PATCH v6 6/9] tpm: fix the missing .owner in tpm_bios_measurements_ops
From: Jarkko Sakkinen @ 2016-11-14 22:22 UTC (permalink / raw)
To: Nayna Jain
Cc: tpmdd-devel, peterhuewe, tpmdd, jgunthorpe, linux-kernel,
linux-security-module
In-Reply-To: <1479117656-12403-7-git-send-email-nayna@linux.vnet.ibm.com>
On Mon, Nov 14, 2016 at 05:00:53AM -0500, Nayna Jain wrote:
> This patch fixes the missing .owner field in
> tpm_bios_measurements_ops definition.
>
> Suggested-by: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
> Signed-off-by: Nayna Jain <nayna@linux.vnet.ibm.com>
Reviewed-by: Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>
/Jarkko
> ---
> drivers/char/tpm/tpm_eventlog.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/drivers/char/tpm/tpm_eventlog.c b/drivers/char/tpm/tpm_eventlog.c
> index f8c42fe..5575ffc 100644
> --- a/drivers/char/tpm/tpm_eventlog.c
> +++ b/drivers/char/tpm/tpm_eventlog.c
> @@ -349,6 +349,7 @@ static int tpm_bios_measurements_open(struct inode *inode,
> }
>
> static const struct file_operations tpm_bios_measurements_ops = {
> + .owner = THIS_MODULE,
> .open = tpm_bios_measurements_open,
> .read = seq_read,
> .llseek = seq_lseek,
> --
> 2.5.0
>
^ permalink raw reply
* Re: [PATCH nft 0/3] src: add nft log flags support
From: Pablo Neira Ayuso @ 2016-11-14 22:21 UTC (permalink / raw)
To: Liping Zhang; +Cc: netfilter-devel, Liping Zhang
In-Reply-To: <1474794421-5365-1-git-send-email-zlpnobody@163.com>
[-- Attachment #1: Type: text/plain, Size: 1099 bytes --]
On Sun, Sep 25, 2016 at 05:06:58PM +0800, Liping Zhang wrote:
> From: Liping Zhang <liping.zhang@spreadtrum.com>
>
> After NF_LOG_XXX is exposed to the userspace, we can set log flags to
> log more things. The following iptables rule:
> # iptables -A OUTPUT -j LOG --log-tcp-sequence --log-tcp-options \
> --log-ip-options --log-uid --log-macdecode
> is equal to the following nft rule:
> # nft add rule filter OUTPUT log tcpseq,tcpopt,ipopt,uid,macdecode
Sorry, I wanted to have a closer look at this but time has been
running up and I didn't manage to get back to this.
So basically, I would like to explore different syntax for this, eg.
log flags tcp sequence,options
log flags ip options
log flags skuid
log flags ether
I think syntax would be larger, but it would look more consistent to
what we have. Worst case is to get them all set. We can provide a
compact version for this:
log flags all
Please, see sketch patch attached for brainstorming.
Would you have a look into this? Thanks and again sorry for not
getting any sooner on this.
[-- Attachment #2: x.patch --]
[-- Type: text/x-diff, Size: 1416 bytes --]
diff --git a/src/parser_bison.y b/src/parser_bison.y
index 91955c187f3f..286290341ffb 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -201,6 +201,8 @@ static void location_update(struct location *loc, struct location *rhs, int n)
%token EXPORT "export"
%token MONITOR "monitor"
+%token ALL "all"
+
%token ACCEPT "accept"
%token DROP "drop"
%token CONTINUE "continue"
@@ -268,6 +270,8 @@ static void location_update(struct location *loc, struct location *rhs, int n)
%token GATEWAY "gateway"
%token MTU "mtu"
+%token OPTIONS "options"
+
%token IP6 "ip6"
%token PRIORITY "priority"
%token FLOWLABEL "flowlabel"
@@ -1530,6 +1534,25 @@ log_arg : PREFIX string
$<stmt>0->log.level = $2;
$<stmt>0->log.flags |= STMT_LOG_LEVEL;
}
+ | FLAGS log_flags
+ {
+ ;
+ }
+ ;
+
+log_flags : TCP log_flags_tcp
+ | IP OPTIONS
+ | SKUID
+ | ETHER
+ | ALL
+ ;
+
+log_flags_tcp : log_flags_tcp COMMA log_flag_tcp
+ | log_flag_tcp
+ ;
+
+log_flag_tcp : SEQUENCE
+ | OPTIONS
;
level_type : string
diff --git a/src/scanner.l b/src/scanner.l
index cd7398b4e534..625023f5257c 100644
--- a/src/scanner.l
+++ b/src/scanner.l
@@ -469,6 +469,9 @@ addrstring ({macaddr}|{ip4addr}|{ip6addr})
"notrack" { return NOTRACK; }
+"options" { return OPTIONS; }
+"all" { return ALL; }
+
"xml" { return XML; }
"json" { return JSON; }
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.