All of lore.kernel.org
 help / color / mirror / Atom feed
* [oe][Patch] systemd: systemd should not depend on systemd-container
From: Max Krummenacher @ 2016-11-14 19:35 UTC (permalink / raw)
  To: openembedded-core; +Cc: Max Krummenacher

Currently systemd depends on systemd-container due to a dangling
symlink deployed with systemd.
Move the symlink to systemd-container.

| DEBUG: systemd contains dangling link /lib/systemd/system/systemd-machined.service
| DEBUG: target found in systemd-container

Signed-off-by: Max Krummenacher <max.krummenacher@toradex.com>
---
 meta/recipes-core/systemd/systemd_230.bb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/recipes-core/systemd/systemd_230.bb b/meta/recipes-core/systemd/systemd_230.bb
index a949fe8..cc196d1 100644
--- a/meta/recipes-core/systemd/systemd_230.bb
+++ b/meta/recipes-core/systemd/systemd_230.bb
@@ -369,6 +369,7 @@ FILES_${PN}-container = "${sysconfdir}/dbus-1/system.d/org.freedesktop.import1.c
                          ${systemd_system_unitdir}/org.freedesktop.machine1.busname \
                          ${systemd_system_unitdir}/systemd-importd.service \
                          ${systemd_system_unitdir}/systemd-machined.service \
+                         ${systemd_system_unitdir}/dbus-org.freedesktop.machine1.service \
                          ${systemd_system_unitdir}/var-lib-machines.mount \
                          ${rootlibexecdir}/systemd/systemd-import \
                          ${rootlibexecdir}/systemd/systemd-importd \
-- 
2.5.5



^ permalink raw reply related

* [PATCH v5 00/23] Support qcom's HSIC USB and rewrite USB2 HS support
From: Stephen Boyd @ 2016-11-14 19:35 UTC (permalink / raw)
  To: linux-arm-kernel
In-Reply-To: <20161111074017.GA21045@b29397-desktop>

Quoting Peter Chen (2016-11-10 23:40:17)
> On Tue, Oct 18, 2016 at 01:51:39PM -0700, Stephen Boyd wrote:
> > Quoting Peter Chen (2016-10-18 02:31:40)
> > > On Mon, Oct 17, 2016 at 06:56:13PM -0700, Stephen Boyd wrote:
> > > > I've also sent separate patches for other minor pieces to make this
> > > > all work. The full tree can be found here[2], hacks and all to get
> > > > things working. I've tested this on the db410c, apq8074 dragonboard,
> > > > and ifc6410 with configfs gadgets and otg cables.
> > > > 
> > > > Patches based on v4.8-rc1
> > 
> > Oops should be v4.9-rc1 here.
> > 
> > > > 
> > > > Changes from v4:
> > > >  * Picked up Acks from Rob
> > > >  * Updated HS phy init sequence DT property to restrict it to offsets
> > > 
> > > I remembered that you got all my acks for chipidea patches, right? I did
> > > not check for this series.
> > 
> > Sorry I've added in one more patch
> > 
> >    usb: chipidea: Emulate OTGSC interrupt enable path
> > 
> > to fix extcon interrupt emulation even further.
> > 
> > > 
> > > Besides, the patch "gpu: Remove depends on RESET_CONTROLLER when not a
> > > provider" [1]  still not be accepted, I need this patch to be merged
> > > first, then apply your chipidea part, otherwise, there is a building
> > > warning.
> > > 
> > > [1] https://patchwork.kernel.org/patch/9322583/
> > 
> > Yes, I'm going to resend that patch now. I hope that David will apply it
> > for -rc2.
> 
> Stephen, just a mind. I have rebased Greg's usb-next tree (v4.9-rc3 on
> it), your GPU fix is still not there.
> 

It looks like the patch is in drm-misc. I think they're going to hold
off on merging it until the next merge window though. fb80016af071 is
the commit in drm-misc tree and in linux-next. I'm not sure anything can
be done here besides a cross tree merge or ignore the warning?

^ permalink raw reply

* Re: [PATCH v5 00/23] Support qcom's HSIC USB and rewrite USB2 HS support
From: Stephen Boyd @ 2016-11-14 19:35 UTC (permalink / raw)
  To: Peter Chen
  Cc: Felipe Balbi, Heikki Krogerus, Arnd Bergmann, Neil Armstrong,
	linux-arm-msm, linux-usb, linux-kernel, Bjorn Andersson,
	devicetree, Rob Herring, Peter Chen, Greg Kroah-Hartman,
	Andy Gross, Ivan T. Ivanov, Kishon Vijay Abraham I,
	linux-arm-kernel
In-Reply-To: <20161111074017.GA21045@b29397-desktop>

Quoting Peter Chen (2016-11-10 23:40:17)
> On Tue, Oct 18, 2016 at 01:51:39PM -0700, Stephen Boyd wrote:
> > Quoting Peter Chen (2016-10-18 02:31:40)
> > > On Mon, Oct 17, 2016 at 06:56:13PM -0700, Stephen Boyd wrote:
> > > > I've also sent separate patches for other minor pieces to make this
> > > > all work. The full tree can be found here[2], hacks and all to get
> > > > things working. I've tested this on the db410c, apq8074 dragonboard,
> > > > and ifc6410 with configfs gadgets and otg cables.
> > > > 
> > > > Patches based on v4.8-rc1
> > 
> > Oops should be v4.9-rc1 here.
> > 
> > > > 
> > > > Changes from v4:
> > > >  * Picked up Acks from Rob
> > > >  * Updated HS phy init sequence DT property to restrict it to offsets
> > > 
> > > I remembered that you got all my acks for chipidea patches, right? I did
> > > not check for this series.
> > 
> > Sorry I've added in one more patch
> > 
> >    usb: chipidea: Emulate OTGSC interrupt enable path
> > 
> > to fix extcon interrupt emulation even further.
> > 
> > > 
> > > Besides, the patch "gpu: Remove depends on RESET_CONTROLLER when not a
> > > provider" [1]  still not be accepted, I need this patch to be merged
> > > first, then apply your chipidea part, otherwise, there is a building
> > > warning.
> > > 
> > > [1] https://patchwork.kernel.org/patch/9322583/
> > 
> > Yes, I'm going to resend that patch now. I hope that David will apply it
> > for -rc2.
> 
> Stephen, just a mind. I have rebased Greg's usb-next tree (v4.9-rc3 on
> it), your GPU fix is still not there.
> 

It looks like the patch is in drm-misc. I think they're going to hold
off on merging it until the next merge window though. fb80016af071 is
the commit in drm-misc tree and in linux-next. I'm not sure anything can
be done here besides a cross tree merge or ignore the warning?

^ permalink raw reply

* Re: [PATCH RFC] mm: Add debug_virt_to_phys()
From: Florian Fainelli @ 2016-11-14 19:23 UTC (permalink / raw)
  To: Laura Abbott, linux-kernel
  Cc: Russell King, Catalin Marinas, Will Deacon, Arnd Bergmann,
	Nicolas Pitre, Chris Brandt, Pratyush Anand, Ard Biesheuvel,
	Mark Rutland, James Morse, Neeraj Upadhyay, Andrew Morton,
	Vlastimil Babka, Michal Hocko, Kirill A. Shutemov,
	Jerome Marchand, Konstantin Khlebnikov, Joonsoo Kim,
	moderated list:ARM PORT,
	open list:GENERIC INCLUDE/ASM HEADER FILES,
	open list:MEMORY MANAGEMENT
In-Reply-To: <aee5072c-4e09-547e-2f58-7b2106bced3c@redhat.com>

On 11/14/2016 10:45 AM, Laura Abbott wrote:
> On 11/11/2016 04:44 PM, Florian Fainelli wrote:
>> When CONFIG_DEBUG_VM is turned on, virt_to_phys() maps to
>> debug_virt_to_phys() which helps catch vmalloc space addresses being
>> passed. This is helpful in debugging bogus drivers that just assume
>> linear mappings all over the place.
>>
>> For ARM, ARM64, Unicore32 and Microblaze, the architectures define
>> __virt_to_phys() as being the functional implementation of the address
>> translation, so we special case the debug stub to call into
>> __virt_to_phys directly.
>>
>> Signed-off-by: Florian Fainelli <f.fainelli@gmail.com>
>> ---
>>  arch/arm/include/asm/memory.h      |  4 ++++
>>  arch/arm64/include/asm/memory.h    |  4 ++++
>>  include/asm-generic/memory_model.h |  4 ++++
>>  mm/debug.c                         | 15 +++++++++++++++
>>  4 files changed, 27 insertions(+)
>>
>> diff --git a/arch/arm/include/asm/memory.h b/arch/arm/include/asm/memory.h
>> index 76cbd9c674df..448dec9b8b00 100644
>> --- a/arch/arm/include/asm/memory.h
>> +++ b/arch/arm/include/asm/memory.h
>> @@ -260,11 +260,15 @@ static inline unsigned long __phys_to_virt(phys_addr_t x)
>>   * translation for translating DMA addresses.  Use the driver
>>   * DMA support - see dma-mapping.h.
>>   */
>> +#ifndef CONFIG_DEBUG_VM
>>  #define virt_to_phys virt_to_phys
>>  static inline phys_addr_t virt_to_phys(const volatile void *x)
>>  {
>>  	return __virt_to_phys((unsigned long)(x));
>>  }
>> +#else
>> +#define virt_to_phys debug_virt_to_phys
>> +#endif
>>  
>>  #define phys_to_virt phys_to_virt
>>  static inline void *phys_to_virt(phys_addr_t x)
>> diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
>> index b71086d25195..c9e436b28523 100644
>> --- a/arch/arm64/include/asm/memory.h
>> +++ b/arch/arm64/include/asm/memory.h
>> @@ -186,11 +186,15 @@ extern u64			kimage_voffset;
>>   * translation for translating DMA addresses.  Use the driver
>>   * DMA support - see dma-mapping.h.
>>   */
>> +#ifndef CONFIG_DEBUG_VM
>>  #define virt_to_phys virt_to_phys
>>  static inline phys_addr_t virt_to_phys(const volatile void *x)
>>  {
>>  	return __virt_to_phys((unsigned long)(x));
>>  }
>> +#else
>> +#define virt_to_phys debug_virt_to_phys
>> +#endif
>>  
>>  #define phys_to_virt phys_to_virt
>>  static inline void *phys_to_virt(phys_addr_t x)
>> diff --git a/include/asm-generic/memory_model.h b/include/asm-generic/memory_model.h
>> index 5148150cc80b..426085757258 100644
>> --- a/include/asm-generic/memory_model.h
>> +++ b/include/asm-generic/memory_model.h
>> @@ -80,6 +80,10 @@
>>  #define page_to_pfn __page_to_pfn
>>  #define pfn_to_page __pfn_to_page
>>  
>> +#ifdef CONFIG_DEBUG_VM
>> +unsigned long debug_virt_to_phys(volatile void *address);
>> +#endif /* CONFIG_DEBUG_VM */
>> +
>>  #endif /* __ASSEMBLY__ */
>>  
>>  #endif
>> diff --git a/mm/debug.c b/mm/debug.c
>> index 9feb699c5d25..72b2ca9b11f4 100644
>> --- a/mm/debug.c
>> +++ b/mm/debug.c
>> @@ -161,4 +161,19 @@ void dump_mm(const struct mm_struct *mm)
>>  	);
>>  }
>>  
>> +#include <asm/memory.h>
>> +#include <linux/mm.h>
>> +
>> +unsigned long debug_virt_to_phys(volatile void *address)
>> +{
>> +	BUG_ON(is_vmalloc_addr((const void *)address));
>> +#if defined(CONFIG_ARM) || defined(CONFIG_ARM64) || defined(CONFIG_UNICORE32) || \
>> +	defined(CONFIG_MICROBLAZE)
>> +	return __virt_to_phys(address);
>> +#else
>> +	return virt_to_phys(address);
>> +#endif
>> +}
>> +EXPORT_SYMBOL(debug_virt_to_phys);
>> +
>>  #endif		/* CONFIG_DEBUG_VM */
>>
> 
> is_vmalloc_addr is necessary but not sufficient. This misses
> cases like module addresses.

Indeed, thanks.

> The x86 version (CONFIG_DEBUG_VIRTUAL)
> bounds checks against the known linear map to catch all cases.
> I'm for a generic approach to this if it can catch all cases
> that an architecture specific version would catch.

For one, my patch causes an early BUG to occur on ARM64 during
arch/arm64/kernel/setup.c::setup_arch when we call
cpu_uninstall_idmap(). I suspect there could be a bunch of little checks
like these where we'd have to have an architecture specific "is this
physical/virtual address valid" that may make a generic implementation
hard to come up with.
-- 
Florian

^ permalink raw reply

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
From: Reza Arbab @ 2016-11-14 19:34 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Andrew Morton,
	Rob Herring, Frank Rowand, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel
In-Reply-To: <87bmxii85s.fsf@concordia.ellerman.id.au>

On Mon, Nov 14, 2016 at 10:59:43PM +1100, Michael Ellerman wrote:
>So I'm not opposed to this, but it is a little vague.
>
>What does the "hotpluggable" property really mean?
>
>Is it just a hint to the operating system? (which may or may not be
>Linux).
>
>Or is it a direction, "this memory must be able to be hotunplugged"?
>
>I think you're intending the former, ie. a hint, which is probably OK.
>But it needs to be documented clearly.

Yes, you've got it right. It's just a hint, not a mandate.

I'm about to send v7 which adds a description of "hotpluggable" in the 
documentation. Hopefully I've explained it well enough there.

-- 
Reza Arbab

^ permalink raw reply

* Re: [PATCH v6 4/4] of/fdt: mark hotpluggable memory
From: Reza Arbab @ 2016-11-14 19:34 UTC (permalink / raw)
  To: Michael Ellerman
  Cc: Benjamin Herrenschmidt, Paul Mackerras, Andrew Morton,
	Rob Herring, Frank Rowand, Thomas Gleixner, Ingo Molnar,
	H. Peter Anvin, linuxppc-dev, linux-mm, devicetree, Bharata B Rao,
	Nathan Fontenot, Stewart Smith, Alistair Popple, Balbir Singh,
	Aneesh Kumar K.V, linux-kernel
In-Reply-To: <87bmxii85s.fsf@concordia.ellerman.id.au>

On Mon, Nov 14, 2016 at 10:59:43PM +1100, Michael Ellerman wrote:
>So I'm not opposed to this, but it is a little vague.
>
>What does the "hotpluggable" property really mean?
>
>Is it just a hint to the operating system? (which may or may not be
>Linux).
>
>Or is it a direction, "this memory must be able to be hotunplugged"?
>
>I think you're intending the former, ie. a hint, which is probably OK.
>But it needs to be documented clearly.

Yes, you've got it right. It's just a hint, not a mandate.

I'm about to send v7 which adds a description of "hotpluggable" in the 
documentation. Hopefully I've explained it well enough there.

-- 
Reza Arbab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply

* Re: [PATCH V2 1/2] pinctrl: tegra: Add DT binding for io pads control
From: Rob Herring @ 2016-11-14 19:34 UTC (permalink / raw)
  To: Laxman Dewangan
  Cc: linus.walleij, mark.rutland, swarren, thierry.reding, gnurou,
	yamada.masahiro, jonathanh, linux-gpio, devicetree, linux-tegra,
	linux-kernel
In-Reply-To: <1478696782-11657-2-git-send-email-ldewangan@nvidia.com>

On Wed, Nov 09, 2016 at 06:36:21PM +0530, Laxman Dewangan wrote:
> NVIDIA Tegra124 and later SoCs support the multi-voltage level and
> low power state of some of its IO pads. The IO pads can work in
> the voltage of the 1.8V and 3.3V of IO voltage from IO power rail
> sources. When IO interfaces are not used then IO pads can be
> configure in low power state to reduce the power consumption from
> that IO pads.
> 
> On Tegra124, the voltage level of IO power rail source is auto
> detected by hardware(SoC) and hence it is only require to configure
> in low power mode if IO pads are not used.
> 
> On T210 onwards, the auto-detection of voltage level from IO power
> rail is removed from SoC and hence SW need to configure the PMC
> register explicitly to set proper voltage in IO pads based on
> IO rail power source voltage.
> 
> Add DT binding document for detailing the DT properties for
> configuring IO pads voltage levels and its power state.
> 
> Signed-off-by: Laxman Dewangan <ldewangan@nvidia.com>
> 
> ---
> Changes from V1:
>  The DT binding document is modified to explain the regulator handle
>  for different IOs and how can it be passed from the DT.
> ---
>  .../bindings/pinctrl/nvidia,tegra-io-pad.txt       | 126 +++++++++++++++++++++
>  1 file changed, 126 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/pinctrl/nvidia,tegra-io-pad.txt
> 
> diff --git a/Documentation/devicetree/bindings/pinctrl/nvidia,tegra-io-pad.txt b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra-io-pad.txt
> new file mode 100644
> index 0000000..6ca961f
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/pinctrl/nvidia,tegra-io-pad.txt
> @@ -0,0 +1,126 @@
> +NVIDIA Tegra PMC IO pad controller
> +
> +NVIDIA Tegra124 and later SoCs support the multi-voltage level and
> +low power state of some of its IO pads. When IO interface are not
> +used then IO pads can be configure in low power state to reduce
> +the power from that IO pads. The IO pads can work in the voltage
> +of the 1.8V and 3.3V of IO voltage from power rail sources.
> +
> +On Tegra124, the voltage of IO power rail source is auto detected by
> +SoC and hence it is only require to configure in low power mode if
> +IO pads are not used.
> +
> +On T210 onwards, the HW based auto-detection for IO voltage is removed
> +and hence SW need to configure the PMC register explicitly, to set proper
> +voltage in IO pads, based on IO rail power source voltage.
> +
> +The voltage configurations and low power state of IO pads should be done
> +in boot if it is not going to change other wise dynamically based on IO
> +rail voltage on that IO pads and usage of IO pads

s/other wise/otherwise/

The end of the sentence is not clear and missing a period.

> +
> +The DT property of the io pads must be under the node of pmc i.e.
> +pmc@7000e400 for Tegra124 onwards.

s/io/IO/

> +
> +Please refer to <pinctrl-bindings.txt> in this directory for details of the
> +common pinctrl bindings used by client devices, including the meaning of the
> +phrase "pin configuration node".
> +
> +Tegra's pin configuration nodes act as a container for an arbitrary number of
> +subnodes. Each of these subnodes represents some desired configuration for an
> +IO pads, or a list of IO pads. This configuration can include the voltage and
> +power enable/disable control
> +
> +The name of each subnode is not important; all subnodes should be enumerated
> +and processed purely based on their content. Each subnode only affects those
> +parameters that are explicitly listed. Unspecified is represented as an absent
> +property,
> +
> +See the TRM to determine which properties and values apply to each IO pads.
> +
> +Required subnode-properties:
> +==========================
> +- pins : An array of strings. Each string contains the name of an IO pads. Valid
> +	 values for these names are listed below.
> +
> +Optional subnode-properties:
> +==========================
> +Following properties are supported from generic pin configuration explained
> +in <dt-bindings/pinctrl/pinctrl-binding.txt>.
> +low-power-enable:		enable low power mode.
> +low-power-disable:		disable low power mode.
> +
> +Valid values for pin for T124 are:
> +	audio, bb, cam, comp, csia, csib, csie, dsi, dsib, dsic, dsid, hdmi,
> +	hsic, hv, lvds, mipi-bias, nand, pex-bias, pex-clk1, pex-clk2,
> +	pex-ctrl, sdmmc1, sdmmc3, sdmmc4, sys-ddc, uart, usb0, usb1, usb2,
> +	usb-bias
> +
> +Valid values for pin for T210 are:
> +	audio, audio-hv, cam, csia, csib, csic, csid, csie, csif,
> +	dbg, debug-nonao, dmic, dp, dsi, dsib, dsic, dsid, emmc, emmc2,
> +	gpio, hdmi, hsic, lvds, mipi-bias, pex-bias, pex-clk1, pex-clk2,
> +	pex-ctrl, sdmmc1, sdmmc3, spi, spi-hv, uart, usb-bias, usb0,
> +	usb1, usb2, usb3.
> +
> +To find out the IO rail voltage for setting the voltage of IO pad by SW,
> +the regulator supply handle must provided from the DT and it is explained
> +in the regulator DT binding document
> +	<devicetree/bindings/regulator/regulator.txt>.
> +For example, for GPIO rail the supply name is vddio-gpio and regulator
> +handle is supplied from DT as
> +	vddio-gpio-supply = <&regulator_xyz>;
> +
> +For T210, following IO pads support the 1.8V/3.3V and the corresponding
> +io voltage pin names are as follows:
> +	audio -> vddio-audio
> +	audio-hv -> vddio-audio-hv
> +	cam ->vddio-cam
> +	dbg -> vddio-dbg
> +	dmic -> vddio-dmic
> +	gpio -> vddio-gpio
> +	pex-ctrl -> vddio-pex-ctrl
> +	sdmmc1 -> vddio-sdmmc1
> +	sdmmc3 -> vddio-sdmmc3
> +	spi -> vddio-spi
> +	spi-hv -> vddio-spi-hv
> +	uart -> vddio-uart
> +
> +Example:
> +	i2c@7000d000 {
> +		pmic@3c {
> +			regulators {
> +				vddio_sdmmc1: ldo2 {
> +					/* Regulator entries for LDO2 */
> +				};
> +
> +				vdd_cam: ldo3 {
> +					/* Regulator entries for LDO3 */
> +				};
> +			};
> +		};
> +	};
> +
> +	pmc@7000e400 {
> +		vddio-cam = <&vdd_cam>;

Missing -supply.

> +		vddio-sdmmc1-supply = <&vddio_sdmmc1>;
> +
> +		pinctrl-names = "default";
> +		pinctrl-0 = <&tegra_io_pad_volt_default>;
> +		tegra_io_pad_volt_default: common {
> +			audio-hv {
> +				pins = "audio-hv";
> +				low-power-disable;
> +			};
> +
> +			gpio {
> +				pins = "gpio";
> +				low-power-disable;
> +			};
> +
> +			audio {
> +				pins = "audio", "dmic", "sdmmc3";

What's the purpose of grouping these?

> +				low-power-enable;
> +			};
> +		};
> +
> +	};
> -- 
> 2.1.4
> 

^ permalink raw reply

* Re: [Qemu-devel] [PATCH 07/13] virtio-scsi: always use dataplane path if ioeventfd is active
From: Paolo Bonzini @ 2016-11-14 19:33 UTC (permalink / raw)
  To: Alex Williamson; +Cc: cornelia.huck, famz, qemu-devel, stefanha, mst
In-Reply-To: <20161114114929.0a45aacf@t450s.home>



On 14/11/2016 19:49, Alex Williamson wrote:
> On Mon, 14 Nov 2016 19:10:54 +0100
> Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
>> On 14/11/2016 18:09, Alex Williamson wrote:
>>> Hmm, fixed yet not fixed.  I get a nice shutdown and it even eliminates
>>> a cpu spike shown in virt-manager at the end of shutdown that was
>>> typical previously, but then I noticed dmesg showing me segfaults, so I
>>> hooked up gdb and:  
>>
>> Well, that I don't get the segfault already says something...  Though 
>> it's not even clear from the backtrace what is causing the segfault.  
>> Let's try the same printf without the change.
> 
> Log starting from virsh shutdown.  The final line occurs just as the VM
> starts spinning on all vcpus.  Thanks,

Ok, this could be a start.  Just for information what is your virtio-win
version?

Paolo

^ permalink raw reply

* Re: [PATCH 1/3] block: fast-path for small and simple direct I/O requests
From: Omar Sandoval @ 2016-11-14 19:33 UTC (permalink / raw)
  To: Jens Axboe; +Cc: axboe, linux-block, linux-fsdevel, hch, Christoph Hellwig
In-Reply-To: <1478927487-12998-2-git-send-email-axboe@fb.com>

On Fri, Nov 11, 2016 at 10:11:25PM -0700, Jens Axboe wrote:
> From: Christoph Hellwig <hch@lst.de>
> 
> This patch adds a small and simple fast patch for small direct I/O
> requests on block devices that don't use AIO.  Between the neat
> bio_iov_iter_get_pages helper that avoids allocating a page array
> for get_user_pages and the on-stack bio and biovec this avoid memory
> allocations and atomic operations entirely in the direct I/O code
> (lower levels might still do memory allocations and will usually
> have at least some atomic operations, though).
> 
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Jens Axboe <axboe@fb.com>
> ---
>  fs/block_dev.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 80 insertions(+)
> 

[snip]

>  static ssize_t
>  blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
>  {
>  	struct file *file = iocb->ki_filp;
>  	struct inode *inode = bdev_file_inode(file);
> +	int nr_pages;
>  
> +	nr_pages = iov_iter_npages(iter, BIO_MAX_PAGES);
> +	if (!nr_pages)
> +		return 0;
> +	if (is_sync_kiocb(iocb) && nr_pages <= DIO_INLINE_BIO_VECS)
> +		return __blkdev_direct_IO_simple(iocb, iter, nr_pages);
>  	return __blockdev_direct_IO(iocb, inode, I_BDEV(inode), iter,
>  				    blkdev_get_block, NULL, NULL,
>  				    DIO_SKIP_DIO_COUNT);

__blockdev_direct_IO() does a few cache prefetches that we're now
bypassing, do we want to do the same in __blkdev_direct_IO_simple()?
That's the stuff added in 65dd2aa90aa1 ("dio: optimize cache misses in
the submission path").

> -- 
> 2.7.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-block" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Omar

^ permalink raw reply

* Re: [PATCH v2 1/2] mfd: pm8xxx: add support to pm8821
From: Srinivas Kandagatla @ 2016-11-14 19:33 UTC (permalink / raw)
  To: Stephen Boyd, Lee Jones
  Cc: bjorn.andersson, Rob Herring, Andy Gross, devicetree,
	linux-kernel, linux-arm-msm, linux-soc, linux-arm-kernel
In-Reply-To: <a1116e3f-32d2-aa2d-9404-1d8147bd8500@codeaurora.org>



On 14/11/16 18:40, Stephen Boyd wrote:
> On 11/14/2016 09:52 AM, Srinivas Kandagatla wrote:
>> diff --git a/drivers/mfd/qcom-pm8xxx.c b/drivers/mfd/qcom-pm8xxx.c
>> index 7f9620e..dc347d3 100644
>> --- a/drivers/mfd/qcom-pm8xxx.c
>> +++ b/drivers/mfd/qcom-pm8xxx.c
>> +
>> +static void pm8821_irq_handler(struct irq_desc *desc)
>> +{
>> +	struct pm_irq_chip *chip = irq_desc_get_handler_data(desc);
>> +	struct irq_chip *irq_chip = irq_desc_get_chip(desc);
>> +	unsigned int master;
>> +	int ret;
>> +
>> +	chained_irq_enter(irq_chip, desc);
>> +	ret = regmap_read(chip->regmap,
>> +			  PM8821_SSBI_REG_ADDR_IRQ_MASTER0, &master);
>> +	if (ret) {
>> +		pr_err("Failed to re:Qad master 0 ret=%d\n", ret);
>
> Hm? vi?
yes..  That was good catch!! will fix it in next version...
>

^ permalink raw reply

* Re: [PATCH] IB/mlx4: Rework special QP creation error path
From: Laurence Oberman @ 2016-11-14 19:33 UTC (permalink / raw)
  To: Bart Van Assche
  Cc: Doug Ledford, Yishai Hadas, linux-rdma-u79uwXL29TY76Z2rM5mHXA
In-Reply-To: <3a41daa7-cfb8-7374-5dab-dfa3b1bf83ff-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>



----- Original Message -----
> From: "Bart Van Assche" <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
> To: "Doug Ledford" <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Cc: "Yishai Hadas" <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Sent: Monday, November 14, 2016 11:44:11 AM
> Subject: [PATCH] IB/mlx4: Rework special QP creation error path
> 
> The special QP creation error path relies on offset_of(struct mlx4_ib_sqp,
> qp) == 0. Remove this assumption because that makes the QP creation
> code easier to understand.
> 
> Signed-off-by: Bart Van Assche <bart.vanassche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
> Cc: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/hw/mlx4/qp.c | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mlx4/qp.c
> b/drivers/infiniband/hw/mlx4/qp.c
> index 570bc86..b63d6be 100644
> --- a/drivers/infiniband/hw/mlx4/qp.c
> +++ b/drivers/infiniband/hw/mlx4/qp.c
> @@ -644,7 +644,7 @@ static int create_qp_common(struct mlx4_ib_dev *dev,
> struct ib_pd *pd,
>  	int qpn;
>  	int err;
>  	struct ib_qp_cap backup_cap;
> -	struct mlx4_ib_sqp *sqp;
> +	struct mlx4_ib_sqp *sqp = NULL;
>  	struct mlx4_ib_qp *qp;
>  	enum mlx4_ib_qp_type qp_type = (enum mlx4_ib_qp_type) init_attr->qp_type;
>  	struct mlx4_ib_cq *mcq;
> @@ -933,7 +933,9 @@ static int create_qp_common(struct mlx4_ib_dev *dev,
> struct ib_pd *pd,
>  		mlx4_db_free(dev->dev, &qp->db);
>  
>  err:
> -	if (!*caller_qp)
> +	if (sqp)
> +		kfree(sqp);
> +	else if (!*caller_qp)
>  		kfree(qp);
>  	return err;
>  }
> --
> 2.10.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
This looks fine to me.

Reviewed-by: Laurence Oberman <loberman-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply

* [PATCH v2 1/2] mfd: pm8xxx: add support to pm8821
From: Srinivas Kandagatla @ 2016-11-14 19:33 UTC (permalink / raw)
  To: linux-arm-kernel
In-Reply-To: <a1116e3f-32d2-aa2d-9404-1d8147bd8500@codeaurora.org>



On 14/11/16 18:40, Stephen Boyd wrote:
> On 11/14/2016 09:52 AM, Srinivas Kandagatla wrote:
>> diff --git a/drivers/mfd/qcom-pm8xxx.c b/drivers/mfd/qcom-pm8xxx.c
>> index 7f9620e..dc347d3 100644
>> --- a/drivers/mfd/qcom-pm8xxx.c
>> +++ b/drivers/mfd/qcom-pm8xxx.c
>> +
>> +static void pm8821_irq_handler(struct irq_desc *desc)
>> +{
>> +	struct pm_irq_chip *chip = irq_desc_get_handler_data(desc);
>> +	struct irq_chip *irq_chip = irq_desc_get_chip(desc);
>> +	unsigned int master;
>> +	int ret;
>> +
>> +	chained_irq_enter(irq_chip, desc);
>> +	ret = regmap_read(chip->regmap,
>> +			  PM8821_SSBI_REG_ADDR_IRQ_MASTER0, &master);
>> +	if (ret) {
>> +		pr_err("Failed to re:Qad master 0 ret=%d\n", ret);
>
> Hm? vi?
yes..  That was good catch!! will fix it in next version...
>

^ permalink raw reply

* Re: [PATCH 1/2] Btrfs: fix file extent corruption
From: Chris Mason @ 2016-11-14 19:32 UTC (permalink / raw)
  To: Josef Bacik, linux-btrfs, kernel-team
In-Reply-To: <1479150382-19273-1-git-send-email-jbacik@fb.com>

On 11/14/2016 02:06 PM, Josef Bacik wrote:
> In order to do hole punching we have a block reserve to hold the reservation we
> need to drop the extents in our range.  Since we could end up dropping a lot of
> extents we set rsv->failfast so we can just loop around again and drop the
> remaining of the range.  Unfortunately we unconditionally fill the hole extents
> in and start from the last extent we encountered, which we may or may not have
> dropped.  So this can result in overlapping file extent entries, which can be
> tripped over in a variety of ways, either by hitting BUG_ON(!ret) in
> fill_holes() after the search, or in btrfs_set_item_key_safe() in
> btrfs_drop_extent() at a later time by an unrelated task.  Fix this by only
> setting drop_end to the last extent we did actually drop.  This way our holes
> are filled in properly for the range that we did drop, and the rest of the range
> that remains to be dropped is actually dropped.  Thanks,
>
> Signed-off-by: Josef Bacik <jbacik@fb.com>

Thanks for tracking this down Josef.  We should mark it for stable too.

-chris

^ permalink raw reply

* Re: [PATCH v4 7/8] v4l: Add signal lock status to source change events
From: Steve Longerbeam @ 2016-11-14 19:32 UTC (permalink / raw)
  To: Steve Longerbeam, Hans Verkuil, lars
  Cc: mchehab, linux-media, linux-kernel, Mauro Carvalho Chehab
In-Reply-To: <de4a622c-0d2b-0233-3f82-958a16319656@mentor.com>



On 11/14/2016 11:19 AM, Steve Longerbeam wrote:
>
>
> On 11/14/2016 03:36 AM, Hans Verkuil wrote:
>> On 08/03/2016 08:03 PM, Steve Longerbeam wrote:
>>> Add a signal lock status change to the source changes bitmask.
>>> This indicates there was a signal lock or unlock event detected
>>> at the input of a video decoder.
>>>
>>> Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
>>> Cc: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
>>>
>>> ---
>>>
>>> v4:
>>> - converted to rst from DocBook
>>>
>>> v3: no changes
>>> v2: no changes
>>> ---
>>>   Documentation/media/uapi/v4l/vidioc-dqevent.rst | 9 +++++++++
>>>   Documentation/media/videodev2.h.rst.exceptions  | 1 +
>>>   include/uapi/linux/videodev2.h                  | 1 +
>>>   3 files changed, 11 insertions(+)
>>>
>>> diff --git a/Documentation/media/uapi/v4l/vidioc-dqevent.rst 
>>> b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
>>> index 73c0d5b..7d8a053 100644
>>> --- a/Documentation/media/uapi/v4l/vidioc-dqevent.rst
>>> +++ b/Documentation/media/uapi/v4l/vidioc-dqevent.rst
>>> @@ -564,6 +564,15 @@ call.
>>>         an input. This can come from an input connector or from a video
>>>         decoder.
>>>   +    -  .. row 2
>>> +
>>> +       -  ``V4L2_EVENT_SRC_CH_LOCK_STATUS``
>>> +
>>> +       -  0x0002
>>> +
>>> +       -  This event gets triggered when there is a signal lock or
>>> +      unlock detected at the input of a video decoder.
>>> +
>>>     Return Value
>>>   ============
>>> diff --git a/Documentation/media/videodev2.h.rst.exceptions 
>>> b/Documentation/media/videodev2.h.rst.exceptions
>>> index 9bb9a6c..f412cc8 100644
>>> --- a/Documentation/media/videodev2.h.rst.exceptions
>>> +++ b/Documentation/media/videodev2.h.rst.exceptions
>>> @@ -453,6 +453,7 @@ replace define V4L2_EVENT_CTRL_CH_FLAGS 
>>> ctrl-changes-flags
>>>   replace define V4L2_EVENT_CTRL_CH_RANGE ctrl-changes-flags
>>>     replace define V4L2_EVENT_SRC_CH_RESOLUTION src-changes-flags
>>> +replace define V4L2_EVENT_SRC_CH_LOCK_STATUS src-changes-flags
>>>     replace define V4L2_EVENT_MD_FL_HAVE_FRAME_SEQ 
>>> v4l2-event-motion-det
>>>   diff --git a/include/uapi/linux/videodev2.h 
>>> b/include/uapi/linux/videodev2.h
>>> index 724f43e..08a153f 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -2078,6 +2078,7 @@ struct v4l2_event_frame_sync {
>>>   };
>>>     #define V4L2_EVENT_SRC_CH_RESOLUTION        (1 << 0)
>>> +#define V4L2_EVENT_SRC_CH_LOCK_STATUS        (1 << 1)
>>>     struct v4l2_event_src_change {
>>>       __u32 changes;
>>>
>> Quoting from an old (July) conversation about this:
>>
>>>>> I'm not entirely sure I like this. Typically losing lock means 
>>>>> that this event
>>>>> is triggered with the V4L2_EVENT_SRC_CH_RESOLUTION flag set, and 
>>>>> userspace has
>>>>> to check the new timings etc., which will fail if there is no lock 
>>>>> anymore.
>>>>>
>>>>> This information is also available through ENUMINPUT.
>>>>>
>>>>> I would need to know more about why you think this is needed, 
>>>>> because I don't
>>>>> see what this adds.
>>> Hi Hans,
>>>
>>> At least on the ADV718x, a source resolution change (from an
>>> autodetected video
>>> standard change) and a signal lock status change are distinct events.
>>> For example
>>> there can be a temporary loss of input signal lock without a change in
>>> detected
>>> input video standard/resolution.
>> OK, but what can the application do with that event? If the glitch 
>> didn't
>> affect the video, then it is pointless.
>
> Hi Hans, that's just it, for i.mx6 it does affect video. On i.mx6 a 
> temporary
> loss of signal from the adv7180 often results in a "split image", or 
> rolling
> image from captured frame to the next, and the only way to recover
> from this failure is to restart the pipeline (stream off -- stream 
> on). So
> the application needs to be informed of this temporary loss of signal
> event in order to restart streaming.
>
>
>>
>> If the lock is lost, then normally you loose video as well. If not, then
>> applications are not interested in the event.
>
> Yes, the lost lock causes a temporary or permanent loss of video,
> but with no other indications from the adv7180 (such as a detected
> video standard change). And on i.mx6, the lost lock often (actually
> usually) causes a split or rolling image.
>

Also it seems the sequence of actions from the application due to a
SRC_CH_LOCK_STATUS event would be in line with the action due to
SRC_CH_RESOLUTION. In other words for SRC_CH_RESOLUTION,
the app requests an update on the detected video standard via
QUERYSTD, and for SRC_CH_LOCK_STATUS, it requests an update
on input status (ENUMINPUTS).

Steve


^ permalink raw reply

* Re: [Qemu-devel] [RFC 2/2] cputlb: drop flush_global flag from tlb_flush
From: Richard Henderson @ 2016-11-14 19:28 UTC (permalink / raw)
  To: Alex Bennée
  Cc: qemu-devel, Paolo Bonzini, Peter Crosthwaite, Aurelien Jarno,
	Peter Maydell, Eduardo Habkost, Edgar E. Iglesias, Yongbok Kim,
	Jia Liu, David Gibson, Alexander Graf, Mark Cave-Ayland,
	Artyom Tarasenko, Guan Xuetao, Max Filippov, open list:ARM,
	open list:PowerPC
In-Reply-To: <20161114174010.31040-3-alex.bennee@linaro.org>

On 11/14/2016 06:40 PM, Alex Bennée wrote:
> We have never has the concept of global TLB entries which would avoid
> the flush so we never actually use this flag. Drop it and make clear
> that tlb_flush is the sledge-hammer it has always been.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---

Reviewed-by: Richard Henderson <rth@twiddle.net>


r~

^ permalink raw reply

* Re: [RFC 2/2] cputlb: drop flush_global flag from tlb_flush
From: Richard Henderson @ 2016-11-14 19:28 UTC (permalink / raw)
  To: Alex Bennée
  Cc: qemu-devel, Paolo Bonzini, Peter Crosthwaite, Aurelien Jarno,
	Peter Maydell, Eduardo Habkost, Edgar E. Iglesias, Yongbok Kim,
	Jia Liu, David Gibson, Alexander Graf, Mark Cave-Ayland,
	Artyom Tarasenko, Guan Xuetao, Max Filippov, open list:ARM,
	open list:PowerPC
In-Reply-To: <20161114174010.31040-3-alex.bennee@linaro.org>

On 11/14/2016 06:40 PM, Alex Bennée wrote:
> We have never has the concept of global TLB entries which would avoid
> the flush so we never actually use this flag. Drop it and make clear
> that tlb_flush is the sledge-hammer it has always been.
>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> ---

Reviewed-by: Richard Henderson <rth@twiddle.net>


r~

^ permalink raw reply

* Re: [Libtirpc-devel] [PATCH rpcbind] Move default state-dir to /run/rpcbind
From: Steve Dickson @ 2016-11-14 19:26 UTC (permalink / raw)
  To: NeilBrown, Linux NFS Mailing List, libtirpc-devel
In-Reply-To: <20161114191234.GV21655@vapier.lan>



On 11/14/2016 02:12 PM, Mike Frysinger wrote:
> On 14 Nov 2016 10:09, NeilBrown wrote:
>> On Sat, Nov 12 2016, Mike Frysinger wrote:
>>> On 11 Nov 2016 14:36, NeilBrown wrote:
>>>> rpcbind can save state in a file to allow restart without forgetting
>>>> about running services.
>>>>
>>>> The default location is currently "/tmp" which is an over-used
>>>> directory that isn't really suitable for system files.
>>>> The modern preferences would be a subdirectory of "/run", which can
>>>> be selected with a ./configure option.  That subdirectory would still need
>>>> to be created by something.
>>> the portable path is /var/cache instead of /run.  i don't think libtirpc
>>> should be configuring itself to assume Linux by default.
>> In principle I agree.  But is /var/cache really a good choice?
>> We don't want the state files to persist over a reboot, and I strongly
>> suspect that /var/cache is designed to do exactly that.
>>
>> Are there agree standards that are broader than Linux that we can look
>> to?
>> FHS defines /var/run (or even /run) but I suspect it is linux-only.
> /var/run should work across systems i believe.  at least BSD's support it.
In the Red Hat distos /var/run is a symbolic link to /run and the systemd
folks have asked us to use /run instead of /var/run

steved.


^ permalink raw reply

* Re: [Qemu-devel] [PATCH 01/21] specs/qcow2: fix bitmap granularity qemu-specific note
From: John Snow @ 2016-11-14 19:26 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy, qemu-block, qemu-devel
  Cc: kwolf, famz, armbru, mreitz, stefanha, pbonzini, den
In-Reply-To: <1478715476-132280-2-git-send-email-vsementsov@virtuozzo.com>



On 11/09/2016 01:17 PM, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  docs/specs/qcow2.txt | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/docs/specs/qcow2.txt b/docs/specs/qcow2.txt
> index 80cdfd0..dda53dd 100644
> --- a/docs/specs/qcow2.txt
> +++ b/docs/specs/qcow2.txt
> @@ -472,8 +472,7 @@ Structure of a bitmap directory entry:
>               17:    granularity_bits
>                      Granularity bits. Valid values: 0 - 63.
>
> -                    Note: Qemu currently doesn't support granularity_bits
> -                    greater than 31.
> +                    Note: Qemu currently supports only values 9 - 31.
>
>                      Granularity is calculated as
>                          granularity = 1 << granularity_bits
>

Reviewed-by: John Snow <jsnow@redhat.com>

^ permalink raw reply

* Re: [PATCH 3/3] backports: empty define for __ro_after_init
From: Arend Van Spriel @ 2016-11-14 19:26 UTC (permalink / raw)
  To: Luis R. Rodriguez, Johannes Berg, Kees Cook; +Cc: backports
In-Reply-To: <20161114185917.GZ13978@wotan.suse.de>

On 14-11-2016 19:59, Luis R. Rodriguez wrote:
> On Sat, Nov 12, 2016 at 10:07:22PM +0100, Johannes Berg wrote:
>> On Sat, 2016-11-12 at 20:18 +0000, Arend van Spriel wrote:
>>> nl80211 now uses section qualifier __ro_after_init. However, this
>>> is not available in kernels before v4.6. Neither is the section
>>> itself hence adding an empty define in backports.
>>
>>> +#if (LINUX_VERSION_CODE < KERNEL_VERSION(4,6,0))
>>> +#define __ro_after_init
>>> +#endif
>>
>> Due to how __ro_after_init works, it *has* to be a macro originally, so
>> you could just ifndef on itself:
>>
>> #ifndef __ro_after_init
>> #define __ro_after_init
>> #endif
>>
>> and be slightly more compatible in case somebody ever backports it.
> 
> Agreed.

Agree as well hence I posted a V2 yesterday ;-)

Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe backports" in

^ permalink raw reply

* Re: [Qemu-devel] [PATCH 18/21] qmp: add x-debug-block-dirty-bitmap-sha256
From: John Snow @ 2016-11-14 19:25 UTC (permalink / raw)
  To: Vladimir Sementsov-Ogievskiy, qemu-block, qemu-devel
  Cc: kwolf, famz, armbru, mreitz, stefanha, pbonzini, den
In-Reply-To: <1478715476-132280-19-git-send-email-vsementsov@virtuozzo.com>



On 11/09/2016 01:17 PM, Vladimir Sementsov-Ogievskiy wrote:
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>  block/dirty-bitmap.c         |  5 +++++
>  blockdev.c                   | 33 +++++++++++++++++++++++++++++++++
>  include/block/dirty-bitmap.h |  2 ++
>  include/qemu/hbitmap.h       |  8 ++++++++
>  qapi/block-core.json         | 26 ++++++++++++++++++++++++++
>  util/hbitmap.c               | 11 +++++++++++
>  6 files changed, 85 insertions(+)
>
> diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
> index fe34d48..775181c 100644
> --- a/block/dirty-bitmap.c
> +++ b/block/dirty-bitmap.c
> @@ -589,3 +589,8 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
>      return bitmap == NULL ? QLIST_FIRST(&bs->dirty_bitmaps) :
>                              QLIST_NEXT(bitmap, list);
>  }
> +
> +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp)
> +{
> +    return hbitmap_sha256(bitmap->bitmap, errp);
> +}
> diff --git a/blockdev.c b/blockdev.c
> index db2bdbb..c028070 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -2811,6 +2811,39 @@ void qmp_block_dirty_bitmap_clear(const char *node, const char *name,
>      aio_context_release(aio_context);
>  }
>
> +/**
> + * Completely clear a bitmap, for the purposes of synchronizing a bitmap
> + * immediately after a full backup operation.
> + */
> +BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
> +                                                              const char *name,
> +                                                              Error **errp)
> +{
> +    AioContext *aio_context;
> +    BdrvDirtyBitmap *bitmap;
> +    BlockDriverState *bs;
> +    BlockDirtyBitmapSha256 *ret = NULL;
> +    char *sha256;
> +
> +    bitmap = block_dirty_bitmap_lookup(node, name, &bs, &aio_context, errp);
> +    if (!bitmap || !bs) {
> +        return NULL;
> +    }
> +
> +    sha256 = bdrv_dirty_bitmap_sha256(bitmap, errp);
> +    if (sha256 == NULL) {
> +        goto out;
> +    }
> +
> +    ret = g_new(BlockDirtyBitmapSha256, 1);
> +    ret->sha256 = sha256;
> +
> +out:
> +    aio_context_release(aio_context);
> +
> +    return ret;
> +}
> +
>  void hmp_drive_del(Monitor *mon, const QDict *qdict)
>  {
>      const char *id = qdict_get_str(qdict, "id");
> diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
> index d71edc4..b022b34 100644
> --- a/include/block/dirty-bitmap.h
> +++ b/include/block/dirty-bitmap.h
> @@ -86,4 +86,6 @@ BdrvDirtyBitmap *bdrv_dirty_bitmap_next(BlockDriverState *bs,
>
>  bool bdrv_has_persistent_bitmaps(BlockDriverState *bs);
>
> +char *bdrv_dirty_bitmap_sha256(const BdrvDirtyBitmap *bitmap, Error **errp);
> +
>  #endif
> diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
> index 5c9dd74..d6f3838 100644
> --- a/include/qemu/hbitmap.h
> +++ b/include/qemu/hbitmap.h
> @@ -240,6 +240,14 @@ void hbitmap_deserialize_ones(HBitmap *hb, uint64_t start, uint64_t count,
>  void hbitmap_deserialize_finish(HBitmap *hb);
>
>  /**
> + * hbitmap_sha256:
> + * @bitmap: HBitmap to operate on.
> + *
> + * Returns SHA256 hash of the last level.
> + */
> +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp);
> +
> +/**
>   * hbitmap_free:
>   * @hb: HBitmap to operate on.
>   *
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 2c33a30..f72cfef 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -1289,6 +1289,32 @@
>    'data': 'BlockDirtyBitmap' }
>
>  ##
> +# @BlockDirtyBitmapSha256:
> +#
> +# SHA256 hash of dirty bitmap data
> +#
> +# @sha256: bitmap SHA256 hash
> +#
> +# Since: 2.8
> +##
> +  { 'struct': 'BlockDirtyBitmapSha256',
> +    'data': {'sha256': 'str'} }
> +
> +##
> +# @x-debug-block-dirty-bitmap-sha256
> +#
> +# Get bitmap SHA256
> +#
> +# Returns: BlockDirtyBitmapSha256 on success
> +#          If @node is not a valid block device, DeviceNotFound
> +#          If @name is not found, GenericError with an explanation
> +#
> +# Since 2.8
> +##
> +  { 'command': 'x-debug-block-dirty-bitmap-sha256',
> +    'data': 'BlockDirtyBitmap', 'returns': 'BlockDirtyBitmapSha256' }
> +
> +##
>  # @blockdev-mirror
>  #
>  # Start mirroring a block device's writes to a new destination.
> diff --git a/util/hbitmap.c b/util/hbitmap.c
> index 72623f4..96af071 100644
> --- a/util/hbitmap.c
> +++ b/util/hbitmap.c
> @@ -13,6 +13,7 @@
>  #include "qemu/hbitmap.h"
>  #include "qemu/host-utils.h"
>  #include "trace.h"
> +#include "crypto/hash.h"
>
>  /* HBitmaps provides an array of bits.  The bits are stored as usual in an
>   * array of unsigned longs, but HBitmap is also optimized to provide fast
> @@ -707,3 +708,13 @@ void hbitmap_free_meta(HBitmap *hb)
>      hbitmap_free(hb->meta);
>      hb->meta = NULL;
>  }
> +
> +char *hbitmap_sha256(const HBitmap *bitmap, Error **errp)
> +{
> +    size_t size = bitmap->sizes[HBITMAP_LEVELS - 1] * sizeof(unsigned long);
> +    char *data = (char *)bitmap->levels[HBITMAP_LEVELS - 1];
> +    char *hash = NULL;
> +    qcrypto_hash_digest(QCRYPTO_HASH_ALG_SHA256, data, size, &hash, errp);

By adding this here, you're adding a linking dependency to 
libqemuutil.a, which causes the test-hbitmap test to fail to link from 
this patch forward.

I'm not sure what the correct approach should be, but at a minimum we 
need to link crypto/hash.o to test/test-hbitmap.o to be able to run that 
test. We get build failures otherwise.

> +
> +    return hash;
> +}
>

^ permalink raw reply

* Re: [PATCH v7 00/17] port branch.c to use ref-filter's printing options
From: Karthik Nayak @ 2016-11-14 19:24 UTC (permalink / raw)
  To: Jacob Keller; +Cc: Git mailing list
In-Reply-To: <CA+P7+xqRjMThZF7u_W6G0sHjFP3j5PMr=TszC6UxL2XCYO+CVA@mail.gmail.com>

Hello,

On Wed, Nov 9, 2016 at 5:45 AM, Jacob Keller <jacob.keller@gmail.com> wrote:
> On Tue, Nov 8, 2016 at 12:11 PM, Karthik Nayak <karthik.188@gmail.com> wrote:
>> This is part of unification of the commands 'git tag -l, git branch -l
>> and git for-each-ref'. This ports over branch.c to use ref-filter's
>> printing options.
>>
>> Initially posted here: $(gmane/279226). It was decided that this series
>> would follow up after refactoring ref-filter parsing mechanism, which
>> is now merged into master (9606218b32344c5c756f7c29349d3845ef60b80c).
>>
>> v1 can be found here: $(gmane/288342)
>> v2 can be found here: $(gmane/288863)
>> v3 can be found here: $(gmane/290299)
>> v4 can be found here: $(gmane/291106)
>> v5b can be found here: $(gmane/292467)
>> v6 can be found here: http://marc.info/?l=git&m=146330914118766&w=2
>>
>
> I reviewed the full series. I found a few minor things I would have
> done differently, but overall I think it looks good. Thanks for the
> hard work and the time invested here. I remember seeing this on the
> list quite some time ago, so it's nice to see it finally come
> together.
>

Thanks for the review. I've gone through all the mails and will follow
up with replies.
Will post the next version as soon as possible. Thanks.

-- 
Regards,
Karthik Nayak

^ permalink raw reply

* [PATCH 2/2] blk-mq: Avoid memory reclaim when remapping queues
From: Gabriel Krisman Bertazi @ 2016-11-14 19:24 UTC (permalink / raw)
  To: axboe
  Cc: linux-block, Gabriel Krisman Bertazi, Brian King, Douglas Miller,
	linux-scsi
In-Reply-To: <1479151478-19725-1-git-send-email-krisman@linux.vnet.ibm.com>

While stressing memory and IO at the same time we changed SMT settings,
we were able to consistently trigger deadlocks in the mm system, which
froze the entire machine.

I think that under memory stress conditions, the large allocations
performed by blk_mq_init_rq_map may trigger a reclaim, which stalls
waiting on the block layer remmaping completion, thus deadlocking the
system.  The trace below was collected after the machine stalled,
waiting for the hotplug event completion.

The simplest fix for this is to make allocations in this path
non-reclaimable, with GFP_NOWAIT.  With this patch, We couldn't hit the
issue anymore.

This should apply on top of Jen's for-next branch cleanly.

 Call Trace:
[c000000f0160aaf0] [c000000f0160ab50] 0xc000000f0160ab50 (unreliable)
[c000000f0160acc0] [c000000000016624] __switch_to+0x2e4/0x430
[c000000f0160ad20] [c000000000b1a880] __schedule+0x310/0x9b0
[c000000f0160ae00] [c000000000b1af68] schedule+0x48/0xc0
[c000000f0160ae30] [c000000000b1b4b0] schedule_preempt_disabled+0x20/0x30
[c000000f0160ae50] [c000000000b1d4fc] __mutex_lock_slowpath+0xec/0x1f0
[c000000f0160aed0] [c000000000b1d678] mutex_lock+0x78/0xa0
[c000000f0160af00] [d000000019413cac] xfs_reclaim_inodes_ag+0x33c/0x380 [xfs]
[c000000f0160b0b0] [d000000019415164] xfs_reclaim_inodes_nr+0x54/0x70 [xfs]
[c000000f0160b0f0] [d0000000194297f8] xfs_fs_free_cached_objects+0x38/0x60 [xfs]
[c000000f0160b120] [c0000000003172c8] super_cache_scan+0x1f8/0x210
[c000000f0160b190] [c00000000026301c] shrink_slab.part.13+0x21c/0x4c0
[c000000f0160b2d0] [c000000000268088] shrink_zone+0x2d8/0x3c0
[c000000f0160b380] [c00000000026834c] do_try_to_free_pages+0x1dc/0x520
[c000000f0160b450] [c00000000026876c] try_to_free_pages+0xdc/0x250
[c000000f0160b4e0] [c000000000251978] __alloc_pages_nodemask+0x868/0x10d0
[c000000f0160b6f0] [c000000000567030] blk_mq_init_rq_map+0x160/0x380
[c000000f0160b7a0] [c00000000056758c] blk_mq_map_swqueue+0x33c/0x360
[c000000f0160b820] [c000000000567904] blk_mq_queue_reinit+0x64/0xb0
[c000000f0160b850] [c00000000056a16c] blk_mq_queue_reinit_notify+0x19c/0x250
[c000000f0160b8a0] [c0000000000f5d38] notifier_call_chain+0x98/0x100
[c000000f0160b8f0] [c0000000000c5fb0] __cpu_notify+0x70/0xe0
[c000000f0160b930] [c0000000000c63c4] notify_prepare+0x44/0xb0
[c000000f0160b9b0] [c0000000000c52f4] cpuhp_invoke_callback+0x84/0x250
[c000000f0160ba10] [c0000000000c570c] cpuhp_up_callbacks+0x5c/0x120
[c000000f0160ba60] [c0000000000c7cb8] _cpu_up+0xf8/0x1d0
[c000000f0160bac0] [c0000000000c7eb0] do_cpu_up+0x120/0x150
[c000000f0160bb40] [c0000000006fe024] cpu_subsys_online+0x64/0xe0
[c000000f0160bb90] [c0000000006f5124] device_online+0xb4/0x120
[c000000f0160bbd0] [c0000000006f5244] online_store+0xb4/0xc0
[c000000f0160bc20] [c0000000006f0a68] dev_attr_store+0x68/0xa0
[c000000f0160bc60] [c0000000003ccc30] sysfs_kf_write+0x80/0xb0
[c000000f0160bca0] [c0000000003cbabc] kernfs_fop_write+0x17c/0x250
[c000000f0160bcf0] [c00000000030fe6c] __vfs_write+0x6c/0x1e0
[c000000f0160bd90] [c000000000311490] vfs_write+0xd0/0x270
[c000000f0160bde0] [c0000000003131fc] SyS_write+0x6c/0x110
[c000000f0160be30] [c000000000009204] system_call+0x38/0xec

Signed-off-by: Gabriel Krisman Bertazi <krisman@linux.vnet.ibm.com>
Cc: Brian King <brking@linux.vnet.ibm.com>
Cc: Douglas Miller <dougmill@linux.vnet.ibm.com>
Cc: linux-block@vger.kernel.org
Cc: linux-scsi@vger.kernel.org
---
 block/blk-mq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 7f7c4ba91adf..3e44303646cb 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1597,7 +1597,7 @@ static struct blk_mq_tags *blk_mq_init_rq_map(struct blk_mq_tag_set *set,
 	INIT_LIST_HEAD(&tags->page_list);
 
 	tags->rqs = kzalloc_node(set->queue_depth * sizeof(struct request *),
-				 GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY,
+				 GFP_NOWAIT | __GFP_NOWARN | __GFP_NORETRY,
 				 set->numa_node);
 	if (!tags->rqs) {
 		blk_mq_free_tags(tags);
@@ -1623,7 +1623,7 @@ static struct blk_mq_tags *blk_mq_init_rq_map(struct blk_mq_tag_set *set,
 
 		do {
 			page = alloc_pages_node(set->numa_node,
-				GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY | __GFP_ZERO,
+				GFP_NOWAIT | __GFP_NOWARN | __GFP_NORETRY | __GFP_ZERO,
 				this_order);
 			if (page)
 				break;
@@ -1644,7 +1644,7 @@ static struct blk_mq_tags *blk_mq_init_rq_map(struct blk_mq_tag_set *set,
 		 * Allow kmemleak to scan these pages as they contain pointers
 		 * to additional allocations like via ops->init_request().
 		 */
-		kmemleak_alloc(p, order_to_size(this_order), 1, GFP_KERNEL);
+		kmemleak_alloc(p, order_to_size(this_order), 1, GFP_NOWAIT);
 		entries_per_page = order_to_size(this_order) / rq_size;
 		to_do = min(entries_per_page, set->queue_depth - i);
 		left -= to_do * rq_size;
-- 
2.7.4


^ permalink raw reply related

* [PATCH 1/2] blk-mq: Fix failed allocation path when mapping queues
From: Gabriel Krisman Bertazi @ 2016-11-14 19:24 UTC (permalink / raw)
  To: axboe
  Cc: linux-block, Gabriel Krisman Bertazi, Brian King, Douglas Miller,
	linux-scsi

In blk_mq_map_swqueue, there is a memory optimization that frees the
tags of a queue that has gone unmapped.  Later, if that hctx is remapped
after another topology change, the tags need to be reallocated.

If this allocation fails, a simple WARN_ON triggers, but the block layer
ends up with an active hctx without any corresponding set of tags.
Then, any income IO to that hctx can trigger an Oops.

I can reproduce it consistently by running IO, flipping CPUs on and off
and eventually injecting a memory allocation failure in that path.

In the fix below, if the system experiences a failed allocation of any
hctx's tags, we remap all the ctxs of that queue to the hctx_0, which
should always keep it's tags.  There is a minor performance hit, since
our mapping just got worse after the error path, but this is
the simplest solution to handle this error path.  The performance hit
will disappear after another successful remap.

I considered dropping the memory optimization all together, but it
seemed a bad trade-off to handle this very specific error case.

My one concern about this patch is if remapping an arbitrary queue to
hctx_0 could result in outstanding requests getting submitted to the
wrong hctx.  I couldn't observe this happening during tests, but I'm not
entirely sure it'll never happen.  I believe the queue will be empty if
we are trying to allocate tags for it, unless it was using another alive
hctx queue and for some reason got reassigned to this new hctx.  is this
possible at all?

This should apply cleanly on top of Jen's for-next branch.

The Oops is the one below:

SP (3fff935ce4d0) is in userspace
1:mon> e
cpu 0x1: Vector: 300 (Data Access) at [c000000fe99eb110]
    pc: c0000000005e868c: __sbitmap_queue_get+0x2c/0x180
    lr: c000000000575328: __bt_get+0x48/0xd0
    sp: c000000fe99eb390
   msr: 900000010280b033
   dar: 28
 dsisr: 40000000
  current = 0xc000000fe9966800
  paca    = 0xc000000007e80300   softe: 0        irq_happened: 0x01
    pid   = 11035, comm = aio-stress
Linux version 4.8.0-rc6+ (root@bean) (gcc version 5.4.0 20160609
(Ubuntu/IBM 5.4.0-6ubuntu1~16.04.2) ) #3 SMP Mon Oct 10 20:16:53 CDT 2016
1:mon> s
[c000000fe99eb3d0] c000000000575328 __bt_get+0x48/0xd0
[c000000fe99eb400] c000000000575838 bt_get.isra.1+0x78/0x2d0
[c000000fe99eb480] c000000000575cb4 blk_mq_get_tag+0x44/0x100
[c000000fe99eb4b0] c00000000056f6f4 __blk_mq_alloc_request+0x44/0x220
[c000000fe99eb500] c000000000570050 blk_mq_map_request+0x100/0x1f0
[c000000fe99eb580] c000000000574650 blk_mq_make_request+0xf0/0x540
[c000000fe99eb640] c000000000561c44 generic_make_request+0x144/0x230
[c000000fe99eb690] c000000000561e00 submit_bio+0xd0/0x200
[c000000fe99eb740] c0000000003ef740 ext4_io_submit+0x90/0xb0
[c000000fe99eb770] c0000000003e95d8 ext4_writepages+0x588/0xdd0
[c000000fe99eb910] c00000000025a9f0 do_writepages+0x60/0xc0
[c000000fe99eb940] c000000000246c88 __filemap_fdatawrite_range+0xf8/0x180
[c000000fe99eb9e0] c000000000246f90 filemap_write_and_wait_range+0x70/0xf0
[c000000fe99eba20] c0000000003dd844 ext4_sync_file+0x214/0x540
[c000000fe99eba80] c000000000364718 vfs_fsync_range+0x78/0x130
[c000000fe99ebad0] c0000000003dd46c ext4_file_write_iter+0x35c/0x430
[c000000fe99ebb90] c00000000038c280 aio_run_iocb+0x3b0/0x450
[c000000fe99ebce0] c00000000038dc28 do_io_submit+0x368/0x730
[c000000fe99ebe30] c000000000009404 system_call+0x38/0xec

Signed-off-by: Gabriel Krisman Bertazi <krisman@linux.vnet.ibm.com>
Cc: Brian King <brking@linux.vnet.ibm.com>
Cc: Douglas Miller <dougmill@linux.vnet.ibm.com>
Cc: linux-block@vger.kernel.org
Cc: linux-scsi@vger.kernel.org
---
 block/blk-mq.c | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

diff --git a/block/blk-mq.c b/block/blk-mq.c
index 3c5aaead71bb..7f7c4ba91adf 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -1862,7 +1862,7 @@ static void blk_mq_init_cpu_queues(struct request_queue *q,
 static void blk_mq_map_swqueue(struct request_queue *q,
 			       const struct cpumask *online_mask)
 {
-	unsigned int i;
+	unsigned int i, hctx_idx;
 	struct blk_mq_hw_ctx *hctx;
 	struct blk_mq_ctx *ctx;
 	struct blk_mq_tag_set *set = q->tag_set;
@@ -1885,6 +1885,15 @@ static void blk_mq_map_swqueue(struct request_queue *q,
 		if (!cpumask_test_cpu(i, online_mask))
 			continue;
 
+		hctx_idx = q->mq_map[i];
+		/* unmapped hw queue can be remapped after CPU topo changed */
+		if (!set->tags[hctx_idx]) {
+			set->tags[hctx_idx] = blk_mq_init_rq_map(set, hctx_idx);
+
+			if (!set->tags[hctx_idx])
+				q->mq_map[i] = 0;
+		}
+
 		ctx = per_cpu_ptr(q->queue_ctx, i);
 		hctx = blk_mq_map_queue(q, i);
 
@@ -1901,7 +1910,10 @@ static void blk_mq_map_swqueue(struct request_queue *q,
 		 * disable it and free the request entries.
 		 */
 		if (!hctx->nr_ctx) {
-			if (set->tags[i]) {
+			/* Never unmap queue 0.  We need it as a
+			 * fallback in case of a new remap fails
+			 * allocation. */
+			if (i && set->tags[i]) {
 				blk_mq_free_rq_map(set, set->tags[i], i);
 				set->tags[i] = NULL;
 			}
@@ -1909,11 +1921,8 @@ static void blk_mq_map_swqueue(struct request_queue *q,
 			continue;
 		}
 
-		/* unmapped hw queue can be remapped after CPU topo changed */
-		if (!set->tags[i])
-			set->tags[i] = blk_mq_init_rq_map(set, i);
 		hctx->tags = set->tags[i];
-		WARN_ON(!hctx->tags);
+		BUG_ON(!hctx->tags);
 
 		/*
 		 * Set the map size to the number of mapped software queues.
-- 
2.7.4


^ permalink raw reply related

* Re: [PATCH] tracing: ignore FTRACE_FL_DISABLED while walking dyn_ftrace records
From: Steven Rostedt @ 2016-11-14 19:24 UTC (permalink / raw)
  To: Alexei Starovoitov
  Cc: Dave Jones, Martin KaFai Lau, linux-kernel, kernel-team
In-Reply-To: <1478560460-3818619-1-git-send-email-ast@fb.com>

On Mon, 7 Nov 2016 15:14:20 -0800
Alexei Starovoitov <ast@fb.com> wrote:

> ftrace_shutdown() checks for sanity of ftrace records
> and if dyn_ftrace->flags is not zero, it will warn.
> It can happen that 'flags' are set to FTRACE_FL_DISABLED at this point,
> since some module was loaded, but before ftrace_module_enable()
> cleared the flags for this module.
> 
> In other words the module.c is doing:
> ftrace_module_init(mod); // calls ftrace_update_code() that sets flags=FTRACE_FL_DISABLED
> ... // here ftrace_shutdown() is called that warns, since
> err = prepare_coming_module(mod); // didn't have a chance to clear FTRACE_FL_DISABLED
> 
> Fix it by ignoring disabled records.
> It's similar to what __ftrace_hash_rec_update() is already doing.
> 

Alexei,

Do you have a clear reproducer of this bug?

-- Steve

> Signed-off-by: Alexei Starovoitov <ast@kernel.org>
> ---
>  kernel/trace/ftrace.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 2050a7652a86..326498baab83 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -2763,7 +2763,7 @@ static int ftrace_shutdown(struct ftrace_ops *ops, int command)
>  		struct dyn_ftrace *rec;
>  
>  		do_for_each_ftrace_rec(pg, rec) {
> -			if (FTRACE_WARN_ON_ONCE(rec->flags))
> +			if (FTRACE_WARN_ON_ONCE(rec->flags & ~FTRACE_FL_DISABLED))
>  				pr_warn("  %pS flags:%lx\n",
>  					(void *)rec->ip, rec->flags);
>  		} while_for_each_ftrace_rec();

^ permalink raw reply

* Re: [RFC PATCH] ARM64: dts: Add support for Meson GXM
From: Kevin Hilman @ 2016-11-14 19:17 UTC (permalink / raw)
  To: Neil Armstrong, Martin Blumenstingl
  Cc: carlo, linux-amlogic, linux-arm-kernel, linux-kernel, devicetree
In-Reply-To: <20161114094411.30199-1-narmstrong@baylibre.com>

Neil Armstrong <narmstrong@baylibre.com> writes:

> Following the Amlogic Linux kernel, it seem the only differences
> between the GXL and GXM SoCs are the CPU Clusters.
>
> Simply add a meson-gxm dtsi and reproduce the P23x to Q20x boards
> dts files since the S905D and S912 SoCs shares the same pinout
> and the P23x and Q20x boards are identical.

Since the P23x and Q20x boards are identical, should we just come up
with a singel dtsi for them all? e.g  move the -p23x.dtsi to -pq2xx.dtsi
or something?

I don't think it's a big deal, but it does seem like we should avoid the
copy/paste wherever possible.

Kevin

^ permalink raw reply


This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.