* [PATCH 01/13] nvme-pci: don't limit DMA segement size
From: Christoph Hellwig @ 2019-06-05 19:08 UTC (permalink / raw)
To: Jens Axboe
Cc: Sebastian Ott, Sagi Grimberg, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, megaraidlinux.pdl,
MPT-FusionLinux.pdl, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605190836.32354-1-hch@lst.de>
NVMe uses PRPs (or optionally unlimited SGLs) for data transfers and
has no specific limit for a single DMA segement. Limiting the size
will cause problems because the block layer assumes PRP-ish devices
using a virt boundary mask don't have a segment limit. And while this
is true, we also really need to tell the DMA mapping layer about it,
otherwise dma-debug will trip over it.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reported-by: Sebastian Ott <sebott@linux.ibm.com>
---
drivers/nvme/host/pci.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index f562154551ce..524d6bd6d095 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2513,6 +2513,12 @@ static void nvme_reset_work(struct work_struct *work)
*/
dev->ctrl.max_hw_sectors = NVME_MAX_KB_SZ << 1;
dev->ctrl.max_segments = NVME_MAX_SEGS;
+
+ /*
+ * Don't limit the IOMMU merged segment size.
+ */
+ dma_set_max_seg_size(dev->dev, 0xffffffff);
+
mutex_unlock(&dev->shutdown_lock);
/*
--
2.20.1
^ permalink raw reply related
* Re: properly communicate queue limits to the DMA layer
From: Jens Axboe @ 2019-06-05 19:17 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Sebastian Ott, Sagi Grimberg, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, megaraidlinux.pdl,
MPT-FusionLinux.pdl, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605190836.32354-1-hch@lst.de>
On 6/5/19 1:08 PM, Christoph Hellwig wrote:
> Hi Jens,
>
> we've always had a bit of a problem communicating the block layer
> queue limits to the DMA layer, which needs to respect them when
> an IOMMU that could merge segments is used. Unfortunately most
> drivers don't get this right. Oddly enough we've been mostly
> getting away with it, although lately dma-debug has been catching
> a few of those issues.
>
> The segment merging fix for devices with PRP-like structures seems
> to have escalated this a bit. The first patch fixes the actual
> report from Sebastian, while the rest fix every drivers that appears
> to have the problem based on a code audit looking for drivers using
> blk_queue_max_segment_size, blk_queue_segment_boundary or
> blk_queue_virt_boundary and calling dma_map_sg eventually. Note
> that for SCSI drivers I've taken the blk_queue_virt_boundary setting
> to the SCSI core, similar to how we did it for the other two settings
> a while ago. This also deals with the fact that the DMA layer
> settings are on a per-device granularity, so the per-device settings
> in a few SCSI drivers can't actually work in an IOMMU environment.
>
> It would be nice to eventually pass these limits as arguments to
> dma_map_sg, but that is a far too big series for the 5.2 merge
> window.
Since I'm heading out shortly and since I think this should make
the next -rc, I'll tentatively queue this up.
--
Jens Axboe
^ permalink raw reply
* Re: properly communicate queue limits to the DMA layer
From: Christoph Hellwig @ 2019-06-05 19:24 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, Sebastian Ott, Sagi Grimberg, Max Gurtovoy,
Bart Van Assche, Ulf Hansson, Alan Stern, Oliver Neukum,
linux-block, linux-rdma, linux-mmc, linux-nvme, linux-scsi,
megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv, linux-usb,
usb-storage, linux-kernel
In-Reply-To: <591cfa1e-fecb-7d00-c855-3b9eb8eb8a2a@kernel.dk>
On Wed, Jun 05, 2019 at 01:17:15PM -0600, Jens Axboe wrote:
> Since I'm heading out shortly and since I think this should make
> the next -rc, I'll tentatively queue this up.
The SCSI bits will need a bit more review, and possibly tweaking
fo megaraid and mpt3sas. But they are really independent of the
other patches, so maybe skip them for now and leave them for Martin
to deal with.
^ permalink raw reply
* Re: [PATCH] revert async probing of VMBus scsi device
From: Christoph Hellwig @ 2019-06-05 19:26 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Christoph Hellwig, linux-scsi, linux-hyperv, Stephen Hemminger
In-Reply-To: <20190605121020.1a41b753@hermes.lan>
On Wed, Jun 05, 2019 at 12:10:20PM -0700, Stephen Hemminger wrote:
> > Sure. But they should not get a way out for just one specific driver.
>
> There are people running new kernels on 6 year old distributions.
> Was every distribution smart enough then? If you think so, then
> this not necessary.
I think you are missing my point. If we want a way to disable this,
we:
a) want it opt-in
b) it needs to for the whole SCSI layer and not just one driver
^ permalink raw reply
* Re: [PATCH] revert async probing of VMBus scsi device
From: David Miller @ 2019-06-05 19:27 UTC (permalink / raw)
To: hch; +Cc: stephen, linux-scsi, linux-hyperv, sthemmin
In-Reply-To: <20190605190722.GA19684@infradead.org>
From: Christoph Hellwig <hch@infradead.org>
Date: Wed, 5 Jun 2019 12:07:23 -0700
> On Wed, Jun 05, 2019 at 12:06:40PM -0700, Stephen Hemminger wrote:
>> > Which is true for every device, and why we use UUIDs or label for
>> > mounts that are supposed to be stable.
>>
>> Not everyone is smart enough to do that.
>
> Sure. But they should not get a way out for just one specific driver.
Agreed.
^ permalink raw reply
* Re: [PATCH] revert async probing of VMBus scsi device
From: Stephen Hemminger @ 2019-06-05 20:06 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-scsi, linux-hyperv, Stephen Hemminger
In-Reply-To: <20190605192647.GA25034@infradead.org>
On Wed, 5 Jun 2019 12:26:47 -0700
Christoph Hellwig <hch@infradead.org> wrote:
> On Wed, Jun 05, 2019 at 12:10:20PM -0700, Stephen Hemminger wrote:
> > > Sure. But they should not get a way out for just one specific driver.
> >
> > There are people running new kernels on 6 year old distributions.
> > Was every distribution smart enough then? If you think so, then
> > this not necessary.
>
> I think you are missing my point. If we want a way to disable this,
> we:
>
> a) want it opt-in
> b) it needs to for the whole SCSI layer and not just one driver
There is some possible issues with how initial images are deployed
with temporary disks. The temp disks come pre-formatted with a blank
NTFS and cloudinit reformats them to ext4 the first time.
Not sure if ordering matters, or if cloudinit is smart enough to do
the right thing. I am trying to get an answer.
It might just be that the first boot should turn off async probing
for the whole scsi layer via kernel cmdline. Or it might not be an issue
if cloudinit was written by developers smart enough to handle moving
disks.
^ permalink raw reply
* Re: [PATCH 08/13] IB/iser: set virt_boundary_mask in the scsi host
From: Jason Gunthorpe @ 2019-06-05 20:22 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, Sebastian Ott, Sagi Grimberg, Max Gurtovoy,
Bart Van Assche, Ulf Hansson, Alan Stern, Oliver Neukum,
linux-block, linux-rdma, linux-mmc, linux-nvme, linux-scsi,
megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv, linux-usb,
usb-storage, linux-kernel
In-Reply-To: <20190605190836.32354-9-hch@lst.de>
On Wed, Jun 05, 2019 at 09:08:31PM +0200, Christoph Hellwig wrote:
> This ensures all proper DMA layer handling is taken care of by the
> SCSI midlayer.
Maybe not entirely related to this series, but it looks like the SCSI
layer is changing the device global dma_set_max_seg_size() - at least
in RDMA the dma device is being shared between many users, so we
really don't want SCSI to make this value smaller.
Can we do something about this?
Wondering about other values too, and the interaction with the new
combining stuff in umem.c
Thanks,
Jason
^ permalink raw reply
* Re: [PATCH] revert async probing of VMBus scsi device
From: Stephen Hemminger @ 2019-06-05 21:20 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-scsi, linux-hyperv, Stephen Hemminger
In-Reply-To: <20190605192647.GA25034@infradead.org>
On Wed, 5 Jun 2019 12:26:47 -0700
Christoph Hellwig <hch@infradead.org> wrote:
> On Wed, Jun 05, 2019 at 12:10:20PM -0700, Stephen Hemminger wrote:
> > > Sure. But they should not get a way out for just one specific driver.
> >
> > There are people running new kernels on 6 year old distributions.
> > Was every distribution smart enough then? If you think so, then
> > this not necessary.
>
> I think you are missing my point. If we want a way to disable this,
> we:
>
> a) want it opt-in
> b) it needs to for the whole SCSI layer and not just one driver
Based on feedback from the Program Manager who handles the distros
the patch for storvsc is not needed.
SCSI device naming was never deterministic anyway.
Cloud-init relies on the same mechanism as the Azure agent to detect the OS and ephemeral disk.
^ permalink raw reply
* Re: [PATCH 08/13] IB/iser: set virt_boundary_mask in the scsi host
From: Sagi Grimberg @ 2019-06-05 23:35 UTC (permalink / raw)
To: Jason Gunthorpe, Christoph Hellwig
Cc: Jens Axboe, Sebastian Ott, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, megaraidlinux.pdl,
MPT-FusionLinux.pdl, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605202235.GC3273@ziepe.ca>
>> This ensures all proper DMA layer handling is taken care of by the
>> SCSI midlayer.
>
> Maybe not entirely related to this series, but it looks like the SCSI
> layer is changing the device global dma_set_max_seg_size() - at least
> in RDMA the dma device is being shared between many users, so we
> really don't want SCSI to make this value smaller.
>
> Can we do something about this?
srp seems to do the right thing:
target_host->max_segment_size = ib_dma_max_seg_size(ibdev);
But iser does not, which means that scsi limits it to:
BLK_MAX_SEGMENT_SIZE (64k)
I can send a fix to iser.
> Wondering about other values too, and the interaction with the new
> combining stuff in umem.c
The only other values AFAICT is the dma_boundary that rdma llds don't
set...
^ permalink raw reply
* Re: [PATCH 10/13] megaraid_sas: set virt_boundary_mask in the scsi host
From: Hannes Reinecke @ 2019-06-06 6:02 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Sebastian Ott, Sagi Grimberg, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, megaraidlinux.pdl,
MPT-FusionLinux.pdl, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605190836.32354-11-hch@lst.de>
On 6/5/19 9:08 PM, Christoph Hellwig wrote:
> This ensures all proper DMA layer handling is taken care of by the
> SCSI midlayer. Note that the effect is global, as the IOMMU merging
> is based off a paramters in struct device. We could still turn if off
> if no PCIe devices are present, but I don't know how to find that out.
>
> Also remove the bogus nomerges flag, merges do take the virt_boundary
> into account.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> drivers/scsi/megaraid/megaraid_sas_base.c | 46 +++++----------------
> drivers/scsi/megaraid/megaraid_sas_fusion.c | 7 ++++
> 2 files changed, 18 insertions(+), 35 deletions(-)
>
> diff --git a/drivers/scsi/megaraid/megaraid_sas_base.c b/drivers/scsi/megaraid/megaraid_sas_base.c
> index 3dd1df472dc6..20b3b3f8bc16 100644
> --- a/drivers/scsi/megaraid/megaraid_sas_base.c
> +++ b/drivers/scsi/megaraid/megaraid_sas_base.c
> @@ -1870,39 +1870,6 @@ void megasas_set_dynamic_target_properties(struct scsi_device *sdev,
> }
> }
>
> -/*
> - * megasas_set_nvme_device_properties -
> - * set nomerges=2
> - * set virtual page boundary = 4K (current mr_nvme_pg_size is 4K).
> - * set maximum io transfer = MDTS of NVME device provided by MR firmware.
> - *
> - * MR firmware provides value in KB. Caller of this function converts
> - * kb into bytes.
> - *
> - * e.a MDTS=5 means 2^5 * nvme page size. (In case of 4K page size,
> - * MR firmware provides value 128 as (32 * 4K) = 128K.
> - *
> - * @sdev: scsi device
> - * @max_io_size: maximum io transfer size
> - *
> - */
> -static inline void
> -megasas_set_nvme_device_properties(struct scsi_device *sdev, u32 max_io_size)
> -{
> - struct megasas_instance *instance;
> - u32 mr_nvme_pg_size;
> -
> - instance = (struct megasas_instance *)sdev->host->hostdata;
> - mr_nvme_pg_size = max_t(u32, instance->nvme_page_size,
> - MR_DEFAULT_NVME_PAGE_SIZE);
> -
> - blk_queue_max_hw_sectors(sdev->request_queue, (max_io_size / 512));
> -
> - blk_queue_flag_set(QUEUE_FLAG_NOMERGES, sdev->request_queue);
> - blk_queue_virt_boundary(sdev->request_queue, mr_nvme_pg_size - 1);
> -}
> -
> -
> /*
> * megasas_set_static_target_properties -
> * Device property set by driver are static and it is not required to be
> @@ -1961,8 +1928,10 @@ static void megasas_set_static_target_properties(struct scsi_device *sdev,
> max_io_size_kb = le32_to_cpu(instance->tgt_prop->max_io_size_kb);
> }
>
> - if (instance->nvme_page_size && max_io_size_kb)
> - megasas_set_nvme_device_properties(sdev, (max_io_size_kb << 10));
> + if (instance->nvme_page_size && max_io_size_kb) {
> + blk_queue_max_hw_sectors(sdev->request_queue,
> + (max_io_size_kb << 10) / 512);
> + }
>
> scsi_change_queue_depth(sdev, device_qd);
>
What happened to the NOMERGES queue flag?
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
hare@suse.de +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
^ permalink raw reply
* Re: [PATCH 08/13] IB/iser: set virt_boundary_mask in the scsi host
From: Christoph Hellwig @ 2019-06-06 6:24 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Christoph Hellwig, Jens Axboe, Sebastian Ott, Sagi Grimberg,
Max Gurtovoy, Bart Van Assche, Ulf Hansson, Alan Stern,
Oliver Neukum, linux-block, linux-rdma, linux-mmc, linux-nvme,
linux-scsi, megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv,
linux-usb, usb-storage, linux-kernel
In-Reply-To: <20190605202235.GC3273@ziepe.ca>
On Wed, Jun 05, 2019 at 05:22:35PM -0300, Jason Gunthorpe wrote:
> On Wed, Jun 05, 2019 at 09:08:31PM +0200, Christoph Hellwig wrote:
> > This ensures all proper DMA layer handling is taken care of by the
> > SCSI midlayer.
>
> Maybe not entirely related to this series, but it looks like the SCSI
> layer is changing the device global dma_set_max_seg_size() - at least
> in RDMA the dma device is being shared between many users, so we
> really don't want SCSI to make this value smaller.
>
> Can we do something about this?
We could do something about it as outlined in my mail - pass the
dma_params explicitly to the dma_map_sg call. But that isn't really
suitable for a short term fix and will take a little more time.
Until we've sorted that out the device paramter needs to be set to
the smallest value supported.
^ permalink raw reply
* Re: [PATCH 10/13] megaraid_sas: set virt_boundary_mask in the scsi host
From: Christoph Hellwig @ 2019-06-06 6:41 UTC (permalink / raw)
To: Hannes Reinecke
Cc: Christoph Hellwig, Jens Axboe, Sebastian Ott, Sagi Grimberg,
Max Gurtovoy, Bart Van Assche, Ulf Hansson, Alan Stern,
Oliver Neukum, linux-block, linux-rdma, linux-mmc, linux-nvme,
linux-scsi, megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv,
linux-usb, usb-storage, linux-kernel
In-Reply-To: <345c3931-0940-7d59-ebc6-fa1ea56c60ac@suse.de>
On Thu, Jun 06, 2019 at 08:02:07AM +0200, Hannes Reinecke wrote:
> > scsi_change_queue_depth(sdev, device_qd);
> >
> What happened to the NOMERGES queue flag?
Quote from the patch description:
"Also remove the bogus nomerges flag, merges do take the virt_boundary
into account."
^ permalink raw reply
* [PATCH v2 0/5] hv: Remove dependencies on guest page size
From: Maya Nakamura @ 2019-06-06 8:00 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
The Linux guest page size and hypervisor page size concepts are
different, even though they happen to be the same value on x86. Hyper-V
code mixes up the two, so this patchset begins to address that by
creating and using a set of Hyper-V specific page definitions.
A major benefit of those new definitions is that they support non-x86
architectures, such as ARM64, that use different page sizes. On ARM64,
the guest page size may not be 4096, and Hyper-V always runs with a page
size of 4096.
In this patchset, the first two patches lay the foundation for the
others, creating definitions and preparing for allocation of memory with
the size and alignment that Hyper-V expects as a page. Patch 3 applies
the page size definition where the guest VM and Hyper-V communicate, and
where the code intends to use the Hyper-V page size. The last two
patches set the ring buffer size to a fixed value, removing the
dependency on the guest page size.
This is the initial set of changes to the Hyper-V code, and future
patches will make additional changes using the same foundation, for
example, replace __vmalloc() and related functions when Hyper-V pages
are intended.
Changes in v2:
- [PATCH 2/5] Replace with a new patch.
Maya Nakamura (5):
x86: hv: hyperv-tlfs.h: Create and use Hyper-V page definitions
x86: hv: hv_init.c: Add functions to allocate/deallocate page for
Hyper-V
hv: vmbus: Replace page definition with Hyper-V specific one
HID: hv: Remove dependencies on PAGE_SIZE for ring buffer
Input: hv: Remove dependencies on PAGE_SIZE for ring buffer
arch/x86/hyperv/hv_init.c | 14 ++++++++++++++
arch/x86/include/asm/hyperv-tlfs.h | 12 +++++++++++-
drivers/hid/hid-hyperv.c | 4 ++--
drivers/hv/hyperv_vmbus.h | 8 ++++----
drivers/input/serio/hyperv-keyboard.c | 4 ++--
5 files changed, 33 insertions(+), 9 deletions(-)
--
2.17.1
^ permalink raw reply
* [PATCH v2 1/5] x86: hv: hyperv-tlfs.h: Create and use Hyper-V page definitions
From: Maya Nakamura @ 2019-06-06 8:03 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
In-Reply-To: <cover.1559807514.git.m.maya.nakamura@gmail.com>
Define HV_HYP_PAGE_SHIFT, HV_HYP_PAGE_SIZE, and HV_HYP_PAGE_MASK because
the Linux guest page size and hypervisor page size concepts are
different, even though they happen to be the same value on x86.
Also, replace PAGE_SIZE with HV_HYP_PAGE_SIZE.
Signed-off-by: Maya Nakamura <m.maya.nakamura@gmail.com>
---
arch/x86/include/asm/hyperv-tlfs.h | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h
index cdf44aa9a501..44bd68aefd00 100644
--- a/arch/x86/include/asm/hyperv-tlfs.h
+++ b/arch/x86/include/asm/hyperv-tlfs.h
@@ -12,6 +12,16 @@
#include <linux/types.h>
#include <asm/page.h>
+/*
+ * While not explicitly listed in the TLFS, Hyper-V always runs with a page size
+ * of 4096. These definitions are used when communicating with Hyper-V using
+ * guest physical pages and guest physical page addresses, since the guest page
+ * size may not be 4096 on all architectures.
+ */
+#define HV_HYP_PAGE_SHIFT 12
+#define HV_HYP_PAGE_SIZE BIT(HV_HYP_PAGE_SHIFT)
+#define HV_HYP_PAGE_MASK (~(HV_HYP_PAGE_SIZE - 1))
+
/*
* The below CPUID leaves are present if VersionAndFeatures.HypervisorPresent
* is set by CPUID(HvCpuIdFunctionVersionAndFeatures).
@@ -841,7 +851,7 @@ union hv_gpa_page_range {
* count is equal with how many entries of union hv_gpa_page_range can
* be populated into the input parameter page.
*/
-#define HV_MAX_FLUSH_REP_COUNT ((PAGE_SIZE - 2 * sizeof(u64)) / \
+#define HV_MAX_FLUSH_REP_COUNT ((HV_HYP_PAGE_SIZE - 2 * sizeof(u64)) / \
sizeof(union hv_gpa_page_range))
struct hv_guest_mapping_flush_list {
--
2.17.1
^ permalink raw reply related
* [PATCH v2 2/5] x86: hv: hv_init.c: Add functions to allocate/deallocate page for Hyper-V
From: Maya Nakamura @ 2019-06-06 8:05 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
In-Reply-To: <cover.1559807514.git.m.maya.nakamura@gmail.com>
Introduce two new functions, hv_alloc_hyperv_page() and
hv_free_hyperv_page(), to allocate/deallocate memory with the size and
alignment that Hyper-V expects as a page. Although currently they are
not used, they are ready to be used to allocate/deallocate memory on x86
when their ARM64 counterparts are implemented, keeping symmetry between
architectures with potentially different guest page sizes.
Signed-off-by: Maya Nakamura <m.maya.nakamura@gmail.com>
---
arch/x86/hyperv/hv_init.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/x86/hyperv/hv_init.c b/arch/x86/hyperv/hv_init.c
index e4ba467a9fc6..84baf0e9a2d4 100644
--- a/arch/x86/hyperv/hv_init.c
+++ b/arch/x86/hyperv/hv_init.c
@@ -98,6 +98,20 @@ EXPORT_SYMBOL_GPL(hyperv_pcpu_input_arg);
u32 hv_max_vp_index;
EXPORT_SYMBOL_GPL(hv_max_vp_index);
+void *hv_alloc_hyperv_page(void)
+{
+ BUILD_BUG_ON(!(PAGE_SIZE == HV_HYP_PAGE_SIZE));
+
+ return (void *)__get_free_page(GFP_KERNEL);
+}
+EXPORT_SYMBOL_GPL(hv_alloc_hyperv_page);
+
+void hv_free_hyperv_page(unsigned long addr)
+{
+ free_page(addr);
+}
+EXPORT_SYMBOL_GPL(hv_free_hyperv_page);
+
static int hv_cpu_init(unsigned int cpu)
{
u64 msr_vp_index;
--
2.17.1
^ permalink raw reply related
* [PATCH v2 3/5] hv: vmbus: Replace page definition with Hyper-V specific one
From: Maya Nakamura @ 2019-06-06 8:06 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
In-Reply-To: <cover.1559807514.git.m.maya.nakamura@gmail.com>
Replace PAGE_SIZE with HV_HYP_PAGE_SIZE because the guest page size may
not be 4096 on all architectures and Hyper-V always runs with a page
size of 4096.
Signed-off-by: Maya Nakamura <m.maya.nakamura@gmail.com>
---
drivers/hv/hyperv_vmbus.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index e5467b821f41..5489b061d261 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -208,11 +208,11 @@ int hv_ringbuffer_read(struct vmbus_channel *channel,
u64 *requestid, bool raw);
/*
- * Maximum channels is determined by the size of the interrupt page
- * which is PAGE_SIZE. 1/2 of PAGE_SIZE is for send endpoint interrupt
- * and the other is receive endpoint interrupt
+ * Maximum channels, 16348, is determined by the size of the interrupt page,
+ * which is HV_HYP_PAGE_SIZE. 1/2 of HV_HYP_PAGE_SIZE is to send endpoint
+ * interrupt, and the other is to receive endpoint interrupt.
*/
-#define MAX_NUM_CHANNELS ((PAGE_SIZE >> 1) << 3) /* 16348 channels */
+#define MAX_NUM_CHANNELS ((HV_HYP_PAGE_SIZE >> 1) << 3)
/* The value here must be in multiple of 32 */
/* TODO: Need to make this configurable */
--
2.17.1
^ permalink raw reply related
* [PATCH v2 4/5] HID: hv: Remove dependencies on PAGE_SIZE for ring buffer
From: Maya Nakamura @ 2019-06-06 8:07 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
In-Reply-To: <cover.1559807514.git.m.maya.nakamura@gmail.com>
Define the ring buffer size as a constant expression because it should
not depend on the guest page size.
Signed-off-by: Maya Nakamura <m.maya.nakamura@gmail.com>
---
drivers/hid/hid-hyperv.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/hid/hid-hyperv.c b/drivers/hid/hid-hyperv.c
index d3311d714d35..e8b154fa38e2 100644
--- a/drivers/hid/hid-hyperv.c
+++ b/drivers/hid/hid-hyperv.c
@@ -112,8 +112,8 @@ struct synthhid_input_report {
#pragma pack(pop)
-#define INPUTVSC_SEND_RING_BUFFER_SIZE (10*PAGE_SIZE)
-#define INPUTVSC_RECV_RING_BUFFER_SIZE (10*PAGE_SIZE)
+#define INPUTVSC_SEND_RING_BUFFER_SIZE (40 * 1024)
+#define INPUTVSC_RECV_RING_BUFFER_SIZE (40 * 1024)
enum pipe_prot_msg_type {
--
2.17.1
^ permalink raw reply related
* [PATCH v2 5/5] Input: hv: Remove dependencies on PAGE_SIZE for ring buffer
From: Maya Nakamura @ 2019-06-06 8:09 UTC (permalink / raw)
To: mikelley, kys, haiyangz, sthemmin, sashal; +Cc: x86, linux-hyperv, linux-kernel
In-Reply-To: <cover.1559807514.git.m.maya.nakamura@gmail.com>
Define the ring buffer size as a constant expression because it should
not depend on the guest page size.
Signed-off-by: Maya Nakamura <m.maya.nakamura@gmail.com>
---
drivers/input/serio/hyperv-keyboard.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/input/serio/hyperv-keyboard.c b/drivers/input/serio/hyperv-keyboard.c
index 7935e52b5435..a3480dbfadd2 100644
--- a/drivers/input/serio/hyperv-keyboard.c
+++ b/drivers/input/serio/hyperv-keyboard.c
@@ -83,8 +83,8 @@ struct synth_kbd_keystroke {
#define HK_MAXIMUM_MESSAGE_SIZE 256
-#define KBD_VSC_SEND_RING_BUFFER_SIZE (10 * PAGE_SIZE)
-#define KBD_VSC_RECV_RING_BUFFER_SIZE (10 * PAGE_SIZE)
+#define KBD_VSC_SEND_RING_BUFFER_SIZE (40 * 1024)
+#define KBD_VSC_RECV_RING_BUFFER_SIZE (40 * 1024)
#define XTKBD_EMUL0 0xe0
#define XTKBD_EMUL1 0xe1
--
2.17.1
^ permalink raw reply related
* Re: [PATCH 08/13] IB/iser: set virt_boundary_mask in the scsi host
From: Jason Gunthorpe @ 2019-06-06 12:59 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, Sebastian Ott, Sagi Grimberg, Max Gurtovoy,
Bart Van Assche, Ulf Hansson, Alan Stern, Oliver Neukum,
linux-block, linux-rdma, linux-mmc, linux-nvme, linux-scsi,
megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv, linux-usb,
usb-storage, linux-kernel
In-Reply-To: <20190606062441.GB26745@lst.de>
On Thu, Jun 06, 2019 at 08:24:41AM +0200, Christoph Hellwig wrote:
> On Wed, Jun 05, 2019 at 05:22:35PM -0300, Jason Gunthorpe wrote:
> > On Wed, Jun 05, 2019 at 09:08:31PM +0200, Christoph Hellwig wrote:
> > > This ensures all proper DMA layer handling is taken care of by the
> > > SCSI midlayer.
> >
> > Maybe not entirely related to this series, but it looks like the SCSI
> > layer is changing the device global dma_set_max_seg_size() - at least
> > in RDMA the dma device is being shared between many users, so we
> > really don't want SCSI to make this value smaller.
> >
> > Can we do something about this?
>
> We could do something about it as outlined in my mail - pass the
> dma_params explicitly to the dma_map_sg call. But that isn't really
> suitable for a short term fix and will take a little more time.
Sounds good to me, having every dma mapping specify its restrictions
makes a lot more sense than a device global setting, IMHO.
In RDMA the restrictions to build a SGL, create a device queue or
build a MR are all a little different.
ie for MRs alignment of the post-IOMMU DMA address is very important
for performance as the MR logic can only build device huge pages out
of properly aligned DMA addresses. While for SGLs we don't care about
this, instead SGLs usually have the 32 bit per-element length limit in
the HW that MRs do not.
> Until we've sorted that out the device paramter needs to be set to
> the smallest value supported.
smallest? largest? We've been setting it to the largest value the
device can handle (ie 2G)
Jason
^ permalink raw reply
* Re: [PATCH 08/13] IB/iser: set virt_boundary_mask in the scsi host
From: Christoph Hellwig @ 2019-06-06 14:19 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Christoph Hellwig, Jens Axboe, Sebastian Ott, Sagi Grimberg,
Max Gurtovoy, Bart Van Assche, Ulf Hansson, Alan Stern,
Oliver Neukum, linux-block, linux-rdma, linux-mmc, linux-nvme,
linux-scsi, megaraidlinux.pdl, MPT-FusionLinux.pdl, linux-hyperv,
linux-usb, usb-storage, linux-kernel
In-Reply-To: <20190606125935.GA17373@ziepe.ca>
On Thu, Jun 06, 2019 at 09:59:35AM -0300, Jason Gunthorpe wrote:
> > Until we've sorted that out the device paramter needs to be set to
> > the smallest value supported.
>
> smallest? largest? We've been setting it to the largest value the
> device can handle (ie 2G)
Well, in general we need the smallest value supported by any ULP,
because if any ULP can't support a larger segment size, we must not
allow the IOMMU to merge it to that size. That being said I can't
really see why any RDMA ULP should limit the size given how the MRs
work.
^ permalink raw reply
* RE: [PATCH 10/13] megaraid_sas: set virt_boundary_mask in the scsi host
From: Kashyap Desai @ 2019-06-06 15:37 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Sebastian Ott, Sagi Grimberg, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, PDL,MEGARAIDLINUX,
PDL-MPT-FUSIONLINUX, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605190836.32354-11-hch@lst.de>
>
> This ensures all proper DMA layer handling is taken care of by the SCSI
> midlayer. Note that the effect is global, as the IOMMU merging is based
> off a
> paramters in struct device. We could still turn if off if no PCIe devices
> are
> present, but I don't know how to find that out.
>
> Also remove the bogus nomerges flag, merges do take the virt_boundary into
> account.
Hi Christoph, Changes for <megaraid_sas> and <mpt3sas> looks good. We want
to confirm few sanity before ACK. BTW, what benefit we will see moving
virt_boundry setting to SCSI mid layer ? Is it just modular approach OR any
functional fix ?
Kashyap
^ permalink raw reply
* Re: [PATCH v3 0/2] Drivers: hv: Move Hyper-V clock/timer code to separate clocksource driver
From: Sasha Levin @ 2019-06-06 16:37 UTC (permalink / raw)
To: Michael Kelley
Cc: will.deacon@arm.com, marc.zyngier@arm.com,
linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org,
linux-kernel@vger.kernel.org, linux-hyperv@vger.kernel.org,
olaf@aepfle.de, apw@canonical.com, vkuznets, jasowang@redhat.com,
marcelo.cerri@canonical.com, Sunil Muthuswamy, KY Srinivasan,
catalin.marinas@arm.com, mark.rutland@arm.com
In-Reply-To: <1558969089-13204-1-git-send-email-mikelley@microsoft.com>
On Mon, May 27, 2019 at 02:59:07PM +0000, Michael Kelley wrote:
>This patch series moves Hyper-V clock/timer code to a separate Hyper-V
>clocksource driver. Previously, Hyper-V clock/timer code and data
>structures were mixed in with other Hyper-V code in the ISA independent
>drivers/hv code as well as in arch dependent code. The new Hyper-V
>clocksource driver is ISA independent, with a just few dependencies on
>arch specific functions. The patch series does not change any behavior
>or functionality -- it only reorganizes the existing code and fixes up
>the linkages. A few places outside of Hyper-V code are fixed up to use
>the new #include file structure.
>
>This restructuring is in response to Marc Zyngier's review comments
>on supporting Hyper-V running on ARM64, and is a good idea in general.
>It increases the amount of code shared between the x86 and ARM64
>architectures, and reduces the size of the new code for supporting
>Hyper-V on ARM64. A new version of the Hyper-V on ARM64 patches will
>follow once this clocksource restructuring is accepted.
>
>The code is diff'ed against Linux 5.2.0-rc1-next-20190524.
>
>Changes in v3:
>* Removed boolean argument to hv_init_clocksource(). Always call
>sched_clock_register, which is needed on ARM64 but a no-op on x86.
>* Removed separate cpuhp setup in hv_stimer_alloc() and instead
>directly call hv_stimer_init() and hv_stimer_cleanup() from
>corresponding VMbus functions. This more closely matches original
>code and avoids clocksource stop/restart problems on ARM64 when
>VMbus code denies CPU offlining request.
>
>Changes in v2:
>* Revised commit short descriptions so the distinction between
>the first and second patches is clearer [GregKH]
>* Renamed new clocksource driver files and functions to use
>existing "timer" and "stimer" names instead of introducing
>"syntimer". [Vitaly Kuznetsov]
>* Introduced CONFIG_HYPER_TIMER to fix build problem when
>CONFIG_HYPERV=m [Vitaly Kuznetsov]
>* Added "Suggested-by: Marc Zyngier"
>
>Michael Kelley (2):
> Drivers: hv: Create Hyper-V clocksource driver from existing
> clockevents code
> Drivers: hv: Move Hyper-V clocksource code to new clocksource driver
>
> MAINTAINERS | 2 +
> arch/x86/entry/vdso/vclock_gettime.c | 1 +
> arch/x86/entry/vdso/vma.c | 2 +-
> arch/x86/hyperv/hv_init.c | 91 +---------
> arch/x86/include/asm/hyperv-tlfs.h | 6 +
> arch/x86/include/asm/mshyperv.h | 81 ++-------
> arch/x86/kernel/cpu/mshyperv.c | 2 +
> arch/x86/kvm/x86.c | 1 +
> drivers/clocksource/Makefile | 1 +
> drivers/clocksource/hyperv_timer.c | 321 +++++++++++++++++++++++++++++++++++
> drivers/hv/Kconfig | 3 +
> drivers/hv/hv.c | 156 +----------------
> drivers/hv/hv_util.c | 1 +
> drivers/hv/hyperv_vmbus.h | 3 -
> drivers/hv/vmbus_drv.c | 42 ++---
> include/clocksource/hyperv_timer.h | 105 ++++++++++++
> 16 files changed, 484 insertions(+), 334 deletions(-)
> create mode 100644 drivers/clocksource/hyperv_timer.c
> create mode 100644 include/clocksource/hyperv_timer.h
Queued for hyperv-next, thanks!
--
Thanks,
Sasha
^ permalink raw reply
* Re: [PATCH v2 1/1] Drivers: hv: vmbus: Break out ISA independent parts of mshyperv.h
From: Sasha Levin @ 2019-06-06 16:50 UTC (permalink / raw)
To: Michael Kelley
Cc: linux-kernel@vger.kernel.org, linux-hyperv@vger.kernel.org,
olaf@aepfle.de, apw@canonical.com, vkuznets, jasowang@redhat.com,
marcelo.cerri@canonical.com, Sunil Muthuswamy, KY Srinivasan
In-Reply-To: <1559175219-17823-1-git-send-email-mikelley@microsoft.com>
On Thu, May 30, 2019 at 12:14:00AM +0000, Michael Kelley wrote:
>Break out parts of mshyperv.h that are ISA independent into a
>separate file in include/asm-generic. This move facilitates
>ARM64 code reusing these definitions and avoids code
>duplication. No functionality or behavior is changed.
>
>Signed-off-by: Michael Kelley <mikelley@microsoft.com>
>Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>---
>Changes in v2:
>* Removed unneeded #includes in asm-generic/mshyperv.h;
>added two #includes that are needed. [Vitaly Kuznetsov]
Queued for hyperv-next, thanks!
--
Thanks,
Sasha
^ permalink raw reply
* Re: [PATCH] revert async probing of VMBus network devices.
From: Stephen Hemminger @ 2019-06-06 20:56 UTC (permalink / raw)
To: netdev; +Cc: linux-hyperv, Stephen Hemminger
In-Reply-To: <20190605185114.12456-1-sthemmin@microsoft.com>
On Wed, 5 Jun 2019 11:51:14 -0700
Stephen Hemminger <stephen@networkplumber.org> wrote:
> Doing asynchronous probing can lead to reordered network device names.
> And because udev doesn't have any useful information to construct a
> persistent name, this causes VM's to sporadically boot with reordered
> device names and no connectivity.
>
> This shows up on the Ubuntu image on larger VM's where 30% of the
> time eth0 and eth1 get swapped.
>
> Note: udev MAC address policy is disabled on Azure images
> because the netvsc and PCI VF will have the same mac address.
>
> Fixes: af0a5646cb8d ("use the new async probing feature for the hyperv drivers")
> Signed-off-by: Stephen Hemminger <sthemmin@microsoft.com>
> ---
> drivers/net/hyperv/netvsc_drv.c | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
> index 06393b215102..1a2c32111106 100644
> --- a/drivers/net/hyperv/netvsc_drv.c
> +++ b/drivers/net/hyperv/netvsc_drv.c
> @@ -2411,9 +2411,6 @@ static struct hv_driver netvsc_drv = {
> .id_table = id_table,
> .probe = netvsc_probe,
> .remove = netvsc_remove,
> - .driver = {
> - .probe_type = PROBE_PREFER_ASYNCHRONOUS,
> - },
> };
>
> /*
Even though storage can handle out of order devices, networking can not.
The network devices in Hyper-V do not have any persistant properties that will
work with existing udev. The current kernel is breaking current distributions.
This patch fixes it, why did you reject it?
^ permalink raw reply
* Re: properly communicate queue limits to the DMA layer
From: Jens Axboe @ 2019-06-07 5:52 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Sebastian Ott, Sagi Grimberg, Max Gurtovoy, Bart Van Assche,
Ulf Hansson, Alan Stern, Oliver Neukum, linux-block, linux-rdma,
linux-mmc, linux-nvme, linux-scsi, megaraidlinux.pdl,
MPT-FusionLinux.pdl, linux-hyperv, linux-usb, usb-storage,
linux-kernel
In-Reply-To: <20190605192405.GA18243@lst.de>
On 6/5/19 1:24 PM, Christoph Hellwig wrote:
> On Wed, Jun 05, 2019 at 01:17:15PM -0600, Jens Axboe wrote:
>> Since I'm heading out shortly and since I think this should make
>> the next -rc, I'll tentatively queue this up.
>
> The SCSI bits will need a bit more review, and possibly tweaking
> fo megaraid and mpt3sas. But they are really independent of the
> other patches, so maybe skip them for now and leave them for Martin
> to deal with.
I dropped the SCSI bits.
--
Jens Axboe
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox