drivres/hv

drivres/hv

[K. Y. Srinivasan]

 
Greg, over the last two months we have sent numerous Hyper-V patches and
these are yet to be comitted (all review comments have been addressed
for these patches). Please let me know if I should resend these patches.

Regards,

K. Y

Re: drivres/hv

[Greg KH]

On Sat, Aug 01, 2015 at 12:46:10PM -0700, K. Y. Srinivasan wrote:
>  
> Greg, over the last two months we have sent numerous Hyper-V patches and
> these are yet to be comitted (all review comments have been addressed
> for these patches). Please let me know if I should resend these patches.

They are in my queue, I've been slow at getting to them, sorry.  If you
want to resend them so that I know exactly which ones to apply, instead
of having to dig through the different 'resend' and 'v2' series, that
would be most appreciated.

thanks,

greg k-h

RE: drivres/hv

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Saturday, August 1, 2015 11:36 AM
> To: KY Srinivasan <kys@microsoft.com>
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org;
> olaf@aepfle.de; apw@canonical.com; vkuznets@redhat.com;
> jasowang@redhat.com
> Subject: Re: drivres/hv
> 
> On Sat, Aug 01, 2015 at 12:46:10PM -0700, K. Y. Srinivasan wrote:
> >
> > Greg, over the last two months we have sent numerous Hyper-V patches
> and
> > these are yet to be comitted (all review comments have been addressed
> > for these patches). Please let me know if I should resend these patches.
> 
> They are in my queue, I've been slow at getting to them, sorry.  If you
> want to resend them so that I know exactly which ones to apply, instead
> of having to dig through the different 'resend' and 'v2' series, that
> would be most appreciated.

Thanks Greg; I can understand. I will send them out right away as one large set.

Regards,

K. Y
> 
> thanks,
> 
> greg k-h

drivres/hv

[K. Y. Srinivasan]

 
Greg, over the last month or more we have sent numerous Hyper-V patches and
these are yet to be comitted (all review comments have been addressed
for these patches). Please let me know if I should resend these patches.

Regards,

K. Y

Re: drivres/hv

[Greg KH]

On Mon, Nov 30, 2015 at 01:15:32PM -0800, K. Y. Srinivasan wrote:
>  
> Greg, over the last month or more we have sent numerous Hyper-V patches and
> these are yet to be comitted (all review comments have been addressed
> for these patches). Please let me know if I should resend these patches.

All should now be applied.

drivres/hv

[K. Y. Srinivasan]

 
Greg,

I recently discovered that the Hyper-V host allocates mmio
memory for some synthetic devices like the virtual framebuffer.
We are currently in the process of implementing this virtual
framebuffer device. As part of the offer message from the host
we are given the mmio region size that needs to be allocatted to
the device. I am told in the current implementation of the firmware,
this mmio resource shows up in the PCI space. What is the best way to
allocate this mmio space for this driver.


Regards,

K. Y

Re: drivres/hv

[Greg KH]

On Wed, Nov 28, 2012 at 12:17:30PM -0800, K. Y. Srinivasan wrote:
>  
> Greg,

You are writing the most unhelpful Subject: lines lately, please be more
descriptive in the future.

> I recently discovered that the Hyper-V host allocates mmio
> memory for some synthetic devices like the virtual framebuffer.
> We are currently in the process of implementing this virtual
> framebuffer device. As part of the offer message from the host
> we are given the mmio region size that needs to be allocatted to
> the device. I am told in the current implementation of the firmware,
> this mmio resource shows up in the PCI space. What is the best way to
> allocate this mmio space for this driver.

I don't understand, does the guest os think this really is mmio memory
and the host just set up up?  Or is the memory being used to pass data
back and forth?  Or something else?

And what do you mean "firmware"?  That usually means UEFI/BIOS to me,
not a hyperv host.

And finally, what does the guest os see as far as the PCI resource space
shows it?  Shouldn't it just think it is a normal PCI device and access
it properly that way?

thanks,

greg k-h

RE: drivres/hv

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Wednesday, November 28, 2012 5:43 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> Subject: Re: drivres/hv
> 
> On Wed, Nov 28, 2012 at 12:17:30PM -0800, K. Y. Srinivasan wrote:
> >
> > Greg,
> 
> You are writing the most unhelpful Subject: lines lately, please be more
> descriptive in the future.

Sorry about that; I will be more descriptive.

> 
> > I recently discovered that the Hyper-V host allocates mmio
> > memory for some synthetic devices like the virtual framebuffer.
> > We are currently in the process of implementing this virtual
> > framebuffer device. As part of the offer message from the host
> > we are given the mmio region size that needs to be allocatted to
> > the device. I am told in the current implementation of the firmware,
> > this mmio resource shows up in the PCI space. What is the best way to
> > allocate this mmio space for this driver.
> 
> I don't understand, does the guest os think this really is mmio memory
> and the host just set up up?  Or is the memory being used to pass data
> back and forth?  Or something else?

>From what I understand, the host is allocating this memory and presenting
this to the guest. This memory is to be used for framebuffer. The guest is
expected to setup appropriate mappings to this region and use that as the 
framebuffer.

> 
> And what do you mean "firmware"?  That usually means UEFI/BIOS to me,
> not a hyperv host.

Hyper-V host presents the firmware to the guest (Hyper-V like KVM supports
full virtualization with selective para-virtualization - PV drivers). As part of the
BIOS presented to the guest this mmio region is allocated.

> 
> And finally, what does the guest os see as far as the PCI resource space
> shows it?  Shouldn't it just think it is a normal PCI device and access
> it properly that way?

This is the crux of the problem. Vmbus and other vmbus based devices are not
PCI  devices. I have implemented vmbus as an ACPI driver since vmbus is an ACPI
enumerated device. I have the size information of the mmio region that is to be 
allocated to the framebuffer device. I am looking at a way to allocate this mmio
region.

> 
> thanks,
> 
> greg k-h
> 

Re: drivres/hv

[Greg KH]

On Wed, Nov 28, 2012 at 11:32:52PM +0000, KY Srinivasan wrote:
> > > I recently discovered that the Hyper-V host allocates mmio
> > > memory for some synthetic devices like the virtual framebuffer.
> > > We are currently in the process of implementing this virtual
> > > framebuffer device. As part of the offer message from the host
> > > we are given the mmio region size that needs to be allocatted to
> > > the device. I am told in the current implementation of the firmware,
> > > this mmio resource shows up in the PCI space. What is the best way to
> > > allocate this mmio space for this driver.
> > 
> > I don't understand, does the guest os think this really is mmio memory
> > and the host just set up up?  Or is the memory being used to pass data
> > back and forth?  Or something else?
> 
> From what I understand, the host is allocating this memory and presenting
> this to the guest. This memory is to be used for framebuffer. The guest is
> expected to setup appropriate mappings to this region and use that as the 
> framebuffer.

What type of framebuffer?  Linux supports a ton of different ones, have
you looked at drivers/video/ to see if one of the ones there will "just
work" with what hyperv is passing to the guest?

If not, you'll have to write a new driver for this, look at the examples
of other framebuffer drivers in that directory for examples.  Also, I
bet the code there will answer your memory questions.

> > And what do you mean "firmware"?  That usually means UEFI/BIOS to me,
> > not a hyperv host.
> 
> Hyper-V host presents the firmware to the guest (Hyper-V like KVM supports
> full virtualization with selective para-virtualization - PV drivers). As part of the
> BIOS presented to the guest this mmio region is allocated.
> 
> > 
> > And finally, what does the guest os see as far as the PCI resource space
> > shows it?  Shouldn't it just think it is a normal PCI device and access
> > it properly that way?
> 
> This is the crux of the problem. Vmbus and other vmbus based devices are not
> PCI  devices. I have implemented vmbus as an ACPI driver since vmbus is an ACPI
> enumerated device. I have the size information of the mmio region that is to be 
> allocated to the framebuffer device. I am looking at a way to allocate this mmio
> region.

Do you have any "hints" as to where it is, if it even is present, or
anything else?  If so, use them and create a new framebuffer device with
that information.

If not, that seems really strange, but hey, it's hyper-v, it wouldn't be
the first time :)

good luck,

greg k-h

RE: drivres/hv

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Wednesday, November 28, 2012 7:05 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> Subject: Re: drivres/hv
> 
> On Wed, Nov 28, 2012 at 11:32:52PM +0000, KY Srinivasan wrote:
> > > > I recently discovered that the Hyper-V host allocates mmio
> > > > memory for some synthetic devices like the virtual framebuffer.
> > > > We are currently in the process of implementing this virtual
> > > > framebuffer device. As part of the offer message from the host
> > > > we are given the mmio region size that needs to be allocatted to
> > > > the device. I am told in the current implementation of the firmware,
> > > > this mmio resource shows up in the PCI space. What is the best way to
> > > > allocate this mmio space for this driver.
> > >
> > > I don't understand, does the guest os think this really is mmio memory
> > > and the host just set up up?  Or is the memory being used to pass data
> > > back and forth?  Or something else?
> >
> > From what I understand, the host is allocating this memory and presenting
> > this to the guest. This memory is to be used for framebuffer. The guest is
> > expected to setup appropriate mappings to this region and use that as the
> > framebuffer.
> 
> What type of framebuffer?  Linux supports a ton of different ones, have
> you looked at drivers/video/ to see if one of the ones there will "just
> work" with what hyperv is passing to the guest?

We are writing a diver for this device. The existing Linux drivers can (and do)
work for the emulated VGA device. What we are implementing is a synthetic
framebuffer driver that will communicate over the vmbus to the host.
 
> 
> If not, you'll have to write a new driver for this, look at the examples
> of other framebuffer drivers in that directory for examples.  Also, I
> bet the code there will answer your memory questions.
> 
> > > And what do you mean "firmware"?  That usually means UEFI/BIOS to me,
> > > not a hyperv host.
> >
> > Hyper-V host presents the firmware to the guest (Hyper-V like KVM supports
> > full virtualization with selective para-virtualization - PV drivers). As part of the
> > BIOS presented to the guest this mmio region is allocated.
> >
> > >
> > > And finally, what does the guest os see as far as the PCI resource space
> > > shows it?  Shouldn't it just think it is a normal PCI device and access
> > > it properly that way?
> >
> > This is the crux of the problem. Vmbus and other vmbus based devices are not
> > PCI  devices. I have implemented vmbus as an ACPI driver since vmbus is an
> ACPI
> > enumerated device. I have the size information of the mmio region that is to be
> > allocated to the framebuffer device. I am looking at a way to allocate this mmio
> > region.
> 
> Do you have any "hints" as to where it is, if it even is present, or
> anything else?  If so, use them and create a new framebuffer device with
> that information.

All I know is that it is present and its size is known. If I have a way of knowing
what range of mmio memory is unclaimed; I could grab that perhaps using
the request_mem_region() call, I know the size that is reserved for this device.
So, is there a way to query the system for the first unclaimed address in the mmio
range.

Thanks,

K. Y
> 

Re: drivres/hv

[Alan Cox]

> All I know is that it is present and its size is known. If I have a way of knowing
> what range of mmio memory is unclaimed; I could grab that perhaps using
> the request_mem_region() call, I know the size that is reserved for this device.
> So, is there a way to query the system for the first unclaimed address in the mmio
> range.

You can add memory region resources yes. See the code in drivers/pci
which allocates mmio space to unassigned BARs and/or moves stuff around.

Alan

drivres/hv

[K. Y. Srinivasan]

 
Greg,

Recently, I had  re-sent a few patches. You have applied all the patches except the balloon driver
related patches. Should I resend the balloon driver  patches. Let me know.

Regards,

K. Y

Re: drivres/hv

[Greg KH]

On Fri, Nov 02, 2012 at 03:11:23PM -0700, K. Y. Srinivasan wrote:
>  
> Greg,
> 
> Recently, I had  re-sent a few patches. You have applied all the patches except the balloon driver
> related patches. Should I resend the balloon driver  patches. Let me know.

I can't apply the balloon driver until you get an ack from the
maintainers of the code you were exporting the symbols from.

Get that, and then resend them, as they are long gone from my "to-apply"
queue.

thanks,

greg k-h

RE: drivres/hv

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Friday, November 02, 2012 7:37 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> Subject: Re: drivres/hv
> 
> On Fri, Nov 02, 2012 at 03:11:23PM -0700, K. Y. Srinivasan wrote:
> >
> > Greg,
> >
> > Recently, I had  re-sent a few patches. You have applied all the patches except
> the balloon driver
> > related patches. Should I resend the balloon driver  patches. Let me know.
> 
> I can't apply the balloon driver until you get an ack from the
> maintainers of the code you were exporting the symbols from.
> 
> Get that, and then resend them, as they are long gone from my "to-apply"
> queue.

I was under  the impression Andrew was ok with the export - I have resent the original email
from Andrew on this. Let me know if this is enough. I will resend the patches once I hear from you.

Regards,

K. Y 

RE: drivres/hv

[KY Srinivasan]

> -----Original Message-----
> From: Greg KH [mailto:gregkh@linuxfoundation.org]
> Sent: Friday, November 02, 2012 7:37 PM
> To: KY Srinivasan
> Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> Subject: Re: drivres/hv
> 
> On Fri, Nov 02, 2012 at 03:11:23PM -0700, K. Y. Srinivasan wrote:
> >
> > Greg,
> >
> > Recently, I had  re-sent a few patches. You have applied all the patches except
> the balloon driver
> > related patches. Should I resend the balloon driver  patches. Let me know.
> 
> I can't apply the balloon driver until you get an ack from the
> maintainers of the code you were exporting the symbols from.
> 
> Get that, and then resend them, as they are long gone from my "to-apply"
> queue.

Greg,

Andrew has checked in my patch that exports the necessary function for Hyper-V
balloon driver:

commit bb2495a71b8499bdfe207738bc88ffa91ebe0b0a
Author: K. Y. Srinivasan <kys@microsoft.com>
Date:   Thu Nov 15 13:37:59 2012 +1100

I had sent the new balloon driver that used this new function a few days ago. Should I
resend the balloon driver. Let me know.

Regards,

K. Y

Re: drivres/hv

[Andrew Morton]

On Thu, 15 Nov 2012 22:22:06 +0000
KY Srinivasan <kys@microsoft.com> wrote:

> 
> 
> > -----Original Message-----
> > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > Sent: Friday, November 02, 2012 7:37 PM
> > To: KY Srinivasan
> > Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> > Subject: Re: drivres/hv
> > 
> > On Fri, Nov 02, 2012 at 03:11:23PM -0700, K. Y. Srinivasan wrote:
> > >
> > > Greg,
> > >
> > > Recently, I had  re-sent a few patches. You have applied all the patches except
> > the balloon driver
> > > related patches. Should I resend the balloon driver  patches. Let me know.
> > 
> > I can't apply the balloon driver until you get an ack from the
> > maintainers of the code you were exporting the symbols from.
> > 
> > Get that, and then resend them, as they are long gone from my "to-apply"
> > queue.
> 
> Greg,
> 
> Andrew has checked in my patch that exports the necessary function for Hyper-V
> balloon driver:
> 
> commit bb2495a71b8499bdfe207738bc88ffa91ebe0b0a
> Author: K. Y. Srinivasan <kys@microsoft.com>
> Date:   Thu Nov 15 13:37:59 2012 +1100
> 
> I had sent the new balloon driver that used this new function a few days ago. Should I
> resend the balloon driver. Let me know.

I'll send this in to Linus later this week, to make life easier for
everyone.  Or Greg can grab it.


From: "K. Y. Srinivasan" <kys@microsoft.com>
Subject: mm: export a function to get vm committed memory

It will be useful to be able to access global memory commitment from
device drivers.  On the Hyper-V platform, the host has a policy engine to
balance the available physical memory amongst all competing virtual
machines hosted on a given node.  This policy engine is driven by a number
of metrics including the memory commitment reported by the guests.  The
balloon driver for Linux on Hyper-V will use this function to retrieve
guest memory commitment.  This function is also used in Xen self
ballooning code.

[akpm@linux-foundation.org: coding-style tweak]
Signed-off-by: K. Y. Srinivasan <kys@microsoft.com>
Cc: Greg KH <greg@kroah.com>
Acked-by: David Rientjes <rientjes@google.com>
Acked-by: Dan Magenheimer <dan.magenheimer@oracle.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Jeremy Fitzhardinge <jeremy@goop.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
---

 drivers/xen/xen-selfballoon.c |    2 +-
 include/linux/mman.h          |    2 ++
 mm/mmap.c                     |   14 ++++++++++++++
 mm/nommu.c                    |   15 +++++++++++++++
 4 files changed, 32 insertions(+), 1 deletion(-)

diff -puN drivers/xen/xen-selfballoon.c~mm-export-a-function-to-get-vm-committed-memory drivers/xen/xen-selfballoon.c
--- a/drivers/xen/xen-selfballoon.c~mm-export-a-function-to-get-vm-committed-memory
+++ a/drivers/xen/xen-selfballoon.c
@@ -222,7 +222,7 @@ static void selfballoon_process(struct w
 	if (xen_selfballooning_enabled) {
 		cur_pages = totalram_pages;
 		tgt_pages = cur_pages; /* default is no change */
-		goal_pages = percpu_counter_read_positive(&vm_committed_as) +
+		goal_pages = vm_memory_committed() +
 				totalreserve_pages +
 				MB2PAGES(selfballoon_reserved_mb);
 #ifdef CONFIG_FRONTSWAP
diff -puN include/linux/mman.h~mm-export-a-function-to-get-vm-committed-memory include/linux/mman.h
--- a/include/linux/mman.h~mm-export-a-function-to-get-vm-committed-memory
+++ a/include/linux/mman.h
@@ -11,6 +11,8 @@ extern int sysctl_overcommit_memory;
 extern int sysctl_overcommit_ratio;
 extern struct percpu_counter vm_committed_as;
 
+unsigned long vm_memory_committed(void);
+
 static inline void vm_acct_memory(long pages)
 {
 	percpu_counter_add(&vm_committed_as, pages);
diff -puN mm/mmap.c~mm-export-a-function-to-get-vm-committed-memory mm/mmap.c
--- a/mm/mmap.c~mm-export-a-function-to-get-vm-committed-memory
+++ a/mm/mmap.c
@@ -89,6 +89,20 @@ int sysctl_max_map_count __read_mostly =
 struct percpu_counter vm_committed_as ____cacheline_aligned_in_smp;
 
 /*
+ * The global memory commitment made in the system can be a metric
+ * that can be used to drive ballooning decisions when Linux is hosted
+ * as a guest. On Hyper-V, the host implements a policy engine for dynamically
+ * balancing memory across competing virtual machines that are hosted.
+ * Several metrics drive this policy engine including the guest reported
+ * memory commitment.
+ */
+unsigned long vm_memory_committed(void)
+{
+	return percpu_counter_read_positive(&vm_committed_as);
+}
+EXPORT_SYMBOL_GPL(vm_memory_committed);
+
+/*
  * Check that a process has enough memory to allocate a new virtual
  * mapping. 0 means there is enough memory for the allocation to
  * succeed and -ENOMEM implies there is not.
diff -puN mm/nommu.c~mm-export-a-function-to-get-vm-committed-memory mm/nommu.c
--- a/mm/nommu.c~mm-export-a-function-to-get-vm-committed-memory
+++ a/mm/nommu.c
@@ -66,6 +66,21 @@ int heap_stack_gap = 0;
 
 atomic_long_t mmap_pages_allocated;
 
+/*
+ * The global memory commitment made in the system can be a metric
+ * that can be used to drive ballooning decisions when Linux is hosted
+ * as a guest. On Hyper-V, the host implements a policy engine for dynamically
+ * balancing memory across competing virtual machines that are hosted.
+ * Several metrics drive this policy engine including the guest reported
+ * memory commitment.
+ */
+unsigned long vm_memory_committed(void)
+{
+	return percpu_counter_read_positive(&vm_committed_as);
+}
+
+EXPORT_SYMBOL_GPL(vm_memory_committed);
+
 EXPORT_SYMBOL(mem_map);
 EXPORT_SYMBOL(num_physpages);
 
_

Re: drivres/hv

[Greg KH]

On Thu, Nov 15, 2012 at 02:34:42PM -0800, Andrew Morton wrote:
> On Thu, 15 Nov 2012 22:22:06 +0000
> KY Srinivasan <kys@microsoft.com> wrote:
> 
> > 
> > 
> > > -----Original Message-----
> > > From: Greg KH [mailto:gregkh@linuxfoundation.org]
> > > Sent: Friday, November 02, 2012 7:37 PM
> > > To: KY Srinivasan
> > > Cc: linux-kernel@vger.kernel.org; devel@linuxdriverproject.org
> > > Subject: Re: drivres/hv
> > > 
> > > On Fri, Nov 02, 2012 at 03:11:23PM -0700, K. Y. Srinivasan wrote:
> > > >
> > > > Greg,
> > > >
> > > > Recently, I had  re-sent a few patches. You have applied all the patches except
> > > the balloon driver
> > > > related patches. Should I resend the balloon driver  patches. Let me know.
> > > 
> > > I can't apply the balloon driver until you get an ack from the
> > > maintainers of the code you were exporting the symbols from.
> > > 
> > > Get that, and then resend them, as they are long gone from my "to-apply"
> > > queue.
> > 
> > Greg,
> > 
> > Andrew has checked in my patch that exports the necessary function for Hyper-V
> > balloon driver:
> > 
> > commit bb2495a71b8499bdfe207738bc88ffa91ebe0b0a
> > Author: K. Y. Srinivasan <kys@microsoft.com>
> > Date:   Thu Nov 15 13:37:59 2012 +1100
> > 
> > I had sent the new balloon driver that used this new function a few days ago. Should I
> > resend the balloon driver. Let me know.
> 
> I'll send this in to Linus later this week, to make life easier for
> everyone.  Or Greg can grab it.

I'll grab it.

thanks,

greg k-h