Patches for v3.8

Patches for v3.8

[Konrad Rzeszutek Wilk]

Heya,

So v3.8 merge window is going to open in two weeks (or perhaps sooner).

I am still fixing some build issues so that they won't surface on linux-next, but
so far this is what I was thinking to push (some to Linus, and some to Jens):

Anything I am missing?

 arch/arm/include/asm/xen/interface.h |    1 +
 arch/arm/xen/enlighten.c             |  124 ++++++++++++---
 arch/x86/include/asm/xen/interface.h |   12 ++-
 arch/x86/include/asm/xen/page.h      |    3 +
 arch/x86/xen/Kconfig                 |   11 ++
 arch/x86/xen/enlighten.c             |  174 +++++++++++++++-----
 arch/x86/xen/irq.c                   |    5 +-
 arch/x86/xen/mmu.c                   |  187 +++++++++++++++++++--
 arch/x86/xen/mmu.h                   |    2 +
 arch/x86/xen/p2m.c                   |    2 +-
 arch/x86/xen/setup.c                 |   64 ++++++--
 arch/x86/xen/smp.c                   |   75 ++++++---
 arch/x86/xen/suspend.c               |    2 +-
 arch/x86/xen/xen-head.S              |   11 +-
 arch/x86/xen/xen-ops.h               |    2 +-
 drivers/block/xen-blkback/blkback.c  |  301 +++++++++++++++++++++++++++++++---
 drivers/block/xen-blkback/common.h   |   16 ++
 drivers/block/xen-blkback/xenbus.c   |   23 +++-
 drivers/block/xen-blkfront.c         |  199 +++++++++++++++++++---
 drivers/xen/Kconfig                  |    3 +
 drivers/xen/Makefile                 |    7 +-
 drivers/xen/balloon.c                |   19 ++-
 drivers/xen/cpu_hotplug.c            |    4 +-
 drivers/xen/events.c                 |    9 +-
 drivers/xen/gntdev.c                 |    3 +-
 drivers/xen/grant-table.c            |   25 +++-
 drivers/xen/privcmd.c                |   72 ++++++++-
 drivers/xen/xen-acpi-pad.c           |  182 ++++++++++++++++++++
 drivers/xen/xen-pciback/pci_stub.c   |  118 +++++++------
 drivers/xen/xenbus/xenbus_client.c   |    3 +-
 drivers/xen/xenbus/xenbus_xs.c       |    1 -
 include/xen/interface/memory.h       |   44 +++++-
 include/xen/interface/physdev.h      |   10 +
 include/xen/interface/platform.h     |   17 ++
 include/xen/xen-ops.h                |    9 +-
 35 files changed, 1484 insertions(+), 256 deletions(-)

Alexander Duyck (1):
      x86/xen: Use __pa_symbol instead of __pa on C visible symbols

Dan Carpenter (1):
      xen/privcmd: fix condition in privcmd_close()

Ian Campbell (6):
      xen: balloon: allow PVMMU interfaces to be compiled out
      xen: arm: enable balloon driver
      xen: correctly use xen_pfn_t in remap_domain_mfn_range.
      xen: arm: implement remap interfaces needed for privcmd mappings.
      xen: x86 pvh: use XENMEM_add_to_physmap_range for foreign gmfn mappings
      xen: include correct header for xen_pfn_t definition.

Jan Beulich (2):
      xen-pciback: simplify and tighten parsing of device IDs
      xen-pciback: reject out of range inputs

Konrad Rzeszutek Wilk (8):
      xen/smp: Move the common CPU init code a bit to prep for PVH patch.
      xen/e820: Coalesce the PVH release/populate logic in the generic case.
      Revert "xen/e820: Coalesce the PVH release/populate logic in the generic case."
      xen/acpi: Fix compile error by missing decleration for xen_domain.
      Merge branch 'stable/pvh.v6.1' into stable/for-linus-3.8
      xen/acpi: Move the xen_running_on_version_or_later function.
      Merge branch 'stable/for-linus-3.8' into HEAD
      Merge branch 'stable/for-jens-3.8' into HEAD

Liu, Jinsong (2):
      xen/acpi: ACPI PAD driver
      xen/acpi: revert pad config check in xen_check_mwait

Mukesh Rathor (6):
      xen/pvh: Support ParaVirtualized Hardware extensions.
      xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
      xen/pvh: Implement MMU changes for PVH.
      xen/pvh: bootup and setup (E820) related changes.
      xen/pvh: balloon and grant changes.
      xen/pvh: /dev/xen/privcmd changes.

Olaf Hering (1):
      xen PVonHVM: use E820_Reserved area for shared_info

Roger Pau Monne (4):
      xen/blkback: Persistent grant maps for xen blk drivers
      xen/blkback: persistent-grants fixes
      xen-blkback: move free persistent grants code
      xen-blkfront: free allocated page

Sachin Kamat (1):
      xen/xenbus: Remove duplicate inclusion of asm/xen/hypervisor.h

Wei Yongjun (1):
      xen/x86: remove duplicated include from enlighten.c

Re: Patches for v3.8

[Jan Beulich]

>>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> Mukesh Rathor (6):
>       xen/pvh: Support ParaVirtualized Hardware extensions.
>       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
>       xen/pvh: Implement MMU changes for PVH.
>       xen/pvh: bootup and setup (E820) related changes.
>       xen/pvh: balloon and grant changes.
>       xen/pvh: /dev/xen/privcmd changes.

As said before - I don't think it is a good idea to push the kernel side
changes into an official release when the hypervisor side ones didn't
even get an initial review yet.

Jan

Re: Patches for v3.8

[Stefano Stabellini]

On Wed, 28 Nov 2012, Jan Beulich wrote:
> >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > Mukesh Rathor (6):
> >       xen/pvh: Support ParaVirtualized Hardware extensions.
> >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> >       xen/pvh: Implement MMU changes for PVH.
> >       xen/pvh: bootup and setup (E820) related changes.
> >       xen/pvh: balloon and grant changes.
> >       xen/pvh: /dev/xen/privcmd changes.
> 
> As said before - I don't think it is a good idea to push the kernel side
> changes into an official release when the hypervisor side ones didn't
> even get an initial review yet.

I agree

Re: Patches for v3.8

[Konrad Rzeszutek Wilk]

On Wed, Nov 28, 2012 at 11:50:54AM +0000, Stefano Stabellini wrote:
> On Wed, 28 Nov 2012, Jan Beulich wrote:
> > >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > > Mukesh Rathor (6):
> > >       xen/pvh: Support ParaVirtualized Hardware extensions.
> > >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> > >       xen/pvh: Implement MMU changes for PVH.
> > >       xen/pvh: bootup and setup (E820) related changes.
> > >       xen/pvh: balloon and grant changes.
> > >       xen/pvh: /dev/xen/privcmd changes.
> > 
> > As said before - I don't think it is a good idea to push the kernel side
> > changes into an official release when the hypervisor side ones didn't
> > even get an initial review yet.
> 
> I agree

That would delay the ARM patches that Ian posted as they are dependent on this.

Re: Patches for v3.8

[Ian Campbell]

On Wed, 2012-11-28 at 14:25 +0000, Konrad Rzeszutek Wilk wrote:
> On Wed, Nov 28, 2012 at 11:50:54AM +0000, Stefano Stabellini wrote:
> > On Wed, 28 Nov 2012, Jan Beulich wrote:
> > > >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > > > Mukesh Rathor (6):
> > > >       xen/pvh: Support ParaVirtualized Hardware extensions.
> > > >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> > > >       xen/pvh: Implement MMU changes for PVH.
> > > >       xen/pvh: bootup and setup (E820) related changes.
> > > >       xen/pvh: balloon and grant changes.
> > > >       xen/pvh: /dev/xen/privcmd changes.
> > > 
> > > As said before - I don't think it is a good idea to push the kernel side
> > > changes into an official release when the hypervisor side ones didn't
> > > even get an initial review yet.
> > 
> > I agree
> 
> That would delay the ARM patches that Ian posted as they are dependent on this.

I'd be happy to produce a series with the generic bits from the pvh
series which are required by the ARM stuff (the privcmd and balloon bits
mainly) and base the arm stuff on that. I think I offered to do this
before.

Ian.

Re: Patches for v3.8

[Konrad Rzeszutek Wilk]

On Wed, Nov 28, 2012 at 02:32:22PM +0000, Ian Campbell wrote:
> On Wed, 2012-11-28 at 14:25 +0000, Konrad Rzeszutek Wilk wrote:
> > On Wed, Nov 28, 2012 at 11:50:54AM +0000, Stefano Stabellini wrote:
> > > On Wed, 28 Nov 2012, Jan Beulich wrote:
> > > > >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > > > > Mukesh Rathor (6):
> > > > >       xen/pvh: Support ParaVirtualized Hardware extensions.
> > > > >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> > > > >       xen/pvh: Implement MMU changes for PVH.
> > > > >       xen/pvh: bootup and setup (E820) related changes.
> > > > >       xen/pvh: balloon and grant changes.
> > > > >       xen/pvh: /dev/xen/privcmd changes.
> > > > 
> > > > As said before - I don't think it is a good idea to push the kernel side
> > > > changes into an official release when the hypervisor side ones didn't
> > > > even get an initial review yet.
> > > 
> > > I agree
> > 
> > That would delay the ARM patches that Ian posted as they are dependent on this.
> 
> I'd be happy to produce a series with the generic bits from the pvh
> series which are required by the ARM stuff (the privcmd and balloon bits
> mainly) and base the arm stuff on that. I think I offered to do this
> before.

OK. The stable/pvh.v6 has yours and Mukesh's patches. If you want to
rebase them on stable/for-linus-3.8 (and if you need to pull in
v3.7-rc4,5,6,7 in it) I am game.

> 
> Ian.
> 
> 

Re: Patches for v3.8

[Ian Campbell]

On Wed, 2012-11-28 at 14:44 +0000, Konrad Rzeszutek Wilk wrote:
> On Wed, Nov 28, 2012 at 02:32:22PM +0000, Ian Campbell wrote:
> > On Wed, 2012-11-28 at 14:25 +0000, Konrad Rzeszutek Wilk wrote:
> > > On Wed, Nov 28, 2012 at 11:50:54AM +0000, Stefano Stabellini wrote:
> > > > On Wed, 28 Nov 2012, Jan Beulich wrote:
> > > > > >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > > > > > Mukesh Rathor (6):
> > > > > >       xen/pvh: Support ParaVirtualized Hardware extensions.
> > > > > >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> > > > > >       xen/pvh: Implement MMU changes for PVH.
> > > > > >       xen/pvh: bootup and setup (E820) related changes.
> > > > > >       xen/pvh: balloon and grant changes.
> > > > > >       xen/pvh: /dev/xen/privcmd changes.
> > > > > 
> > > > > As said before - I don't think it is a good idea to push the kernel side
> > > > > changes into an official release when the hypervisor side ones didn't
> > > > > even get an initial review yet.
> > > > 
> > > > I agree
> > > 
> > > That would delay the ARM patches that Ian posted as they are dependent on this.
> > 
> > I'd be happy to produce a series with the generic bits from the pvh
> > series which are required by the ARM stuff (the privcmd and balloon bits
> > mainly) and base the arm stuff on that. I think I offered to do this
> > before.
> 
> OK. The stable/pvh.v6 has yours and Mukesh's patches. If you want to
> rebase them on stable/for-linus-3.8 (and if you need to pull in
> v3.7-rc4,5,6,7 in it) I am game.

I'll take a look over the next couple of days.

Ian.

Re: Patches for v3.8

[Mukesh Rathor]

On Wed, 28 Nov 2012 08:24:04 +0000
"Jan Beulich" <JBeulich@suse.com> wrote:

> >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk
> >>> <konrad.wilk@oracle.com> wrote:
> > Mukesh Rathor (6):
> >       xen/pvh: Support ParaVirtualized Hardware extensions.
> >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> >       xen/pvh: Implement MMU changes for PVH.
> >       xen/pvh: bootup and setup (E820) related changes.
> >       xen/pvh: balloon and grant changes.
> >       xen/pvh: /dev/xen/privcmd changes.
> 
> As said before - I don't think it is a good idea to push the kernel
> side changes into an official release when the hypervisor side ones
> didn't even get an initial review yet.

On the xen patch side, I've refrshed my tree. I realized I had undone
the union of pv_domain and hvm_domain since I used fields from both.
I'm now redoing that to make that a union and add a struct in
hvm_domain for fields from pv_domain that are used by PVH. 

Thanks,
Mukesh

Re: Patches for v3.8

[Jan Beulich]

>>> On 29.11.12 at 02:50, Mukesh Rathor <mukesh.rathor@oracle.com> wrote:
> On Wed, 28 Nov 2012 08:24:04 +0000
> "Jan Beulich" <JBeulich@suse.com> wrote:
> 
>> >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk
>> >>> <konrad.wilk@oracle.com> wrote:
>> > Mukesh Rathor (6):
>> >       xen/pvh: Support ParaVirtualized Hardware extensions.
>> >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
>> >       xen/pvh: Implement MMU changes for PVH.
>> >       xen/pvh: bootup and setup (E820) related changes.
>> >       xen/pvh: balloon and grant changes.
>> >       xen/pvh: /dev/xen/privcmd changes.
>> 
>> As said before - I don't think it is a good idea to push the kernel
>> side changes into an official release when the hypervisor side ones
>> didn't even get an initial review yet.
> 
> On the xen patch side, I've refrshed my tree. I realized I had undone
> the union of pv_domain and hvm_domain since I used fields from both.
> I'm now redoing that to make that a union and add a struct in
> hvm_domain for fields from pv_domain that are used by PVH. 

Perhaps better to move out of the union just the shared fields?

Jan

Re: Patches for v3.8

[Ian Campbell]

On Wed, 2012-11-28 at 16:00 +0000, Ian Campbell wrote:
> I'll take a look over the next couple of days.

Just looking into this now and in
konrad/stable/pvh.v6:arch/x86/xen/mmu.c in xen_remap_domain_mfn_range we
have:


	if (xen_feature(XENFEAT_auto_translated_physmap))
		return -EINVAL;

...

	if (xen_feature(XENFEAT_auto_translated_physmap)) {
		/* We need to update the local page tables and the xen HAP */
		return pvh_remap_gmfn_range(vma, addr, mfn, nr, prot, domid, pages);
	}

So it seems like the second one will never be reached...

AFAICT this was already the case when support was added in
de81067d4ef5b434b47f63e11448b3b09cfc0b5e.

Ian.

Re: Patches for v3.8

[Konrad Rzeszutek Wilk]

On Thu, Nov 29, 2012 at 11:51:27AM +0000, Ian Campbell wrote:
> On Wed, 2012-11-28 at 16:00 +0000, Ian Campbell wrote:
> > I'll take a look over the next couple of days.
> 
> Just looking into this now and in
> konrad/stable/pvh.v6:arch/x86/xen/mmu.c in xen_remap_domain_mfn_range we
> have:
> 
> 
> 	if (xen_feature(XENFEAT_auto_translated_physmap))
> 		return -EINVAL;
> 
> ...
> 
> 	if (xen_feature(XENFEAT_auto_translated_physmap)) {
> 		/* We need to update the local page tables and the xen HAP */
> 		return pvh_remap_gmfn_range(vma, addr, mfn, nr, prot, domid, pages);
> 	}
> 
> So it seems like the second one will never be reached...

Yikes! I wonder if I did a mismerge/rebase incorrectly? Mukesh, do you
recall if the original patchset had the first check removed?

> 
> AFAICT this was already the case when support was added in
> de81067d4ef5b434b47f63e11448b3b09cfc0b5e.
> 
> Ian.
> 

Re: Patches for v3.8

[Ian Campbell]

On Wed, 2012-11-28 at 16:00 +0000, Ian Campbell wrote:
> I'll take a look over the next couple of days.

Here we are. I've basically turned your stable/pvh.v6 branch over so
that the ARM stuff comes first (along with required bits of pvh).

I have pushed two branches to my tree at
git://xenbits.xen.org/people/ianc/linux.git.

The first is arm-privcmd-for-3.8 and contains the ARM stuff destined for
3.8.

The second is pvh.v7 which is based on arm-privcmd-for-3.8 and adds the
pvh stuff.

Pull requests for both are below.

The diff between the pvh branch and stable/pvh.v6 corresponds exactly to
rebasing on top of "xen PVonHVM: use E820_Reserved area for shared_info"
and the pcifront changes.

I didn't try and fix up the two checks of xenfeat thinggummy in
xen_remap_domain_mfn_range that I pointed to earlier.

Since I was rebasing things anyway I did feel free clean up a few
snaggles in the history:

I skipped "xen/e820: Coalesce the PVH release/populate logic in the
generic case." and its immediate revert.

I folded "xen: include correct header for xen_pfn_t definition." into
"xen: correctly use xen_pfn_t in remap_domain_mfn_range."

Ian.


The following changes since commit b3e40b72bb24237b0aee9f6ba2e9f88dd4ff3c0a:

  xen-pciback: reject out of range inputs (2012-11-02 11:04:03 -0400)

are available in the git repository at:
  git://xenbits.xen.org/people/ianc/linux.git arm-privcmd-for-3.8

Ian Campbell (5):
      xen: add pages parameter to xen_remap_domain_mfn_range
      xen: balloon: allow PVMMU interfaces to be compiled out
      xen: arm: enable balloon driver
      xen: correctly use xen_pfn_t in remap_domain_mfn_range.
      xen: arm: implement remap interfaces needed for privcmd mappings.

Mukesh Rathor (1):
      xen: privcmd: support autotranslated physmap guests.

 arch/arm/include/asm/xen/interface.h |    1 +
 arch/arm/xen/enlighten.c             |  123 ++++++++++++++++++++++++++++------
 arch/x86/include/asm/xen/interface.h |    1 +
 arch/x86/xen/Kconfig                 |    1 +
 arch/x86/xen/mmu.c                   |   17 ++++-
 drivers/xen/Kconfig                  |    3 +
 drivers/xen/Makefile                 |    4 +-
 drivers/xen/balloon.c                |    5 +-
 drivers/xen/privcmd.c                |   72 +++++++++++++++++++-
 include/xen/interface/memory.h       |   44 ++++++++++++-
 include/xen/xen-ops.h                |    8 ++-
 11 files changed, 246 insertions(+), 33 deletions(-)

-------------------------------------------------------------------------------------


The following changes since commit f832da068b0aadb15f747f6427b6bf945f525ba4:

  xen: arm: implement remap interfaces needed for privcmd mappings. (2012-11-29 14:00:19 +0000)

are available in the git repository at:
  git://xenbits.xen.org/people/ianc/linux.git pvh.v7

Ian Campbell (1):
      xen: x86 pvh: use XENMEM_add_to_physmap_range for foreign gmfn mappings

Konrad Rzeszutek Wilk (2):
      xen/smp: Move the common CPU init code a bit to prep for PVH patch.
      xen/pvh: balloon and grant changes.

Mukesh Rathor (4):
      xen/pvh: Support ParaVirtualized Hardware extensions.
      xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
      xen/pvh: Implement MMU changes for PVH.
      xen/pvh: bootup and setup (E820) related changes.

Wei Yongjun (1):
      xen/x86: remove duplicated include from enlighten.c

 arch/x86/include/asm/xen/interface.h |   11 ++-
 arch/x86/include/asm/xen/page.h      |    3 +
 arch/x86/xen/Kconfig                 |   10 ++
 arch/x86/xen/enlighten.c             |   76 +++++++++++++----
 arch/x86/xen/irq.c                   |    5 +-
 arch/x86/xen/mmu.c                   |  155 ++++++++++++++++++++++++++++++++--
 arch/x86/xen/mmu.h                   |    2 +
 arch/x86/xen/p2m.c                   |    2 +-
 arch/x86/xen/setup.c                 |   64 +++++++++++---
 arch/x86/xen/smp.c                   |   75 +++++++++++------
 arch/x86/xen/xen-head.S              |   11 ++-
 drivers/xen/balloon.c                |   14 ++-
 drivers/xen/cpu_hotplug.c            |    4 +-
 drivers/xen/events.c                 |    9 ++-
 drivers/xen/gntdev.c                 |    3 +-
 drivers/xen/grant-table.c            |   25 +++++-
 drivers/xen/xenbus/xenbus_client.c   |    3 +-
 include/xen/interface/physdev.h      |   10 ++
 18 files changed, 403 insertions(+), 79 deletions(-)

Re: Patches for v3.8

[Mukesh Rathor]

On Thu, 29 Nov 2012 07:28:34 +0000
"Jan Beulich" <JBeulich@suse.com> wrote:

> >>> On 29.11.12 at 02:50, Mukesh Rathor <mukesh.rathor@oracle.com>
> >>> wrote:
> > On Wed, 28 Nov 2012 08:24:04 +0000
> > "Jan Beulich" <JBeulich@suse.com> wrote:
> > 
> >> >>> On 27.11.12 at 22:31, Konrad Rzeszutek Wilk
> >> >>> <konrad.wilk@oracle.com> wrote:
> >> > Mukesh Rathor (6):
> >> >       xen/pvh: Support ParaVirtualized Hardware extensions.
> >> >       xen/pvh: Extend vcpu_guest_context, p2m, event, and XenBus.
> >> >       xen/pvh: Implement MMU changes for PVH.
> >> >       xen/pvh: bootup and setup (E820) related changes.
> >> >       xen/pvh: balloon and grant changes.
> >> >       xen/pvh: /dev/xen/privcmd changes.
> >> 
> >> As said before - I don't think it is a good idea to push the kernel
> >> side changes into an official release when the hypervisor side ones
> >> didn't even get an initial review yet.
> > 
> > On the xen patch side, I've refrshed my tree. I realized I had
> > undone the union of pv_domain and hvm_domain since I used fields
> > from both. I'm now redoing that to make that a union and add a
> > struct in hvm_domain for fields from pv_domain that are used by
> > PVH. 
> 
> Perhaps better to move out of the union just the shared fields?
> 

Well, not sure which is better. Those fields are not many, and only used
by PVH and not HVM. So, for now I just put it part of hvm_vcpu.
That way it's less confusing and obvious to anyone reading the code or
debugging. BTW, earlier I meant pv/hvm_vcpu and not pv/hvm_domain. If
it makes struct size smaller I could create pv_pvh_vcpu {} and put
those fields there, but that involves lot of unnecessary code changes.

thanks
mukesh