[PATCH 00/15] ia64/pv_ops take 5

[PATCH 00/15] ia64/pv_ops take 5

[Isaku Yamahata]

Hi. This patchset implements ia64/pv_ops support which is the
framework for virtualization support.
Now all the comments so far have been addressed, but only a few exceptions.

On x86 various ways to support virtualization were proposed, and
eventually pv_ops won. So on ia64 the pv_ops strategy is appropriate too.
Later I'll post the patchset which implements xen domU based on
ia64/pv_ops. Currently only ia64/xen pv_ops implementation exists,
but I believe ia64/kvm can also obtain benefits from this ia64/pv_ops
frame work.
linux/ia64 has the machine vector interface and another approach might be
to utilize it. However pv_ops is better for some reasons. Please see
the thread of the mailing list from
http://www.spinics.net/lists/linux-ia64/msg05238.html


This patchset does the followings.
- some preparation work. Mainly make the kernel virtualization friendly.
- introduce pv_info which describes the execution environment.
- introduce macros for hand written assembly code to paravirtualize
  hand written code to replace sensitive/privileged instructions.
- introduce various hooks to replace the implementation with
  paravirtualized.
  They are defined as function tables called, pv_init_ops, pv_cpu_ops,
  pv_irq_ops, pv_iosapic_ops and, pv_time_ops.
  They represent
    pv_init_ops: hooks for various initialization.
    pv_cpu_ops: hooks for ia64 intrinsics.
    pv_irq_ops: hooks for irq related operations.
    pv_iosapic_ops: hooks for paravirtualized iosapic.
    pv_time_ops: hooks for steal time accounting


The working full source is available from
http://people.valinux.co.jp/~yamahata/xen-ia64/linux-2.6-xen-ia64.git/
branch: xen-ia64-2008apr30

At this phase, we don't address the following issues.
Those will be addressed after the first merge.
- optimization by binary patch
  In fact, we had the patch to do that, but we intentionally dropped
  for patch size/readability/cleanness.
- freeing the unused pages, i.e. pages for unused ivt.S.

Changes from take 4:
- refined NR_IRQS paravirtualization according to Jes Soresen's comment
- rebased and fixed the breakage due to the upstream change.
- fixed various compilation errors.

Changes from take 3:
- split the patch set into pv_op part and xen domU part.
- many clean ups.
- introduced pv_ops: pv_cpu_ops and pv_time_ops.

Changes from take 2:
- many clean ups following to comments.
- clean up:assembly instruction macro.
- introduced pv_ops: pv_info, pv_init_ops, pv_iosapic_ops, pv_irq_ops.

Changes from take 1:
Single IVT source code. compile multitimes using assembler macros.

thanks,

Diffstat:
 arch/ia64/Makefile                 |    6 +
 arch/ia64/kernel/Makefile          |   44 +++++
 arch/ia64/kernel/entry.S           |  115 +++++++----
 arch/ia64/kernel/iosapic.c         |   45 +++--
 arch/ia64/kernel/irq_ia64.c        |   19 ++-
 arch/ia64/kernel/ivt.S             |  227 +++++++++++-----------
 arch/ia64/kernel/minstate.h        |   13 +-
 arch/ia64/kernel/nr-irqs.c         |   24 +++
 arch/ia64/kernel/paravirt.c        |  369 ++++++++++++++++++++++++++++++++++++
 arch/ia64/kernel/paravirt_inst.h   |   29 +++
 arch/ia64/kernel/paravirtentry.S   |   60 ++++++
 arch/ia64/kernel/setup.c           |   10 +
 arch/ia64/kernel/smpboot.c         |    2 +
 arch/ia64/kernel/time.c            |   23 +++
 include/asm-ia64/Kbuild            |    2 +-
 include/asm-ia64/gcc_intrin.h      |   24 ++--
 include/asm-ia64/hw_irq.h          |   23 ++-
 include/asm-ia64/intel_intrin.h    |   41 ++--
 include/asm-ia64/intrinsics.h      |   55 ++++++
 include/asm-ia64/iosapic.h         |   18 ++-
 include/asm-ia64/irq.h             |    9 +-
 include/asm-ia64/mmu_context.h     |    6 +-
 include/asm-ia64/native/inst.h     |  180 ++++++++++++++++++
 include/asm-ia64/native/irq.h      |   35 ++++
 include/asm-ia64/paravirt.h        |  252 ++++++++++++++++++++++++
 include/asm-ia64/paravirt_privop.h |  114 +++++++++++
 include/asm-ia64/smp.h             |    2 +
 include/asm-ia64/system.h          |   10 +-
 28 files changed, 1520 insertions(+), 237 deletions(-)

Re: [PATCH 00/15] ia64/pv_ops take 5

[Isaku Yamahata]

Hi Tony.
Could you give me comments/hints to make progress?
What do you think about those patches?
I believe that ia64/kvm and other ia64 virtualization technologies 
also can get much benefit from ia64/pv_ops and that no performance
penalties exist with CONFIG_PARAVIRT=n.

Regarding to the ia64/xen specific patches, should they go through
the xen maintainer like the ia64/kvm patches did?

thanks,

On Wed, Apr 30, 2008 at 09:29:13PM +0900, Isaku Yamahata wrote:
> Hi. This patchset implements ia64/pv_ops support which is the
> framework for virtualization support.
> Now all the comments so far have been addressed, but only a few exceptions.
> 
> On x86 various ways to support virtualization were proposed, and
> eventually pv_ops won. So on ia64 the pv_ops strategy is appropriate too.
> Later I'll post the patchset which implements xen domU based on
> ia64/pv_ops. Currently only ia64/xen pv_ops implementation exists,
> but I believe ia64/kvm can also obtain benefits from this ia64/pv_ops
> frame work.
> linux/ia64 has the machine vector interface and another approach might be
> to utilize it. However pv_ops is better for some reasons. Please see
> the thread of the mailing list from
> http://www.spinics.net/lists/linux-ia64/msg05238.html
> 
> 
> This patchset does the followings.
> - some preparation work. Mainly make the kernel virtualization friendly.
> - introduce pv_info which describes the execution environment.
> - introduce macros for hand written assembly code to paravirtualize
>   hand written code to replace sensitive/privileged instructions.
> - introduce various hooks to replace the implementation with
>   paravirtualized.
>   They are defined as function tables called, pv_init_ops, pv_cpu_ops,
>   pv_irq_ops, pv_iosapic_ops and, pv_time_ops.
>   They represent
>     pv_init_ops: hooks for various initialization.
>     pv_cpu_ops: hooks for ia64 intrinsics.
>     pv_irq_ops: hooks for irq related operations.
>     pv_iosapic_ops: hooks for paravirtualized iosapic.
>     pv_time_ops: hooks for steal time accounting
> 
> 
> The working full source is available from
> http://people.valinux.co.jp/~yamahata/xen-ia64/linux-2.6-xen-ia64.git/
> branch: xen-ia64-2008apr30
> 
> At this phase, we don't address the following issues.
> Those will be addressed after the first merge.
> - optimization by binary patch
>   In fact, we had the patch to do that, but we intentionally dropped
>   for patch size/readability/cleanness.
> - freeing the unused pages, i.e. pages for unused ivt.S.
> 
> Changes from take 4:
> - refined NR_IRQS paravirtualization according to Jes Soresen's comment
> - rebased and fixed the breakage due to the upstream change.
> - fixed various compilation errors.
> 
> Changes from take 3:
> - split the patch set into pv_op part and xen domU part.
> - many clean ups.
> - introduced pv_ops: pv_cpu_ops and pv_time_ops.
> 
> Changes from take 2:
> - many clean ups following to comments.
> - clean up:assembly instruction macro.
> - introduced pv_ops: pv_info, pv_init_ops, pv_iosapic_ops, pv_irq_ops.
> 
> Changes from take 1:
> Single IVT source code. compile multitimes using assembler macros.
> 
> thanks,
> 
> Diffstat:
>  arch/ia64/Makefile                 |    6 +
>  arch/ia64/kernel/Makefile          |   44 +++++
>  arch/ia64/kernel/entry.S           |  115 +++++++----
>  arch/ia64/kernel/iosapic.c         |   45 +++--
>  arch/ia64/kernel/irq_ia64.c        |   19 ++-
>  arch/ia64/kernel/ivt.S             |  227 +++++++++++-----------
>  arch/ia64/kernel/minstate.h        |   13 +-
>  arch/ia64/kernel/nr-irqs.c         |   24 +++
>  arch/ia64/kernel/paravirt.c        |  369 ++++++++++++++++++++++++++++++++++++
>  arch/ia64/kernel/paravirt_inst.h   |   29 +++
>  arch/ia64/kernel/paravirtentry.S   |   60 ++++++
>  arch/ia64/kernel/setup.c           |   10 +
>  arch/ia64/kernel/smpboot.c         |    2 +
>  arch/ia64/kernel/time.c            |   23 +++
>  include/asm-ia64/Kbuild            |    2 +-
>  include/asm-ia64/gcc_intrin.h      |   24 ++--
>  include/asm-ia64/hw_irq.h          |   23 ++-
>  include/asm-ia64/intel_intrin.h    |   41 ++--
>  include/asm-ia64/intrinsics.h      |   55 ++++++
>  include/asm-ia64/iosapic.h         |   18 ++-
>  include/asm-ia64/irq.h             |    9 +-
>  include/asm-ia64/mmu_context.h     |    6 +-
>  include/asm-ia64/native/inst.h     |  180 ++++++++++++++++++
>  include/asm-ia64/native/irq.h      |   35 ++++
>  include/asm-ia64/paravirt.h        |  252 ++++++++++++++++++++++++
>  include/asm-ia64/paravirt_privop.h |  114 +++++++++++
>  include/asm-ia64/smp.h             |    2 +
>  include/asm-ia64/system.h          |   10 +-
>  28 files changed, 1520 insertions(+), 237 deletions(-)

-- 
yamahata

RE: [PATCH 00/15] ia64/pv_ops take 5

[Luck, Tony]

I started looking at this patch set.

Parts 1-9 applied ok, but part10 (entry.S) failed to apply because of
recent changes to this file to fix the problems with warnings when
trying to get locks with interrupts blocked.

I thought this would be a good point to test the bisectability
of this patch set and started a build with just parts 1-9 applied.
The build was perfectly clean, but the kernel hung during boot
(possibly when first trying to run user processes by the look
of the last few messages on the console.

The code with parts 1-9 applied still looks readable, but I
think we may need some documentation later to help with maintainability.
Perhaps even a tool that can check to make sure people don't add
direct uses of instructions to .S files that need to be
paravirtualized.

I'm talking to some of the virtualization people here to
see if they can help me set up a system so that I will be
able to test out the non-native execution paths.

-Tony

Re: [PATCH 00/15] ia64/pv_ops take 5

[Isaku Yamahata]

On Fri, May 16, 2008 at 05:06:21PM -0700, Luck, Tony wrote:
> I started looking at this patch set.

Thank you for your time.


> Parts 1-9 applied ok, but part10 (entry.S) failed to apply because of
> recent changes to this file to fix the problems with warnings when
> trying to get locks with interrupts blocked.
> 
> I thought this would be a good point to test the bisectability
> of this patch set and started a build with just parts 1-9 applied.
> The build was perfectly clean, but the kernel hung during boot
> (possibly when first trying to run user processes by the look
> of the last few messages on the console.

I'll rebase/debug the patches and resend them.
My native case test is done under Xen HVM(full virtualized)
domain.


> The code with parts 1-9 applied still looks readable, but I
> think we may need some documentation later to help with maintainability.

I'll write it. It will be under linux-2.6/Documentation/ia64/.

  
> Perhaps even a tool that can check to make sure people don't add
> direct uses of instructions to .S files that need to be
> paravirtualized.

Yes, I also really want such a tool and I've thought of it.
I want to prevent people from accidently breaking paravirtualization
and only a way to do so would be by enforcing.
I think it can be easily achived by CPP macro trick.
At worst a simple .s (or .o) parser.

-- 
yamahata

Re: [PATCH 00/15] ia64/pv_ops take 5

[Jeremy Fitzhardinge]

Isaku Yamahata wrote:
> Yes, I also really want such a tool and I've thought of it.
> I want to prevent people from accidently breaking paravirtualization
> and only a way to do so would be by enforcing.
> I think it can be easily achived by CPP macro trick.
> At worst a simple .s (or .o) parser.
>   

For what its worth, this hasn't been a problem on x86 at all.  All the 
"interesting" instructions are used via macros/inline functions, and 
there have been no instances (that I know of, at least) of naked ones 
appearing anywhere.  And all the gatekeeper maintainers are definitely 
aware of the paravirt constraints, so I don't think any would make it 
past review.

    J