All of lore.kernel.org
 help / color / mirror / Atom feed
* RE: Genapic
@ 2005-05-19 21:10 Puthiyaparambil, Aravindh
  2005-05-20 19:07 ` Genapic Andi Kleen
  0 siblings, 1 reply; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-19 21:10 UTC (permalink / raw)
  To: natasha, Keir Fraser
  Cc: ak, xen-devel, Davis, Jason, Vessey, Bruce A, Subrahmanian, Raj

Natasha, thanks for bringing order back to the discussion :-)

> The question is where to go from here to enable Xen to run on sub-arch

> machines.  Given the impending move to Xen 3.0, I think it would be 
> best to pull in the needed Linux-i386 genapic code near-term.

I agree that this is the best direction to follow.

> This code may very well not be sufficient for newer x86-64 machines

Yes, I think this is true.

> Personally, I don't know.  I also don't know if the x86-64 code can be

> easily ported to work on the i386 machines currently being targeted.
> If it can then a move to the x86-64 style may be a good move for the 
> future.  Or maybe we need to find a way for both to coexist within
Xen.

We have a few options for getting both i386 and x86_64 genapics working
in Xen:

1. Merging the two genapic models into one unified model. This will be a
sizeable effort and so is feasible only post Xen 3.0. 

2. Create a x86_64 sub-architecture for i386 genapic. But I am not sure
how much work will be involved or if this is the best direction to
follow. 

3. Mimic the Linux kernel and have separate genapic models.

> Does this sound like a viable way forward?  Or have I missed
something?
 
For the moment I think it is best we follow Natasha's suggestion and
work towards bringing in the i386 genapic mode. 

Comments?

Aravindh

 
> 
> 
> On Wed, 18 May 2005 17:54:43 +0100, Keir Fraser
<Keir.Fraser@cl.cam.ac.uk>
> wrote:
> 
> >
> > On 18 May 2005, at 16:45, Puthiyaparambil, Aravindh wrote:
> >
> >> Genapic is done differently for i386 and x86_64 in the LK. In i386
it
> is
> >> machine specific and in x86_64 it is APIC model specific. Do you
want
> >> Xen to mimic that? Further more both i386 and x86_64 have separate
> >> apic.c, io_apic.c (just to name a couple) files. Xen shares them
> between
> >> the two. Will that cause any problems?
> >>
> >> I am wondering if it is possible to have a unified genapic for Xen.
Or
> >> is it way too complicated?
> >
> > Shared platform code (apics and the like) has been remarkably
simple.
> > This is to be expected:  processor differences aside, i386 and
x86/64
> > are identical platforms. So, for example, fixing Xen's smpboot.c
> > (derived from Linux/i386) for x86/64 required one or two lines to be
> > changed. Same for ACPI parsing, MP-table parsing, and so on. I
wonder
> > why i386 and x86/64 have been allowed to diverge so wildly in Linux?
:-)
> >
> >   -- Keir

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-24 19:58 Puthiyaparambil, Aravindh
  2005-05-24 20:22 ` Genapic Keir Fraser
  2005-05-25 15:21 ` Genapic Andi Kleen
  0 siblings, 2 replies; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-24 19:58 UTC (permalink / raw)
  To: Andi Kleen, Natasha Jarymowycz
  Cc: xen-devel, Davis, Jason, Vessey, Bruce A, Subrahmanian, Raj

I am fine with going down the x86_64 genapic route for Xen. The only
thing is that I know that IRQ overrides are not present in the x86_64
genapic. So that might have to be brought over for the ES7000.

I would like to start working on bringing the x86_64 code over. Will
this clash with anyone else?

Aravindh

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, May 20, 2005 3:03 PM
To: Natasha Jarymowycz
Cc: Keir Fraser; Puthiyaparambil, Aravindh;
xen-devel@lists.xensource.com; Davis, Jason; Vessey, Bruce A;
Subrahmanian, Raj
Subject: Re: [Xen-devel] Genapic

On Wed, May 18, 2005 at 02:45:49PM -0500, Natasha Jarymowycz wrote:
> Based upon the discussion so far, I think this is where xen stands:
> 
> Xen is currently based upon linux-i386 code, including support for
> mach-specific functionality in asm-x86/mach-xxx subdirs.
> 
> Xen is responsible for initializing local APICs and so it must include
> support for these machines (it can't be deferred to linux).
> 
> Linux (dom 0) may or may not need to include support for these
machines
> as well depending upon other features (timers? numa?) which may be
> added in the future.  This is not an issue for Xen 3.0
> 
> Adding genapic support to Xen a la linux-i386 should be trivial given
> the current state of the code.  [Genapic allows the appropriate
mach-xxx
> code to be chosen during run time.  This is considered very important
> from a distro stand point]
> 
> Linux-x86-64 code is very different from i386 from a code organization
> standpoint.  Using apic as an example, it chooses between flat or
> clustered rather than es7000 vs summit vs xxx.

The behaviour is basically the same. The reason i386 is so different
has more historical reasons and that it supports subarchitectures
for non PC like x86 systems that need much more changes (like NUMAQ
or pc98). genapic actually doesnt support these, but the i386
subarchitectures had to be still fit into that framework.

I doubt Xen will ever run on any of these.

On x86-64 only the APIC differences are abstracted this way,
which is enough for all PC Like systems (including Summit and ES7000) 

For Xen I would recommend to go with the simpler model from x86-64.

> 
> sub-arch machines.  Given the impending move to Xen 3.0, I think it
would
> be best to pull in the needed Linux-i386 genapic code near-term.
> This code may very well not be sufficient for newer x86-64 machines

It should, but the x86-64 code is much cleaner and should
be sufficient.

> Personally, I don't know.  I also don't know if the x86-64 code can be
> easily ported to work on the i386 machines currently being targeted.

It should.

-Andi

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-18 18:26 Nakajima, Jun
  0 siblings, 0 replies; 21+ messages in thread
From: Nakajima, Jun @ 2005-05-18 18:26 UTC (permalink / raw)
  To: Puthiyaparambil, Aravindh, Andi Kleen, Keir Fraser
  Cc: Subrahmanian, Raj, Davis, Jason, xen-devel, Vessey, Bruce A

Puthiyaparambil, Aravindh wrote:
> I am a little confused here. When you say XenLinux I am assuming you
> are talking about the Linux user kernels which have Xen patches. Why
> would we even turn on the mach specific features in the user kernels?
> 
Xen assumes particular mach specific featurs, such as timer, memory
reporting, IRQ, interrutp handling, which don't necessarily work with
such mach specific features as found on real machines.

> I would think that that the mach specific code needs to reside in the
> Xen hypervisor itself. It at the moment has unified apic.c/io_apic.c
> for x86_32 and x86_64 platforms. Don't we to add genapic code in
> there? Or am I wrong on this? Maybe I need more coffee :-)
I talked about the generic platform support, not genapic per se. We have
moved ACPI, PCI, and part of io_apic (which required apic.c) handling to
dom0 to do better platform detection and configuration. Xen still owns
IO APICs, and some operatons must be done by Xen.  

Jun

> 
> Aravindh
> 

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-18 17:44 Puthiyaparambil, Aravindh
  0 siblings, 0 replies; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-18 17:44 UTC (permalink / raw)
  To: Nakajima, Jun, Andi Kleen, Keir Fraser
  Cc: Subrahmanian, Raj, Davis, Jason, xen-devel, Vessey, Bruce A

I am a little confused here. When you say XenLinux I am assuming you are
talking about the Linux user kernels which have Xen patches. Why would
we even turn on the mach specific features in the user kernels?

I would think that that the mach specific code needs to reside in the
Xen hypervisor itself. It at the moment has unified apic.c/io_apic.c for
x86_32 and x86_64 platforms. Don't we to add genapic code in there? Or
am I wrong on this? Maybe I need more coffee :-)

Aravindh

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com
[mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Nakajima,
Jun
Sent: Wednesday, May 18, 2005 1:22 PM
To: Andi Kleen; Keir Fraser
Cc: Subrahmanian, Raj; Davis, Jason; xen-devel@lists.xensource.com;
Vessey, Bruce A
Subject: RE: [Xen-devel] Genapic

Andi Kleen wrote:
>> Shared platform code (apics and the like) has been remarkably simple.
>> This is to be expected:  processor differences aside, i386 and x86/64
>> are identical platforms. So, for example, fixing Xen's smpboot.c
> 
> No, they are not. x86-64 is a vastly simplified "modern x86 only with
> minimal bugs" platform, while i386 is a "workarounds for every bug
> under the sun and all ancient features plus some non PC architectures"
> mixed bag of stuff. This leads to many differences in details.
> 
> One bigger difference is that x86-64 is also normally NUMA enabled,
> while i386 normally is not. There are some others too.
> 
> -Andi
> 
I think in the long-term we want to have a unified code for
pic.c/io_apic.c in XenLinux, but we don't want to redo the dubugging and
testing that have already been done by Linux side (I know it's very
painful). We also want the same platform support in XenLinux as the
native Linux does; if we unify them today, we probably lose some
machines. This leads us to separate apic.c/io_apic.c for x86 and x86-64
XenLinux (that's the current plan), because debugging on various
platforms is more painful (even not realistic) than maintaining the
different code for x86 and x86-64 at this point. 

Jun

> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-18 17:21 Nakajima, Jun
  0 siblings, 0 replies; 21+ messages in thread
From: Nakajima, Jun @ 2005-05-18 17:21 UTC (permalink / raw)
  To: Andi Kleen, Keir Fraser
  Cc: Subrahmanian, Raj, Davis, Jason, xen-devel, Vessey, Bruce A

Andi Kleen wrote:
>> Shared platform code (apics and the like) has been remarkably simple.
>> This is to be expected:  processor differences aside, i386 and x86/64
>> are identical platforms. So, for example, fixing Xen's smpboot.c
> 
> No, they are not. x86-64 is a vastly simplified "modern x86 only with
> minimal bugs" platform, while i386 is a "workarounds for every bug
> under the sun and all ancient features plus some non PC architectures"
> mixed bag of stuff. This leads to many differences in details.
> 
> One bigger difference is that x86-64 is also normally NUMA enabled,
> while i386 normally is not. There are some others too.
> 
> -Andi
> 
I think in the long-term we want to have a unified code for
pic.c/io_apic.c in XenLinux, but we don't want to redo the dubugging and
testing that have already been done by Linux side (I know it's very
painful). We also want the same platform support in XenLinux as the
native Linux does; if we unify them today, we probably lose some
machines. This leads us to separate apic.c/io_apic.c for x86 and x86-64
XenLinux (that's the current plan), because debugging on various
platforms is more painful (even not realistic) than maintaining the
different code for x86 and x86-64 at this point. 

Jun

> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-18 15:45 Puthiyaparambil, Aravindh
  2005-05-18 16:36 ` Genapic Andi Kleen
  2005-05-18 16:54 ` Genapic Keir Fraser
  0 siblings, 2 replies; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-18 15:45 UTC (permalink / raw)
  To: Keir Fraser
  Cc: ak, xen-devel, Davis, Jason, Vessey, Bruce A, Subrahmanian, Raj

Genapic is done differently for i386 and x86_64 in the LK. In i386 it is
machine specific and in x86_64 it is APIC model specific. Do you want
Xen to mimic that? Further more both i386 and x86_64 have separate
apic.c, io_apic.c (just to name a couple) files. Xen shares them between
the two. Will that cause any problems? 

I am wondering if it is possible to have a unified genapic for Xen. Or
is it way too complicated?

Aravindh

-----Original Message-----
From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
Sent: Wednesday, May 18, 2005 11:25 AM
To: Puthiyaparambil, Aravindh
Cc: Davis, Jason; xen-devel@lists.xensource.com; Vessey, Bruce A;
Subrahmanian, Raj
Subject: Re: [Xen-devel] Genapic


On 18 May 2005, at 14:59, Puthiyaparambil, Aravindh wrote:

> But for now I shall bring in the mach-es7000 as a compile time option.

I didn't realise that most of the genapic targets are trivially 
run-time selectable. That being the case, I'm happy for you to make 
es7000 run-time selected right now.

  -- Keir

^ permalink raw reply	[flat|nested] 21+ messages in thread
* RE: Genapic
@ 2005-05-18 13:59 Puthiyaparambil, Aravindh
  2005-05-18 15:24 ` Genapic Keir Fraser
  0 siblings, 1 reply; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-18 13:59 UTC (permalink / raw)
  To: Keir Fraser; +Cc: xen-devel, Davis, Jason, Vessey, Bruce A, Subrahmanian, Raj

Genapic loosely refers to the generic sub architecture for i386 and
x86_64 platforms though it is implemented differently for the two. It
makes switching between different mach types a run-time option. This is
what Andi Kleen had to say about it when he submitted the patch to the
LK.

   [PATCH] generic subarchitecture for ia32
   
   From: Andi Kleen <ak@muc.de>
   
   This patch adds an generic x86 subarchitecture. It is intended to
provide
   an dynamic interface for APIC drivers. There are already three
subarchitectures
   (bigsmp, summit, default) that only differ in how they drive the
local APIC.
   A fourth - Unisys ES7000 - is scheduled to be merged soon.
   
   The subarchitecture concept separated this nicely, but it has the big
   drawback that they are compile time options. A Linux vendor cannot
   ship own binary kernel rpms for all of these machines. Runtime
probing
   is needed instead.
   
   This patch adds a new "generic" subarchitecture that just acts as a
   dynamic switching layer for APIC drivers. It only tries to virtualize
   the APICs, no attempt is made to cover further incompatiblities.
   This means machines like the Visual Workstation, pc9800 or
   Voyager are not covered; but these are unlikely to be supported by
   binary distributions anyways.
   
   The generic arch reuses the existing interface in mach_ipi /
mach_mpparse.h /
   mach_apic.h and just pulls it using some macros into an "struct
genapic"
   object. The main APIC code does not recognize it, it is all hidden
   in the mach-generic include files.
   
   Auto detection of APIC types is supported in the usual way used by
   existing ports like Summit - checking ACPI or mptables for specific
   signatures - or it can be specified by the user using a new "apic="
   boot option. I also moved the DMI scan to before the generic
   subarchitecture probe, so DMI could be used in future too to probe
   specific machines.
   
   Some minor hacks were needed to avoid circular declaration of a few
   symbols, but overall it's fairly clean.
   
   The patch has been tested on a Summit machine, an generic 4 virtual
CPUs
   Xeon and on an ES7000.


But for now I shall bring in the mach-es7000 as a compile time option.

Aravindh

-----Original Message-----
From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] 
Sent: Wednesday, May 18, 2005 3:34 AM
To: Puthiyaparambil, Aravindh
Cc: xen-devel@lists.xensource.com; Vessey, Bruce A; Subrahmanian, Raj
Subject: Re: [Xen-devel] Genapic


On 17 May 2005, at 22:18, Puthiyaparambil, Aravindh wrote:

> Now that Xen has platform code has been upgraded to be based off of
the
> LK 2.6 code base, it seems that the best way of introducing other
> platform specific code would be to add "genapic" support. Is anybody
> working on this? Is the unstable tree ready for this?

I'm not sure what 'genapic' support is, but if you mean porting things 
like mach-es7000 across to Xen, then yes: I think we are now ready for 
that.

Longer term I'd like to make switching between the different mach types 
a run-time rather than compile-time decision. But that will require a 
fair bit of modification to the standard Linux way of doing things -- 
for initial 3.0 I think we'll stick to compile-time switches for 
selecting esoteric platform configurations.

  -- Keir

^ permalink raw reply	[flat|nested] 21+ messages in thread
* Genapic
@ 2005-05-17 21:18 Puthiyaparambil, Aravindh
  2005-05-18  7:34 ` Genapic Keir Fraser
  0 siblings, 1 reply; 21+ messages in thread
From: Puthiyaparambil, Aravindh @ 2005-05-17 21:18 UTC (permalink / raw)
  To: xen-devel; +Cc: Subrahmanian, Raj, Vessey, Bruce A

Now that Xen has platform code has been upgraded to be based off of the
LK 2.6 code base, it seems that the best way of introducing other
platform specific code would be to add "genapic" support. Is anybody
working on this? Is the unstable tree ready for this?

Aravindh

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2005-05-25 15:26 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-05-19 21:10 Genapic Puthiyaparambil, Aravindh
2005-05-20 19:07 ` Genapic Andi Kleen
  -- strict thread matches above, loose matches on Subject: below --
2005-05-24 19:58 Genapic Puthiyaparambil, Aravindh
2005-05-24 20:22 ` Genapic Keir Fraser
2005-05-25 15:26   ` Genapic Andi Kleen
2005-05-25 15:21 ` Genapic Andi Kleen
2005-05-18 18:26 Genapic Nakajima, Jun
2005-05-18 17:44 Genapic Puthiyaparambil, Aravindh
2005-05-18 17:21 Genapic Nakajima, Jun
2005-05-18 15:45 Genapic Puthiyaparambil, Aravindh
2005-05-18 16:36 ` Genapic Andi Kleen
2005-05-18 16:54 ` Genapic Keir Fraser
2005-05-18 16:56   ` Genapic Andi Kleen
2005-05-18 19:45   ` Genapic Natasha Jarymowycz
2005-05-20 19:02     ` Genapic Andi Kleen
2005-05-18 13:59 Genapic Puthiyaparambil, Aravindh
2005-05-18 15:24 ` Genapic Keir Fraser
2005-05-17 21:18 Genapic Puthiyaparambil, Aravindh
2005-05-18  7:34 ` Genapic Keir Fraser
2005-05-18 13:57   ` Genapic Andi Kleen
2005-05-18 15:23     ` Genapic Keir Fraser

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.