linux/arch/xen/i386 or linux/arch/i386/xen

linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

A while ago (I think at the Xummit), I recall someone
saying that the structure of the files in xenlinux may
change soon so that the xen-specific files would move to
a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
instead of linux/arch/xen/i386).

Is this still planned?  If so, what is the timing?
If it's in progress, I'd like to get in the loop...

Thanks,
Dan

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> A while ago (I think at the Xummit), I recall someone
> saying that the structure of the files in xenlinux may
> change soon so that the xen-specific files would move to
> a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
> instead of linux/arch/xen/i386).

I said I would do that.  I started, and got preempted.  I'm actually
just finally clearing my plate to get back to it this week.

thanks,
-chris

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

There are a number of "architecture independent" files
and headers.  Where are you planning on putting those?

Also, what is the expected timing of the change (e.g.
pre-Xen-3.0 release or post-)?

I'm starting some work in this code for ia64 and would
prefer to map to the "future" directory structure if
possible.

Thanks,
Dan

> -----Original Message-----
> From: Chris Wright [mailto:chrisw@osdl.org] 
> Sent: Monday, May 16, 2005 12:08 PM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: xen-devel
> Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> 
> * Magenheimer, Dan (HP Labs Fort Collins) 
> (dan.magenheimer@hp.com) wrote:
> > A while ago (I think at the Xummit), I recall someone
> > saying that the structure of the files in xenlinux may
> > change soon so that the xen-specific files would move to
> > a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
> > instead of linux/arch/xen/i386).
> 
> I said I would do that.  I started, and got preempted.  I'm actually
> just finally clearing my plate to get back to it this week.
> 
> thanks,
> -chris
> 

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Mark Williamson]

> There are a number of "architecture independent" files
> and headers.  Where are you planning on putting those?

The last plan I heard was to put "core" Xen files under drivers/xen.  It seems 
kinda "backwards" putting arch/xen under there but it seems like the most 
reasonable place for code that's Xen-specific but CPU-neutral.

Cheers,
Mark

> Also, what is the expected timing of the change (e.g.
> pre-Xen-3.0 release or post-)?
>
> I'm starting some work in this code for ia64 and would
> prefer to map to the "future" directory structure if
> possible.
>
> Thanks,
> Dan
>
> > -----Original Message-----
> > From: Chris Wright [mailto:chrisw@osdl.org]
> > Sent: Monday, May 16, 2005 12:08 PM
> > To: Magenheimer, Dan (HP Labs Fort Collins)
> > Cc: xen-devel
> > Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> >
> > * Magenheimer, Dan (HP Labs Fort Collins)
> >
> > (dan.magenheimer@hp.com) wrote:
> > > A while ago (I think at the Xummit), I recall someone
> > > saying that the structure of the files in xenlinux may
> > > change soon so that the xen-specific files would move to
> > > a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
> > > instead of linux/arch/xen/i386).
> >
> > I said I would do that.  I started, and got preempted.  I'm actually
> > just finally clearing my plate to get back to it this week.
> >
> > thanks,
> > -chris
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

Hmmm... kinda ugly.  The right place for the (non-driver)
files is probably in a top-level "xen" directory, but I'll
bet it would take a very long lkml discussion to make that
happen. 

> -----Original Message-----
> From: maw48@hermes.cam.ac.uk [mailto:maw48@hermes.cam.ac.uk] 
> On Behalf Of Mark Williamson
> Sent: Monday, May 16, 2005 4:21 PM
> To: xen-devel@lists.xensource.com
> Cc: Magenheimer, Dan (HP Labs Fort Collins); Chris Wright
> Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> 
> > There are a number of "architecture independent" files
> > and headers.  Where are you planning on putting those?
> 
> The last plan I heard was to put "core" Xen files under 
> drivers/xen.  It seems 
> kinda "backwards" putting arch/xen under there but it seems 
> like the most 
> reasonable place for code that's Xen-specific but CPU-neutral.
> 
> Cheers,
> Mark
> 
> > Also, what is the expected timing of the change (e.g.
> > pre-Xen-3.0 release or post-)?
> >
> > I'm starting some work in this code for ia64 and would
> > prefer to map to the "future" directory structure if
> > possible.
> >
> > Thanks,
> > Dan
> >
> > > -----Original Message-----
> > > From: Chris Wright [mailto:chrisw@osdl.org]
> > > Sent: Monday, May 16, 2005 12:08 PM
> > > To: Magenheimer, Dan (HP Labs Fort Collins)
> > > Cc: xen-devel
> > > Subject: Re: [Xen-devel] linux/arch/xen/i386 or 
> linux/arch/i386/xen
> > >
> > > * Magenheimer, Dan (HP Labs Fort Collins)
> > >
> > > (dan.magenheimer@hp.com) wrote:
> > > > A while ago (I think at the Xummit), I recall someone
> > > > saying that the structure of the files in xenlinux may
> > > > change soon so that the xen-specific files would move to
> > > > a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
> > > > instead of linux/arch/xen/i386).
> > >
> > > I said I would do that.  I started, and got preempted.  
> I'm actually
> > > just finally clearing my plate to get back to it this week.
> > >
> > > thanks,
> > > -chris
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Mark Williamson]

> Hmmm... kinda ugly.  The right place for the (non-driver)
> files is probably in a top-level "xen" directory, but I'll

Well...  Most of the things that would end up under drivers/xen would be a bit 
driver like" anyhow...  Things like the console, the control interface... 
There's nothing stopping grant tables, kexec, some event channel stuff, etc 
going in there too.  It does seem a bit "backwards" but I guess all those 
things are "drivers" in some sense...

> bet it would take a very long lkml discussion to make that
> happen.

Heh, I'd love to see the fight that ensued if someone suggested that at this 
point :-)  Maybe one day...

Cheers,
Mark

> > -----Original Message-----
> > From: maw48@hermes.cam.ac.uk [mailto:maw48@hermes.cam.ac.uk]
> > On Behalf Of Mark Williamson
> > Sent: Monday, May 16, 2005 4:21 PM
> > To: xen-devel@lists.xensource.com
> > Cc: Magenheimer, Dan (HP Labs Fort Collins); Chris Wright
> > Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> >
> > > There are a number of "architecture independent" files
> > > and headers.  Where are you planning on putting those?
> >
> > The last plan I heard was to put "core" Xen files under
> > drivers/xen.  It seems
> > kinda "backwards" putting arch/xen under there but it seems
> > like the most
> > reasonable place for code that's Xen-specific but CPU-neutral.
> >
> > Cheers,
> > Mark
> >
> > > Also, what is the expected timing of the change (e.g.
> > > pre-Xen-3.0 release or post-)?
> > >
> > > I'm starting some work in this code for ia64 and would
> > > prefer to map to the "future" directory structure if
> > > possible.
> > >
> > > Thanks,
> > > Dan
> > >
> > > > -----Original Message-----
> > > > From: Chris Wright [mailto:chrisw@osdl.org]
> > > > Sent: Monday, May 16, 2005 12:08 PM
> > > > To: Magenheimer, Dan (HP Labs Fort Collins)
> > > > Cc: xen-devel
> > > > Subject: Re: [Xen-devel] linux/arch/xen/i386 or
> >
> > linux/arch/i386/xen
> >
> > > > * Magenheimer, Dan (HP Labs Fort Collins)
> > > >
> > > > (dan.magenheimer@hp.com) wrote:
> > > > > A while ago (I think at the Xummit), I recall someone
> > > > > saying that the structure of the files in xenlinux may
> > > > > change soon so that the xen-specific files would move to
> > > > > a subdirectory or arch/<arch> (e.g. linux/arch/i386/xen
> > > > > instead of linux/arch/xen/i386).
> > > >
> > > > I said I would do that.  I started, and got preempted.
> >
> > I'm actually
> >
> > > > just finally clearing my plate to get back to it this week.
> > > >
> > > > thanks,
> > > > -chris
> > >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xensource.com
> > > http://lists.xensource.com/xen-devel

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Mark Williamson (mark.williamson@cl.cam.ac.uk) wrote:
> > Hmmm... kinda ugly.  The right place for the (non-driver)
> > files is probably in a top-level "xen" directory, but I'll
> 
> Well...  Most of the things that would end up under drivers/xen would be a bit 
> driver like" anyhow...  Things like the console, the control interface... 
> There's nothing stopping grant tables, kexec, some event channel stuff, etc 
> going in there too.  It does seem a bit "backwards" but I guess all those 
> things are "drivers" in some sense...

Yeah, and the primary point is to eliminate as much copied code as
possible (at arch level).  So drivers/xen is closest fit (and not
entirely w/out precedence).

> > bet it would take a very long lkml discussion to make that
> > happen.
> 
> Heh, I'd love to see the fight that ensued if someone suggested that at this 
> point :-)  Maybe one day...

Heh ;-)

thanks,
-chris

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

Where are xen-generic header files going to go?
Where are arch-specific header files going to go?

Ar you going to try to get the structure right for
multiple arch's rather than just invert the structure?

> -----Original Message-----
> From: Chris Wright [mailto:chrisw@osdl.org] 
> Sent: Monday, May 16, 2005 5:23 PM
> To: Mark Williamson
> Cc: Magenheimer, Dan (HP Labs Fort Collins); Chris Wright; 
> xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> 
> * Mark Williamson (mark.williamson@cl.cam.ac.uk) wrote:
> > > Hmmm... kinda ugly.  The right place for the (non-driver)
> > > files is probably in a top-level "xen" directory, but I'll
> > 
> > Well...  Most of the things that would end up under 
> drivers/xen would be a bit 
> > driver like" anyhow...  Things like the console, the 
> control interface... 
> > There's nothing stopping grant tables, kexec, some event 
> channel stuff, etc 
> > going in there too.  It does seem a bit "backwards" but I 
> guess all those 
> > things are "drivers" in some sense...
> 
> Yeah, and the primary point is to eliminate as much copied code as
> possible (at arch level).  So drivers/xen is closest fit (and not
> entirely w/out precedence).
> 
> > > bet it would take a very long lkml discussion to make that
> > > happen.
> > 
> > Heh, I'd love to see the fight that ensued if someone 
> suggested that at this 
> > point :-)  Maybe one day...
> 
> Heh ;-)
> 
> thanks,
> -chris
> 

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> Where are xen-generic header files going to go?

Will include/xen/ work for you?

> Where are arch-specific header files going to go?

That's part of sub-arch (this just needs to get include order right).

> Ar you going to try to get the structure right for
> multiple arch's rather than just invert the structure?

The structure should be similar for multiple arch's, but it won't
fallout from doing one arch.  Already I was doing i386 separate from
x86_64.  

thanks,
-chris

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

> Will include/xen/ work for you?

OK with me, but will that be as hard to get by the Linux community
as linux/xen for the generic C code?  If so, maybe
include/asm-generic/xen/file.h?

> That's part of sub-arch (this just needs to get include order right).

So for example include/asm-ia64/xen/file.h?

Also, what about arch-specific C code?  Hopefully this
will be minimized in drivers (by handling arch-specific
features with macros) but for most of the other xen-specific
files (e.g. setup.c) there will almost certainly need to
be arch-specific subdirectories...

So drivers/xen/arch/ia64/file.c or
drivers/xen/ia64/file.c or ??

Thanks,
Dan

> -----Original Message-----
> From: Chris Wright [mailto:chrisw@osdl.org] 
> Sent: Tuesday, May 17, 2005 11:58 AM
> To: Magenheimer, Dan (HP Labs Fort Collins)
> Cc: Chris Wright; Mark Williamson; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> 
> * Magenheimer, Dan (HP Labs Fort Collins) 
> (dan.magenheimer@hp.com) wrote:
> > Where are xen-generic header files going to go?
> 
> Will include/xen/ work for you?
> 
> > Where are arch-specific header files going to go?
> 
> That's part of sub-arch (this just needs to get include order right).
> 
> > Ar you going to try to get the structure right for
> > multiple arch's rather than just invert the structure?
> 
> The structure should be similar for multiple arch's, but it won't
> fallout from doing one arch.  Already I was doing i386 separate from
> x86_64.  
> 
> thanks,
> -chris
> 

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> > Will include/xen/ work for you?
> 
> OK with me, but will that be as hard to get by the Linux community
> as linux/xen for the generic C code?  If so, maybe
> include/asm-generic/xen/file.h?

Shouldn't be an issue.

> > That's part of sub-arch (this just needs to get include order right).
> 
> So for example include/asm-ia64/xen/file.h?

Yup, basically (s/xen/mach-xen/).

> Also, what about arch-specific C code?  Hopefully this
> will be minimized in drivers (by handling arch-specific
> features with macros) but for most of the other xen-specific
> files (e.g. setup.c) there will almost certainly need to
> be arch-specific subdirectories...
> 
> So drivers/xen/arch/ia64/file.c or
> drivers/xen/ia64/file.c or ??

Arch specific code should be in sub-arch.

thanks,
-chris

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

> > OK with me, but will that be as hard to get by the Linux community
> > as linux/xen for the generic C code?  If so, maybe
> > include/asm-generic/xen/file.h?
> 
> Shouldn't be an issue.
> 
> > > That's part of sub-arch (this just needs to get include 
> order right).
> > 
> > So for example include/asm-ia64/xen/file.h?
> 
> Yup, basically (s/xen/mach-xen/).
> 
> > Also, what about arch-specific C code?  Hopefully this
> > will be minimized in drivers (by handling arch-specific
> > features with macros) but for most of the other xen-specific
> > files (e.g. setup.c) there will almost certainly need to
> > be arch-specific subdirectories...
> > 
> > So drivers/xen/arch/ia64/file.c or
> > drivers/xen/ia64/file.c or ??
> 
> Arch specific code should be in sub-arch.

Be aware that some linux arch's don't use subarch's or the
mach-xxx syntax.  This latter appears to be limited to i386,
arm, mips and a couple others.  I don't understand the difference
well enough to know if this will matter in the long run,
but it might be worth checking/discussing with a maintainer
of one of the "other" arch's (e.g. ia64).

Dan

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Vincent Hanquez]

On Tue, May 17, 2005 at 11:14:35AM -0700, Chris Wright wrote:
> [snip]

So let's recap:

xen arch code in		arch/$(ARCH)/mach-xen/
xen generic code in		drivers/xen/core/
xen arch includes in		include/asm-$(ARCH)/mach-xen/
xen generic includes in		include/asm-xen/

Now for arch that don't use mach-XX syntax we can probably make a
xen directory instead of a mach-xen directory, but I'ld rather go with
something consistant.

As well we can probably host a public tree in cambridge for anybody
interested in the files reorganisation.

Any others thought, comments ?

-- 
Vincent Hanquez

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

Um, one other minor semantic issue.  The semantics of
using mach-xxx may be inappropriate.  If my understanding
is correct, two mach-xxx's cannot both be built, e.g. one
cannot build a kernel which supports (for i386) both
mach-es7000 and mach-voyager.

There have been various discussions on this list about
"transparent paravirtualization", i.e. the ability for
a paravirtualized kernel to run both as a guest of Xen
and on bare metal.  This is definitely an objective of
Xen/ia64.  Nobody has tried it for Xen/x86, but if it
can be done, I'm sure commercial companies and distros
would be eager to utilize it (one less set of bits to
support).

In many ways, a "xen" subdirectory is much more like
a "pci" or "math-emu" subdirectory, than a subarch.
For example, mach-es7000 and xen may need to co-exist
in the same kernel.

So, mach-xen may be a poor choice.  A subtle distinction
perhaps but when dealing with Linux kernel developers,
purity of thinking may avoid future patch submission
arguments.

So I'd vote for:

xen arch code in		arch/$(ARCH)/xen/
xen generic code in		drivers/xen/core/
xen arch includes in		include/asm-$(ARCH)/xen/
xen generic includes in		include/asm-xen/

though I realize this is not a democracy :-)

> -----Original Message-----
> From: Vincent Hanquez [mailto:vincent.hanquez@cl.cam.ac.uk] 
> Sent: Wednesday, May 18, 2005 4:15 AM
> To: Chris Wright
> Cc: Magenheimer, Dan (HP Labs Fort Collins); 
> xen-devel@lists.xensource.com; Mark Williamson
> Subject: Re: [Xen-devel] linux/arch/xen/i386 or linux/arch/i386/xen
> 
> On Tue, May 17, 2005 at 11:14:35AM -0700, Chris Wright wrote:
> > [snip]
> 
> So let's recap:
> 
> xen arch code in		arch/$(ARCH)/mach-xen/
> xen generic code in		drivers/xen/core/
> xen arch includes in		include/asm-$(ARCH)/mach-xen/
> xen generic includes in		include/asm-xen/
> 
> Now for arch that don't use mach-XX syntax we can probably make a
> xen directory instead of a mach-xen directory, but I'ld rather go with
> something consistant.
> 
> As well we can probably host a public tree in cambridge for anybody
> interested in the files reorganisation.
> 
> Any others thought, comments ?
> 
> -- 
> Vincent Hanquez
> 

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> Um, one other minor semantic issue.  The semantics of
> using mach-xxx may be inappropriate.  If my understanding
> is correct, two mach-xxx's cannot both be built, e.g. one
> cannot build a kernel which supports (for i386) both
> mach-es7000 and mach-voyager.

Yes, I had thought about this.  It's not clear to me it's problematic yet.
In many cases, it's exactly the bits that would become irrelevant that
are in the sub-arch (irq, smpboot, timers, etc.), whereas the generic
arch code would be sufficient running xenlinux on one of the sub-arches.

> There have been various discussions on this list about
> "transparent paravirtualization", i.e. the ability for
> a paravirtualized kernel to run both as a guest of Xen
> and on bare metal.  This is definitely an objective of
> Xen/ia64.  Nobody has tried it for Xen/x86, but if it
> can be done, I'm sure commercial companies and distros
> would be eager to utilize it (one less set of bits to
> support).

Hmm, this would be interesting/problematic.  Hadn't considered that
part.

> In many ways, a "xen" subdirectory is much more like
> a "pci" or "math-emu" subdirectory, than a subarch.
> For example, mach-es7000 and xen may need to co-exist
> in the same kernel.
> 
> So, mach-xen may be a poor choice.  A subtle distinction
> perhaps but when dealing with Linux kernel developers,
> purity of thinking may avoid future patch submission
> arguments.

The primary goal is to avoid code duplication to keep maintenance sane.
Clean way to do that is what really matters.

> So I'd vote for:
> 
> xen arch code in		arch/$(ARCH)/xen/

that's effectively sub-arch

> xen generic code in		drivers/xen/core/

yup (although no need for core/ subdir)

> xen arch includes in		include/asm-$(ARCH)/xen/

again, effectively sub-arch

> xen generic includes in		include/asm-xen/

or just include/xen

> though I realize this is not a democracy :-)

heh ;-)

thanks,
-chris

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Vincent Hanquez (vincent.hanquez@cl.cam.ac.uk) wrote:
> On Tue, May 17, 2005 at 11:14:35AM -0700, Chris Wright wrote:
> > [snip]
> 
> So let's recap:
> 
> xen arch code in		arch/$(ARCH)/mach-xen/
> xen generic code in		drivers/xen/core/
> xen arch includes in		include/asm-$(ARCH)/mach-xen/
> xen generic includes in		include/asm-xen/
> 
> Now for arch that don't use mach-XX syntax we can probably make a
> xen directory instead of a mach-xen directory, but I'ld rather go with
> something consistant.

yes, it needs to be consistant, partly just for ease of working with
kbuild (otherwise we rebuild an unnecessary infrastructure).  Andi, for
example, was open to the idea of adding sub-arch to x86_64 to support
xen.  He just hadn't needed sub-arch support yet.

> As well we can probably host a public tree in cambridge for anybody
> interested in the files reorganisation.
> 
> Any others thought, comments ?

Do you have a list of more of the consolidation/cleanups like the
user_mode/user_mode_vm bits that you sent upstream?

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

> > So I'd vote for:
> > 
> > xen arch code in		arch/$(ARCH)/xen/
> 
> that's effectively sub-arch

The difference is admittedly very subtle (though probably
not to some Linux kernel developer purists).  The question
is whether xen is subsidiary architecture (which uses
the mach- prefix) or whether it is functionality that can
be turned on or off (no mach- prefix).

And since this will appear hard-coded in include lines
in source files, we do need to decide one way or the
other.

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> > > So I'd vote for:
> > > 
> > > xen arch code in		arch/$(ARCH)/xen/
> > 
> > that's effectively sub-arch
> 
> The difference is admittedly very subtle (though probably
> not to some Linux kernel developer purists).  The question
> is whether xen is subsidiary architecture (which uses
> the mach- prefix) or whether it is functionality that can
> be turned on or off (no mach- prefix).

OK, how about one step at a time.  It's already a huge step to move
things around (between Kconfig, and tangled source, and headers...).
The advantage of move towards a known target (sub-arch) is there's
infrastructure in place to support it already.  I don't think it's a
dead-end to go there and then look towards the issues you brought up.

> And since this will appear hard-coded in include lines
> in source files, we do need to decide one way or the
> other.

Things move all the time, it's just a simple matter of patching ;-)
Or did I misunderstand you?

thanks,
-chris

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Christian Limpach]

On 5/17/05, Chris Wright <chrisw@osdl.org> wrote:
> * Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> > Where are xen-generic header files going to go?
> 
> Will include/xen/ work for you?

If that's acceptable, then I think that any of arch/xen (without the
arch specific subdirectories), xen (toplevel xen, how does it compare
in significance/ubiquity to crypto, fs, ipc, net, security or sound?)
or {kernel,mm,drivers/pci}/xen would seem just as reasonable and make
more sense than drivers/xen...

    christian

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Zachary Amsden]

Chris Wright wrote:

>* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
>  
>
>>>>So I'd vote for:
>>>>
>>>>xen arch code in		arch/$(ARCH)/xen/
>>>>        
>>>>
>>>that's effectively sub-arch
>>>      
>>>
>>The difference is admittedly very subtle (though probably
>>not to some Linux kernel developer purists).  The question
>>is whether xen is subsidiary architecture (which uses
>>the mach- prefix) or whether it is functionality that can
>>be turned on or off (no mach- prefix).
>>    
>>
>
>OK, how about one step at a time.  It's already a huge step to move
>things around (between Kconfig, and tangled source, and headers...).
>The advantage of move towards a known target (sub-arch) is there's
>infrastructure in place to support it already.  I don't think it's a
>dead-end to go there and then look towards the issues you brought up.
>  
>

Xen needs it's own mechanisms for implementing TLB flushes and other 
hardware type operations.  Those operations are hardcoded in header 
files in the asm-i386 directory.  It is just plain weird to have a 
completely separate set of functions and macros, say 
xen_local_save_flags().  Many of these macros and functions are used in 
common kernel code, and it would be rather taboo to #ifdef the kernel 
all over the place; #ifdef'ing the asm-i386 code for Xen support is also 
fairly ugly.

The mach-xen approach has the advantage that functions like this, 
local_save_flags can be moved into mach-default.  Then if you build a 
Xen subarchitecture kernel, you can override any definitions you need by 
placing an alternative implementation in include/asm-i386/mach-xen - 
this system is already in place and works quite nicely.

If you've already got the sub-arch prefix, you also get your own 
arch/i386/mach-xen directory for xen specific support code for 
bootstrapping, reboot, shutdown, and a bunch of other platform type 
operations.

If you think about virtualization as a platform, classical 
virtualization is where the platform is a clone of a physical machine.  
Para-virtualization is where the platform presented to the operating 
system is shifted, but the underlying CPU/MMU hardware is the same.  
That is arguably exactly what sub-architecture support is supposed to 
provide.

Zach

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Vincent Hanquez]

On Wed, May 18, 2005 at 09:53:31AM -0700, Chris Wright wrote:
> Do you have a list of more of the consolidation/cleanups like the
> user_mode/user_mode_vm bits that you sent upstream?

For the moment nothing to submit that won't be too xen specific.
If you have any ideas however, feel free to submit it.

-- 
Vincent Hanquez

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Vincent Hanquez]

On Wed, May 18, 2005 at 09:47:48AM -0700, Chris Wright wrote:
> * Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> > Um, one other minor semantic issue.  The semantics of
> > using mach-xxx may be inappropriate.  If my understanding
> > is correct, two mach-xxx's cannot both be built, e.g. one
> > cannot build a kernel which supports (for i386) both
> > mach-es7000 and mach-voyager.
> 
> Yes, I had thought about this.  It's not clear to me it's problematic yet.

I guess it could be, as if I understood correctly Keir's later post, we
would like to run on other subarch. 

> The primary goal is to avoid code duplication to keep maintenance sane.
> Clean way to do that is what really matters.

indeed.

> > xen generic code in		drivers/xen/core/
> 
> yup (although no need for core/ subdir)

there's already some directories in drivers/xen/ for xen's drivers.
so keeping core/ would probably be clearer.

-- 
Vincent Hanquez

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

> OK, how about one step at a time.  It's already a huge step to move
> things around (between Kconfig, and tangled source, and headers...).
> The advantage of move towards a known target (sub-arch) is there's
> infrastructure in place to support it already.  I don't think it's a
> dead-end to go there and then look towards the issues you brought up.

It's OK with me.  Sometimes two-step changes are better than one.
It would be nice to have finished the second step before Xen support
finds its way into the mainstream kernel though (IMHO).

> > And since this will appear hard-coded in include lines
> > in source files, we do need to decide one way or the
> > other.
> 
> Things move all the time, it's just a simple matter of patching ;-)
> Or did I misunderstand you?

Yeah, I was just trying to pre-answer the question:  "Does this
silly naming issue really matter?"  with, yes, a fair number of
files will have to change.

Dan

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Zachary Amsden]

Vincent Hanquez wrote:

>On Wed, May 18, 2005 at 09:47:48AM -0700, Chris Wright wrote:
>  
>
>>* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
>>    
>>
>>>Um, one other minor semantic issue.  The semantics of
>>>using mach-xxx may be inappropriate.  If my understanding
>>>is correct, two mach-xxx's cannot both be built, e.g. one
>>>cannot build a kernel which supports (for i386) both
>>>mach-es7000 and mach-voyager.
>>>      
>>>
>>Yes, I had thought about this.  It's not clear to me it's problematic yet.
>>    
>>
>
>I guess it could be, as if I understood correctly Keir's later post, we
>would like to run on other subarch. 
>  
>

I see, you would like domain zero to support other subarchs?

I guarantee you it is not going to be voyager!  There should be no issue 
for user domains, but could domain zero could potentially need to know 
things about the hardware subarch, say for some large NUMA systems?

Zach

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Vincent Hanquez]

On Wed, May 18, 2005 at 12:48:03PM -0700, Zachary Amsden wrote:
> I see, you would like domain zero to support other subarchs?
> 
> I guarantee you it is not going to be voyager!  There should be no issue 
> for user domains, but could domain zero could potentially need to know 
> things about the hardware subarch, say for some large NUMA systems?

I think we don't want to support the voyager subarch, but maybe the
bigsmp and es7000.

-- 
Vincent Hanquez

RE: linux/arch/xen/i386 or linux/arch/i386/xen

[Magenheimer, Dan (HP Labs Fort Collins)]

> > Will include/xen/ work for you?
> 
> If that's acceptable, then I think that any of arch/xen (without the
> arch specific subdirectories), xen (toplevel xen, how does it compare
> in significance/ubiquity to crypto, fs, ipc, net, security or sound?)
> or {kernel,mm,drivers/pci}/xen would seem just as reasonable and make
> more sense than drivers/xen...
> 
>     christian

After some thought (and perhaps for the first time :-)  I agree
with Christian.

Chris, any chance you can try out the idea of a top-level
xen directory on Linus?  It's really where it belongs...
the xen drivers aren't really drivers anyway, more like IPC
to provide virtual drivers.  And so much of the functionality
is cross-arch AND even potentially cross-sub-arch

It's probably a political non-starter, but can't hurt to ask.

Re: linux/arch/xen/i386 or linux/arch/i386/xen

[Chris Wright]

* Magenheimer, Dan (HP Labs Fort Collins) (dan.magenheimer@hp.com) wrote:
> Chris, any chance you can try out the idea of a top-level
> xen directory on Linus?  It's really where it belongs...
> the xen drivers aren't really drivers anyway, more like IPC
> to provide virtual drivers.  And so much of the functionality
> is cross-arch AND even potentially cross-sub-arch
> 
> It's probably a political non-starter, but can't hurt to ask.

Sure, I'm actually more concerned about getting the arch bit correct.
Where the generic stuff goes shouldn't be a major issue IMHO.

thanks,
-chris