Re: [RFC, Announce] Unified x86 architecture, arch/x86

[Arnd Bergmann <arnd@arndb.de>]

On Saturday 21 July 2007, Thomas Gleixner wrote:
> The topic of sharing more x86 code has been discussed on LKML a number 
> of times. Various approaches were discussed and we decided to advance 
> the discussion by implementing a full solution that brings the 
> transition to a shared tree to completion.

Great stuff. I've worked on doing the same for s390 and powerpc
in the past, and really think it's the right thing to do. I've
even started my own x86 merge two or three times in the past
but never got very far because of the quickly moving source.

> In this initial implementation the old arch/i386 and arch/x86_64 trees 
> are removed _immediately_, in the same commit, and all future x86 
> development goes on in the new, shared tree. So the transition right now 
> is one atomic operation.
> 
> As a next step we plan to generate a gradual, fully bisectable, fully
> working switchover from the current code to the fully populated
> arch/x86 tree. It will result in about 1000-2000 commits. We are
> releasing our current solution because it 100% represents the finally
> resulting arch/x86 source tree already, and we first wanted to make
> sure that the new architecture layout works fine and folks are happy
> before we go and do the (even more complex) fine-grained work.

I don't think it's really good to do it this way, or maybe I'm still
misunderstanding where you're going. If you really want to end
up with the exact set of files that you have your tree now, I see
absolutely zero point in making it bisectable. On the contrary,
there is nothing particularly complicated in it, so once it has
seen some amount of testing it can better get merged in one
big changeset. I'm just not convinced that it actully is what
we want to end up with.

In my experience, it's very helpful to have a single set of header
files, and merging the two versions of one header usually exposes
bugs that have been fixed in only one of the two, so you get
to fix actual bugs in the process.

In the s390 merge, I also started out in an attempt to guarantee
unchanged object files, much like what you describe. However, it
turned out that fixing it in the process is actually easier.
Either way, 'diff -D __x86_64__' is a great tool for a start, you
should try it out to see how easy it is to merge a lot of files.

To put it into perspective, I think the s390 merge was a lot easier
than the x86 merge, because there is only a very limited set of
hardware configurations for s390 compared to others. We ended up
doing the full merge with three people within less than a week
and no separate files at all.

OTOH, the powerpc merge is now going into its third year, mostly
because it was started with the intention to remove all cruft
in the process and to only allow sane code into the new architecture.

The steps that I'd suggest instead are:

* merge all exported header files of the two architectures. This
  alone is a worthy goal, because it allows us to get rid of
  the ugly code for deciding which version to use in installed
  headers and elsewhere.

* Merge the remaining header files, to end up with a single
  include/asm-x86 directory.

* Come up with a model that integrates the machine type selection
  of i386 with the way we build things on x86_64. One way would
  be to make X86_64 another platform next to X86_PC, X86_VOYAGER
  and the others.

* Create an arch/x86/Kconfig that handles the new common
  configuration

* Create an arch/x86/Makefile that descends into ../i386/* and
  ../x86_64/* instead of its subdirectories.

* Merge the arch/x86/* subdirectories, one at a time, starting with
  the low-hanging fruit like oprofile or pci, and do the hard
  ones like mm and kernel last.

Unfortunately, I don't think I'll spend much time on this, so I
don't get to decide on it, but you asked for feedback ;-)

	Arnd <><