Re: Re: changeset 13403 ...

Re: Re: changeset 13403 ...

[Joe Bonasera]

I'm dropping in on this part way through but
out of context the following sounds dangerous.

For example, Solaris domains assume at start
of time, that only pages mapped in the start
area are in use in any way. All other pages
go immediately in our free lists. If you change
this protocol, then we'll have backwards
incompatibility issues to fix.

> Date: Mon, 29 Jan 2007 16:35:48 +0100
> From: Gerd Hoffmann <kraxel@suse.de>
> Subject: Re: [Xen-devel] Re: changeset 13403 ...
> To: Keir Fraser <keir@xensource.com>
> Cc: Xen devel list <xen-devel@lists.xensource.com>
> Message-ID: <45BE1454.9030101@suse.de>
> Content-Type: text/plain; charset=ISO-8859-1
> 
>   Hi,
> 
>> You may even know that they only map part of
>> the guest physical memory and so choose to locate the intermediate
>> pagetables outside that mapped area.
> 
> Good point, that solves at least a part of the problem, as the read-only
> mapping is needed one-way only then ...
> 
> thanks,
>   Gerd
> 

Re: Re: changeset 13403 ...

[Keir Fraser]

We're not proposing to change that. This out-of-the-way allocation would be
a very temporary measure to transfer us from the current pagetables to the
builder pagetables. The memory area will be unused by the time the
chain-booted kernel starts to execute.

 -- Keir

On 29/1/07 18:36, "Joe Bonasera" <joe.bonasera@sun.com> wrote:

> 
> I'm dropping in on this part way through but
> out of context the following sounds dangerous.
> 
> For example, Solaris domains assume at start
> of time, that only pages mapped in the start
> area are in use in any way. All other pages
> go immediately in our free lists. If you change
> this protocol, then we'll have backwards
> incompatibility issues to fix.
> 
>> Date: Mon, 29 Jan 2007 16:35:48 +0100
>> From: Gerd Hoffmann <kraxel@suse.de>
>> Subject: Re: [Xen-devel] Re: changeset 13403 ...
>> To: Keir Fraser <keir@xensource.com>
>> Cc: Xen devel list <xen-devel@lists.xensource.com>
>> Message-ID: <45BE1454.9030101@suse.de>
>> Content-Type: text/plain; charset=ISO-8859-1
>> 
>>   Hi,
>> 
>>> You may even know that they only map part of
>>> the guest physical memory and so choose to locate the intermediate
>>> pagetables outside that mapped area.
>> 
>> Good point, that solves at least a part of the problem, as the read-only
>> mapping is needed one-way only then ...
>> 
>> thanks,
>>   Gerd
>> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Re: changeset 13403 ...

[Gerd Hoffmann]

  Hi,

> I'm dropping in on this part way through but
> out of context the following sounds dangerous.
> 
> For example, Solaris domains assume at start
> of time, that only pages mapped in the start
> area are in use in any way. All other pages
> go immediately in our free lists. If you change
> this protocol, then we'll have backwards
> incompatibility issues to fix.

It's another protocol change: you can't switch from one self-consistent
set of page tables to another set of self-consistent page tables (where
the old set has only the old pagetables but not the new ones mapped
read-only, same for the new set and the old tables) any more.

That was added during the 3.0.3 cycle (I think) and now removed again.
domU kexec was the only user of that feature as far I know.  I've found
a way to get away without it.  I'm using a small intermediate page table
as suggested by Keir for one page table switch, the second page switch
to the final page table layout for the new kernel is easier to handle
due memory being mapped partly only in the builders initial memory layout.

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Re: changeset 13403 ...

[Keir Fraser]

On 1/2/07 16:10, "Gerd Hoffmann" <kraxel@suse.de> wrote:

> It's another protocol change: you can't switch from one self-consistent
> set of page tables to another set of self-consistent page tables (where
> the old set has only the old pagetables but not the new ones mapped
> read-only, same for the new set and the old tables) any more.

Yeah, it turned out to be a pain in the butt feature to support (due to
interactions with all the different paging modes we support, including
shadow modes) and it was basically broken in lots of configurations (e.g.,
try using it while live migrating? It wasn't gonna work! :-). This was
exacerbated by the fact the feature was rarely tested, since it's only used
by the ongoing kexec domU development work.

 -- Keir

Re: changeset 13403 ...

[Keir Fraser]

On 29/1/07 13:40, "Gerd Hoffmann" <kraxel@suse.de> wrote:

> Just noticed that domU kexec doesn't work any more, changeset 13403 is
> the culpit.  Hmm.  No, it isn't easy to work-around ...
> 
> cheers,

Two pagetable switches in a single multicall? First switch is to a pagetable
which maps only the multicall structure. This small intermediate table has
to be mapped read-only in the original pagetables and in the final
pagetables, but it will only be a few pages of memory (4 in the worst case).

 -- Keir

Re: Re: changeset 13403 ...

[Keir Fraser]

On 29/1/07 13:48, "Keir Fraser" <keir@xensource.com> wrote:

> On 29/1/07 13:40, "Gerd Hoffmann" <kraxel@suse.de> wrote:
> 
>> Just noticed that domU kexec doesn't work any more, changeset 13403 is
>> the culpit.  Hmm.  No, it isn't easy to work-around ...
>> 
>> cheers,
> 
> Two pagetable switches in a single multicall? First switch is to a pagetable
> which maps only the multicall structure. This small intermediate table has
> to be mapped read-only in the original pagetables and in the final
> pagetables, but it will only be a few pages of memory (4 in the worst case).

Oh, it'll need a hypercall transfer page too. And it's a mmuext_op list that
you need, not a multicall. The transfer page and mmuext_op list can be
placed in adjacent pages so that you don't need any more intermediate
pagetable pages.

 -- Keir

Re: Re: changeset 13403 ...

[Gerd Hoffmann]

Keir Fraser wrote:
> 
> 
> On 29/1/07 13:48, "Keir Fraser" <keir@xensource.com> wrote:
> 
>> On 29/1/07 13:40, "Gerd Hoffmann" <kraxel@suse.de> wrote:
>>
>>> Just noticed that domU kexec doesn't work any more, changeset 13403 is
>>> the culpit.  Hmm.  No, it isn't easy to work-around ...
>>>
>>> cheers,
>> Two pagetable switches in a single multicall? First switch is to a pagetable
>> which maps only the multicall structure. This small intermediate table has
>> to be mapped read-only in the original pagetables and in the final
>> pagetables, but it will only be a few pages of memory (4 in the worst case).

Helps a bit, but doesn't solve the fundamental problem that I can't use
the page tables created by the domain-builder as-is.

> Oh, it'll need a hypercall transfer page too. And it's a mmuext_op list that
> you need, not a multicall. The transfer page and mmuext_op list can be
> placed in adjacent pages so that you don't need any more intermediate
> pagetable pages.

Hmm.  hypercall transfer page?  The one with the int 82h instructions?

Is the mmuext_op list copyed over before running it?  If so, the it
should not be required to be mapped all the time, right?  In that case I
could get away with an empty page as page directory?  Or maybe even
baseptr=NULL?

cheers,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>

Re: Re: changeset 13403 ...

[Keir Fraser]

On 29/1/07 14:47, "Gerd Hoffmann" <kraxel@suse.de> wrote:

>>> Two pagetable switches in a single multicall? First switch is to a pagetable
>>> which maps only the multicall structure. This small intermediate table has
>>> to be mapped read-only in the original pagetables and in the final
>>> pagetables, but it will only be a few pages of memory (4 in the worst case).
> 
> Helps a bit, but doesn't solve the fundamental problem that I can't use
> the page tables created by the domain-builder as-is.

That's not fundamental. You know where those pagetables are. You can walk
them and you can modify them. You may even know that they only map part of
the guest physical memory and so choose to locate the intermediate
pagetables outside that mapped area. Then perhaps you can get away with
needing to modify the builder pagetables at all?

>> Oh, it'll need a hypercall transfer page too. And it's a mmuext_op list that
>> you need, not a multicall. The transfer page and mmuext_op list can be
>> placed in adjacent pages so that you don't need any more intermediate
>> pagetable pages.
> 
> Hmm.  hypercall transfer page?  The one with the int 82h instructions?

Yes. But actually I meant an mmuext_op array.

> Is the mmuext_op list copyed over before running it?  If so, the it
> should not be required to be mapped all the time, right?

It gets written back to with hypercall result. Mmuext_op arrays also get
read from for each individual sub-operation.

> In that case I
> could get away with an empty page as page directory?  Or maybe even
> baseptr=NULL?

No and no, I'm afraid. It's not quite that easy.

 -- Keir

Re: Re: changeset 13403 ...

[Gerd Hoffmann]

  Hi,

> You may even know that they only map part of
> the guest physical memory and so choose to locate the intermediate
> pagetables outside that mapped area.

Good point, that solves at least a part of the problem, as the read-only
mapping is needed one-way only then ...

thanks,
  Gerd

-- 
Gerd Hoffmann <kraxel@suse.de>