About arm32 address translation

About arm32 address translation

[Chen Baozi]

[-- Attachment #1.1: Type: text/plain, Size: 275 bytes --]

Hi Ian,

I am trying to add supporting dumping 32-bit guest on arm32 to see what happened in my dom0 kernel when booting. It seems gvirt_to_maddr() is not workable under arm32. But why has it been implemented both in include/asm-arm/arm{32,64}/page.h?

Thanks.

Baozi

[-- Attachment #1.2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: About arm32 address translation

[Ian Campbell]

On Wed, 2013-10-16 at 17:20 +0800, Chen Baozi wrote:
> Hi Ian,
> 
> I am trying to add supporting dumping 32-bit guest on arm32 to see
> what happened in my dom0 kernel when booting. It seems
> gvirt_to_maddr() is not workable under arm32.

What makes you say that, it should work, or else it is buggy and we've
not noticed, but it is used in copy to/from user routines so I doubt
that.

>  But why has it been implemented both in
> include/asm-arm/arm{32,64}/page.h?
> 
> Thanks.
> 
> Baozi

Re: About arm32 address translation

[Chen Baozi]

On Oct 16, 2013, at 5:25 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> On Wed, 2013-10-16 at 17:20 +0800, Chen Baozi wrote:
>> Hi Ian,
>> 
>> I am trying to add supporting dumping 32-bit guest on arm32 to see
>> what happened in my dom0 kernel when booting. It seems
>> gvirt_to_maddr() is not workable under arm32.
> 
> What makes you say that, it should work, or else it is buggy and we've
> not noticed, but it is used in copy to/from user routines so I doubt
> that.

Ok, I'll look into it. The output shows me "Failed to convert stack to physical address". In this case, it must be gvirt_to_maddr to return -EFAULT.

Thanks.

Baozi

> 
>> But why has it been implemented both in
>> include/asm-arm/arm{32,64}/page.h?
>> 
>> Thanks.
>> 
>> Baozi
> 
> 
> 

Re: About arm32 address translation

[Ian Campbell]

On Wed, 2013-10-16 at 18:02 +0800, Chen Baozi wrote:
> On Oct 16, 2013, at 5:25 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> 
> > On Wed, 2013-10-16 at 17:20 +0800, Chen Baozi wrote:
> >> Hi Ian,
> >> 
> >> I am trying to add supporting dumping 32-bit guest on arm32 to see
> >> what happened in my dom0 kernel when booting. It seems
> >> gvirt_to_maddr() is not workable under arm32.
> > 
> > What makes you say that, it should work, or else it is buggy and we've
> > not noticed, but it is used in copy to/from user routines so I doubt
> > that.
> 
> Ok, I'll look into it. The output shows me "Failed to convert stack to
> physical address". In this case, it must be gvirt_to_maddr to return
> -EFAULT.

That suggests that the guest's SP doesn't have a mapping in the guest's
PTs. What's the full trace?


Ian

Re: About arm32 address translation

[Chen Baozi]

On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
> On Wed, 2013-10-16 at 18:02 +0800, Chen Baozi wrote:
> > On Oct 16, 2013, at 5:25 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> > 
> > > On Wed, 2013-10-16 at 17:20 +0800, Chen Baozi wrote:
> > >> Hi Ian,
> > >> 
> > >> I am trying to add supporting dumping 32-bit guest on arm32 to see
> > >> what happened in my dom0 kernel when booting. It seems
> > >> gvirt_to_maddr() is not workable under arm32.
> > > 
> > > What makes you say that, it should work, or else it is buggy and we've
> > > not noticed, but it is used in copy to/from user routines so I doubt
> > > that.
> > 
> > Ok, I'll look into it. The output shows me "Failed to convert stack to
> > physical address". In this case, it must be gvirt_to_maddr to return
> > -EFAULT.
> 
> That suggests that the guest's SP doesn't have a mapping in the guest's
> PTs. What's the full trace?

See http://pastebin.com/DYL5gkAQ

Thanks.

Baozi

Re: About arm32 address translation

[Chen Baozi]

On Oct 16, 2013, at 6:24 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> On Wed, 2013-10-16 at 18:02 +0800, Chen Baozi wrote:
>> On Oct 16, 2013, at 5:25 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
>> 
>>> On Wed, 2013-10-16 at 17:20 +0800, Chen Baozi wrote:
>>>> Hi Ian,
>>>> 
>>>> I am trying to add supporting dumping 32-bit guest on arm32 to see
>>>> what happened in my dom0 kernel when booting. It seems
>>>> gvirt_to_maddr() is not workable under arm32.
>>> 
>>> What makes you say that, it should work, or else it is buggy and we've
>>> not noticed, but it is used in copy to/from user routines so I doubt
>>> that.
>> 
>> Ok, I'll look into it. The output shows me "Failed to convert stack to
>> physical address". In this case, it must be gvirt_to_maddr to return
>> -EFAULT.
> 
> That suggests that the guest's SP doesn't have a mapping in the guest's
> PTs. What's the full trace?

I may have missed lots of work when trying to enable this, for this is my first time to work on tracing related work, ;-)

Cheers,

Baozi

Re: About arm32 address translation

[Ian Campbell]

On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
> On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:

> > That suggests that the guest's SP doesn't have a mapping in the guest's
> > PTs. What's the full trace?
> 
> See http://pastebin.com/DYL5gkAQ

Yeah, not a whole lot to go on is there ;-)

Converting the PC (c02c1544) to a code line with gdb or addr2line would
be a good start I think.

You could also consider hacking up an earlyprintk via Xen by dropping a
xen_raw_console_write into vprintk_emit once the buffer is formatted and
see if that gets you any useful console output.

(Somebody really needs to make a patch for proper Xen capable
earlyprintk for arm...)

Ian.

Re: About arm32 address translation

[Chen Baozi]

On Oct 16, 2013, at 6:41 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:

> On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
>> On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
> 
>>> That suggests that the guest's SP doesn't have a mapping in the guest's
>>> PTs. What's the full trace?
>> 
>> See http://pastebin.com/DYL5gkAQ
> 
> Yeah, not a whole lot to go on is there ;-)
> 
> Converting the PC (c02c1544) to a code line with gdb or addr2line would
> be a good start I think.

Hmmm, I've posted a mail about this yesterday. The addr2line points 0xc021544 to __loop_delay in arch/arm/lib/delay-loop.S of linux kernel.

> 
> You could also consider hacking up an earlyprintk via Xen by dropping a
> xen_raw_console_write into vprintk_emit once the buffer is formatted and
> see if that gets you any useful console output.

Ok, I'll have a try.

Thanks.

Baozi

> 
> (Somebody really needs to make a patch for proper Xen capable
> earlyprintk for arm...)
> 
> Ian.
> 

Re: About arm32 address translation

[Ian Campbell]

On Wed, 2013-10-16 at 18:47 +0800, Chen Baozi wrote:
> On Oct 16, 2013, at 6:41 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:
> 
> > On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
> >> On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
> > 
> >>> That suggests that the guest's SP doesn't have a mapping in the guest's
> >>> PTs. What's the full trace?
> >> 
> >> See http://pastebin.com/DYL5gkAQ
> > 
> > Yeah, not a whole lot to go on is there ;-)
> > 
> > Converting the PC (c02c1544) to a code line with gdb or addr2line would
> > be a good start I think.
> 
> Hmmm, I've posted a mail about this yesterday. The addr2line points
> 0xc021544 to __loop_delay in arch/arm/lib/delay-loop.S of linux
> kernel.

Oh, so maybe this is just a CPU sitting in the idle loop. Odd that the
stack isn't valid, not sure why that would be. You could make
gvirt_to_maddr log the failed PAR register which we could then decode
using the manual.

You could also try dump_guest_s1_walk() to see if the guest PTs make
sense.

It is likely to be stuck in the idle loop because it is waiting for some
device to fire an interrupt. Getting some console logging might help
determine which device that is.

> 
> > 
> > You could also consider hacking up an earlyprintk via Xen by dropping a
> > xen_raw_console_write into vprintk_emit once the buffer is formatted and
> > see if that gets you any useful console output.
> 
> Ok, I'll have a try.
> 
> Thanks.
> 
> Baozi
> 
> > 
> > (Somebody really needs to make a patch for proper Xen capable
> > earlyprintk for arm...)
> > 
> > Ian.
> > 
> 

Re: About arm32 address translation

[Chen Baozi]

On Wed, Oct 16, 2013 at 11:41:18AM +0100, Ian Campbell wrote:
> On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
> > On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
> 
> > > That suggests that the guest's SP doesn't have a mapping in the guest's
> > > PTs. What's the full trace?
> > 
> > See http://pastebin.com/DYL5gkAQ
> 
> Yeah, not a whole lot to go on is there ;-)
> 
> Converting the PC (c02c1544) to a code line with gdb or addr2line would
> be a good start I think.
> 
> You could also consider hacking up an earlyprintk via Xen by dropping a
> xen_raw_console_write into vprintk_emit once the buffer is formatted and
> see if that gets you any useful console output.

The hack is workable! See http://pastebin.com/iSGJnKSg

It looks like that dom0 has got some issues on "uart3", which is "skipped" due
to its disabled status in DT.

Cheers,

Baozi

Re: About arm32 address translation

[Chen Baozi]

On Oct 16, 2013, at 7:44 PM, Chen Baozi <baozich@gmail.com> wrote:

> On Wed, Oct 16, 2013 at 11:41:18AM +0100, Ian Campbell wrote:
>> On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
>>> On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
>> 
>>>> That suggests that the guest's SP doesn't have a mapping in the guest's
>>>> PTs. What's the full trace?
>>> 
>>> See http://pastebin.com/DYL5gkAQ
>> 
>> Yeah, not a whole lot to go on is there ;-)
>> 
>> Converting the PC (c02c1544) to a code line with gdb or addr2line would
>> be a good start I think.
>> 
>> You could also consider hacking up an earlyprintk via Xen by dropping a
>> xen_raw_console_write into vprintk_emit once the buffer is formatted and
>> see if that gets you any useful console output.
> 
> The hack is workable! See http://pastebin.com/iSGJnKSg
> 
> It looks like that dom0 has got some issues on "uart3", which is "skipped" due
> to its disabled status in DT.

Julien,

Will the DT node of uart3 be populated to dom0 but not mapped?

Thanks,

Baozi

Re: About arm32 address translation

[David Vrabel]

On 16/10/13 11:41, Ian Campbell wrote:
> 
> (Somebody really needs to make a patch for proper Xen capable
> earlyprintk for arm...)

The existing xenboot console should just work, yes?  It just needs
plumbing into ARM's early console setup?

David

Re: About arm32 address translation

[Julien Grall]

On 10/16/2013 01:02 PM, Chen Baozi wrote:
> 
> On Oct 16, 2013, at 7:44 PM, Chen Baozi <baozich@gmail.com> wrote:
> 
>> On Wed, Oct 16, 2013 at 11:41:18AM +0100, Ian Campbell wrote:
>>> On Wed, 2013-10-16 at 18:26 +0800, Chen Baozi wrote:
>>>> On Wed, Oct 16, 2013 at 11:24:47AM +0100, Ian Campbell wrote:
>>>
>>>>> That suggests that the guest's SP doesn't have a mapping in the guest's
>>>>> PTs. What's the full trace?
>>>>
>>>> See http://pastebin.com/DYL5gkAQ
>>>
>>> Yeah, not a whole lot to go on is there ;-)
>>>
>>> Converting the PC (c02c1544) to a code line with gdb or addr2line would
>>> be a good start I think.
>>>
>>> You could also consider hacking up an earlyprintk via Xen by dropping a
>>> xen_raw_console_write into vprintk_emit once the buffer is formatted and
>>> see if that gets you any useful console output.
>>
>> The hack is workable! See http://pastebin.com/iSGJnKSg
>>
>> It looks like that dom0 has got some issues on "uart3", which is "skipped" due
>> to its disabled status in DT.
> 
> Julien,
> 
> Will the DT node of uart3 be populated to dom0 but not mapped?

If it's the UART used by Xen, the UART is removed from the device tree
and a small UART is emulated by Xen (search arch/arm/vuart.c).

For what I see, Linux is unable to map the memory region (so _mpu_rt_va
= NULL) but he is trying to use it later... It's definitely an error
from Linux side and not Xen.

-- 
Julien Grall

Re: About arm32 address translation

[Ian Campbell]

On Wed, 2013-10-16 at 13:54 +0100, David Vrabel wrote:
> On 16/10/13 11:41, Ian Campbell wrote:
> > 
> > (Somebody really needs to make a patch for proper Xen capable
> > earlyprintk for arm...)
> 
> The existing xenboot console should just work, yes?  It just needs
> plumbing into ARM's early console setup?

In principal yes, however 32-bit ARM's "earlyprintk" option takes no
parameters and simply enables a set of hardcoded per-platform UART
routines, so it needs a bit more thought...

64-bit arm kernel allows for the selection of which earlyprintk so it
should be reasonable easy. In fact, I think I wrote that patch
already...  Now where did I put it...

Ian.