Re: [ARM:PATCH v1 1/1] Add Odroid-XU (Exynos5410) support

[Suriyan Ramasami <suriyan.r@gmail.com>]

Hello Ian/Julien,
    I did some more debugging and found this weirdness. It has me a
little baffled.

I added xen_raw_printk() as suggested by Ian in earlier posts as a
mechanism for debugging domU (xen to be compiled with debug=y), and I
have been slapping myself silly as to why I didn't try this before! It
definitely gave me output that I could then hone in on!

The times that domU was not booting up, it crashed with a "divide by
zero" in the linux kernel because of the arch_timer_rate being 0. I
would see the warning "Architected timer frequency not available" in
the kernel log (xen output)

This is indeed baffling! Especially as I am setting CNTFRQ to the the
right value, and I know it is not 0!

The times that domU boots up, it does read the CNTFRQ value correctly!

This is the call trace that happens in drivers/clocksource/arm_arch_timer.c

It comes into the function, and finds arch_timer_rate as 0.
It tries to read clock_frequency property in the device tree. It is not set.
It checks if cntbase is set. It is not set.
It then calls arch_timer_get_cntfrq() to issue the mrc call.
I have printed out the return value of arch_timer_get_cntfrq() and
sometimes I get a 0 and sometimes I get 24000000.

arch_timer_get_cntfrq is nothing but:
static inline u32 arch_timer_get_cntfrq(void)
{
        u32 val;
        asm volatile("mrc p15, 0, %0, c14, c0, 0" : "=r" (val));
        return val;
}

Are the mrc calls trapped by XEN?

Any pointers are appreciated!
- Suriyan


On Mon, Jul 28, 2014 at 11:06 AM, Suriyan Ramasami <suriyan.r@gmail.com> wrote:
> Hello Julien,
>
> On Mon, Jul 28, 2014 at 5:53 AM, Julien Grall <julien.grall@linaro.org> wrote:
>> Hi Suriyan,
>>
>> On 07/27/2014 07:12 PM, Suriyan Ramasami wrote:
>>>>
>>>>> I was wondering if dom0 is touching the physical CPUs by mistake (mainly
>>>>> because we map some regions used to control them).
>>>>> It might be worse to check that no cpufreq or power management is called
>>>>> on DOM0 at any time.
>>>>
>>> I shall dig into this some more.
>>> Thanks!
>>
>> I would give a try with:
>>         - Xen: 2 CPUs
>>         - DOM0: 1 VCPU
>>         - Guest: 1 VCPU
>>
>> You can restrict the number of DOM0 VCPUs with dom0_max_vcpus=n where n
>> is the number of VCPUs for DOM0.
>>
>> If it's not failing then the error is in Linux.
>>
> I did the above suggestion and I tried bringing up domU atleast 30
> times. None of the attempts succeeded - domU gave no console output. I
> also tried with a XEN compile with exynos5_specific_mapping not being
> called for the Odroid-XU (don't know if this is of any relevance)
>
> Does this imply the problem is with Xen 4.5 dev branch?
>
> Relevant logs:
> ------------ 8< ------------
> XEN:
> ...
> (XEN) Command line: sync_console console=dtuart
> dtuart=/serial@12C20000 earlyprintk=xen dom0_mem=512M dom0_max_vcpus=1
> ...
> (XEN) Bringing up CPU1
> (XEN) exynos5.c:126: Power: 0 status: 0
> (XEN) exynos5.c:130: Waiting for power status to change to 3
> (XEN) exynos5.c:138: Power status changed to 3!
> (XEN) CPU 1 booted.
> (XEN) Brought up 2 CPUs
>
> DOM0:
> ...
> [    0.000000] Kernel command line: console=hvc0 earlyprintk debug
> clk_ignore_unused root=/dev/mmcblk0p2 rootwait rw
> ...
> [    0.038352] Xen: initializing cpu0
> [    0.038414] Setting up static identity map for 0x604940c8 - 0x60494114
> [    0.039487] Brought up 1 CPUs
> [    0.039496] SMP: Total of 1 processors activated.
> [    0.039503] CPU: All CPU(s) started in SVC mode.
>
> root@dom0:~/Debug# xl vcpu-list
> Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
> Domain-0                             0     0    0   r--     123.3  all
> domU2                                2     0    1   r--     670.9  all
>
> And domU2's sample function from xenctx 2 is as follows: (domU2 is
> linux mainline)
> root@dom0:~/Debug# ./domUtrace.sh 2
> 0xc0054c74 do_raw_spin_unlock
> 0xc00085d0 gic_handle_irq
> 0xc006c670 tick_periodic
> 0xc00085d0 gic_handle_irq
> 0xc006789c get_xtime_and_monotonic_and_sleep_offset
> 0xc00085d0 gic_handle_irq
> 0xc0054a90 do_raw_spin_lock
> 0xc0021934 irq_exit
> 0xc00085d0 gic_handle_irq
> 0xc0054c60 do_raw_spin_unlock
> 0xc00455e8 calc_global_load
> 0xc00085d0 gic_handle_irq
> 0xc006175c cpu_needs_another_gp
> 0xc00085d0 gic_handle_irq
> 0xc003b1b8 hrtimer_run_queues
> 0xc00085d0 gic_handle_irq
> ----------- 8< ----------------
>
> Thanks for all the pointers so far!
>
>> Regards,
>>
>> --
>> Julien Grall