Re: [Qemu-devel] Poking a sun4v machine

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> Tried to boot QEMU Niagara machine with the firmware from the
> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
> , and it dies very early.
> The reason: in translate.c
>
> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>
> and the dc->mem_idx is initialized like this:
>
>    if (env1->tl > 0) {
>        return MMU_NUCLEUS_IDX;
>    } else if (cpu_hypervisor_mode(env1)) {
>        return MMU_HYPV_IDX;
>    } else if (cpu_supervisor_mode(env1)) {
>        return MMU_KERNEL_IDX;
>    } else {
>        return MMU_USER_IDX;
>    }
>
> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
> still super- and hyper-visor bits are set, so both supervisor(dc) and
> hypervisor(dc) must return 1 which is impossible in the current
> implementation.

I don't think this is needed. The MMU index tells which TLB is used
for guest virtual to host address translations, during tl == MAXTL we
want to use hypervisor mode translations.

>
> What would be the proper way to fix it? Make mem_idx bitmap, add two
> more variables to DisasContext, or ...?
>
> Some other findings/questions:
>
>    /* Sun4v generic Niagara machine */
>    {
>        .default_cpu_model = "Sun UltraSparc T1",
>        .console_serial_base = 0xfff0c2c000ULL,
>
> Where is this address coming from? The OpenSPARC Niagara machine has a
> "dumb serial" at 0x1f10000000ULL.

I think I actually used Ontario machine definitions.

>
> And the biggest issue: UA2005 (as well as UA2007) describe a totally
> different format for a MMU TTE entry than the one sun4u CPU are using.
> I think the best way to handle it would be splitting off Niagara
> machine, and #defining MMU bits differently for sun4u and sun4v
> machines.
>
> Do we the cases in qemu where more than two (qemu-system-xxx and
> qemu-system-xxx64) binaries are produced?
> Would the name qemu-system-sun4v fit the naming convention?
>
> Artyom
>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Artyom Tarasenko]

On Tue, May 1, 2012 at 11:19 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>> Tried to boot QEMU Niagara machine with the firmware from the
>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>> , and it dies very early.
>> The reason: in translate.c
>>
>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>
>> and the dc->mem_idx is initialized like this:
>>
>>    if (env1->tl > 0) {
>>        return MMU_NUCLEUS_IDX;
>>    } else if (cpu_hypervisor_mode(env1)) {
>>        return MMU_HYPV_IDX;
>>    } else if (cpu_supervisor_mode(env1)) {
>>        return MMU_KERNEL_IDX;
>>    } else {
>>        return MMU_USER_IDX;
>>    }
>>
>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>> hypervisor(dc) must return 1 which is impossible in the current
>> implementation.
>
> I don't think this is needed. The MMU index tells which TLB is used
> for guest virtual to host address translations, during tl == MAXTL we
> want to use hypervisor mode translations.

Is just about the address translations? I thought it affects TB cache
as well, which has to be invalidated between nucleus, super- and
hypervisor switches?

>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>> more variables to DisasContext, or ...?
>>

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Tue, May 1, 2012 at 13:33, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> On Tue, May 1, 2012 at 11:19 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> Tried to boot QEMU Niagara machine with the firmware from the
>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>> , and it dies very early.
>>> The reason: in translate.c
>>>
>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>
>>> and the dc->mem_idx is initialized like this:
>>>
>>>    if (env1->tl > 0) {
>>>        return MMU_NUCLEUS_IDX;
>>>    } else if (cpu_hypervisor_mode(env1)) {
>>>        return MMU_HYPV_IDX;
>>>    } else if (cpu_supervisor_mode(env1)) {
>>>        return MMU_KERNEL_IDX;
>>>    } else {
>>>        return MMU_USER_IDX;
>>>    }
>>>
>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>> hypervisor(dc) must return 1 which is impossible in the current
>>> implementation.
>>
>> I don't think this is needed. The MMU index tells which TLB is used
>> for guest virtual to host address translations, during tl == MAXTL we
>> want to use hypervisor mode translations.
>
> Is just about the address translations? I thought it affects TB cache
> as well, which has to be invalidated between nucleus, super- and
> hypervisor switches?

This is (should be) handled by cpu_get_tb_cpu_state(). But it looks
like only supervisor mode (PS_PRIV) is taken into account, not others.

Also when D/I MMU is enabled or disabled, TLB is flushed, so there's
no need to store these bits.

I'm not sure TL is interesting either.

>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>> more variables to DisasContext, or ...?
>>>

Re: [Qemu-devel] Poking a sun4v machine

[Artyom Tarasenko]

On Tue, May 1, 2012 at 3:54 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
> On Tue, May 1, 2012 at 13:33, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>> On Tue, May 1, 2012 at 11:19 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>> , and it dies very early.
>>>> The reason: in translate.c
>>>>
>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>
>>>> and the dc->mem_idx is initialized like this:
>>>>
>>>>    if (env1->tl > 0) {
>>>>        return MMU_NUCLEUS_IDX;
>>>>    } else if (cpu_hypervisor_mode(env1)) {
>>>>        return MMU_HYPV_IDX;
>>>>    } else if (cpu_supervisor_mode(env1)) {
>>>>        return MMU_KERNEL_IDX;
>>>>    } else {
>>>>        return MMU_USER_IDX;
>>>>    }
>>>>
>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>> implementation.
>>>
>>> I don't think this is needed. The MMU index tells which TLB is used
>>> for guest virtual to host address translations, during tl == MAXTL we
>>> want to use hypervisor mode translations.
>>
>> Is just about the address translations? I thought it affects TB cache
>> as well, which has to be invalidated between nucleus, super- and
>> hypervisor switches?
>
> This is (should be) handled by cpu_get_tb_cpu_state(). But it looks
> like only supervisor mode (PS_PRIV) is taken into account, not others.
>
> Also when D/I MMU is enabled or disabled, TLB is flushed, so there's
> no need to store these bits.
>
> I'm not sure TL is interesting either.

And what about the MMU contexts? UA2005 does have 3 ones: Primary,
Secondary and Nucleus.
Is it not relevant here?

-- 
Regards,
Artyom Tarasenko

solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Wed, May 2, 2012 at 2:50 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> On Tue, May 1, 2012 at 3:54 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>> On Tue, May 1, 2012 at 13:33, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> On Tue, May 1, 2012 at 11:19 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>> On Mon, Apr 30, 2012 at 16:39, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>> , and it dies very early.
>>>>> The reason: in translate.c
>>>>>
>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>
>>>>> and the dc->mem_idx is initialized like this:
>>>>>
>>>>>    if (env1->tl > 0) {
>>>>>        return MMU_NUCLEUS_IDX;
>>>>>    } else if (cpu_hypervisor_mode(env1)) {
>>>>>        return MMU_HYPV_IDX;
>>>>>    } else if (cpu_supervisor_mode(env1)) {
>>>>>        return MMU_KERNEL_IDX;
>>>>>    } else {
>>>>>        return MMU_USER_IDX;
>>>>>    }
>>>>>
>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>> implementation.
>>>>
>>>> I don't think this is needed. The MMU index tells which TLB is used
>>>> for guest virtual to host address translations, during tl == MAXTL we
>>>> want to use hypervisor mode translations.
>>>
>>> Is just about the address translations? I thought it affects TB cache
>>> as well, which has to be invalidated between nucleus, super- and
>>> hypervisor switches?
>>
>> This is (should be) handled by cpu_get_tb_cpu_state(). But it looks
>> like only supervisor mode (PS_PRIV) is taken into account, not others.
>>
>> Also when D/I MMU is enabled or disabled, TLB is flushed, so there's
>> no need to store these bits.
>>
>> I'm not sure TL is interesting either.
>
> And what about the MMU contexts? UA2005 does have 3 ones: Primary,
> Secondary and Nucleus.
> Is it not relevant here?

We also flush the TLB when the context registers are changed. This is
not optimal, we'd want to add context numbers to TB flags but there's
not enough space. New flags or context numbers should be added, they
would be useful for other architectures too. Probably only primary
context is interesting for Sparc guests, flushes for secondary and
nucleus could be OK. Sparc32 MMU also has context numbers which is
handled by flushing ATM.

>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Andreas Färber]

Am 30.04.2012 19:38, schrieb Artyom Tarasenko:
> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>> Tried to boot QEMU Niagara machine with the firmware from the
>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>> , and it dies very early.
>>> The reason: in translate.c
>>>
>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>
>>> and the dc->mem_idx is initialized like this:
>>>
>>>     if (env1->tl > 0) {
>>>         return MMU_NUCLEUS_IDX;
>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>         return MMU_HYPV_IDX;
>>>     } else if (cpu_supervisor_mode(env1)) {
>>>         return MMU_KERNEL_IDX;
>>>     } else {
>>>         return MMU_USER_IDX;
>>>     }
>>>
>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>> hypervisor(dc) must return 1 which is impossible in the current
>>> implementation.
>>>
>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>> more variables to DisasContext, or ...?
>>>
>>> Some other findings/questions:
>>>
>>>     /* Sun4v generic Niagara machine */
>>>     {
>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>
>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>> "dumb serial" at 0x1f10000000ULL.
>>>
>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>> I think the best way to handle it would be splitting off Niagara
>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>> machines.
>>>
>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>> qemu-system-xxx64) binaries are produced?
>>> Would the name qemu-system-sun4v fit the naming convention?
>>
>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>> Better avoid it for sparc in the first place.
>>
>> Instead, you should add a callback function pointer to SPARCCPUClass
>> that you initialize based on CPU model so that is behaves differently at
>> runtime rather than at compile time.
>> Or if it's just about the class_init then after the Hard Freeze I can
>> start polishing my subclasses for sparc so that you can add a special
>> class_init for Niagara.
> 
> But this would mean that the defines from
> #define TTE_NFO_BIT (1ULL << 60)
> to
> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
> 
> inclusive would need to be replaced with functions and variables?
> Sounds like a further performance regression for sun4u?

First of all, I wouldn't use the term performance regression for a
target such as sparc64 that has never really worked before. ;-)

I don't know the details of the SPARC implementation or architecture.
What I am telling you is that there are ways to implement sun4u and
sun4v inside one executable; whether you use a new function or extend a
macro with an additional parameter is up to you and Blue. My guess is
that unlike ARM you won't need to mix different cores at runtime for the
time being.

In ARM we have a macro IS_M() that distinguishes between the Cortex-M
and Cortex-A families rather than moving Cortex-M to its own executable.
The Freescale Vybrid MCUs are going to combine the two in one SoC, i.e.
in one qemu-system-arm executable simultaneously.

> And would it be possible to have a different register set for an
> inherited SPARCCPUClass ?

You can add new state fields to an inherited SPARCCPU.

If you need them in TCG as offsets from env, then CPUSPARCState might be
an easier place to add them but there you cannot distinguish between the
models.

To an inherited SPARCCPUClass you can add fields for saving constant,
e.g., reset values for registers of the model.

The unanswered question is whether SPARC should follow the x86 model
with declarative (and user-specified) definitions (in that case
extending SPARCCPUClass) or the ARM model with imperative per-model init
functions (in that case probably not extending SPARCCPU at all).

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Poking a sun4v machine

[Artyom Tarasenko]

On Tue, May 1, 2012 at 10:35 AM, Andreas Färber <afaerber@suse.de> wrote:
> Am 30.04.2012 19:38, schrieb Artyom Tarasenko:
>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>> , and it dies very early.
>>>> The reason: in translate.c
>>>>
>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>
>>>> and the dc->mem_idx is initialized like this:
>>>>
>>>>     if (env1->tl > 0) {
>>>>         return MMU_NUCLEUS_IDX;
>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>         return MMU_HYPV_IDX;
>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>         return MMU_KERNEL_IDX;
>>>>     } else {
>>>>         return MMU_USER_IDX;
>>>>     }
>>>>
>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>> implementation.
>>>>
>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>> more variables to DisasContext, or ...?
>>>>
>>>> Some other findings/questions:
>>>>
>>>>     /* Sun4v generic Niagara machine */
>>>>     {
>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>
>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>> "dumb serial" at 0x1f10000000ULL.
>>>>
>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>> I think the best way to handle it would be splitting off Niagara
>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>> machines.
>>>>
>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>> qemu-system-xxx64) binaries are produced?
>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>
>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>> Better avoid it for sparc in the first place.
>>>
>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>> that you initialize based on CPU model so that is behaves differently at
>>> runtime rather than at compile time.
>>> Or if it's just about the class_init then after the Hard Freeze I can
>>> start polishing my subclasses for sparc so that you can add a special
>>> class_init for Niagara.
>>
>> But this would mean that the defines from
>> #define TTE_NFO_BIT (1ULL << 60)
>> to
>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>
>> inclusive would need to be replaced with functions and variables?
>> Sounds like a further performance regression for sun4u?
>
> First of all, I wouldn't use the term performance regression for a
> target such as sparc64 that has never really worked before. ;-)

Well, it works good enough for me. ;-) Also until the recent PCI
changes it was able to boot HelenOS out of the box.

> I don't know the details of the SPARC implementation or architecture.
> What I am telling you is that there are ways to implement sun4u and
> sun4v inside one executable; whether you use a new function or extend a
> macro with an additional parameter is up to you and Blue. My guess is
> that unlike ARM you won't need to mix different cores at runtime for the
> time being.
>
> In ARM we have a macro IS_M() that distinguishes between the Cortex-M
> and Cortex-A families rather than moving Cortex-M to its own executable.
> The Freescale Vybrid MCUs are going to combine the two in one SoC, i.e.
> in one qemu-system-arm executable simultaneously.
>
>> And would it be possible to have a different register set for an
>> inherited SPARCCPUClass ?
>
> You can add new state fields to an inherited SPARCCPU.

Can the inheritance hierarchy have any depth? Then we could think of
having SPARC->(SPARC32,SPARC64)
and SPARC64 -> (sun4u, sun4v, Fujitsu). But I still don't see the
advantages of having it configurable at runtime.

> If you need them in TCG as offsets from env, then CPUSPARCState might be
> an easier place to add them but there you cannot distinguish between the
> models.
>
> To an inherited SPARCCPUClass you can add fields for saving constant,
> e.g., reset values for registers of the model.
>
> The unanswered question is whether SPARC should follow the x86 model
> with declarative (and user-specified) definitions (in that case
> extending SPARCCPUClass) or the ARM model with imperative per-model init
> functions (in that case probably not extending SPARCCPU at all).
>
> Andreas
>
> --
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg



-- 
Regards,
Artyom Tarasenko

solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>> Tried to boot QEMU Niagara machine with the firmware from the
>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>> , and it dies very early.
>>> The reason: in translate.c
>>>
>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>
>>> and the dc->mem_idx is initialized like this:
>>>
>>>     if (env1->tl > 0) {
>>>         return MMU_NUCLEUS_IDX;
>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>         return MMU_HYPV_IDX;
>>>     } else if (cpu_supervisor_mode(env1)) {
>>>         return MMU_KERNEL_IDX;
>>>     } else {
>>>         return MMU_USER_IDX;
>>>     }
>>>
>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>> hypervisor(dc) must return 1 which is impossible in the current
>>> implementation.
>>>
>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>> more variables to DisasContext, or ...?
>>>
>>> Some other findings/questions:
>>>
>>>     /* Sun4v generic Niagara machine */
>>>     {
>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>
>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>> "dumb serial" at 0x1f10000000ULL.
>>>
>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>> I think the best way to handle it would be splitting off Niagara
>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>> machines.
>>>
>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>> qemu-system-xxx64) binaries are produced?
>>> Would the name qemu-system-sun4v fit the naming convention?
>>
>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>> Better avoid it for sparc in the first place.
>>
>> Instead, you should add a callback function pointer to SPARCCPUClass
>> that you initialize based on CPU model so that is behaves differently at
>> runtime rather than at compile time.
>> Or if it's just about the class_init then after the Hard Freeze I can
>> start polishing my subclasses for sparc so that you can add a special
>> class_init for Niagara.
>
> But this would mean that the defines from
> #define TTE_NFO_BIT (1ULL << 60)
> to
> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>
> inclusive would need to be replaced with functions and variables?
> Sounds like a further performance regression for sun4u?

There could be parallel definitions for sun4u (actually UltraSparc-III
onwards the MMU is again different) and sun4v.

At tlb_fill(), different implementations can be selected based on MMU
model. For ASI accesses, we can add conditional code but for higher
performance, some checks can be moved to translation time.

>
> And would it be possible to have a different register set for an
> inherited SPARCCPUClass ?
> Also trap handling and related cpu registers behavior has to be quite different.

Not really, we already handle some (but not all cases).

>
> Artyom
>
>> Helpers such as cpu_hypervisor_mode() will need to be changed to take a
>> SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim
>> solution sparc_env_get_cpu() can be used. (examples on the list for sh4)
>>
>> Andreas
>>
>> --
>> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
>
>
>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Artyom Tarasenko]

On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>> , and it dies very early.
>>>> The reason: in translate.c
>>>>
>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>
>>>> and the dc->mem_idx is initialized like this:
>>>>
>>>>     if (env1->tl > 0) {
>>>>         return MMU_NUCLEUS_IDX;
>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>         return MMU_HYPV_IDX;
>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>         return MMU_KERNEL_IDX;
>>>>     } else {
>>>>         return MMU_USER_IDX;
>>>>     }
>>>>
>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>> implementation.
>>>>
>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>> more variables to DisasContext, or ...?
>>>>
>>>> Some other findings/questions:
>>>>
>>>>     /* Sun4v generic Niagara machine */
>>>>     {
>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>
>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>> "dumb serial" at 0x1f10000000ULL.
>>>>
>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>> I think the best way to handle it would be splitting off Niagara
>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>> machines.
>>>>
>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>> qemu-system-xxx64) binaries are produced?
>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>
>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>> Better avoid it for sparc in the first place.
>>>
>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>> that you initialize based on CPU model so that is behaves differently at
>>> runtime rather than at compile time.
>>> Or if it's just about the class_init then after the Hard Freeze I can
>>> start polishing my subclasses for sparc so that you can add a special
>>> class_init for Niagara.
>>
>> But this would mean that the defines from
>> #define TTE_NFO_BIT (1ULL << 60)
>> to
>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>
>> inclusive would need to be replaced with functions and variables?
>> Sounds like a further performance regression for sun4u?
>
> There could be parallel definitions for sun4u (actually UltraSparc-III
> onwards the MMU is again different) and sun4v.
>
> At tlb_fill(), different implementations can be selected based on MMU
> model. For ASI accesses, we can add conditional code but for higher
> performance, some checks can be moved to translation time.

Can be done, but what is the gain of having it runtime configurable?

>> And would it be possible to have a different register set for an
>> inherited SPARCCPUClass ?
>> Also trap handling and related cpu registers behavior has to be quite different.
>
> Not really, we already handle some (but not all cases).

You mean by not handling AG? The UA2005 doesn't have IG and MG either,
but that's the easy part. Instead it has a few register banks
switchable with the GL register pointer.
cpu_change_pstate should probably have another parameter (new_GL)
which is only valid for sun4v.
And, depending on a trap type, env->htba has to be taken instead of
env->tbr. To me it looks like at the end do_interrupt will have less
common parts between sun4u and sun4v than specific ones.

>>> Helpers such as cpu_hypervisor_mode() will need to be changed to take a
>>> SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim
>>> solution sparc_env_get_cpu() can be used. (examples on the list for sh4)
>>>
>>> Andreas
>>>
>>> --
>>> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>>> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
>>

-- 
Regards,
Artyom Tarasenko

solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>> , and it dies very early.
>>>>> The reason: in translate.c
>>>>>
>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>
>>>>> and the dc->mem_idx is initialized like this:
>>>>>
>>>>>     if (env1->tl > 0) {
>>>>>         return MMU_NUCLEUS_IDX;
>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>         return MMU_HYPV_IDX;
>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>         return MMU_KERNEL_IDX;
>>>>>     } else {
>>>>>         return MMU_USER_IDX;
>>>>>     }
>>>>>
>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>> implementation.
>>>>>
>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>> more variables to DisasContext, or ...?
>>>>>
>>>>> Some other findings/questions:
>>>>>
>>>>>     /* Sun4v generic Niagara machine */
>>>>>     {
>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>
>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>
>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>> I think the best way to handle it would be splitting off Niagara
>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>> machines.
>>>>>
>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>> qemu-system-xxx64) binaries are produced?
>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>
>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>> Better avoid it for sparc in the first place.
>>>>
>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>> that you initialize based on CPU model so that is behaves differently at
>>>> runtime rather than at compile time.
>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>> start polishing my subclasses for sparc so that you can add a special
>>>> class_init for Niagara.
>>>
>>> But this would mean that the defines from
>>> #define TTE_NFO_BIT (1ULL << 60)
>>> to
>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>
>>> inclusive would need to be replaced with functions and variables?
>>> Sounds like a further performance regression for sun4u?
>>
>> There could be parallel definitions for sun4u (actually UltraSparc-III
>> onwards the MMU is again different) and sun4v.
>>
>> At tlb_fill(), different implementations can be selected based on MMU
>> model. For ASI accesses, we can add conditional code but for higher
>> performance, some checks can be moved to translation time.
>
> Can be done, but what is the gain of having it runtime configurable?

I was thinking of code like this in:

switch (env->mmu_model) {
case MMU_US2:
   return tlb_fill_us2(..);
case MMU_US3:
   return tlb_fill_us3(..);
case MMU_US4:
   return tlb_fill_us4(..);
case MMU_T1:
   return tlb_fill_t1(..);
case MMU_T2:
   return tlb_fill_t2(..);
}

The perfomance cost shouldn't be too high. Alternatively a function
pointer could be set up.

>
>>> And would it be possible to have a different register set for an
>>> inherited SPARCCPUClass ?
>>> Also trap handling and related cpu registers behavior has to be quite different.
>>
>> Not really, we already handle some (but not all cases).
>
> You mean by not handling AG? The UA2005 doesn't have IG and MG either,
> but that's the easy part. Instead it has a few register banks
> switchable with the GL register pointer.

Yes, we can always provide the register bank, older models just access
some of those.

> cpu_change_pstate should probably have another parameter (new_GL)
> which is only valid for sun4v.
> And, depending on a trap type, env->htba has to be taken instead of
> env->tbr. To me it looks like at the end do_interrupt will have less
> common parts between sun4u and sun4v than specific ones.

Same as tlb_fill(), switch() or function pointer. The functions are different.

This is unavoidable (unless maybe in the future the TLB handling can
be pushed partially higher so mmu_idx parameters can be eliminated)
and the performance cost is not great.

>
>>>> Helpers such as cpu_hypervisor_mode() will need to be changed to take a
>>>> SPARCCPU *cpu rather than CPUSPARCState *env argument; as an interim
>>>> solution sparc_env_get_cpu() can be used. (examples on the list for sh4)
>>>>
>>>> Andreas
>>>>
>>>> --
>>>> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>>>> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
>>>
>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Artyom Tarasenko]

On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>>> , and it dies very early.
>>>>>> The reason: in translate.c
>>>>>>
>>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>>
>>>>>> and the dc->mem_idx is initialized like this:
>>>>>>
>>>>>>     if (env1->tl > 0) {
>>>>>>         return MMU_NUCLEUS_IDX;
>>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>>         return MMU_HYPV_IDX;
>>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>>         return MMU_KERNEL_IDX;
>>>>>>     } else {
>>>>>>         return MMU_USER_IDX;
>>>>>>     }
>>>>>>
>>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>>> implementation.
>>>>>>
>>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>>> more variables to DisasContext, or ...?
>>>>>>
>>>>>> Some other findings/questions:
>>>>>>
>>>>>>     /* Sun4v generic Niagara machine */
>>>>>>     {
>>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>>
>>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>>
>>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>>> I think the best way to handle it would be splitting off Niagara
>>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>>> machines.
>>>>>>
>>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>>> qemu-system-xxx64) binaries are produced?
>>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>>
>>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>>> Better avoid it for sparc in the first place.
>>>>>
>>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>>> that you initialize based on CPU model so that is behaves differently at
>>>>> runtime rather than at compile time.
>>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>>> start polishing my subclasses for sparc so that you can add a special
>>>>> class_init for Niagara.
>>>>
>>>> But this would mean that the defines from
>>>> #define TTE_NFO_BIT (1ULL << 60)
>>>> to
>>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>>
>>>> inclusive would need to be replaced with functions and variables?
>>>> Sounds like a further performance regression for sun4u?
>>>
>>> There could be parallel definitions for sun4u (actually UltraSparc-III
>>> onwards the MMU is again different) and sun4v.
>>>
>>> At tlb_fill(), different implementations can be selected based on MMU
>>> model. For ASI accesses, we can add conditional code but for higher
>>> performance, some checks can be moved to translation time.
>>
>> Can be done, but what is the gain of having it runtime configurable?
>
> I was thinking of code like this in:
>
> switch (env->mmu_model) {
> case MMU_US2:
>   return tlb_fill_us2(..);
> case MMU_US3:
>   return tlb_fill_us3(..);
> case MMU_US4:
>   return tlb_fill_us4(..);
> case MMU_T1:
>   return tlb_fill_t1(..);
> case MMU_T2:
>   return tlb_fill_t2(..);
> }
>
> The perfomance cost shouldn't be too high. Alternatively a function
> pointer could be set up.

Actually I was more worried about get_physical_address_* than filling,
there we would have to use variables instead of constants and
functions instead of macros.

> Yes, we can always provide the register bank, older models just access
> some of those.
>
>> cpu_change_pstate should probably have another parameter (new_GL)
>> which is only valid for sun4v.
>> And, depending on a trap type, env->htba has to be taken instead of
>> env->tbr. To me it looks like at the end do_interrupt will have less
>> common parts between sun4u and sun4v than specific ones.
>
> Same as tlb_fill(), switch() or function pointer. The functions are different.
>
> This is unavoidable (unless maybe in the future the TLB handling can
> be pushed partially higher so mmu_idx parameters can be eliminated)
> and the performance cost is not great.

So, altogether you'd still prefer run-time checks over having
qemu-system-sun4v (or -sparc64v) ?

-- 
Regards,
Artyom Tarasenko

solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>>>> , and it dies very early.
>>>>>>> The reason: in translate.c
>>>>>>>
>>>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>>>
>>>>>>> and the dc->mem_idx is initialized like this:
>>>>>>>
>>>>>>>     if (env1->tl > 0) {
>>>>>>>         return MMU_NUCLEUS_IDX;
>>>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>>>         return MMU_HYPV_IDX;
>>>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>>>         return MMU_KERNEL_IDX;
>>>>>>>     } else {
>>>>>>>         return MMU_USER_IDX;
>>>>>>>     }
>>>>>>>
>>>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>>>> implementation.
>>>>>>>
>>>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>>>> more variables to DisasContext, or ...?
>>>>>>>
>>>>>>> Some other findings/questions:
>>>>>>>
>>>>>>>     /* Sun4v generic Niagara machine */
>>>>>>>     {
>>>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>>>
>>>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>>>
>>>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>>>> I think the best way to handle it would be splitting off Niagara
>>>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>>>> machines.
>>>>>>>
>>>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>>>> qemu-system-xxx64) binaries are produced?
>>>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>>>
>>>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>>>> Better avoid it for sparc in the first place.
>>>>>>
>>>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>>>> that you initialize based on CPU model so that is behaves differently at
>>>>>> runtime rather than at compile time.
>>>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>>>> start polishing my subclasses for sparc so that you can add a special
>>>>>> class_init for Niagara.
>>>>>
>>>>> But this would mean that the defines from
>>>>> #define TTE_NFO_BIT (1ULL << 60)
>>>>> to
>>>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>>>
>>>>> inclusive would need to be replaced with functions and variables?
>>>>> Sounds like a further performance regression for sun4u?
>>>>
>>>> There could be parallel definitions for sun4u (actually UltraSparc-III
>>>> onwards the MMU is again different) and sun4v.
>>>>
>>>> At tlb_fill(), different implementations can be selected based on MMU
>>>> model. For ASI accesses, we can add conditional code but for higher
>>>> performance, some checks can be moved to translation time.
>>>
>>> Can be done, but what is the gain of having it runtime configurable?
>>
>> I was thinking of code like this in:
>>
>> switch (env->mmu_model) {
>> case MMU_US2:
>>   return tlb_fill_us2(..);
>> case MMU_US3:
>>   return tlb_fill_us3(..);
>> case MMU_US4:
>>   return tlb_fill_us4(..);
>> case MMU_T1:
>>   return tlb_fill_t1(..);
>> case MMU_T2:
>>   return tlb_fill_t2(..);
>> }
>>
>> The perfomance cost shouldn't be too high. Alternatively a function
>> pointer could be set up.
>
> Actually I was more worried about get_physical_address_* than filling,
> there we would have to use variables instead of constants and
> functions instead of macros.

Preferably entirely different functions with constants.

>
>> Yes, we can always provide the register bank, older models just access
>> some of those.
>>
>>> cpu_change_pstate should probably have another parameter (new_GL)
>>> which is only valid for sun4v.
>>> And, depending on a trap type, env->htba has to be taken instead of
>>> env->tbr. To me it looks like at the end do_interrupt will have less
>>> common parts between sun4u and sun4v than specific ones.
>>
>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>
>> This is unavoidable (unless maybe in the future the TLB handling can
>> be pushed partially higher so mmu_idx parameters can be eliminated)
>> and the performance cost is not great.
>
> So, altogether you'd still prefer run-time checks over having
> qemu-system-sun4v (or -sparc64v) ?

Yes. Architectures are not meant to handle small issues like this.
Should performance become a problem, there are a plenty of lower
hanging fruits where to start optimizing.

Even in this case, rather than the new architecture solution, it could
be possible to build separate TLB handlers which call directly the
correct MMU functions without switches and these would be selected at
translation time or earlier. For the PPCEMB case, maybe the memory API
could be changed to handle different page sizes without loss of
performance, I don't know. Devices should not depend on
TARGET_PAGE_SIZE.

>
> --
> Regards,
> Artyom Tarasenko
>
> solaris/sparc under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Poking a sun4v machine

[Alexander Graf]

Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:

> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>>>>> , and it dies very early.
>>>>>>>> The reason: in translate.c
>>>>>>>> 
>>>>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>>>> 
>>>>>>>> and the dc->mem_idx is initialized like this:
>>>>>>>> 
>>>>>>>>     if (env1->tl > 0) {
>>>>>>>>         return MMU_NUCLEUS_IDX;
>>>>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>>>>         return MMU_HYPV_IDX;
>>>>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>>>>         return MMU_KERNEL_IDX;
>>>>>>>>     } else {
>>>>>>>>         return MMU_USER_IDX;
>>>>>>>>     }
>>>>>>>> 
>>>>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>>>>> implementation.
>>>>>>>> 
>>>>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>>>>> more variables to DisasContext, or ...?
>>>>>>>> 
>>>>>>>> Some other findings/questions:
>>>>>>>> 
>>>>>>>>     /* Sun4v generic Niagara machine */
>>>>>>>>     {
>>>>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>>>> 
>>>>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>>>> 
>>>>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>>>>> I think the best way to handle it would be splitting off Niagara
>>>>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>>>>> machines.
>>>>>>>> 
>>>>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>>>>> qemu-system-xxx64) binaries are produced?
>>>>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>>>> 
>>>>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>>>>> Better avoid it for sparc in the first place.
>>>>>>> 
>>>>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>>>>> that you initialize based on CPU model so that is behaves differently at
>>>>>>> runtime rather than at compile time.
>>>>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>>>>> start polishing my subclasses for sparc so that you can add a special
>>>>>>> class_init for Niagara.
>>>>>> 
>>>>>> But this would mean that the defines from
>>>>>> #define TTE_NFO_BIT (1ULL << 60)
>>>>>> to
>>>>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>>>> 
>>>>>> inclusive would need to be replaced with functions and variables?
>>>>>> Sounds like a further performance regression for sun4u?
>>>>> 
>>>>> There could be parallel definitions for sun4u (actually UltraSparc-III
>>>>> onwards the MMU is again different) and sun4v.
>>>>> 
>>>>> At tlb_fill(), different implementations can be selected based on MMU
>>>>> model. For ASI accesses, we can add conditional code but for higher
>>>>> performance, some checks can be moved to translation time.
>>>> 
>>>> Can be done, but what is the gain of having it runtime configurable?
>>> 
>>> I was thinking of code like this in:
>>> 
>>> switch (env->mmu_model) {
>>> case MMU_US2:
>>>   return tlb_fill_us2(..);
>>> case MMU_US3:
>>>   return tlb_fill_us3(..);
>>> case MMU_US4:
>>>   return tlb_fill_us4(..);
>>> case MMU_T1:
>>>   return tlb_fill_t1(..);
>>> case MMU_T2:
>>>   return tlb_fill_t2(..);
>>> }
>>> 
>>> The perfomance cost shouldn't be too high. Alternatively a function
>>> pointer could be set up.
>> 
>> Actually I was more worried about get_physical_address_* than filling,
>> there we would have to use variables instead of constants and
>> functions instead of macros.
> 
> Preferably entirely different functions with constants.
> 
>> 
>>> Yes, we can always provide the register bank, older models just access
>>> some of those.
>>> 
>>>> cpu_change_pstate should probably have another parameter (new_GL)
>>>> which is only valid for sun4v.
>>>> And, depending on a trap type, env->htba has to be taken instead of
>>>> env->tbr. To me it looks like at the end do_interrupt will have less
>>>> common parts between sun4u and sun4v than specific ones.
>>> 
>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>> 
>>> This is unavoidable (unless maybe in the future the TLB handling can
>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>> and the performance cost is not great.
>> 
>> So, altogether you'd still prefer run-time checks over having
>> qemu-system-sun4v (or -sparc64v) ?
> 
> Yes. Architectures are not meant to handle small issues like this.
> Should performance become a problem, there are a plenty of lower
> hanging fruits where to start optimizing.
> 
> Even in this case, rather than the new architecture solution, it could
> be possible to build separate TLB handlers which call directly the
> correct MMU functions without switches and these would be selected at
> translation time or earlier. For the PPCEMB case, maybe the memory API
> could be changed to handle different page sizes without loss of
> performance, I don't know. Devices should not depend on
> TARGET_PAGE_SIZE.

It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.

That'd hurt performance quite a bit. The softmmu already is a very big chunk of execution time on ppc and zi really don't want that number to go up.


Alex

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Sun, May 6, 2012 at 8:58 AM, Alexander Graf <agraf@suse.de> wrote:
>
>
> Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:
>
>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>>>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>>>>>> , and it dies very early.
>>>>>>>>> The reason: in translate.c
>>>>>>>>>
>>>>>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>>>>>
>>>>>>>>> and the dc->mem_idx is initialized like this:
>>>>>>>>>
>>>>>>>>>     if (env1->tl > 0) {
>>>>>>>>>         return MMU_NUCLEUS_IDX;
>>>>>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>>>>>         return MMU_HYPV_IDX;
>>>>>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>>>>>         return MMU_KERNEL_IDX;
>>>>>>>>>     } else {
>>>>>>>>>         return MMU_USER_IDX;
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>>>>>> implementation.
>>>>>>>>>
>>>>>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>>>>>> more variables to DisasContext, or ...?
>>>>>>>>>
>>>>>>>>> Some other findings/questions:
>>>>>>>>>
>>>>>>>>>     /* Sun4v generic Niagara machine */
>>>>>>>>>     {
>>>>>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>>>>>
>>>>>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>>>>>
>>>>>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>>>>>> I think the best way to handle it would be splitting off Niagara
>>>>>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>>>>>> machines.
>>>>>>>>>
>>>>>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>>>>>> qemu-system-xxx64) binaries are produced?
>>>>>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>>>>>
>>>>>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>>>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>>>>>> Better avoid it for sparc in the first place.
>>>>>>>>
>>>>>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>>>>>> that you initialize based on CPU model so that is behaves differently at
>>>>>>>> runtime rather than at compile time.
>>>>>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>>>>>> start polishing my subclasses for sparc so that you can add a special
>>>>>>>> class_init for Niagara.
>>>>>>>
>>>>>>> But this would mean that the defines from
>>>>>>> #define TTE_NFO_BIT (1ULL << 60)
>>>>>>> to
>>>>>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>>>>>
>>>>>>> inclusive would need to be replaced with functions and variables?
>>>>>>> Sounds like a further performance regression for sun4u?
>>>>>>
>>>>>> There could be parallel definitions for sun4u (actually UltraSparc-III
>>>>>> onwards the MMU is again different) and sun4v.
>>>>>>
>>>>>> At tlb_fill(), different implementations can be selected based on MMU
>>>>>> model. For ASI accesses, we can add conditional code but for higher
>>>>>> performance, some checks can be moved to translation time.
>>>>>
>>>>> Can be done, but what is the gain of having it runtime configurable?
>>>>
>>>> I was thinking of code like this in:
>>>>
>>>> switch (env->mmu_model) {
>>>> case MMU_US2:
>>>>   return tlb_fill_us2(..);
>>>> case MMU_US3:
>>>>   return tlb_fill_us3(..);
>>>> case MMU_US4:
>>>>   return tlb_fill_us4(..);
>>>> case MMU_T1:
>>>>   return tlb_fill_t1(..);
>>>> case MMU_T2:
>>>>   return tlb_fill_t2(..);
>>>> }
>>>>
>>>> The perfomance cost shouldn't be too high. Alternatively a function
>>>> pointer could be set up.
>>>
>>> Actually I was more worried about get_physical_address_* than filling,
>>> there we would have to use variables instead of constants and
>>> functions instead of macros.
>>
>> Preferably entirely different functions with constants.
>>
>>>
>>>> Yes, we can always provide the register bank, older models just access
>>>> some of those.
>>>>
>>>>> cpu_change_pstate should probably have another parameter (new_GL)
>>>>> which is only valid for sun4v.
>>>>> And, depending on a trap type, env->htba has to be taken instead of
>>>>> env->tbr. To me it looks like at the end do_interrupt will have less
>>>>> common parts between sun4u and sun4v than specific ones.
>>>>
>>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>>>
>>>> This is unavoidable (unless maybe in the future the TLB handling can
>>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>>> and the performance cost is not great.
>>>
>>> So, altogether you'd still prefer run-time checks over having
>>> qemu-system-sun4v (or -sparc64v) ?
>>
>> Yes. Architectures are not meant to handle small issues like this.
>> Should performance become a problem, there are a plenty of lower
>> hanging fruits where to start optimizing.
>>
>> Even in this case, rather than the new architecture solution, it could
>> be possible to build separate TLB handlers which call directly the
>> correct MMU functions without switches and these would be selected at
>> translation time or earlier. For the PPCEMB case, maybe the memory API
>> could be changed to handle different page sizes without loss of
>> performance, I don't know. Devices should not depend on
>> TARGET_PAGE_SIZE.
>
> It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.
>
> That'd hurt performance quite a bit. The softmmu already is a very big chunk of execution time on ppc and zi really don't want that number to go up.

Yes, but that's not what I proposed. Now the translator arranges a
call to for example qemu_ld32u which uses this fixed TLB size.
Instead, it should generate calls to qemu_ld32u_ppcemb (which should
use 1k pages) or qemu_ld32u_ppc (4k?) as needed, maybe also combining
with MMU_IDX they would expand to qemu_ld32u_ppc_hypv etc. Obviously
this would need big changes everywhere and MMU_IDX change should be
compared with the negative cache effects of having many separate,
often used functions in the hot path.

Memory API (or actually the related pieces in exec.c) hardcodes
TARGET_PAGE_SIZE assumptions in many places, but it might be possible
to make this dynamic.

>
>
> Alex
>

Re: [Qemu-devel] Poking a sun4v machine

[Alexander Graf]

Am 06.05.2012 um 11:13 schrieb Blue Swirl <blauwirbel@gmail.com>:

> On Sun, May 6, 2012 at 8:58 AM, Alexander Graf <agraf@suse.de> wrote:
>> 
>> 
>> Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:
>> 
>>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>> On Tue, May 1, 2012 at 11:25 AM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>>>> On Mon, Apr 30, 2012 at 17:38, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>>>> On Mon, Apr 30, 2012 at 7:15 PM, Andreas Färber <afaerber@suse.de> wrote:
>>>>>>>>> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>>>>>>>>>> Tried to boot QEMU Niagara machine with the firmware from the
>>>>>>>>>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>>>>>>>>>> , and it dies very early.
>>>>>>>>>> The reason: in translate.c
>>>>>>>>>> 
>>>>>>>>>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>>>>>>>>>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>>>>>>>>>> 
>>>>>>>>>> and the dc->mem_idx is initialized like this:
>>>>>>>>>> 
>>>>>>>>>>     if (env1->tl > 0) {
>>>>>>>>>>         return MMU_NUCLEUS_IDX;
>>>>>>>>>>     } else if (cpu_hypervisor_mode(env1)) {
>>>>>>>>>>         return MMU_HYPV_IDX;
>>>>>>>>>>     } else if (cpu_supervisor_mode(env1)) {
>>>>>>>>>>         return MMU_KERNEL_IDX;
>>>>>>>>>>     } else {
>>>>>>>>>>         return MMU_USER_IDX;
>>>>>>>>>>     }
>>>>>>>>>> 
>>>>>>>>>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>>>>>>>>>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>>>>>>>>>> hypervisor(dc) must return 1 which is impossible in the current
>>>>>>>>>> implementation.
>>>>>>>>>> 
>>>>>>>>>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>>>>>>>>>> more variables to DisasContext, or ...?
>>>>>>>>>> 
>>>>>>>>>> Some other findings/questions:
>>>>>>>>>> 
>>>>>>>>>>     /* Sun4v generic Niagara machine */
>>>>>>>>>>     {
>>>>>>>>>>         .default_cpu_model = "Sun UltraSparc T1",
>>>>>>>>>>         .console_serial_base = 0xfff0c2c000ULL,
>>>>>>>>>> 
>>>>>>>>>> Where is this address coming from? The OpenSPARC Niagara machine has a
>>>>>>>>>> "dumb serial" at 0x1f10000000ULL.
>>>>>>>>>> 
>>>>>>>>>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>>>>>>>>>> different format for a MMU TTE entry than the one sun4u CPU are using.
>>>>>>>>>> I think the best way to handle it would be splitting off Niagara
>>>>>>>>>> machine, and #defining MMU bits differently for sun4u and sun4v
>>>>>>>>>> machines.
>>>>>>>>>> 
>>>>>>>>>> Do we the cases in qemu where more than two (qemu-system-xxx and
>>>>>>>>>> qemu-system-xxx64) binaries are produced?
>>>>>>>>>> Would the name qemu-system-sun4v fit the naming convention?
>>>>>>>>> 
>>>>>>>>> We have such a case for ppc (ppcemb) and it is kind of a maintenance
>>>>>>>>> nightmare - I'm working towards getting rid of it with my QOM CPU work.
>>>>>>>>> Better avoid it for sparc in the first place.
>>>>>>>>> 
>>>>>>>>> Instead, you should add a callback function pointer to SPARCCPUClass
>>>>>>>>> that you initialize based on CPU model so that is behaves differently at
>>>>>>>>> runtime rather than at compile time.
>>>>>>>>> Or if it's just about the class_init then after the Hard Freeze I can
>>>>>>>>> start polishing my subclasses for sparc so that you can add a special
>>>>>>>>> class_init for Niagara.
>>>>>>>> 
>>>>>>>> But this would mean that the defines from
>>>>>>>> #define TTE_NFO_BIT (1ULL << 60)
>>>>>>>> to
>>>>>>>> #define TTE_PGSIZE(tte)     (((tte) >> 61) & 3ULL)
>>>>>>>> 
>>>>>>>> inclusive would need to be replaced with functions and variables?
>>>>>>>> Sounds like a further performance regression for sun4u?
>>>>>>> 
>>>>>>> There could be parallel definitions for sun4u (actually UltraSparc-III
>>>>>>> onwards the MMU is again different) and sun4v.
>>>>>>> 
>>>>>>> At tlb_fill(), different implementations can be selected based on MMU
>>>>>>> model. For ASI accesses, we can add conditional code but for higher
>>>>>>> performance, some checks can be moved to translation time.
>>>>>> 
>>>>>> Can be done, but what is the gain of having it runtime configurable?
>>>>> 
>>>>> I was thinking of code like this in:
>>>>> 
>>>>> switch (env->mmu_model) {
>>>>> case MMU_US2:
>>>>>   return tlb_fill_us2(..);
>>>>> case MMU_US3:
>>>>>   return tlb_fill_us3(..);
>>>>> case MMU_US4:
>>>>>   return tlb_fill_us4(..);
>>>>> case MMU_T1:
>>>>>   return tlb_fill_t1(..);
>>>>> case MMU_T2:
>>>>>   return tlb_fill_t2(..);
>>>>> }
>>>>> 
>>>>> The perfomance cost shouldn't be too high. Alternatively a function
>>>>> pointer could be set up.
>>>> 
>>>> Actually I was more worried about get_physical_address_* than filling,
>>>> there we would have to use variables instead of constants and
>>>> functions instead of macros.
>>> 
>>> Preferably entirely different functions with constants.
>>> 
>>>> 
>>>>> Yes, we can always provide the register bank, older models just access
>>>>> some of those.
>>>>> 
>>>>>> cpu_change_pstate should probably have another parameter (new_GL)
>>>>>> which is only valid for sun4v.
>>>>>> And, depending on a trap type, env->htba has to be taken instead of
>>>>>> env->tbr. To me it looks like at the end do_interrupt will have less
>>>>>> common parts between sun4u and sun4v than specific ones.
>>>>> 
>>>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>>>> 
>>>>> This is unavoidable (unless maybe in the future the TLB handling can
>>>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>>>> and the performance cost is not great.
>>>> 
>>>> So, altogether you'd still prefer run-time checks over having
>>>> qemu-system-sun4v (or -sparc64v) ?
>>> 
>>> Yes. Architectures are not meant to handle small issues like this.
>>> Should performance become a problem, there are a plenty of lower
>>> hanging fruits where to start optimizing.
>>> 
>>> Even in this case, rather than the new architecture solution, it could
>>> be possible to build separate TLB handlers which call directly the
>>> correct MMU functions without switches and these would be selected at
>>> translation time or earlier. For the PPCEMB case, maybe the memory API
>>> could be changed to handle different page sizes without loss of
>>> performance, I don't know. Devices should not depend on
>>> TARGET_PAGE_SIZE.
>> 
>> It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.
>> 
>> That'd hurt performance quite a bit. The softmmu already is a very big chunk of execution time on ppc and zi really don't want that number to go up.
> 
> Yes, but that's not what I proposed. Now the translator arranges a
> call to for example qemu_ld32u which uses this fixed TLB size.
> Instead, it should generate calls to qemu_ld32u_ppcemb (which should
> use 1k pages) or qemu_ld32u_ppc (4k?) as needed, maybe also combining
> with MMU_IDX they would expand to qemu_ld32u_ppc_hypv etc. Obviously
> this would need big changes everywhere and MMU_IDX change should be
> compared with the negative cache effects of having many separate,
> often used functions in the hot path.

Ah, I see :). Yeah, that'd work and sounds like a great idea!

Alex

> 
> Memory API (or actually the related pieces in exec.c) hardcodes
> TARGET_PAGE_SIZE assumptions in many places, but it might be possible
> to make this dynamic.
> 
>> 
>> 
>> Alex
>> 

Re: [Qemu-devel] Poking a sun4v machine

[Andreas Färber]

Am 06.05.2012 10:58, schrieb Alexander Graf:
> Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:
>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>> To me it looks like at the end do_interrupt will have less
>>>>> common parts between sun4u and sun4v than specific ones.
>>>>
>>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>>>
>>>> This is unavoidable (unless maybe in the future the TLB handling can
>>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>>> and the performance cost is not great.
[...]
>> Architectures are not meant to handle small issues like this.
>> Should performance become a problem, there are a plenty of lower
>> hanging fruits where to start optimizing.
>>
>> Even in this case, rather than the new architecture solution, it could
>> be possible to build separate TLB handlers which call directly the
>> correct MMU functions without switches and these would be selected at
>> translation time or earlier. For the PPCEMB case, maybe the memory API
>> could be changed to handle different page sizes without loss of
>> performance, I don't know. Devices should not depend on
>> TARGET_PAGE_SIZE.
> 
> It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.

Due to the TLB being a common property of CPUs yet dependent on target
sizes, I had the idea of breaking it out from CPU(Arch)State so that we
can have a different inheritance hierarchy than for CPUs. But that's
just an unevaluated idea so far and more than 138 commits in the future.

Same problem for breakpoints and watchpoints btw.

ppcemb is no priority for me, but in order to move the very basic halted
and interrupt_request fields to CPUState, for a VMState post_load hook
we need to be able to tlb_flush() based on CPUState rather than
CPUArchState. Currently just a pointer hack on qom-cpu branch; as an
interim solution I might just defer that to target code to workaround
the problem...

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Poking a sun4v machine

[Blue Swirl]

On Sun, May 6, 2012 at 10:18 AM, Andreas Färber <afaerber@suse.de> wrote:
> Am 06.05.2012 10:58, schrieb Alexander Graf:
>> Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:
>>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>> To me it looks like at the end do_interrupt will have less
>>>>>> common parts between sun4u and sun4v than specific ones.
>>>>>
>>>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>>>>
>>>>> This is unavoidable (unless maybe in the future the TLB handling can
>>>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>>>> and the performance cost is not great.
> [...]
>>> Architectures are not meant to handle small issues like this.
>>> Should performance become a problem, there are a plenty of lower
>>> hanging fruits where to start optimizing.
>>>
>>> Even in this case, rather than the new architecture solution, it could
>>> be possible to build separate TLB handlers which call directly the
>>> correct MMU functions without switches and these would be selected at
>>> translation time or earlier. For the PPCEMB case, maybe the memory API
>>> could be changed to handle different page sizes without loss of
>>> performance, I don't know. Devices should not depend on
>>> TARGET_PAGE_SIZE.
>>
>> It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.
>
> Due to the TLB being a common property of CPUs yet dependent on target
> sizes, I had the idea of breaking it out from CPU(Arch)State so that we
> can have a different inheritance hierarchy than for CPUs. But that's
> just an unevaluated idea so far and more than 138 commits in the future.

Yes, this is in part what I wanted with cputlb.[ch] change.

> Same problem for breakpoints and watchpoints btw.
>
> ppcemb is no priority for me, but in order to move the very basic halted
> and interrupt_request fields to CPUState, for a VMState post_load hook
> we need to be able to tlb_flush() based on CPUState rather than
> CPUArchState. Currently just a pointer hack on qom-cpu branch; as an
> interim solution I might just defer that to target code to workaround
> the problem...

Could the TLB become a separate object? The TLB functions could use a
pointer to struct CPUCommonTLB (containing approximately what
CPU_COMMON_TLB in cpu-defs.h has now) instead of CPUState. Though we
need to pass a CPUState to TLB fill function anyway.

>
> Andreas
>
> --
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Poking a sun4v machine

[Andreas Färber]

Am 06.05.2012 12:58, schrieb Blue Swirl:
> On Sun, May 6, 2012 at 10:18 AM, Andreas Färber <afaerber@suse.de> wrote:
>> Am 06.05.2012 10:58, schrieb Alexander Graf:
>>> Am 06.05.2012 um 10:29 schrieb Blue Swirl <blauwirbel@gmail.com>:
>>>> On Wed, May 2, 2012 at 2:38 PM, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>> On Tue, May 1, 2012 at 4:06 PM, Blue Swirl <blauwirbel@gmail.com> wrote:
>>>>>> On Tue, May 1, 2012 at 13:54, Artyom Tarasenko <atar4qemu@gmail.com> wrote:
>>>>>>> To me it looks like at the end do_interrupt will have less
>>>>>>> common parts between sun4u and sun4v than specific ones.
>>>>>>
>>>>>> Same as tlb_fill(), switch() or function pointer. The functions are different.
>>>>>>
>>>>>> This is unavoidable (unless maybe in the future the TLB handling can
>>>>>> be pushed partially higher so mmu_idx parameters can be eliminated)
>>>>>> and the performance cost is not great.
>> [...]
>>>> Architectures are not meant to handle small issues like this.
>>>> Should performance become a problem, there are a plenty of lower
>>>> hanging fruits where to start optimizing.
>>>>
>>>> Even in this case, rather than the new architecture solution, it could
>>>> be possible to build separate TLB handlers which call directly the
>>>> correct MMU functions without switches and these would be selected at
>>>> translation time or earlier. For the PPCEMB case, maybe the memory API
>>>> could be changed to handle different page sizes without loss of
>>>> performance, I don't know. Devices should not depend on
>>>> TARGET_PAGE_SIZE.
>>>
>>> It's not a matter of an API. The main problem is that the QEMU TLB has to be fine grained enough to handle 1k faults, so it has to be in 1k-steps in its current design.
>>
>> Due to the TLB being a common property of CPUs yet dependent on target
>> sizes, I had the idea of breaking it out from CPU(Arch)State so that we
>> can have a different inheritance hierarchy than for CPUs. But that's
>> just an unevaluated idea so far and more than 138 commits in the future.
> 
> Yes, this is in part what I wanted with cputlb.[ch] change.
> 
>> Same problem for breakpoints and watchpoints btw.
>>
>> ppcemb is no priority for me, but in order to move the very basic halted
>> and interrupt_request fields to CPUState, for a VMState post_load hook
>> we need to be able to tlb_flush() based on CPUState rather than
>> CPUArchState. Currently just a pointer hack on qom-cpu branch; as an
>> interim solution I might just defer that to target code to workaround
>> the problem...
> 
> Could the TLB become a separate object?

That's what I meant with "breaking it out from CPU(Arch)State". :)

/-F

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] Poking a sun4v machine

[Alexander Graf]

On 30.04.2012, at 19:15, Andreas Färber wrote:

> Am 30.04.2012 18:39, schrieb Artyom Tarasenko:
>> Tried to boot QEMU Niagara machine with the firmware from the
>> OpenSPARC T1 emulator ( www.opensparc.net/opensparc-t1/download.html )
>> , and it dies very early.
>> The reason: in translate.c
>> 
>> #define hypervisor(dc) (dc->mem_idx == MMU_HYPV_IDX)
>> #define supervisor(dc) (dc->mem_idx >= MMU_KERNEL_IDX)
>> 
>> and the dc->mem_idx is initialized like this:
>> 
>>    if (env1->tl > 0) {
>>        return MMU_NUCLEUS_IDX;
>>    } else if (cpu_hypervisor_mode(env1)) {
>>        return MMU_HYPV_IDX;
>>    } else if (cpu_supervisor_mode(env1)) {
>>        return MMU_KERNEL_IDX;
>>    } else {
>>        return MMU_USER_IDX;
>>    }
>> 
>> Which seems to be conceptually incorrect. After reset tl == MAXTL, but
>> still super- and hyper-visor bits are set, so both supervisor(dc) and
>> hypervisor(dc) must return 1 which is impossible in the current
>> implementation.
>> 
>> What would be the proper way to fix it? Make mem_idx bitmap, add two
>> more variables to DisasContext, or ...?
>> 
>> Some other findings/questions:
>> 
>>    /* Sun4v generic Niagara machine */
>>    {
>>        .default_cpu_model = "Sun UltraSparc T1",
>>        .console_serial_base = 0xfff0c2c000ULL,
>> 
>> Where is this address coming from? The OpenSPARC Niagara machine has a
>> "dumb serial" at 0x1f10000000ULL.
>> 
>> And the biggest issue: UA2005 (as well as UA2007) describe a totally
>> different format for a MMU TTE entry than the one sun4u CPU are using.
>> I think the best way to handle it would be splitting off Niagara
>> machine, and #defining MMU bits differently for sun4u and sun4v
>> machines.
>> 
>> Do we the cases in qemu where more than two (qemu-system-xxx and
>> qemu-system-xxx64) binaries are produced?
>> Would the name qemu-system-sun4v fit the naming convention?
> 
> We have such a case for ppc (ppcemb) and it is kind of a maintenance
> nightmare - I'm working towards getting rid of it with my QOM CPU work.
> Better avoid it for sparc in the first place.

The big differentiator of ppcemb is that TARGET_PAGE_SIZE is 1k. That's a setting I really don't want to enable for anyone not running 440 code.


Alex