[Qemu-devel] Adding SMP support for Sparc Target

[Qemu-devel] Adding SMP support for Sparc Target

[Damien Hilloulin]

Hello everyone,

I'm a newcomer in QEMU and my goal would be to port an existing system 
simulator using another emulator to QEMU.
Some work has already been done, and Sparc has been the main target so 
far because of its simplicity (and because we have a very good support 
for Sparc with the other emulator).
QEMU is great, open-source (contrary to the other emulator we have been 
using in the past), and that's why we are aiming at using it.

However, it seems that the Sparc targets doesn't really support SMP/CMT 
as of now. So I am considering two possibilities:
- adding SMP support in QEMU for the Sparc targets (and contribute it to 
QEMU :) )
- switch our focus to another arch wich supports SMP/CMT in QEMU e.g 
x86, ARM (lot of work for us : ( )...

So that's why I'm writing to you, to judge how feasible it is to add the 
SMP/CMT support for the Sparc Targets.
I'm not a Sparc guru at all, so I don't really know what would be the 
steps to add this support.
Do you have any idea about how to approach it? Do you know where I can 
find some good documentation that could help me?
How much time it would take for someone having 3 or 4 hours per day to 
dedicate it?


Thanks for any help,

Best Regards,
Damien.

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Artyom Tarasenko]

Hello Damien,

On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
<damien.hilloulin@epfl.ch> wrote:
> Hello everyone,
>
> I'm a newcomer in QEMU and my goal would be to port an existing system
> simulator using another emulator to QEMU.
> Some work has already been done, and Sparc has been the main target so far
> because of its simplicity (and because we have a very good support for Sparc
> with the other emulator).
> QEMU is great, open-source (contrary to the other emulator we have been
> using in the past), and that's why we are aiming at using it.
>
> However, it seems that the Sparc targets doesn't really support SMP/CMT as
> of now. So I am considering two possibilities:
> - adding SMP support in QEMU for the Sparc targets (and contribute it to
> QEMU :) )

Do you mean a) emulating multiple guest cores on in a single host
thread, or b) emulating multiple guest cores in multiple host threads?

The former (a) should be relative easy for a sun4m platform: just have
to put the CPUs at the proper place in the system bus and fill the CPU
Module Ids (MIDs) with the proper data.

It would bring no performance increase though. In fact the guest OS
would likely run slower because the speed of an emulated CPU would
decrease like 1/N, and utilization of multiple CPUs by a guest OS is
probably scale like ~ log N, where N is the number of CPUs emulated.

If you mean b), things get more complicated because TCG can currently
utilize just one host thread. There was an attempt to do utilize
multiple threads for an ARM target:
http://sourceforge.net/p/coremu/home/Home

It would be interesting to hear what the TCG experts would say. Adding
Richard to CC.

Artyom

-- 
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Alex Bennée]

Artyom Tarasenko <atar4qemu@gmail.com> writes:

> Hello Damien,
>
> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
> <damien.hilloulin@epfl.ch> wrote:
>> Hello everyone,
>>
>> I'm a newcomer in QEMU and my goal would be to port an existing system
>> simulator using another emulator to QEMU.
<snip>
>> However, it seems that the Sparc targets doesn't really support SMP/CMT as
>> of now. So I am considering two possibilities:
>> - adding SMP support in QEMU for the Sparc targets (and contribute it to
>> QEMU :) )
>
> Do you mean a) emulating multiple guest cores on in a single host
> thread, or b) emulating multiple guest cores in multiple host threads?
<snip>
> If you mean b), things get more complicated because TCG can currently
> utilize just one host thread. There was an attempt to do utilize
> multiple threads for an ARM target:
> http://sourceforge.net/p/coremu/home/Home
>
> It would be interesting to hear what the TCG experts would say. Adding
> Richard to CC.

There is a desire to fix this but a distinct lack of cycles. It's not a
small job and will require quite a bit of preparatory work to map out an
approach and then fix it.

Having said that I'm sure someone mentioned they had done some work on
this on one of the KVM conference calls. Unfortunately I didn't catch
their names as my phone kept dumping me of the call. Does anyone
remember who that was? 

-- 
Alex Bennée

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Damien Hilloulin]

Le 06/11/2014 16:27, Artyom Tarasenko a écrit :
> Hello Damien,
>
> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
> <damien.hilloulin@epfl.ch> wrote:
>> Hello everyone,
>>
>> I'm a newcomer in QEMU and my goal would be to port an existing system
>> simulator using another emulator to QEMU.
>> Some work has already been done, and Sparc has been the main target so far
>> because of its simplicity (and because we have a very good support for Sparc
>> with the other emulator).
>> QEMU is great, open-source (contrary to the other emulator we have been
>> using in the past), and that's why we are aiming at using it.
>>
>> However, it seems that the Sparc targets doesn't really support SMP/CMT as
>> of now. So I am considering two possibilities:
>> - adding SMP support in QEMU for the Sparc targets (and contribute it to
>> QEMU :) )
> Do you mean a) emulating multiple guest cores on in a single host
> thread, or b) emulating multiple guest cores in multiple host threads?
a) Would be enough for us (but b) would be amazing) !
> The former (a) should be relative easy for a sun4m platform: just have
> to put the CPUs at the proper place in the system bus and fill the CPU
> Module Ids (MIDs) with the proper data.
Could you please explain it with some more details? I don't really know 
how to do that (yet)...
We are really interested in such a support for Sparc64, so I think that 
it would be for sun4u machines only.
Would the changes would be the same or would there be more work?

> It would bring no performance increase though. In fact the guest OS
> would likely run slower because the speed of an emulated CPU would
> decrease like 1/N, and utilization of multiple CPUs by a guest OS is
> probably scale like ~ log N, where N is the number of CPUs emulated.
>
> If you mean b), things get more complicated because TCG can currently
> utilize just one host thread. There was an attempt to do utilize
> multiple threads for an ARM target:
> http://sourceforge.net/p/coremu/home/Home
>
> It would be interesting to hear what the TCG experts would say. Adding
> Richard to CC.
>
> Artyom
>
Speeding up the simulation would be really interesting for us, but would 
be in the next iteration of changes. Would be great if we had both the 
features and speed :)

Thanks for your help, it seems really promising to me!

Damien.

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Artyom Tarasenko]

On Thu, Nov 6, 2014 at 6:36 PM, Damien Hilloulin
<damien.hilloulin@epfl.ch> wrote:
> Le 06/11/2014 16:27, Artyom Tarasenko a écrit :
>>
>> Hello Damien,
>>
>> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
>> <damien.hilloulin@epfl.ch> wrote:
>>>
>>> Hello everyone,
>>>
>>> I'm a newcomer in QEMU and my goal would be to port an existing system
>>> simulator using another emulator to QEMU.
>>> Some work has already been done, and Sparc has been the main target so
>>> far
>>> because of its simplicity (and because we have a very good support for
>>> Sparc
>>> with the other emulator).
>>> QEMU is great, open-source (contrary to the other emulator we have been
>>> using in the past), and that's why we are aiming at using it.
>>>
>>> However, it seems that the Sparc targets doesn't really support SMP/CMT
>>> as
>>> of now. So I am considering two possibilities:
>>> - adding SMP support in QEMU for the Sparc targets (and contribute it to
>>> QEMU :) )
>>
>> Do you mean a) emulating multiple guest cores on in a single host
>> thread, or b) emulating multiple guest cores in multiple host threads?
>
> a) Would be enough for us (but b) would be amazing) !
>>
>> The former (a) should be relative easy for a sun4m platform: just have
>> to put the CPUs at the proper place in the system bus and fill the CPU
>> Module Ids (MIDs) with the proper data.
>
> Could you please explain it with some more details? I don't really know how
> to do that (yet)...
> We are really interested in such a support for Sparc64, so I think that it
> would be for sun4u machines only.
> Would the changes would be the same or would there be more work?

Taking into account the overall status of sun4u emulation, it is a bit
more work.
AFAIR the SMP-related registers of Ultrasparc CPUs/chipsets are not implemented.
Additionally there are currently no sun4u SMP boards emulated in QEMU,
but I think this is a smaller issue.

Also some support in OpenBIOS might be necessary. Mark can surely tell more.

Artyom

-- 
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Damien Hilloulin]

Le 06/11/2014 19:23, Artyom Tarasenko a écrit :
> On Thu, Nov 6, 2014 at 6:36 PM, Damien Hilloulin
> <damien.hilloulin@epfl.ch> wrote:
>> Le 06/11/2014 16:27, Artyom Tarasenko a écrit :
>>> Hello Damien,
>>>
>>> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
>>> <damien.hilloulin@epfl.ch> wrote:
>>>> Hello everyone,
>>>>
>>>> I'm a newcomer in QEMU and my goal would be to port an existing system
>>>> simulator using another emulator to QEMU.
>>>> Some work has already been done, and Sparc has been the main target so
>>>> far
>>>> because of its simplicity (and because we have a very good support for
>>>> Sparc
>>>> with the other emulator).
>>>> QEMU is great, open-source (contrary to the other emulator we have been
>>>> using in the past), and that's why we are aiming at using it.
>>>>
>>>> However, it seems that the Sparc targets doesn't really support SMP/CMT
>>>> as
>>>> of now. So I am considering two possibilities:
>>>> - adding SMP support in QEMU for the Sparc targets (and contribute it to
>>>> QEMU :) )
>>> Do you mean a) emulating multiple guest cores on in a single host
>>> thread, or b) emulating multiple guest cores in multiple host threads?
>> a) Would be enough for us (but b) would be amazing) !
>>> The former (a) should be relative easy for a sun4m platform: just have
>>> to put the CPUs at the proper place in the system bus and fill the CPU
>>> Module Ids (MIDs) with the proper data.
>> Could you please explain it with some more details? I don't really know how
>> to do that (yet)...
>> We are really interested in such a support for Sparc64, so I think that it
>> would be for sun4u machines only.
>> Would the changes would be the same or would there be more work?
> Taking into account the overall status of sun4u emulation, it is a bit
> more work.
> AFAIR the SMP-related registers of Ultrasparc CPUs/chipsets are not implemented.
> Additionally there are currently no sun4u SMP boards emulated in QEMU,
> but I think this is a smaller issue.
>
> Also some support in OpenBIOS might be necessary. Mark can surely tell more.
>
> Artyom
>
Ok... Quite bad that's not as easy than for Sparc32, but I think that's 
still better for us than focusing on ARM direclty.
Would someone have a little bit of documentation about those topics 
(specific registers/interrupts/bios)?
Any help is welcome!

Damien.

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Artyom Tarasenko]

On Thu, Nov 6, 2014 at 11:05 PM, Damien Hilloulin
<damien.hilloulin@epfl.ch> wrote:
> Le 06/11/2014 19:23, Artyom Tarasenko a écrit :
>
>> On Thu, Nov 6, 2014 at 6:36 PM, Damien Hilloulin
>> <damien.hilloulin@epfl.ch> wrote:
>>>
>>> Le 06/11/2014 16:27, Artyom Tarasenko a écrit :
>>>>
>>>> Hello Damien,
>>>>
>>>> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
>>>> <damien.hilloulin@epfl.ch> wrote:
>>>>>
>>>>> Hello everyone,
>>>>>
>>>>> I'm a newcomer in QEMU and my goal would be to port an existing system
>>>>> simulator using another emulator to QEMU.
>>>>> Some work has already been done, and Sparc has been the main target so
>>>>> far
>>>>> because of its simplicity (and because we have a very good support for
>>>>> Sparc
>>>>> with the other emulator).
>>>>> QEMU is great, open-source (contrary to the other emulator we have been
>>>>> using in the past), and that's why we are aiming at using it.
>>>>>
>>>>> However, it seems that the Sparc targets doesn't really support SMP/CMT
>>>>> as
>>>>> of now. So I am considering two possibilities:
>>>>> - adding SMP support in QEMU for the Sparc targets (and contribute it
>>>>> to
>>>>> QEMU :) )
>>>>
>>>> Do you mean a) emulating multiple guest cores on in a single host
>>>> thread, or b) emulating multiple guest cores in multiple host threads?
>>>
>>> a) Would be enough for us (but b) would be amazing) !
>>>>
>>>> The former (a) should be relative easy for a sun4m platform: just have
>>>> to put the CPUs at the proper place in the system bus and fill the CPU
>>>> Module Ids (MIDs) with the proper data.
>>>
>>> Could you please explain it with some more details? I don't really know
>>> how
>>> to do that (yet)...
>>> We are really interested in such a support for Sparc64, so I think that
>>> it
>>> would be for sun4u machines only.
>>> Would the changes would be the same or would there be more work?
>>
>> Taking into account the overall status of sun4u emulation, it is a bit
>> more work.
>> AFAIR the SMP-related registers of Ultrasparc CPUs/chipsets are not
>> implemented.
>> Additionally there are currently no sun4u SMP boards emulated in QEMU,
>> but I think this is a smaller issue.
>>
>> Also some support in OpenBIOS might be necessary. Mark can surely tell
>> more.
>>
>> Artyom
>>
> Ok... Quite bad that's not as easy than for Sparc32, but I think that's
> still better for us than focusing on ARM direclty.
> Would someone have a little bit of documentation about those topics
> (specific registers/interrupts/bios)?

http://people.freebsd.org/~jake/docs/802-7220-02.pdf

The cross-calls are described in chapter 7.12.

The interrupt processing of the Ultrasparc IIi CPUs is described here:

http://people.freebsd.org/~jake/docs/805-0087.pdf

The other documents from http://people.freebsd.org/~jake/docs/ may be
helpful as well.

Artyom


-- 
Regards,
Artyom Tarasenko

SPARC and PPC PReP under qemu blog: http://tyom.blogspot.com/search/label/qemu

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Damien Hilloulin]

Le 07/11/2014 09:05, Artyom Tarasenko a écrit :
> On Thu, Nov 6, 2014 at 11:05 PM, Damien Hilloulin
> <damien.hilloulin@epfl.ch> wrote:
>> Le 06/11/2014 19:23, Artyom Tarasenko a écrit :
>>
>>> On Thu, Nov 6, 2014 at 6:36 PM, Damien Hilloulin
>>> <damien.hilloulin@epfl.ch> wrote:
>>>> Le 06/11/2014 16:27, Artyom Tarasenko a écrit :
>>>>> Hello Damien,
>>>>>
>>>>> On Thu, Nov 6, 2014 at 8:38 AM, Damien Hilloulin
>>>>> <damien.hilloulin@epfl.ch> wrote:
>>>>>> Hello everyone,
>>>>>>
>>>>>> I'm a newcomer in QEMU and my goal would be to port an existing system
>>>>>> simulator using another emulator to QEMU.
>>>>>> Some work has already been done, and Sparc has been the main target so
>>>>>> far
>>>>>> because of its simplicity (and because we have a very good support for
>>>>>> Sparc
>>>>>> with the other emulator).
>>>>>> QEMU is great, open-source (contrary to the other emulator we have been
>>>>>> using in the past), and that's why we are aiming at using it.
>>>>>>
>>>>>> However, it seems that the Sparc targets doesn't really support SMP/CMT
>>>>>> as
>>>>>> of now. So I am considering two possibilities:
>>>>>> - adding SMP support in QEMU for the Sparc targets (and contribute it
>>>>>> to
>>>>>> QEMU :) )
>>>>> Do you mean a) emulating multiple guest cores on in a single host
>>>>> thread, or b) emulating multiple guest cores in multiple host threads?
>>>> a) Would be enough for us (but b) would be amazing) !
>>>>> The former (a) should be relative easy for a sun4m platform: just have
>>>>> to put the CPUs at the proper place in the system bus and fill the CPU
>>>>> Module Ids (MIDs) with the proper data.
>>>> Could you please explain it with some more details? I don't really know
>>>> how
>>>> to do that (yet)...
>>>> We are really interested in such a support for Sparc64, so I think that
>>>> it
>>>> would be for sun4u machines only.
>>>> Would the changes would be the same or would there be more work?
>>> Taking into account the overall status of sun4u emulation, it is a bit
>>> more work.
>>> AFAIR the SMP-related registers of Ultrasparc CPUs/chipsets are not
>>> implemented.
>>> Additionally there are currently no sun4u SMP boards emulated in QEMU,
>>> but I think this is a smaller issue.
>>>
>>> Also some support in OpenBIOS might be necessary. Mark can surely tell
>>> more.
>>>
>>> Artyom
>>>
>> Ok... Quite bad that's not as easy than for Sparc32, but I think that's
>> still better for us than focusing on ARM direclty.
>> Would someone have a little bit of documentation about those topics
>> (specific registers/interrupts/bios)?
> http://people.freebsd.org/~jake/docs/802-7220-02.pdf
>
> The cross-calls are described in chapter 7.12.
>
> The interrupt processing of the Ultrasparc IIi CPUs is described here:
>
> http://people.freebsd.org/~jake/docs/805-0087.pdf
>
> The other documents from http://people.freebsd.org/~jake/docs/ may be
> helpful as well.
>
> Artyom
>
>
Thanks a lot for your help!
I will look at that!

Damien.

Re: [Qemu-devel] Adding SMP support for Sparc Target

[Mark Cave-Ayland]

On 06/11/14 18:23, Artyom Tarasenko wrote:

>> Could you please explain it with some more details? I don't really know how
>> to do that (yet)...
>> We are really interested in such a support for Sparc64, so I think that it
>> would be for sun4u machines only.
>> Would the changes would be the same or would there be more work?
> 
> Taking into account the overall status of sun4u emulation, it is a bit
> more work.
> AFAIR the SMP-related registers of Ultrasparc CPUs/chipsets are not implemented.
> Additionally there are currently no sun4u SMP boards emulated in QEMU,
> but I think this is a smaller issue.
> 
> Also some support in OpenBIOS might be necessary. Mark can surely tell more.

Hi Damien,

Great to see some interest in sun4u SMP support! I think Artyom's
summary above is about right - as a starting point you'll want to change
the sun4u init function to allow multiple CPUs and pass the details to
OpenBIOS so that more than one CPU node is added to the device tree.

With that in place I would expect multiple CPUs to be visible to the
guest, and so then it's a matter of implementing additional registers
required for CPU operation. As Artyom points out, there are still copies
of the old Sun documentation you can find lying around and I've also had
good support from the NetBSD/OpenBSD projects when trying to debug
low-level kernel issues during boot.

Feel free to CC me on any patches and I look forward to seeing the
results of your work :)


ATB,

Mark.