[Qemu-devel] TCG baremetal tests repo

[Qemu-devel] TCG baremetal tests repo

[Alexander Spyridakis]

[-- Attachment #1: Type: text/plain, Size: 891 bytes --]

Hello all,

You can find the latest tcg atomic test payload in the following repo:
> git clone https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git

You also need an arm baremetal cross-compiler like arm-none-gnueabi- (arm)
and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in the current
multithreading tcg repo, the atomic test was modified to work also on the
vexpress machine model.

To run it:
> make vexpress (or virt/virt64 for other targets)
> ../mttcg/arm-softmmu/qemu-system-arm -nographic -M vexpress-a15 -kernel
build-vexpress/image-vexpress.axf -smp 4

On my machine it takes around 30 seconds for one run of the test and the
results vary from as low as 5 to 30 errors per vCPU per 10 million
iterations (no errors with KVM). It is also very interesting to note, that
the current test finishes faster on upstream qemu than multithreaded qemu.

Best regards.

[-- Attachment #2: Type: text/html, Size: 1075 bytes --]

Re: [Qemu-devel] TCG baremetal tests repo

[Alex Bennée]

Alexander Spyridakis <a.spyridakis@virtualopensystems.com> writes:

> Hello all,
>
> You can find the latest tcg atomic test payload in the following repo:
>> git clone
>https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git

Both git and Chrome complain about your SSL certificates btw.

>
> You also need an arm baremetal cross-compiler like arm-none-gnueabi- (arm)
> and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in the current
> multithreading tcg repo, the atomic test was modified to work also on the
> vexpress machine model.
>
> To run it:
>> make vexpress (or virt/virt64 for other targets)
>> ../mttcg/arm-softmmu/qemu-system-arm -nographic -M vexpress-a15 -kernel
> build-vexpress/image-vexpress.axf -smp 4
>
> On my machine it takes around 30 seconds for one run of the test and the
> results vary from as low as 5 to 30 errors per vCPU per 10 million
> iterations (no errors with KVM). It is also very interesting to note, that
> the current test finishes faster on upstream qemu than multithreaded
> qemu.

Interesting. Thanks for putting the tests online.

>
> Best regards.

-- 
Alex Bennée

Re: [Qemu-devel] TCG baremetal tests repo

[Alex Bennée]

Alexander Spyridakis <a.spyridakis@virtualopensystems.com> writes:

> Hello all,
>
> You can find the latest tcg atomic test payload in the following repo:
>> git clone https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git
>
> You also need an arm baremetal cross-compiler like arm-none-gnueabi- (arm)
> and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in the current
> multithreading tcg repo, the atomic test was modified to work also on the
> vexpress machine model.

I sent a patch to fix the PSCI hang case to wake up the sleeping CPU. I
couldn't figure out how the vexpress code was waking it's CPUs. Do they
just start powered on?

>
> To run it:
>> make vexpress (or virt/virt64 for other targets)
>> ../mttcg/arm-softmmu/qemu-system-arm -nographic -M vexpress-a15 -kernel
> build-vexpress/image-vexpress.axf -smp 4
>
> On my machine it takes around 30 seconds for one run of the test and the
> results vary from as low as 5 to 30 errors per vCPU per 10 million
> iterations (no errors with KVM). It is also very interesting to note, that
> the current test finishes faster on upstream qemu than multithreaded qemu.
>
> Best regards.

-- 
Alex Bennée

Re: [Qemu-devel] TCG baremetal tests repo

[Peter Maydell]

On 24 June 2015 at 17:39, Alex Bennée <alex.bennee@linaro.org> wrote:
>
> Alexander Spyridakis <a.spyridakis@virtualopensystems.com> writes:
>> You can find the latest tcg atomic test payload in the following repo:
>>> git clone https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git
>>
>> You also need an arm baremetal cross-compiler like arm-none-gnueabi- (arm)
>> and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in the current
>> multithreading tcg repo, the atomic test was modified to work also on the
>> vexpress machine model.
>
> I sent a patch to fix the PSCI hang case to wake up the sleeping CPU. I
> couldn't figure out how the vexpress code was waking it's CPUs. Do they
> just start powered on?

Yes -- our vexpress model is "like hardware", so all CPUs leap into
the boot firmware at once, and the firmware deals with putting the
secondaries into a pen to be released in a controlled manner later.
[assuming you're not using -kernel; if you are then boot.c has the
secondary pen code]

-- PMM

Re: [Qemu-devel] TCG baremetal tests repo

[Alex Bennée]

Peter Maydell <peter.maydell@linaro.org> writes:

> On 24 June 2015 at 17:39, Alex Bennée <alex.bennee@linaro.org> wrote:
>>
>> Alexander Spyridakis <a.spyridakis@virtualopensystems.com> writes:
>>> You can find the latest tcg atomic test payload in the following repo:
>>>> git clone https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git
>>>
>>> You also need an arm baremetal cross-compiler like arm-none-gnueabi- (arm)
>>> and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in the current
>>> multithreading tcg repo, the atomic test was modified to work also on the
>>> vexpress machine model.
>>
>> I sent a patch to fix the PSCI hang case to wake up the sleeping CPU. I
>> couldn't figure out how the vexpress code was waking it's CPUs. Do they
>> just start powered on?
>
> Yes -- our vexpress model is "like hardware", so all CPUs leap into
> the boot firmware at once, and the firmware deals with putting the
> secondaries into a pen to be released in a controlled manner later.
> [assuming you're not using -kernel; if you are then boot.c has the
> secondary pen code]

Ahh good. No mystery to solve then ;-)

-- 
Alex Bennée

Re: [Qemu-devel] TCG baremetal tests repo

[Frederic Konrad]

On 22/06/2015 12:54, Alexander Spyridakis wrote:
> Hello all,
>
> You can find the latest tcg atomic test payload in the following repo:
> > git clone https://git.virtualopensystems.com/dev/tcg_baremetal_tests.git
>
> You also need an arm baremetal cross-compiler like arm-none-gnueabi- 
> (arm) and the usual aarch64-linux-gnu- (arm64). Due to a PSCI bug in 
> the current multithreading tcg repo, the atomic test was modified to 
> work also on the vexpress machine model.
>
> To run it:
> > make vexpress (or virt/virt64 for other targets)
> > ../mttcg/arm-softmmu/qemu-system-arm -nographic -M vexpress-a15 
> -kernel build-vexpress/image-vexpress.axf -smp 4
>
> On my machine it takes around 30 seconds for one run of the test and 
> the results vary from as low as 5 to 30 errors per vCPU per 10 million 
> iterations (no errors with KVM). It is also very interesting to note, 
> that the current test finishes faster on upstream qemu than 
> multithreaded qemu.
>
> Best regards.

Hi,

I just tested this with vexpress, seems ATOMIC is not defined by default 
it uses:

void non_atomic_lock(int *lock_var)
{
     while (*lock_var != 0);
     *lock_var = 1;
}

void non_atomic_unlock(int *lock_var)
{
     *lock_var = 0;
}

instead of:

void atomic_lock(int *lock_var)
{
     while (__sync_lock_test_and_set(lock_var, 1));
}

void atomic_unlock(int *lock_var)
{
     __sync_lock_release(lock_var);
}

It doesn't cause any errors upstream but a lot on mttcg and mttcg is 
faster in this
case.

I don't have any error when I use ATOMIC like this:

diff --git a/helpers.h b/helpers.h
index b5810ad..427659f 100644
--- a/helpers.h
+++ b/helpers.h
@@ -36,13 +36,8 @@
  #define SYS_CFGCTR_WRITE   0x40000000
  #define SYS_CFG_SHUTDOWN   0x00800000

-#ifdef ATOMIC
  #define LOCK   atomic_lock
  #define UNLOCK atomic_unlock
-#else
-#define LOCK   non_atomic_lock
-#define UNLOCK non_atomic_unlock
-#endif

  int online_cpus;
  int global_lock;

but it's slower than upstream which I think is normal. We can have two CPUs
fighting for the lock in mttcg but not in upstream as VCPUs doesn't run 
at the same
time.

Fred

Re: [Qemu-devel] TCG baremetal tests repo

[Alexander Spyridakis]

On 25 June 2015 at 18:01, Frederic Konrad <fred.konrad@greensocs.com> wrote:
> I just tested this with vexpress, seems ATOMIC is not defined by default it
> uses:

Wow nice catch! Somehow between revision the define was no longer
propagated and ATOMIC was not defined at all.

Tested with 8 cores and no more errors. As you said performance is
still slower but probably due to the continuous contesting of the
lock.

Thanks.