[Xenomai-help] Atomic 64 bit write access on PCIe device

[Xenomai-help] Atomic 64 bit write access on PCIe device

[M. Koehrer]

Hi everybody,

perhaps this issue is off-topic, but I expect that there are many experts on that issue reading the Xenomai list...
I have a Core2Duo Intel CPU (32 bit mode) and a proprietary PCI Express I/O board plugged in.
This PCI Express I/O board has some 64 bit registers that are mapped into address space via mmap.
Now, I want to write atomically (one PCIe access) to one of the 64 bit registers.
Unfortunately, the Intel CPUs does not have an atomic 64 bit write operations when running the CPU in 32 bit mode.
When doing something like

volatile unsigned long long *ull = register_address;
*ull = my_new_register_value;

I see the gcc is generating two 32 bit write accesses. Of course this will not lead to an atomic 64 bit write access.

Any idea, how I can write 64 bit integers atomically to such an 64 bit register?


Thanks for any feedback!

Regards

Mathias

-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT FÜR ALLE NEUEINSTEIGER
Jetzt bei Arcor: günstig und schnell mit DSL - das All-Inclusive-Paket
für clevere Doppel-Sparer, nur  29,95 Euro  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Atomic 64 bit write access on PCIe device

[Gilles Chanteperdrix]

On Feb 20, 2008 9:25 AM, M. Koehrer <mathias_koehrer@domain.hid> wrote:
> Hi everybody,
>
> perhaps this issue is off-topic, but I expect that there are many experts on that issue reading the Xenomai list...
> I have a Core2Duo Intel CPU (32 bit mode) and a proprietary PCI Express I/O board plugged in.
> This PCI Express I/O board has some 64 bit registers that are mapped into address space via mmap.
> Now, I want to write atomically (one PCIe access) to one of the 64 bit registers.
> Unfortunately, the Intel CPUs does not have an atomic 64 bit write operations when running the CPU in 32 bit mode.
> When doing something like
>
> volatile unsigned long long *ull = register_address;
> *ull = my_new_register_value;
>
> I see the gcc is generating two 32 bit write accesses. Of course this will not lead to an atomic 64 bit write access.
>
> Any idea, how I can write 64 bit integers atomically to such an 64 bit register?
>
>
> Thanks for any feedback!

Maybe using a spinlock would be possible ?
I do not think that using inline assembly would work (I would expect
the assembler to refuse 64 bits instructions when in 32 bits mode).
Why not compiling the kernel and Xenomai in 64 bits mode ?

-- 
                                               Gilles Chanteperdrix

Re: [Xenomai-help] Atomic 64 bit write access on PCIe device

[Philippe Gerum]

M. Koehrer wrote:
> Hi everybody,
> 
> perhaps this issue is off-topic, but I expect that there are many experts on that issue reading the Xenomai list...
> I have a Core2Duo Intel CPU (32 bit mode) and a proprietary PCI Express I/O board plugged in.
> This PCI Express I/O board has some 64 bit registers that are mapped into address space via mmap.
> Now, I want to write atomically (one PCIe access) to one of the 64 bit registers.
> Unfortunately, the Intel CPUs does not have an atomic 64 bit write operations when running the CPU in 32 bit mode.
> When doing something like
> 
> volatile unsigned long long *ull = register_address;
> *ull = my_new_register_value;
> 
> I see the gcc is generating two 32 bit write accesses. Of course this will not lead to an atomic 64 bit write access.
> 
> Any idea, how I can write 64 bit integers atomically to such an 64 bit register?
>

Assuming this register may only be accessed from userland through
mmapped memory (and not concurrently from a kernel driver), and that you
may write this value as 2 x 32bit as far as the hw is concerned but you
want to avoid software access races doing so, something like this would
probably do:

iopl(3)
...
mutex_lock(&lock); /* prevent SMP race */
asm volatile("cli"); /* prevent local preemption */
*ull = my_new_register_value;
asm volatile("sti");
mutex_unlock(&lock);

Yeah, that's pretty ugly. Yum.

> 
> Thanks for any feedback!
> 
> Regards
> 
> Mathias
> 


-- 
Philippe.

Re: [Xenomai-help] Atomic 64 bit write access on PCIe device

[Fillod Stephane]

M. Koehrer wrote:
>perhaps this issue is off-topic, but I expect that there are many
experts on that issue reading the Xenomai list...
>I have a Core2Duo Intel CPU (32 bit mode) and a proprietary PCI Express
I/O board plugged in.
>This PCI Express I/O board has some 64 bit registers that are mapped
into address space via mmap.
>Now, I want to write atomically (one PCIe access) to one of the 64 bit
registers.
>Unfortunately, the Intel CPUs does not have an atomic 64 bit write
operations when running the CPU in 32 bit mode.

I'm no expert, but did you tried already using FP instructions, which is
a common
trick to make use of 64 bits I/O in code optimization (before SIMD era).

How about the following:

void do_write64(void *p, unsigned long long x)
{
        volatile union {
                unsigned long long ull;
                double d;
        } u;

        u.ull = x;
        *(volatile double *)p = u.d;
}

Disassemble that code, you should see some fldl and fstpl.
-- 
Stephane

Re: [Xenomai-help] Atomic 64 bit write access on PCIe device

[M. Koehrer]

Hi Stephane,

thanks for your mail.
> 
> I'm no expert, but did you tried already using FP instructions, which is
> a common
> trick to make use of 64 bits I/O in code optimization (before SIMD era).
> 
> How about the following:
> 
> void do_write64(void *p, unsigned long long x)
> {
>         volatile union {
>                 unsigned long long ull;
>                 double d;
>         } u;
> 
>         u.ull = x;
>         *(volatile double *)p = u.d;
> }
> 
> Disassemble that code, you should see some fldl and fstpl.

First I thought of using the FPU for this issue.
Hoever, I am not sure if it is allowed to use the FPU here as I think there are some 64 bit integer values
that will be modified by fldl (e.g. if they represent some kind of NaN).
According to Intel's instruction set reference for fld:

"Pushes the source operand onto the FPU register stack. The source operand can be in
single-precision, double-precision, or double extended-precision floating-point
format. If the source operand is in single-precision or double-precision floating-point
format, it is automatically converted to the double extended-precision floating-point
format before being pushed on the stack."

In my case I have a double precision which will be converted to the double extended-precision
format.
Thus, I can imagine that there are some 64 bit long longs that will not be handled correctly.


Regards

Mathias


-- 
Mathias Koehrer
mathias_koehrer@domain.hid


Viel oder wenig? Schnell oder langsam? Unbegrenzt surfen + telefonieren
ohne Zeit- und Volumenbegrenzung? DAS TOP ANGEBOT FÜR ALLE NEUEINSTEIGER
Jetzt bei Arcor: günstig und schnell mit DSL - das All-Inclusive-Paket
für clevere Doppel-Sparer, nur  29,95 Euro  inkl. DSL- und ISDN-Grundgebühr!
http://www.arcor.de/rd/emf-dsl-2

Re: [Xenomai-help] Atomic 64 bit write access on PCIe device

[Fillod Stephane]

M. Koehrer wrote:
>First I thought of using the FPU for this issue.
>Hoever, I am not sure if it is allowed to use the FPU here as I think
there are some 64 bit integer values
>that will be modified by fldl (e.g. if they represent some kind of
NaN).

Might be. To be checked whether there are some other safer FP
instructions.

Another idea would be to use MMX and the MOVQ instruction for example.

Regards,
-- 
Stephane