quick question on 64-bit values with 32-bit inline assembly

quick question on 64-bit values with 32-bit inline assembly

[Chris Friesen]

Hi all,

We're running a 64-bit kernel and 32-bit userspace.  We've got some code 
that is trying to get a 64-bit timestamp in userspace.

The following code seems to work fine in the kernel but in userspace it 
appears to be swapping the two words in the result.

gethrtime(void)
{
    unsigned long long result;

    asm volatile ("rdhwr %0,$31" : "=r" (result));
    return result;
}

Do I need to do something special because userspace is 32-bit?  If so, 
can someone point me to a reference?

Thanks,

Chris Friesen

Re: quick question on 64-bit values with 32-bit inline assembly

[Ralf Baechle]

On Mon, Jan 21, 2008 at 12:09:37PM -0600, Chris Friesen wrote:

> We're running a 64-bit kernel and 32-bit userspace.  We've got some code 
> that is trying to get a 64-bit timestamp in userspace.
>
> The following code seems to work fine in the kernel but in userspace it 
> appears to be swapping the two words in the result.
>
> gethrtime(void)
> {
>    unsigned long long result;
>
>    asm volatile ("rdhwr %0,$31" : "=r" (result));

Ah, Cavium.

>    return result;
> }
>
> Do I need to do something special because userspace is 32-bit?  If so, can 
> someone point me to a reference?

Ouch.  You found a nasty special case.  Normally 32-bit userspace should
not use 64-bit values but since you're running a 64-bit kernel.

unsigned long long gethrtime(void)
{
	unsigned long result;

	asm volatile(
	"	.set	mips64r2		\n"
	"	rdhwr	%M0, $31		\n"
	"	sll	%L0, %M0, 0		\n"
	"	dsra	%M0, 32			\n"
	"	.set	mips0			\n"
	: "=r" (result));

	return result;
}

Note this wouldn't possibly work on a 32-bit kernel because 32-bit kernels
will corrupt the upper 32-bit of integer registers so you might lose the
result value before you can stash it away.  Also 32-bit kernels don't allow
the execution of 64-bit instructions, not even on 64-bit processors.

  Ralf

Re: quick question on 64-bit values with 32-bit inline assembly

[Geert Uytterhoeven]

On Tue, 22 Jan 2008, Ralf Baechle wrote:
> On Mon, Jan 21, 2008 at 12:09:37PM -0600, Chris Friesen wrote:
> > We're running a 64-bit kernel and 32-bit userspace.  We've got some code 
> > that is trying to get a 64-bit timestamp in userspace.
> >
> > The following code seems to work fine in the kernel but in userspace it 
> > appears to be swapping the two words in the result.
> >
> > gethrtime(void)
> > {
> >    unsigned long long result;
> >
> >    asm volatile ("rdhwr %0,$31" : "=r" (result));
> 
> Ah, Cavium.
> 
> >    return result;
> > }
> >
> > Do I need to do something special because userspace is 32-bit?  If so, can 
> > someone point me to a reference?
> 
> Ouch.  You found a nasty special case.  Normally 32-bit userspace should
> not use 64-bit values but since you're running a 64-bit kernel.
> 
> unsigned long long gethrtime(void)
> {
> 	unsigned long result;
        ^^^^^^^^^^^^^
	unsigned long long

> 	asm volatile(
> 	"	.set	mips64r2		\n"
> 	"	rdhwr	%M0, $31		\n"
> 	"	sll	%L0, %M0, 0		\n"
> 	"	dsra	%M0, 32			\n"
> 	"	.set	mips0			\n"
> 	: "=r" (result));
> 
> 	return result;
> }
> 
> Note this wouldn't possibly work on a 32-bit kernel because 32-bit kernels
> will corrupt the upper 32-bit of integer registers so you might lose the
> result value before you can stash it away.  Also 32-bit kernels don't allow
> the execution of 64-bit instructions, not even on 64-bit processors.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: quick question on 64-bit values with 32-bit inline assembly

[Ralf Baechle]

On Tue, Jan 22, 2008 at 07:02:02PM +0100, Geert Uytterhoeven wrote:

> > unsigned long long gethrtime(void)
> > {
> > 	unsigned long result;
>         ^^^^^^^^^^^^^
> 	unsigned long long

Obviously, indeed.

Thanks :-)

  Ralf

Re: quick question on 64-bit values with 32-bit inline assembly

[Chris Friesen]

Ralf Baechle wrote:
> On Mon, Jan 21, 2008 at 12:09:37PM -0600, Chris Friesen wrote:

>>We're running a 64-bit kernel and 32-bit userspace.  We've got some code 
>>that is trying to get a 64-bit timestamp in userspace.
>>
>>The following code seems to work fine in the kernel but in userspace it 
>>appears to be swapping the two words in the result.
>>
>>gethrtime(void)
>>{
>>   unsigned long long result;
>>
>>   asm volatile ("rdhwr %0,$31" : "=r" (result));

> Ah, Cavium.

Yes indeed.  Any peculiarities that we should be watching out for? 
Previous mailing list threads would be great.

> Ouch.  You found a nasty special case.  Normally 32-bit userspace should
> not use 64-bit values but since you're running a 64-bit kernel.

I haven't done mips in years and was a bit surprised that the 
instruction set didn't provide for ways to read high and low words of a 
64-bit value the way that ppc32 does.

> unsigned long long gethrtime(void)
> {
> 	unsigned long long result;
> 
> 	asm volatile(
> 	"	.set	mips64r2		\n"
> 	"	rdhwr	%M0, $31		\n"
> 	"	sll	%L0, %M0, 0		\n"
> 	"	dsra	%M0, 32			\n"
> 	"	.set	mips0			\n"
> 	: "=r" (result));
> 
> 	return result;
> }
> 
> Note this wouldn't possibly work on a 32-bit kernel because 32-bit kernels
> will corrupt the upper 32-bit of integer registers so you might lose the
> result value before you can stash it away.  Also 32-bit kernels don't allow
> the execution of 64-bit instructions, not even on 64-bit processors.

I was a bit worried looking at the mips32 architecture manuals...didn't 
realize that you could just flip to 64-bit mode like that.

Thanks for all the help.

Chris

Re: quick question on 64-bit values with 32-bit inline assembly

[Ralf Baechle]

On Tue, Jan 22, 2008 at 12:55:45PM -0600, Chris Friesen wrote:

>>> gethrtime(void)
>>> {
>>>   unsigned long long result;
>>>
>>>   asm volatile ("rdhwr %0,$31" : "=r" (result));
>
>> Ah, Cavium.
>
> Yes indeed.  Any peculiarities that we should be watching out for? Previous 
> mailing list threads would be great.

Cavium so far received little coverage on this list - but seems you're
about to start this.  The reason why I was able to identify Cavium is that
afaics only Cavium is the only 64-bit CPU to implement a 64-bit timer in
hardware register $31.

The definition of rdhwr is generic and I think if anybody has considered
this specific interaction of ABI and processor architecture then it was
probably found not to implement such a special read because it is messy
in more than one way.

  Ralf

Re: quick question on 64-bit values with 32-bit inline assembly

[M. Warner Losh]

In message: <20080122200751.GA2672@linux-mips.org>
            Ralf Baechle <ralf@linux-mips.org> writes:
: On Tue, Jan 22, 2008 at 12:55:45PM -0600, Chris Friesen wrote:
: 
: >>> gethrtime(void)
: >>> {
: >>>   unsigned long long result;
: >>>
: >>>   asm volatile ("rdhwr %0,$31" : "=r" (result));
: >
: >> Ah, Cavium.
: >
: > Yes indeed.  Any peculiarities that we should be watching out for? Previous 
: > mailing list threads would be great.
: 
: Cavium so far received little coverage on this list - but seems you're
: about to start this.  The reason why I was able to identify Cavium is that
: afaics only Cavium is the only 64-bit CPU to implement a 64-bit timer in
: hardware register $31.
: 
: The definition of rdhwr is generic and I think if anybody has considered
: this specific interaction of ABI and processor architecture then it was
: probably found not to implement such a special read because it is messy
: in more than one way.

When 64-bit operations are enabled, you get all 64-bits.  When they
aren't, only the lower 32-bits of the counter are provided (with sign
extension).  So when operating in 32-bit mode, saving the upper 32
bits are not necessary (or even possible).

Warner

Re: quick question on 64-bit values with 32-bit inline assembly

[Ralf Baechle]

On Mon, Jan 28, 2008 at 02:02:45PM -0700, M. Warner Losh wrote:

> : this specific interaction of ABI and processor architecture then it was
> : probably found not to implement such a special read because it is messy
> : in more than one way.
> 
> When 64-bit operations are enabled, you get all 64-bits.  When they
> aren't, only the lower 32-bits of the counter are provided (with sign
> extension).  So when operating in 32-bit mode, saving the upper 32
> bits are not necessary (or even possible).

The architecture manual doesn't make a difference between 32-bit and
64-bit for rdhwr.  My reading is the entire 64-bit would have to be
transfered.

  Ralf

Re: quick question on 64-bit values with 32-bit inline assembly

[M. Warner Losh]

In message: <20080128211803.GA20434@linux-mips.org>
            Ralf Baechle <ralf@linux-mips.org> writes:
: On Mon, Jan 28, 2008 at 02:02:45PM -0700, M. Warner Losh wrote:
: 
: > : this specific interaction of ABI and processor architecture then it was
: > : probably found not to implement such a special read because it is messy
: > : in more than one way.
: > 
: > When 64-bit operations are enabled, you get all 64-bits.  When they
: > aren't, only the lower 32-bits of the counter are provided (with sign
: > extension).  So when operating in 32-bit mode, saving the upper 32
: > bits are not necessary (or even possible).
: 
: The architecture manual doesn't make a difference between 32-bit and
: 64-bit for rdhwr.  My reading is the entire 64-bit would have to be
: transfered.

Hmmm, the manual I have specifically calls out the difference...

Has cavium provided a public version?

Warner

Re: quick question on 64-bit values with 32-bit inline assembly

[Maciej W. Rozycki]

On Mon, 28 Jan 2008, M. Warner Losh wrote:

> : The architecture manual doesn't make a difference between 32-bit and
> : 64-bit for rdhwr.  My reading is the entire 64-bit would have to be
> : transfered.
> 
> Hmmm, the manual I have specifically calls out the difference...

 Concerning implementation-specific registers number 30 and 31 the MIPS64 
architecture manual states that if the register in question is 64-bit and 
64-bit operations are enabled then it is copied as is and otherwise it is 
sign-extended from the bit #31.  Note that the kernel mode implies 64-bit 
operations enabled.

  Maciej