sizeof(long) different under windows x64 and linux x64

sizeof(long) different under windows x64 and linux x64

[James Harper]

Under gcc on an x64 system, sizeof(long) = 8
Under windows ddk on an x64 system, sizeof(long) = 4

Are the xen header files written with the assumption that sizeof(long) =
8? If so, this would explain why i'm getting EINVAL from the hypervisor
under windows x64...

James

Re: sizeof(long) different under windows x64 and linux x64

[Daniel Stodden]

On Wed, 2008-01-23 at 10:01 +1100, James Harper wrote:
> Under gcc on an x64 system, sizeof(long) = 8
> Under windows ddk on an x64 system, sizeof(long) = 4

true for windows c compilers, independent of the ddk.

> Are the xen header files written with the assumption that sizeof(long) =
> 8? 

well, gcc built the xen binary under this assumption, right?

> If so, this would explain why i'm getting EINVAL from the hypervisor
> under windows x64...

indeed. it's still about the add_to_physmap issue you described? 
will need a fix then to adjust the xen_ulong_t typedef accordingly on
x86_64.

correlated question from my side: what is xen_ulong_t good for?
shouldn't the public headers use either stdint or a more descriptive
typedef? ulong_t sounds pretty redundant to me, especially when
typedef'd to an 'unsigned long'.

regards,
daniel

-- 
dns@somacoma.net

Wire up your home and stay there.

RE: sizeof(long) different under windows x64 and linuxx64

[James Harper]

> On Wed, 2008-01-23 at 10:01 +1100, James Harper wrote:
> > Under gcc on an x64 system, sizeof(long) = 8
> > Under windows ddk on an x64 system, sizeof(long) = 4
> 
> true for windows c compilers, independent of the ddk.

Yes. sizeof(long) = 4 always under windows x32 and x64.

> > If so, this would explain why i'm getting EINVAL from the hypervisor
> > under windows x64...
> 
> indeed. it's still about the add_to_physmap issue you described?
> will need a fix then to adjust the xen_ulong_t typedef accordingly on
> x86_64.
> 
> correlated question from my side: what is xen_ulong_t good for?
> shouldn't the public headers use either stdint or a more descriptive
> typedef? ulong_t sounds pretty redundant to me, especially when
> typedef'd to an 'unsigned long'.

The enormity of this problem is just sinking in... Xen makes use of a
type (long) that it assumes is 32 bits under a 32 bit arch, and 64 bits
under a 64 bit arch. Windows has no such native type (except for
pointer), but that can be solved via some #if statements.

'long' and 'unsigned long' is used all over the place inside
xen/include/public. I would need to create this new type, and replace
all occurrences of 'long' with it in the 8750-ish lines of .h files.

Any suggestions as to what the type should be called? Maybe a bit of
typedef'ing around xen_ulong_t and xen_long_t would work?

Thanks

James

RE: sizeof(long) different under windows x64 and linuxx64

[James Harper]

> The enormity of this problem is just sinking in... Xen makes use of a
> type (long) that it assumes is 32 bits under a 32 bit arch, and 64
bits
> under a 64 bit arch. Windows has no such native type (except for
> pointer), but that can be solved via some #if statements.
> 
> 'long' and 'unsigned long' is used all over the place inside
> xen/include/public. I would need to create this new type, and replace
> all occurrences of 'long' with it in the 8750-ish lines of .h files.

Having another look, there are 248 occurrences of the word 'long' in the
.h files under xen/include/public. A search and replace on that is
probably going to break things (eg it would find the word 'belong' too),
but a search and replace on 'unsigned long' (203 occurrences) is
probably safe, leaving 45 'long's to be manually inspected. Nothing like
as big a job as it could have been.

I'll tackle it tonight!

James

RE: sizeof(long) different under windows x64 and linuxx64

[Daniel Stodden]

On Wed, 2008-01-23 at 10:34 +1100, James Harper wrote:

> The enormity of this problem is just sinking in... Xen makes use of a
> type (long) that it assumes is 32 bits under a 32 bit arch, and 64 bits
> under a 64 bit arch. Windows has no such native type (except for
> pointer), but that can be solved via some #if statements.
> 
> 'long' and 'unsigned long' is used all over the place inside
> xen/include/public. I would need to create this new type, and replace
> all occurrences of 'long' with it in the 8750-ish lines of .h files.
> 
> Any suggestions as to what the type should be called? Maybe a bit of
> typedef'ing around xen_ulong_t and xen_long_t would work?

well, i don't see yet where a new type would be needed.

it's only public/ which matters. and for the foreseeable time being,
from that it's only the subset which matters to frontends (although i
agree that fixing the full PV interface would be cleaner).

under the assumption that xen_ulong_t is to be interpreted as 'ulong as
what xen considers a ulong': fixing the xen_(u)long_t to (u)int64_t
should be sufficient, because that's what xen/gcc will always want it to
be on x86_64.

means you need to split the typedef among xen-32/64.h.

anything else?

regards,
daniel

-- 
dns@somacoma.net

Wire up your home and stay there.

Re: sizeof(long) different under windows x64 and linuxx64

[Samuel Thibault]

James Harper, le Wed 23 Jan 2008 10:34:39 +1100, a écrit :
> 'long' and 'unsigned long' is used all over the place inside
> xen/include/public. I would need to create this new type, and replace
> all occurrences of 'long' with it in the 8750-ish lines of .h files.
> 
> Any suggestions as to what the type should be called?

Just the standard name: uintptr_t

Samuel

RE: sizeof(long) different under windows x64 and linuxx64

[James Harper]

> >
> > Any suggestions as to what the type should be called?
> 
> Just the standard name: uintptr_t
> 

Can you qualify 'standard' in that context (eg Linux, gcc, POSIX, etc).
I don't ever remember hearing of that field before. Is it a type defined
as 'an integer with the same width as a pointer'?

Thanks

James

Re: sizeof(long) different under windows x64 and linuxx64

[Samuel Thibault]

James Harper, le Wed 23 Jan 2008 11:55:50 +1100, a écrit :
> > >
> > > Any suggestions as to what the type should be called?
> > 
> > Just the standard name: uintptr_t
> > 
> 
> Can you qualify 'standard' in that context (eg Linux, gcc, POSIX, etc).

C99

> I don't ever remember hearing of that field before. Is it a type defined
> as 'an integer with the same width as a pointer'?

« The following type designates an unsigned integer type with the property
that any valid pointer to void can be converted to this type, then
converted back to pointer to void, and the result will compare equal to
the original pointer: uintptr_t »

Samuel

RE: sizeof(long) different under windows x64 and linuxx64

[James Harper]

> James Harper, le Wed 23 Jan 2008 11:55:50 +1100, a écrit :
> > > >
> > > > Any suggestions as to what the type should be called?
> > >
> > > Just the standard name: uintptr_t
> > >
> >
> > Can you qualify 'standard' in that context (eg Linux, gcc, POSIX, etc).
> 
> C99

That'll do nicely :)

> > I don't ever remember hearing of that field before. Is it a type defined
> > as 'an integer with the same width as a pointer'?
> 
> « The following type designates an unsigned integer type with the property
> that any valid pointer to void can be converted to this type, then
> converted back to pointer to void, and the result will compare equal to
> the original pointer: uintptr_t »

Perfect. Thanks.

James

RE: sizeof(long) different under windows x64 and linuxx64

[James Harper]

> > > > Just the standard name: uintptr_t
> > > >
> > >
> > > Can you qualify 'standard' in that context (eg Linux, gcc, POSIX,
> etc).
> >
> > C99
> 
> That'll do nicely :)

Can I get some feedback from the people-who-make-decisions if a patch
changing all 'unsigned long's in the public .h files to uintptr_t's
would be accepted? If not, what type should I use?

Under Windows, I'm already using typedef's to create int8_t, uint8_t,
etc, so I don't have a problem if the uintptr_t type isn't valid in
windows natively either.

James  

Re: sizeof(long) different under windows x64 and linuxx64

[Keir Fraser]

On 23/1/08 02:30, "James Harper" <james.harper@bendigoit.com.au> wrote:

>> That'll do nicely :)
> 
> Can I get some feedback from the people-who-make-decisions if a patch
> changing all 'unsigned long's in the public .h files to uintptr_t's
> would be accepted? If not, what type should I use?
> 
> Under Windows, I'm already using typedef's to create int8_t, uint8_t,
> etc, so I don't have a problem if the uintptr_t type isn't valid in
> windows natively either.

See my previous mail. Yes, we might, but if we don't there is hardly much
downside for you. In this case, well, I suppose it would make the headers
clearer. The approach for upstream would be to typedef a xen_uintptr_t,
def'ed to unsigned long in old values of XEN_INTERFACE_VERSION and uintptr_t
in the latest XEN_INTERFACE_VERSION.

 -- Keir

Re: sizeof(long) different under windows x64 and linuxx64

[John Levon]

On Wed, Jan 23, 2008 at 01:30:47PM +1100, James Harper wrote:

> Can I get some feedback from the people-who-make-decisions if a patch
> changing all 'unsigned long's in the public .h files to uintptr_t's
> would be accepted? If not, what type should I use?

Shouldn't it be changed appropriately? That is, uintptr_t for values
that actually do, or can, hold a "pointer" value, and something else for
other 'longs'?

john

Re: sizeof(long) different under windows x64 and linuxx64

[Keir Fraser]

On 23/1/08 14:25, "John Levon" <levon@movementarian.org> wrote:

>> Can I get some feedback from the people-who-make-decisions if a patch
>> changing all 'unsigned long's in the public .h files to uintptr_t's
>> would be accepted? If not, what type should I use?
> 
> Shouldn't it be changed appropriately? That is, uintptr_t for values
> that actually do, or can, hold a "pointer" value, and something else for
> other 'longs'?

We could just call it xen_long or xen_ulong. That would hide the underlying
uintptr_t, which is needed anyway for XEN_INTERFACE_VERSION backward
source-level compatibility.

 -- Keir

Re: sizeof(long) different under windows x64 and linuxx64

[Christoph Egger]

On Wednesday 23 January 2008 15:33:20 Keir Fraser wrote:
> On 23/1/08 14:25, "John Levon" <levon@movementarian.org> wrote:
> >> Can I get some feedback from the people-who-make-decisions if a patch
> >> changing all 'unsigned long's in the public .h files to uintptr_t's
> >> would be accepted? If not, what type should I use?
> >
> > Shouldn't it be changed appropriately? That is, uintptr_t for values
> > that actually do, or can, hold a "pointer" value, and something else for
> > other 'longs'?
>
> We could just call it xen_long or xen_ulong. That would hide the underlying
> uintptr_t, which is needed anyway for XEN_INTERFACE_VERSION backward
> source-level compatibility.

When that ends up for uintptr_t being "unsigned long long" on 64bit platforms
and "unsigned long" on 32bit platforms, then you did it in a portable way.

BTW: C99 also defines intptr_t for whatever that might be useful.

Christoph

-- 
AMD Saxony, Dresden, Germany
Operating System Research Center

Legal Information:
AMD Saxony Limited Liability Company & Co. KG
Sitz (Geschäftsanschrift):
   Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
Registergericht Dresden: HRA 4896
vertretungsberechtigter Komplementär:
   AMD Saxony LLC (Sitz Wilmington, Delaware, USA)
Geschäftsführer der AMD Saxony LLC:
   Dr. Hans-R. Deppe, Thomas McCoy

Re: sizeof(long) different under windows x64 and linuxx64

[Christoph Egger]

On Wednesday 23 January 2008 02:38:37 James Harper wrote:
> > James Harper, le Wed 23 Jan 2008 11:55:50 +1100, a écrit :
> > > > > Any suggestions as to what the type should be called?
> > > >
> > > > Just the standard name: uintptr_t
> > >
> > > Can you qualify 'standard' in that context (eg Linux, gcc, POSIX, etc).
> >
> > C99
>
> That'll do nicely :)
>
> > > I don't ever remember hearing of that field before. Is it a type
> > > defined as 'an integer with the same width as a pointer'?
> >
> > « The following type designates an unsigned integer type with the
> > property that any valid pointer to void can be converted to this type,
> > then converted back to pointer to void, and the result will compare equal
> > to the original pointer: uintptr_t »
>
> Perfect. Thanks.

C99 introduced uintptr_t/intptr_t for portability:
http://www.unix.org/version2/whatsnew/lp64_wp.html

Christoph


-- 
AMD Saxony, Dresden, Germany
Operating System Research Center

Legal Information:
AMD Saxony Limited Liability Company & Co. KG
Sitz (Geschäftsanschrift):
   Wilschdorfer Landstr. 101, 01109 Dresden, Deutschland
Registergericht Dresden: HRA 4896
vertretungsberechtigter Komplementär:
   AMD Saxony LLC (Sitz Wilmington, Delaware, USA)
Geschäftsführer der AMD Saxony LLC:
   Dr. Hans-R. Deppe, Thomas McCoy

Re: sizeof(long) different under windows x64 and linux x64

[Andy Grover]

On Wed, 2008-01-23 at 10:01 +1100, James Harper wrote:
> Under gcc on an x64 system, sizeof(long) = 8
> Under windows ddk on an x64 system, sizeof(long) = 4
> 
> Are the xen header files written with the assumption that sizeof(long) =
> 8? If so, this would explain why i'm getting EINVAL from the hypervisor
> under windows x64...

I think it's very hard to be LP64/LLP64-clean without actual testing on
both.

I'm sure patches would be accepted to clean things up. We imported a
private version of the xen headers into the winpv tree hg so maybe we
should just go ahead and modify that, and then we can pull out a diff to
submit against xen-unstable when we know all the issues are cleaned up?

Most of the 700+ warnings on x64 Windows winpv build is probably because
of this :)

Regards -- Andy

Re: sizeof(long) different under windows x64 and linux x64

[Keir Fraser]

On 22/1/08 23:39, "Andy Grover" <andy.grover@oracle.com> wrote:

>> Are the xen header files written with the assumption that sizeof(long) =
>> 8? If so, this would explain why i'm getting EINVAL from the hypervisor
>> under windows x64...
> 
> I think it's very hard to be LP64/LLP64-clean without actual testing on
> both.
> 
> I'm sure patches would be accepted to clean things up. We imported a
> private version of the xen headers into the winpv tree hg so maybe we
> should just go ahead and modify that, and then we can pull out a diff to
> submit against xen-unstable when we know all the issues are cleaned up?

Maybe. I don't see why people are worried about having privately diverged
copies of the headers for their own target environment though. They're not
going to magically become incompatible with future versions of Xen. You need
only update your private headers, and then only incrementally, when you want
to use new features of future Xen versions.

 -- Keir

Re: sizeof(long) different under windows x64 and linux x64

[Chris Morrow]

Andy Grover wrote:

> I think it's very hard to be LP64/LLP64-clean without actual testing on
> both.

Printf format statements need to reflect whether the 64 bit type is a 
long or a long long.

uintptr_t foo
printf("%lx", foo) or printf("%llx", foo);

This can be a pain to resolve. /usr/include/inttypes.h has defines that 
are supposed to help with this, but they are ugly to use.

-- 
Chris Morrow	YottaYotta Inc. email: cmorrow@yottayotta.com
phone: (780) 989 6814           web:   http://www.yottayotta.com

Re: sizeof(long) different under windows x64 and linux x64

[Keir Fraser]

I doubt anyone printf's hypercall struct fields directly. If they do, we can
provide a suitable PRIxxx macro. Yeah, they're a bit ugly but it's the right
answer to this problem. We use PRIxxx macros extensively in Xen code
already.

 -- Keir

On 23/1/08 20:41, "Chris Morrow" <cmorrow@YottaYotta.com> wrote:

> Andy Grover wrote:
> 
>> I think it's very hard to be LP64/LLP64-clean without actual testing on
>> both.
> 
> Printf format statements need to reflect whether the 64 bit type is a
> long or a long long.
> 
> uintptr_t foo
> printf("%lx", foo) or printf("%llx", foo);
> 
> This can be a pain to resolve. /usr/include/inttypes.h has defines that
> are supposed to help with this, but they are ugly to use.