RFC: 32 bits as smallest atomic size.

RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

In PPC land we have an issue where cmpxchg operations can only be
applied to strictly aligned 32 and 64 bit quantities.

We understand on x86 the use of the packed attribute has certain
benefits, but in the common areas we believe that portability should
be the primary concern in the common code.

I can only assume that much debate will come from this as we find
alignment and size issues all over.

The particular offender at the moment is:
  include/public/grant_table.h flags 76 u16 flags;

It is used with atomic methods such as cmpxchg_user() and clear_bit().

Any thoughts on how to proceed with this would be appreciated.

-JX

BTW: is clear_bit() as implemeted for x86 have soem alignment issues?
My expertise in this space is limited.





-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Fri, 25 Mar 2005, Jimi Xenidis wrote:

> We understand on x86 the use of the packed attribute has certain
> benefits, but in the common areas we believe that portability should
> be the primary concern in the common code.

I would actually like to see packed go away forever, as it is a headache 
for me each time I deal with changes. Plan 9 compilers don't like this 
type of thing. 


ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

I expect we can fix that up in the ppc macros: if the atomic access is 
sub-32-bit aligned then round the address down to 32-bit boundary and 
do a 32-bit cmpxchg.

  -- Keir

On 25 Mar 2005, at 21:25, Jimi Xenidis wrote:

>
> In PPC land we have an issue where cmpxchg operations can only be
> applied to strictly aligned 32 and 64 bit quantities.
>
> We understand on x86 the use of the packed attribute has certain
> benefits, but in the common areas we believe that portability should
> be the primary concern in the common code.
>
> I can only assume that much debate will come from this as we find
> alignment and size issues all over.
>
> The particular offender at the moment is:
>   include/public/grant_table.h flags 76 u16 flags;
>
> It is used with atomic methods such as cmpxchg_user() and clear_bit().
>
> Any thoughts on how to proceed with this would be appreciated.
>
> -JX
>
> BTW: is clear_bit() as implemeted for x86 have soem alignment issues?
> My expertise in this space is limited.
>
>
>
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real 
> users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

>>>>> "KF" == Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:

 KF> I expect we can fix that up in the ppc macros: if the atomic access is 
 KF> sub-32-bit aligned then round the address down to 32-bit boundary and 
 KF> do a 32-bit cmpxchg.

A few issues:
  1) this assumes that the quantity is wholy contained in our atomic
     unit, by using packed there is no way to guarantee that.

  2) it would have to be a runtime decision on all PPC atomic
     operations that could fail with no real failure path available.

We could also program it out, consider this alternative:
    typedef struct {
        struct {
	    u16     flags;      /* 0 */
	    domid_t domid;      /* 2 */
	} atomic;
        u32     frame;      /* 4 */
    } grant_entry_t; /* 8 bytes */

Few things to note:
  1) packed is necessary since the ABI will do the right thing
  2) would simplify the code at grant_table.c:162
  3) will have better performance since all access are guranteed to be
     aligned.

I understand the usefulness of packed, but its gratuitous use
unecessarily complicates other architectures, and can hurt even x86 if
it results in a data that "straddles" cachelines.

-JX


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 25 Mar 2005, at 23:49, Jimi Xenidis wrote:

>>>>>> "KF" == Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
>
>  KF> I expect we can fix that up in the ppc macros: if the atomic 
> access is
>  KF> sub-32-bit aligned then round the address down to 32-bit boundary 
> and
>  KF> do a 32-bit cmpxchg.
>
> A few issues:
>   1) this assumes that the quantity is wholy contained in our atomic
>      unit, by using packed there is no way to guarantee that.

We always naturally align fields, even in packed structs. The compiler 
doesn't do it for us, but we check it manually.

>   2) it would have to be a runtime decision on all PPC atomic
>      operations that could fail with no real failure path available.

It may have to be a run-time decision unless we can come up with some 
clever trick. I don't understand what you mean about there being no 
failure path.

> We could also program it out, consider this alternative:
>     typedef struct {
>         struct {
> 	    u16     flags;      /* 0 */
> 	    domid_t domid;      /* 2 */
> 	} atomic;
>         u32     frame;      /* 4 */
>     } grant_entry_t; /* 8 bytes */

This may make sense since we do update those two fields as a pair. We 
have a big update to the grant table code about to be checked in. We 
shoudl hold off doing this kind of fix until after the update has been 
committed.


> Few things to note:
>   1) packed is necessary since the ABI will do the right thing
>   2) would simplify the code at grant_table.c:162
>   3) will have better performance since all access are guranteed to be
>      aligned.

We always naturally align fields. We may do *subword* accesses 
sometimes, but they are always naturally aligned and will never 
straddle a cacheline. I think that subword accesses should get fixed up 
in your ppc macros. In teh specific case of the grant table structures 
let's look closer once teh big update is in the tree.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

>>>>> "KF" == Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:

 KF> On 25 Mar 2005, at 23:49, Jimi Xenidis wrote:

 >> A few issues:
 >> 1) this assumes that the quantity is wholy contained in our atomic
 >> unit, by using packed there is no way to guarantee that.

 KF> We always naturally align fields, even in packed structs. The compiler 
 KF> doesn't do it for us, but we check it manually.

hmm, is it padding you are trying to avoid?
Perhaps not using packed and adding the gcc option '-Wpadded' which
will automagically catch the offenders for you might be more
appropriate?

Depending on manually checking can result in performance and
correctness issues that are difficult to track.

 >> 2) it would have to be a runtime decision on all PPC atomic
 >> operations that could fail with no real failure path available.

 KF> It may have to be a run-time decision unless we can come up with some 
 KF> clever trick. I don't understand what you mean about there being no 
 KF> failure path.

These are atomic operation, usually used in contexts where printing or
any other feedback is unavailable.


 KF> This may make sense since we do update those two fields as a pair. We 
 KF> have a big update to the grant table code about to be checked in. We 
 KF> shoudl hold off doing this kind of fix until after the update has been 
 KF> committed.

Agreed, its not like I'm even at the point of running this code :)
However, some attention to endianness would also be appreciated,
example common/grant_table.c:870 (and friends)
        /* Merge two 16-bit values into a 32-bit combined update. */
        /* NB. Endianness! */
        prev_scombo = scombo = ((u32)sdom << 16) | (u32)sflags;

 KF> We always naturally align fields. We may do *subword* accesses 
 KF> sometimes, but they are always naturally aligned and will never 
 KF> straddle a cacheline. I think that subword accesses should get fixed up 
 KF> in your ppc macros.

Could we not simply avoid *subword* accesses in common code as a rule?
It costs x86 nothing to do this and results in portable code.


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 26 Mar 2005, at 14:13, Jimi Xenidis wrote:

>  KF> We always naturally align fields, even in packed structs. The 
> compiler
>  KF> doesn't do it for us, but we check it manually.
>
> hmm, is it padding you are trying to avoid?
> Perhaps not using packed and adding the gcc option '-Wpadded' which
> will automagically catch the offenders for you might be more
> appropriate?
>
> Depending on manually checking can result in performance and
> correctness issues that are difficult to track.

Mostly it's used for structs that are part of the public Xen interface. 
I don't want different compilers packing fields in different ways.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Fri, 25 Mar 2005, Jimi Xenidis wrote:

> Few things to note:
>   1) packed is necessary since the ABI will do the right thing
                 un-necessary?

ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

>>>>> "RGM" == Ronald G Minnich <rminnich@lanl.gov> writes:

 RGM> On Fri, 25 Mar 2005, Jimi Xenidis wrote:

 >> Few things to note:
 >> 1) packed is necessary since the ABI will do the right thing
 RGM>                  un-necessary?
oops, yes un-necessary.
-JX


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Sat, 26 Mar 2005, Jimi Xenidis wrote:

> Could we not simply avoid *subword* accesses in common code as a rule?
> It costs x86 nothing to do this and results in portable code.

it would help me too ...

ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 26 Mar 2005, at 17:31, Jimi Xenidis wrote:

>>>>>> "RGM" == Ronald G Minnich <rminnich@lanl.gov> writes:
>
>  RGM> On Fri, 25 Mar 2005, Jimi Xenidis wrote:
>
>>> Few things to note:
>>> 1) packed is necessary since the ABI will do the right thing
>  RGM>                  un-necessary?
> oops, yes un-necessary.
> -JX

I prefer to pack things explicitly rather than rely on ABI padding. My 
aim is to be able to document the interfaces separately from the C 
source, and that will include knowing the offset of each field, and 
that's easiest if the compiler isn't automatically helping by paddin 
things unexpectedly. If PACKED is too painful on a particular arch you 
can always stub it out -- it's only a macro after all.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 26 Mar 2005, at 17:49, Keir Fraser wrote:

> I prefer to pack things explicitly rather than rely on ABI padding. My 
> aim is to be able to document the interfaces separately from the C 
> source, and that will include knowing the offset of each field, and 
> that's easiest if the compiler isn't automatically helping by paddin 
> things unexpectedly. If PACKED is too painful on a particular arch you 
> can always stub it out -- it's only a macro after all.

Actually, one of the more interesting thing to discuss imo is how to 
structure the public interface headers to support multiple 
architectures. Is it feasible to strive for much commonality, or should 
each arch have its own public headers, or what? I don't have a good 
feel for what the best solution is going to be...

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

>>>>> "KF" == Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:

 KF> On 26 Mar 2005, at 17:49, Keir Fraser wrote:

 KF> Actually, one of the more interesting thing to discuss imo is how to 
 KF> structure the public interface headers to support multiple 
 KF> architectures. Is it feasible to strive for much commonality, or should 
 KF> each arch have its own public headers, or what? I don't have a good 
 KF> feel for what the best solution is going to be...

Considering that on PPC we'll be trying to support domains of several
different ABIs (ELF, XCOFF and MachO) not to mention 32 bit apps on
64bit machines, we are extremely sensitive (and sympathetic) to your
goals.

On x86 you have Windows, and their (intentionally) incompatible 64 bit
ABI :(

The current use of specific sized types is good for this goal.
Typically, defining these data structures with a member order that
removes padding, or padding explicitly to disambiguate structure
layout, is a good strategy (-Wpadded can help here).

I think we will disable PACKED on PPC and see how that goes, as we
detect padded areas we will submit patches that may shuffle structure
members to achieve the above goals.

WRT to subword accesses, we will continue to eveluate them and
generate some patches for your approval, rather than attempt to
shuffle bits around at runtime.

-JX


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[David Hopwood]

Keir Fraser wrote:
> On 26 Mar 2005, at 17:31, Jimi Xenidis wrote:
> 
>>>> Few things to note:
>>>> 1) packed is [un]necessary since the ABI will do the right thing
> 
> I prefer to pack things explicitly rather than rely on ABI padding.

If you use -Wpadded when compiling with gcc then you're not relying on
ABI padding; you're automatically checking that it is not used. That
would seem to be precisely what is required here.

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Sat, 26 Mar 2005, Keir Fraser wrote:

> Actually, one of the more interesting thing to discuss imo is how to structure
> the public interface headers to support multiple architectures. 

As the headers are written now, they are not portable across different C 
compilers, which is a concern to me. If your C compiler doesn't act just 
like gcc, you're going to have to translate the headers. 

> Is it feasible to strive for much commonality, or should each arch have
> its own public headers, or what? I don't have a good feel for what the
> best solution is going to be...

It is feasible to strive for commonality, I think. 

I would prefer headers that make no use of any gcc magic. It is a bit of a
problem for me each time xen rotates headers as I have to translate them
all over again.

My issues with the Plan 9 port have all revolved around portability from 
x86 to x86, due to the gcc-isms in the headers. 

ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 27 Mar 2005, at 00:27, David Hopwood wrote:

> Keir Fraser wrote:
>> On 26 Mar 2005, at 17:31, Jimi Xenidis wrote:
>>>>> Few things to note:
>>>>> 1) packed is [un]necessary since the ABI will do the right thing
>> I prefer to pack things explicitly rather than rely on ABI padding.
>
> If you use -Wpadded when compiling with gcc then you're not relying on
> ABI padding; you're automatically checking that it is not used. That
> would seem to be precisely what is required here.

If I could specify it on a per-struct basis then it would be perfect. 
But I do want the normal padding rules to be applied for non-public 
struct definitions.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 27 Mar 2005, at 04:34, Ronald G. Minnich wrote:

> My issues with the Plan 9 port have all revolved around portability 
> from
> x86 to x86, due to the gcc-isms in the headers.

I don't think there are that many gcc-isms, apart from use of PACKED 
(please correct me if I'm wrong). You can always define that to nothing 
if you need to - I'd hope that no compiler adds padding since all 
fields should be naturally aligned. I don't see us moving to a model 
where we define macros on char arrays anytime soon. :-) But perhaps we 
could include a script to generate such macros from our structure 
definitions....

I know Kip Macy iss including our public headers into FreeBSD with 
-ansi, so we can't be that far from ISO code.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Keir Fraser]

On 27 Mar 2005, at 11:56, Keir Fraser wrote:

> I don't think there are that many gcc-isms, apart from use of PACKED 
> (please correct me if I'm wrong). You can always define that to 
> nothing if you need to - I'd hope that no compiler adds padding since 
> all fields should be naturally aligned. I don't see us moving to a 
> model where we define macros on char arrays anytime soon. :-) But 
> perhaps we could include a script to generate such macros from our 
> structure definitions....

Actually this would get round the need to manually determine field 
offsets -- we could get rid of PACKED, let fields fall as they may, and 
then run the headers through gcc to get field offsets for those who 
need them. There are a very few cases where we actually really care 
about two fields being directly adjacent, but I could add annotations 
in the header files and run a script to check that the placement 
constraints are satisfied.

Already the field offset comments are broken in a few places in 
unstable, so perhaps this is a better way to go.

  -- Keir



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[David Hopwood]

Keir Fraser wrote:
> On 27 Mar 2005, at 00:27, David Hopwood wrote:
>> Keir Fraser wrote:
>>> On 26 Mar 2005, at 17:31, Jimi Xenidis wrote:
>>>
>>>>>> Few things to note:
>>>>>> 1) packed is [un]necessary since the ABI will do the right thing
>>>
>>> I prefer to pack things explicitly rather than rely on ABI padding.
>>
>> If you use -Wpadded when compiling with gcc then you're not relying on
>> ABI padding; you're automatically checking that it is not used. That
>> would seem to be precisely what is required here.
> 
> If I could specify it on a per-struct basis then it would be perfect.

There is no way to selectively enable/disable warnings in gcc:
<http://gcc.gnu.org/bugzilla/show_bug.cgi?id=9049>  :-(

OTOH, spurious warnings for padding in private structures cause no real
harm. If only a few private structures are involved, you could add dummy
padding to those anyway.

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>



-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Sun, 27 Mar 2005, Keir Fraser wrote:

> 
> On 27 Mar 2005, at 04:34, Ronald G. Minnich wrote:
> 
> > My issues with the Plan 9 port have all revolved around portability from
> > x86 to x86, due to the gcc-isms in the headers.
> 
> I don't think there are that many gcc-isms, apart from use of PACKED (please
> correct me if I'm wrong). 

that's the biggie. And I don't see the need in many cases. 

> You can always define that to nothing if you need to
> - I'd hope that no compiler adds padding since all fields should be naturally
> aligned. 

They are not in all cases. 

I will try to find a simple example later today. 

thanks

ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Ronald G. Minnich]

On Sun, 27 Mar 2005, Keir Fraser wrote:

> Already the field offset comments are broken in a few places in unstable, so
> perhaps this is a better way to go.

it's an idea.  A few years ago I wrote drivers for SCI and had a mixed 
linux/alpha freebsd/pentium cluster. It's a little tricky to get everyone 
happily addressing each other's memories across that boundary, but I did 
it by making all struct fields unsigned long long and letting the 
compilers do the right thing. I didn't use a single __attribute__ 
directive, but at the same time, these fairly different machines with 
different OSes had shared memory. I still don't see why xen can not do the 
same.

ron


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click

Re: RFC: 32 bits as smallest atomic size.

[Kurt Garloff]

[-- Attachment #1.1: Type: text/plain, Size: 596 bytes --]

Hi Keir,

On Sat, Mar 26, 2005 at 05:07:23PM +0000, Keir Fraser wrote:
> Mostly it's used for structs that are part of the public Xen interface. 
> I don't want different compilers packing fields in different ways.

The layout of (non-packed) structures is defined by the C ABI.
All compilers producing e.g. elf_i386 code should thus use the
same.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>             [Koeln, DE]
Physics:Plasma modeling <garloff@plasimo.phys.tue.nl> [TU Eindhoven, NL]
Linux: SUSE Labs (Director)    <garloff@suse.de>            [Novell Inc]

[-- Attachment #1.2: Type: application/pgp-signature, Size: 189 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: RFC: 32 bits as smallest atomic size.

[Hollis Blanchard]

[Resending to really correct list this time.]

On Fri, 2005-03-25 at 21:43 +0000, Keir Fraser wrote:
> I expect we can fix that up in the ppc macros: if the atomic access is 
> sub-32-bit aligned then round the address down to 32-bit boundary and 
> do a 32-bit cmpxchg.

>From a quick survey of Linux headers, it seems at least the following
architectures only support 4/8-byte atomic instructions: ppc(64),
sparc(64), parisc, mips, alpha.

The following architectures support 1/2-byte atomic instructions:
x86(64), ia64.

Since so many architectures don't allow non-word-sized accesses (a link
error is triggered), common Linux code does not make such calls. Could
we please have the same policy?

Right now there is a single problematic cmpxchg user, grant_entry_t, and
the impact of the fix is tiny (make the 'flags' member 4 bytes). Is that
too large an inconvenience?

-- 
Hollis Blanchard
IBM Linux Technology Center

RE: RFC: 32 bits as smallest atomic size.

[Ian Pratt]

> Right now there is a single problematic cmpxchg user, 
> grant_entry_t, and the impact of the fix is tiny (make the 
> 'flags' member 4 bytes). Is that too large an inconvenience?

That would make grant table entries 12 bytes rather than 8 bytes, which
is not ideal given that you can end up needing quite a lot of them,
particularly when the cacheline packing is considered. 

Perhaps the best thing to do would be to change that particular instance
of cmpxchg into cmpxchg_16, where we handle the subword case on
architectures which can't do it directly by rounding down the address,
doing a read, insert, and cmpxchg. 

This seems a better fix, at least to me.

Ian

RE: RFC: 32 bits as smallest atomic size.

[Hollis Blanchard]

On Tue, 2005-03-29 at 23:00 +0100, Ian Pratt wrote:
> > Right now there is a single problematic cmpxchg user, 
> > grant_entry_t, and the impact of the fix is tiny (make the 
> > 'flags' member 4 bytes). Is that too large an inconvenience?
> 
> That would make grant table entries 12 bytes rather than 8 bytes, which
> is not ideal given that you can end up needing quite a lot of them,
> particularly when the cacheline packing is considered. 
> 
> Perhaps the best thing to do would be to change that particular instance
> of cmpxchg into cmpxchg_16, where we handle the subword case on
> architectures which can't do it directly by rounding down the address,
> doing a read, insert, and cmpxchg. 

If you look closely, that code is a hack that Keir mentioned would be
replaced soon. Two 16-bit quantities are being concatenated (in an
endian-unsafe manner), then passed as a 32-bit quantity to cmpxchg,
using the address of the first 16-bit value.

I was looking for a more general policy though. There are a number of
atomic operations (including xchg, cmpxchg, bitops' *_bit, and atomic_*)
that will require extra code (to write and run) on many architectures,
if the types used are not native word size. Why shouldn't Xen follow
Linux's example here?

-- 
Hollis Blanchard
IBM Linux Technology Center

RE: RFC: 32 bits as smallest atomic size.

[Jimi Xenidis]

>>>>> "IP" == Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk> writes:

 >> Right now there is a single problematic cmpxchg user, 
 >> grant_entry_t, and the impact of the fix is tiny (make the 
 >> 'flags' member 4 bytes). Is that too large an inconvenience?

 IP> That would make grant table entries 12 bytes rather than 8 bytes, which
 IP> is not ideal given that you can end up needing quite a lot of them,
 IP> particularly when the cacheline packing is considered. 

Actually, the lock field can be 1 bit for all I care, as long as the
atomic method to change it word sized.

This is quite a reasonable request.
-JX