Re: [rfc][patch 3/3] x86: optimise barriers

[Jarek Poplawski <jarkao2@o2.pl>]

On Fri, Oct 12, 2007 at 12:42:38PM +0200, Nick Piggin wrote:
> On Fri, Oct 12, 2007 at 11:55:05AM +0200, Jarek Poplawski wrote:
> > On Fri, Oct 12, 2007 at 10:57:33AM +0200, Nick Piggin wrote:
> > > 
> > > I don't know quite what you're saying... the CPUs could probably get
> > > performance by having weakly ordered loads, OTOH I think the Intel
> > > ones might already do this speculatively so they appear in order but
> > > essentially have the performance of weak order.
> > 
> > I meant: if there is any reordering possible this should be quite
> > distinctly visible.
> 
> It's not. Not in the cases where it is explicitly allowed and actively
> exploited (loads passing stores), but most definitely not distinctly
> visible in errata cases that have slipped through all the V&V.
> 
> 
> > because why would any vendor enable such nasty
> > things if not for performance. But now I start to doubt: of course
> > there is such a possibility someone makes this reordering for some
> > other reasons which could be so rare it's hard to check.
> 
> Yes: it isn't the explicitly allowed reorderings that we care
> about here (because obviously we're retaining the barriers for those).
> It would be cases of bugs in the CPUs meaning they don't follow the
> standard. But how far do you take your mistrust of a CPU? You could
> ask gcc to insert locked ops between every load and store operation?
> Or keep it switched off to ensure no bugs ;)

I'm not sure of your point, but it seems we don't differ here, and
after all there is quirks code for such things.

> 
> 
> > Anyway, it seems any heavy testing such as yours, should give us the
> > same informations years earlier than any vendors manual and then any
> > gain is multiplied by millions of users. Then only still doubtful
> > cases could be treated with additional caution and some debugging
> > code.
> 
> Firstly, while it can be possible to write a code to show up reordering,
> it is really hard (ie. impossible) to guarantee no reordering happens. For
> example, it may have only showed up on SMT+SMP P4 CPUs with some obscure
> interactions between threads and cores involving more than 2 threads.

I'm not sure how much this all above is consistent wrt. this earlier
opinion:

> [...] If you can actually come up with a test
> case that triggers load/load or store/store reordering, I'm sure
> Intel / AMD would like to see it ;)

It seems, after testing only (plus no official spec against this idea),
you could be almost sure there is no such test possible. And, if it
were done a few years ago, you think it still should be not enough to
make a decision on changing this smp_rmb because of lack of official
specs? Besides, there is probably so much features guessing in arch
and drivers sections, this reorder testing should look as solid as a
math proof wrt. them.

> 
> Secondly, even if we were sure that no current implementations reordered
> loads, we don't want to go outside the bounds of the specification
> because we might break on some future CPUs. This isn't a big performance
> win.

I don't agree with this - IMO we should care only about currently used
CPUs, and test each time the new ones.


> > > All existing processors as far as we know are in-order WRT loads vs
> > > loads and stores vs stores. It was just a matter of getting the docs
> > > clarified, which gives us more confidence that we're correct and a
> > > reasonable guarnatee of forward compatibility.
> > 
> > After reading this Intel's legal information I don't think you should
> > feel so much more forward confident...
> 
> Yes, but that's the same way I feel after reading *any* legal "information" ;)
> 

Strange... I feel exactly opposite. Are you sure you've chosen the
right job (...and the right system)?

Jarek P.