SAL PCI config space

SAL PCI config space

[Matthew Wilcox]

ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL().
This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ
and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant.  So is there
any reason we shouldn't use SAL_CALL_REENTRANT?

(at the moment, we're protected by pci_lock so it wouldn't matter,
but I'm just about to submit a patch to do away with that)

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: SAL PCI config space

[Alex Williamson]

On Fri, 2004-04-30 at 11:51, Matthew Wilcox wrote:
> ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use SAL_CALL().
> This takes an IRQ-safe spinlock, but SAL 3.2 says that SAL_PCI_CONFIG_READ
> and SAL_PCI_CONFIG_WRITE are both MP-safe and reentrant.  So is there
> any reason we shouldn't use SAL_CALL_REENTRANT?
> 

   I thought there was going to be further clarification on this, but
I'm not sure what SAL rev it would be in.  On HP boxes, we can't preform
multiple config accesses to the same root bus at the same time.  So far,
the SAL_CALL lock has protected us from needing to worry about it.  I'd
suspect other boxes might have similar restrictions.

	Alex

> (at the moment, we're protected by pci_lock so it wouldn't matter,
> but I'm just about to submit a patch to do away with that)

RE: SAL PCI config space

[Luck, Tony]

>ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use 
>SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that 
>SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and
>reentrant.  So is there any reason we shouldn't use SAL_CALL_REENTRANT?

Theoretically there might be older SAL implementations where these
aren't re-entrant ... so to be really safe, you'd have to check that
the SAL version is >= 3.2 before using the SAL_CALL_REENTRANT form.

-Tony

Re: SAL PCI config space

[Matthew Wilcox]

On Fri, Apr 30, 2004 at 11:04:44AM -0700, Luck, Tony wrote:
> >ia64_sal_pci_config_read() and ia64_sal_pci_config_write() use 
> >SAL_CALL(). This takes an IRQ-safe spinlock, but SAL 3.2 says that 
> >SAL_PCI_CONFIG_READ and SAL_PCI_CONFIG_WRITE are both MP-safe and
> >reentrant.  So is there any reason we shouldn't use SAL_CALL_REENTRANT?
> 
> Theoretically there might be older SAL implementations where these
> aren't re-entrant ... so to be really safe, you'd have to check that
> the SAL version is >= 3.2 before using the SAL_CALL_REENTRANT form.

Gack.  I went and dug up some older SAL specs.

SAL 3.1 is ambiguous.  On page 9-4, it says "Yes" under "Re-entrancy
requirement" for these calls, but then on 9-18 and 9-19, it says,
"Good programming practices dictate that indexed accesses to the
configuration space be serialized in order to be MP-safe."

(this seems rather looney to me.  how can you have something that's
reentrant but not MP-safe?  I think it's an unintended carry-over from
SAL 3.0)

SAL 3.0 states "SAL procedures are not re-entrant with respect to any
runtime service (including itself)."

So what's actually shipping in systems?  David's against runtime checks
due to some of the buggy firmware hp's managed to ship.  I agree with
him that it would have been much more sensible had the SAL spec specifid
new calls rather than changing the semantics of existing calls.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: SAL PCI config space

[Luck, Tony]

>So what's actually shipping in systems?  David's against runtime checks
>due to some of the buggy firmware hp's managed to ship.  I agree with
>him that it would have been much more sensible had the SAL 
>spec specifid new calls rather than changing the semantics of existing
calls.

It's not whats shipping that's the big problem ... it's the people
still running BigSur, Lion and other prototype systems that will
never have their firmware upgraded to SAL3.2 spec.

Apart from general cleanliness of not having functions that are
re-entrant and MP safe uselessly spin on a lock to serialize, do
you see any tangible benefits from switching these to use the
SAL_CALL_REENTRANT method?  Surely config space accesses aren't
in anyone's critical path?

-Tony

RE: SAL PCI config space

[John Lee]

Most of the SAL calls except the pure reading ones w/o affecting any
state have to be reentrant and MP safe, don't they?
Multiple OSs/executables on different Soft partitions do not share
others' lock variables, unless otherwise all SAL calls of soft
partitions are redirected to one hosting OS, which sounds not
reasonable.
So, SAL_CALL_REENTRANT could be a solution for the kernel itself, but
not for partitioned platforms.

Thanks,
John
 

-----Original Message-----
From: linux-ia64-owner@vger.kernel.org
[mailto:linux-ia64-owner@vger.kernel.org] On Behalf Of Luck, Tony
Sent: Friday, April 30, 2004 1:09 PM
To: Matthew Wilcox
Cc: linux-ia64@vger.kernel.org
Subject: RE: SAL PCI config space

>So what's actually shipping in systems?  David's against runtime checks
>due to some of the buggy firmware hp's managed to ship.  I agree with
>him that it would have been much more sensible had the SAL 
>spec specifid new calls rather than changing the semantics of existing
calls.

It's not whats shipping that's the big problem ... it's the people
still running BigSur, Lion and other prototype systems that will
never have their firmware upgraded to SAL3.2 spec.

Apart from general cleanliness of not having functions that are
re-entrant and MP safe uselessly spin on a lock to serialize, do
you see any tangible benefits from switching these to use the
SAL_CALL_REENTRANT method?  Surely config space accesses aren't
in anyone's critical path?

-Tony
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SAL PCI config space

[Grant Grundler]

On Fri, Apr 30, 2004 at 01:27:14PM -0700, John Lee wrote:
> Most of the SAL calls except the pure reading ones w/o affecting any
> state have to be reentrant and MP safe, don't they?
> Multiple OSs/executables on different Soft partitions do not share
> others' lock variables, unless otherwise all SAL calls of soft
> partitions are redirected to one hosting OS, which sounds not
> reasonable.

Yes. I think that's why the SAL spec changed because all the vendors
want to support virtual machines.
But that doesn't help the older boxes which do not implement
the newer SAL spec or have "non-compliant" implementations.

> So, SAL_CALL_REENTRANT could be a solution for the kernel itself, but
> not for partitioned platforms.

Sorry - this statement doesn't make sense to me.
You meant s/SAL_CALL_REENTRANT/spinlocks/ above maybe?

grant

> 
> Thanks,
> John
>  
> 
> -----Original Message-----
> From: linux-ia64-owner@vger.kernel.org
> [mailto:linux-ia64-owner@vger.kernel.org] On Behalf Of Luck, Tony
> Sent: Friday, April 30, 2004 1:09 PM
> To: Matthew Wilcox
> Cc: linux-ia64@vger.kernel.org
> Subject: RE: SAL PCI config space
> 
> >So what's actually shipping in systems?  David's against runtime checks
> >due to some of the buggy firmware hp's managed to ship.  I agree with
> >him that it would have been much more sensible had the SAL 
> >spec specifid new calls rather than changing the semantics of existing
> calls.
> 
> It's not whats shipping that's the big problem ... it's the people
> still running BigSur, Lion and other prototype systems that will
> never have their firmware upgraded to SAL3.2 spec.
> 
> Apart from general cleanliness of not having functions that are
> re-entrant and MP safe uselessly spin on a lock to serialize, do
> you see any tangible benefits from switching these to use the
> SAL_CALL_REENTRANT method?  Surely config space accesses aren't
> in anyone's critical path?
> 
> -Tony
> -
> To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: SAL PCI config space

[Luck, Tony]

>> Multiple OSs/executables on different Soft partitions do not share
>> others' lock variables, unless otherwise all SAL calls of soft
>> partitions are redirected to one hosting OS, which sounds not
>> reasonable.
>
>Yes. I think that's why the SAL spec changed because all the vendors
>want to support virtual machines.
>But that doesn't help the older boxes which do not implement
>the newer SAL spec or have "non-compliant" implementations.

There would be other issues besides whether some SAL calls
are re-entrant to running multiple OS in soft partitions. E.g.
MCA

-Tony

RE: SAL PCI config space

[John Lee]

> -----Original Message-----
> From: Grant Grundler [mailto:iod00d@hp.com] 
> Sent: Friday, April 30, 2004 2:46 PM
> To: John Lee
> Cc: Luck, Tony; Matthew Wilcox; linux-ia64@vger.kernel.org
> Subject: Re: SAL PCI config space
> 
> On Fri, Apr 30, 2004 at 01:27:14PM -0700, John Lee wrote:
> > Most of the SAL calls except the pure reading ones w/o affecting any
> > state have to be reentrant and MP safe, don't they?
> > Multiple OSs/executables on different Soft partitions do not share
> > others' lock variables, unless otherwise all SAL calls of soft
> > partitions are redirected to one hosting OS, which sounds not
> > reasonable.
> 
> Yes. I think that's why the SAL spec changed because all the vendors
> want to support virtual machines.

Right. So it's SAL implementers' responsibility to make them mp-safe.

> But that doesn't help the older boxes which do not implement
> the newer SAL spec or have "non-compliant" implementations.
> 
> > So, SAL_CALL_REENTRANT could be a solution for the kernel 
> itself, but
> > not for partitioned platforms.
> 
> Sorry - this statement doesn't make sense to me.
> You meant s/SAL_CALL_REENTRANT/spinlocks/ above maybe?
> 
> grant

For Linux's own backward compatibility with older machines, I'm with
Tony - SAL_CALL_REENTRANT -, until we all agree not to worry about them.

Thanks,
John

RE: SAL PCI config space

[John Lee]

> -----Original Message-----
> From: Luck, Tony [mailto:tony.luck@intel.com] 
> Sent: Friday, April 30, 2004 2:59 PM
> To: Grant Grundler; John Lee
> Cc: Matthew Wilcox; linux-ia64@vger.kernel.org
> Subject: RE: SAL PCI config space
> 
> >> Multiple OSs/executables on different Soft partitions do not share
> >> others' lock variables, unless otherwise all SAL calls of soft
> >> partitions are redirected to one hosting OS, which sounds not
> >> reasonable.
> >
> >Yes. I think that's why the SAL spec changed because all the vendors
> >want to support virtual machines.
> >But that doesn't help the older boxes which do not implement
> >the newer SAL spec or have "non-compliant" implementations.
> 
> There would be other issues besides whether some SAL calls
> are re-entrant to running multiple OS in soft partitions. E.g.
> MCA

I'm already there. RAS for soft partitions ;)
Any forum or discussion on that front would be helpful if you could
invite us.

Thanks,
John Lee