Re: [PATCH V4] PCI: Extend ACS configurability

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Jun 25, 2024 at 06:26:00PM +0200, Lukas Wunner wrote:
> On Tue, Jun 25, 2024 at 09:01:50PM +0530, Vidya Sagar wrote:
> > Add a kernel command-line option 'config_acs' to directly control all the
> > ACS bits for specific devices, which allows the operator to setup the
> > right level of isolation to achieve the desired P2P configuration.
> 
> An example wouldn't hurt, here and in kernel-parameters.txt.
> 
> 
> > ACS offers a range of security choices controlling how traffic is
> > allowed to go directly between two devices. Some popular choices:
> >   - Full prevention
> >   - Translated requests can be direct, with various options
> >   - Asymmetric direct traffic, A can reach B but not the reverse
> >   - All traffic can be direct
> > Along with some other less common ones for special topologies.
> 
> I'm wondering whether it would make more sense to let users choose
> between those "higher-level" options, instead of giving direct access
> to bits (and thus risking users to choose an incorrect setting).

It doesn't seem like the kernel has enough information to do that, or
at least describing enough information in the command line would be
more complex than this.

> Also, would it be possible to automatically change ACS settings
> when enabling or disabling P2PDMA?

No, as the commit said the ACS settings are required at early boot
before iommu_groups are formed. They cannot be changed dynamically
with today's kernel.

> The representation chosen here (as a command line option) seems
> questionable:
> 
> We're going to add more user-controllable options going forward.
> E.g., when introducing IDE, we'll have to let user space choose
> whether encryption should be enabled for certain PCIe devices.
> That's because encryption isn't for free, so can't be enabled
> opportunistically.  (The number of crypto engines on a CPU is
> limited and enabling encryption consumes energy.)

But that isn't part of ACS, so what is wrong with having ACS its own
configurable and other PCI functions can do what is appropriate for
them?

I do encourage people to avoid using the kernel command line. ACS is
forced into that because of the iommu_group issue.

> What about exposing such user configurable settings with sysctl?

I think sysctl is mostly deprecated in favour of sysfs.

An ide file in the sysfs to control the IDE stuff makes alot of sense
to me.

Jason