Expose ltr/obff interface by sysfs

Expose ltr/obff interface by sysfs

[Hao, Xudong]

Hi, 

I'm working on virtualization Xen/KVM. I saw there are ltr/obff enabling/disabling function in pci.c, but no called till now. I want to know if anybody(driver developer) are working for using it? Can driver change the LTR latency value dynamically?

/*
LTR(Latency tolerance reporting) allows devices to send messages to the root complex indicating their latency tolerance for snooped & unsnooped memory transactions.
OBFF (optimized buffer flush/fill), where supported, can help improve energy efficiency by giving devices information about when interrupts and other activity will have a reduced power impact.
*/

One way to control ltr/obff is used by driver, however, I'm considering that in virtualization, how guest OS driver control them. I have an idea that expose an inode interface by sysfs, like "reset" inode implemented in pci-sysfs.c, so that system user/administrator can enable/disable ltr/obff or set latency value on userspace, but not limited on driver. Comments?

< pls CC me when reply this mail, thanks >

Best Regards,
Xudong Hao


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel&#174; Ethernet, visit http://communities.intel.com/community/wired

Re: Expose ltr/obff interface by sysfs

[Jesse Barnes]

[-- Attachment #1: Type: text/plain, Size: 1673 bytes --]

On Fri, 6 Apr 2012 02:43:59 +0000
"Hao, Xudong" <xudong.hao@intel.com> wrote:

> Hi, 
> 
> I'm working on virtualization Xen/KVM. I saw there are ltr/obff enabling/disabling function in pci.c, but no called till now. I want to know if anybody(driver developer) are working for using it? Can driver change the LTR latency value dynamically?

I believe the value is writable, but I'd expect some devices to
misbehave if the value were programmed too high.  Performance would
also suffer if the value were set too high, at least for IOPS sensitive
devices.

> /*
> LTR(Latency tolerance reporting) allows devices to send messages to the root complex indicating their latency tolerance for snooped & unsnooped memory transactions.
> OBFF (optimized buffer flush/fill), where supported, can help improve energy efficiency by giving devices information about when interrupts and other activity will have a reduced power impact.
> */
> 
> One way to control ltr/obff is used by driver, however, I'm considering that in virtualization, how guest OS driver control them. I have an idea that expose an inode interface by sysfs, like "reset" inode implemented in pci-sysfs.c, so that system user/administrator can enable/disable ltr/obff or set latency value on userspace, but not limited on driver. Comments?

Given how device specific these extensions are, I'd expect you'd need
to know about each specific device anyway, which is why I think the
control belongs in the driver.  I don't see why you'd need to
enable/disable/change these functions when assigning a device from one
guest to another...

-- 
Jesse Barnes, Intel Open Source Technology Center

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Expose ltr/obff interface by sysfs

[Don Dutile]

On 04/06/2012 11:26 AM, Jesse Barnes wrote:
> On Fri, 6 Apr 2012 02:43:59 +0000
> "Hao, Xudong"<xudong.hao@intel.com>  wrote:
>
>> Hi,
>>
>> I'm working on virtualization Xen/KVM. I saw there are ltr/obff enabling/disabling function in pci.c, but no called till now. I want to know if anybody(driver developer) are working for using it? Can driver change the LTR latency value dynamically?
>
> I believe the value is writable, but I'd expect some devices to
> misbehave if the value were programmed too high.  Performance would
> also suffer if the value were set too high, at least for IOPS sensitive
> devices.
>
>> /*
>> LTR(Latency tolerance reporting) allows devices to send messages to the root complex indicating their latency tolerance for snooped&  unsnooped memory transactions.
>> OBFF (optimized buffer flush/fill), where supported, can help improve energy efficiency by giving devices information about when interrupts and other activity will have a reduced power impact.
>> */
>>
>> One way to control ltr/obff is used by driver, however, I'm considering that in virtualization, how guest OS driver control them. I have an idea that expose an inode interface by sysfs, like "reset" inode implemented in pci-sysfs.c, so that system user/administrator can enable/disable ltr/obff or set latency value on userspace, but not limited on driver. Comments?
>
> Given how device specific these extensions are, I'd expect you'd need
> to know about each specific device anyway, which is why I think the
> control belongs in the driver.  I don't see why you'd need to
> enable/disable/change these functions when assigning a device from one
> guest to another...
>

and looking at this code last week, I noticed the ltr, obff & ido
functions should be doing pcie_cap_has_*() checks before
accessing registers that may not exist, as is done in
pci_[save,restore]_pcie_state() ...

in drivers/pci/pci.c:
#define pcie_cap_has_devctl2(type, flags)               \
                 ((flags & PCI_EXP_FLAGS_VERS) > 1)
#define pcie_cap_has_lnkctl2(type, flags)               \
                 ((flags & PCI_EXP_FLAGS_VERS) > 1)
#define pcie_cap_has_sltctl2(type, flags)               \
                 ((flags & PCI_EXP_FLAGS_VERS) > 1)

Re: Expose ltr/obff interface by sysfs

[Konrad Rzeszutek Wilk]

On Fri, Apr 06, 2012 at 02:43:59AM +0000, Hao, Xudong wrote:
> Hi, 
> 
> I'm working on virtualization Xen/KVM. I saw there are ltr/obff enabling/disabling function in pci.c, but no called till now. I want to know if anybody(driver developer) are working for using it? Can driver change the LTR latency value dynamically?
> 
> /*
> LTR(Latency tolerance reporting) allows devices to send messages to the root complex indicating their latency tolerance for snooped & unsnooped memory transactions.
> OBFF (optimized buffer flush/fill), where supported, can help improve energy efficiency by giving devices information about when interrupts and other activity will have a reduced power impact.
> */
> 
> One way to control ltr/obff is used by driver, however, I'm considering that in virtualization, how guest OS driver control them. I have an idea that expose an inode interface by sysfs, like "reset" inode implemented in pci-sysfs.c, so that system user/administrator can enable/disable ltr/obff or set latency value on userspace, but not limited on driver. Comments?

So right now the driver inside the guest can probably see it, but can't change them.
(As those requests end up being filtered).

But there is nothing wrong with your changing those values from within the host.

But a better question is - why should this be done - especially from the guest which
has a limited view of the machine? The machine might be running a lot of other
requests so the OBFF inside the guest could be invalid.


> 
> < pls CC me when reply this mail, thanks >
> 
> Best Regards,
> Xudong Hao
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-pci" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel&#174; Ethernet, visit http://communities.intel.com/community/wired

RE: Expose ltr/obff interface by sysfs

[Hao, Xudong]

> But a better question is - why should this be done - especially from the guest
> which has a limited view of the machine? The machine might be running a lot of
> other requests so the OBFF inside the guest could be invalid.
>
Maybe I did not describe it clearly, the original idea is host admin can control the inode interface but not guest.

> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Saturday, April 07, 2012 2:40 AM
> To: Hao, Xudong
> Cc: linux-pci@vger.kernel.org; netdev@vger.kernel.org;
> e1000-devel@lists.sourceforge.net; Jesse Barnes
> Subject: Re: Expose ltr/obff interface by sysfs
> 
> On Fri, Apr 06, 2012 at 02:43:59AM +0000, Hao, Xudong wrote:
> > Hi,
> >
> > I'm working on virtualization Xen/KVM. I saw there are ltr/obff
> enabling/disabling function in pci.c, but no called till now. I want to know if
> anybody(driver developer) are working for using it? Can driver change the LTR
> latency value dynamically?
> >
> > /*
> > LTR(Latency tolerance reporting) allows devices to send messages to the root
> complex indicating their latency tolerance for snooped & unsnooped memory
> transactions.
> > OBFF (optimized buffer flush/fill), where supported, can help improve energy
> efficiency by giving devices information about when interrupts and other
> activity will have a reduced power impact.
> > */
> >
> > One way to control ltr/obff is used by driver, however, I'm considering that in
> virtualization, how guest OS driver control them. I have an idea that expose an
> inode interface by sysfs, like "reset" inode implemented in pci-sysfs.c, so that
> system user/administrator can enable/disable ltr/obff or set latency value on
> userspace, but not limited on driver. Comments?
> 
> So right now the driver inside the guest can probably see it, but can't change
> them.
> (As those requests end up being filtered).
> 
> But there is nothing wrong with your changing those values from within the
> host.
> 
> But a better question is - why should this be done - especially from the guest
> which has a limited view of the machine? The machine might be running a lot of
> other requests so the OBFF inside the guest could be invalid.
> 
> 
> >
> > < pls CC me when reply this mail, thanks >
> >
> > Best Regards,
> > Xudong Hao
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-pci"
> > in the body of a message to majordomo@vger.kernel.org More majordomo
> > info at  http://vger.kernel.org/majordomo-info.html