Re: PCI userland access non-mmap APIs, kernel access to legacy space

[Jesse Barnes <jesse.barnes@intel.com>]

On Sunday, June 17, 2007 7:16:44 Benjamin Herrenschmidt wrote:
> Hi folks !
>
> Before I start actually writing code, I want to do a quick brain dump
> here to see if there are any objections.
>
> So a problem we have is that on non-x86, we expect to be able to
> access PCI IO space via mmap'ing of sysfs based "resource" files.
>
> However, there are various PCI host bridges out there that don't make
> that a practical solution. So far, I have two cases on ppc of bridges
> that either use indirect access for the IO space or require specific
> HW workarounds on accesses that involve spinlocks (to properly deal
> with the non-posted aspect of IO transactions). In addition, Lennert
> tells me that similar issues exist on ARM.
>
> Thus, I would like to introduce an additional read/write interface to
> sysfs "resource" files (and then get Ian's libpciaccess use that when
> mmap is unavailable).
>
> In fact, it's funny in a way becauase the "legacy" files interface is
> already read/write based (exclusively even, mmap isn't supported) for
> legacy IOs.
>
> I beleive that doing so is better than trying to extend /dev/port to
> understand multiple domains, >64K IO space, etc... etc...., because
> you don't have to understand all the remapping tricks that the
> operating system might be doing behind the scene, just open the
> resource file (or the legacy file of that bus for legacy accesses)
> and read/write to it.
>
> Any objection ?

Sounds like a good idea.  Originally, the "legacy_io" file was 
implemented this way because I knew of machines that couldn't properly 
handle it as a mmappable interface.  But when I pinged people about the 
resource files, I didn't hear anything, so they were just done with 
mmap.

But adding read/write is fine with me.  In fact, having read/write hooks 
for the memory backed resource files is also nice, since it makes 
dealing with them on the command line a bit easier.

> Also, while at it, I would like to introduce a pair of in-kernel
> interfaces:

What's the other half of the pair?

> int pci_translate_legacy_resource(struct pci_bus *bus, struct
> resource *res);
>
> Which takes a resource containing an absolute legacy range on that
> bus (for example, an IO resource with the VGA port numbers, or a
> memory resource with the VGA text mode aperture). That resource is
> then "converted" for the given PCI bus. For x86, that function would
> boil down to "return 0;".
>
> The reason is that on multiple domains machines, it's actually not
> trivial to figure out where the legacy space of a given domain is
> mapped, if available at all. The above function is a way to solve
> that problem, allowing things like vgacon to work in multi-domains
> setup without half of the current arch specific hacks in there.
>
> That would be also allowed to fail (some platforms can't give access
> to legacy memory or legacy IOs at all).
>
> We could probably even implement the pci sysfs "legacy" interface on
> top of these as well, thus making it generic to all platforms.
>
> Any objection there too ?

Yeah, sounds like a good additional abstraction for that interface.  
Should make porting it to other arches easier (I talked with kyle about 
this last week, so maybe this'll give him motivation to do it for 
parisc).

Jesse