Endpoint-to-Endpoint communication - Handshake Problem Enumeration

[Barry Grussling <barry@grussling.com>]

Hello Everyone,

I am working on a device with a PLX PCIe switch and multiple
PCIe endpoints enumerated off an ARM RP.  I am playing with
endpoint-to-endpoint communications.  I have verified after
the PCIe bus is enumerated that I can send TLPs from
one endpoint to an enumerated BAR on another endpoint
and the TLP is directed to the other endpoint.  I have further
verified that the RP receivies nothing as desired.

The issue I am looking for some advice on is how to tell
each endpoint what all the other endpoints BAR address is.
For the sake of argument, lets say I have N modules and
each module needs to know the enumerated endpoint
address of the other N-1 modules Bar0 address.

The brute force/prototype method I have been testing is
scrape /proc/iomem after enumeration and write to each
card's registers via /dev/mem the address of all other
cards.  This works, but is not something I would put
on my resume.  I am now trying to "do it right".  This
scraping also assumes bus addresses and host addresses
are the same.  This seems to be true on my architecture.

My preliminary driver design has a common base layer
that each endpoint's driver registers with when it is probed.
After the BAR addresses are assigned the endpoint driver
tells the common base layer its assigned address and
the base layer writes that into registers in all the
other registered endpoints via their respective drivers.

Before I go off boldly and and completely re-invent the
wheel I was wondering if something like this had
been contemplated or created yet, perhaps via
callback hooks or some other PCIe framework
thingy I am not aware of.  Or is there another way
to achieve the same goal?

Anyone have any feedback?

Thanks for your time!

Barry