[igt-dev] [RFC] IGT device scanning and selection

[igt-dev] [RFC] IGT device scanning and selection

[Kempczynski, Zbigniew]

Hi all.

IGT device management is a little bit old. More than one GPU on the machine 
are available on the market from years so there's time to propose new device
management in IGT.

Selection using bus-device or functional specifiers.
---
1. bus-device selection
   Current devices are mostly PCI-Express devices, but we should be able to
   open to specify device using any bus selection. For example:
   --device PCI:00:02.0
   would select PCI device on 00 bus, 02 device and function 0.

   --device USB:002:003
   would select device 003 on bus 002.
  
   Future buses can be specified by extending --device argument parsing.

2. Functional selection
   CI environments shouldn't rely on bus-device. Same devices (GPUs) may be 
   inserted into different slots so functional selection is required. 
   
   Let's imagine machine have 5 cards:
   00:02:0 - VendorX, product: A
   00:03:0 - VendorY, product: B
   00:04:0 - VendorX, product: A
   00:05:0 - VendorY, product: C
   00:06:0 - VendorX, product: D

   We should be able to select same device regardless order on the bus.
   For example:
   --device VendorX:A 
   --device VendorX:A:0 
   will select first A card of VendorX

   --device VendorY:A:1
   will select second A card of VendorX

   If vendor would like to have its own parsing method for the product
   overriding parsing will be possible. For example:
   --device VendorX:GPU-Card:0
   VendorX must then register its own function to handle 'GPU-Card:0'
   string on his own. 

===

Device management
---
New device management should allow user to see current devices. 
I propose IGT tool called 'lsgpu' which would discover devices in 
any registered bus topologies scanning methods.

---
Best,
Zbigniew Kempczyński





_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Ser, Simon]

On Tue, 2019-06-04 at 08:09 +0000, Kempczynski, Zbigniew wrote:
> Hi all.
> 
> IGT device management is a little bit old. More than one GPU on the machine 
> are available on the market from years so there's time to propose new device
> management in IGT.

Nice! There is definitely room for improvement here, thanks for working
on this!

I think we already have an env variable for (some of) this:
IGT_FORCE_DRIVER. It's not great though.

> Selection using bus-device or functional specifiers.
> ---
> 1. bus-device selection
>    Current devices are mostly PCI-Express devices, but we should be able to
>    open to specify device using any bus selection. For example:
>    --device PCI:00:02.0
>    would select PCI device on 00 bus, 02 device and function 0.
> 
>    --device USB:002:003
>    would select device 003 on bus 002.

Why not specify the DRM node path, instead of this?

>    Future buses can be specified by extending --device argument parsing.

Some devices like the Raspberry Pi use platform buses. vc4 uses IGT, so
it would be nice to support that.

> 2. Functional selection
>    CI environments shouldn't rely on bus-device. Same devices (GPUs) may be 
>    inserted into different slots so functional selection is required. 
>    
>    Let's imagine machine have 5 cards:
>    00:02:0 - VendorX, product: A
>    00:03:0 - VendorY, product: B
>    00:04:0 - VendorX, product: A
>    00:05:0 - VendorY, product: C
>    00:06:0 - VendorX, product: D
> 
>    We should be able to select same device regardless order on the bus.
>    For example:
>    --device VendorX:A 
>    --device VendorX:A:0 
>    will select first A card of VendorX
> 
>    --device VendorY:A:1
>    will select second A card of VendorX

Hmm. What defines the order between the "first" and the "second" one?
This doesn't seem like it's "regardless order on the bus" to me? Am I
missing something?

>    If vendor would like to have its own parsing method for the product
>    overriding parsing will be possible. For example:
>    --device VendorX:GPU-Card:0
>    VendorX must then register its own function to handle 'GPU-Card:0'
>    string on his own. 

What is the use-case for this?

> 
> ===
> 
> Device management
> ---
> New device management should allow user to see current devices. 
> I propose IGT tool called 'lsgpu' which would discover devices in 
> any registered bus topologies scanning methods.

I believe we can do everything with libdrm's drmGetDevice.

Note that some utilities (e.g. https://github.com/ascent12/drm_info)
already expose this functionality to some extend.
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Arkadiusz Hiler]

On Tue, Jun 04, 2019 at 08:09:55AM +0000, Kempczynski, Zbigniew wrote:
> Hi all.
> 
> IGT device management is a little bit old. More than one GPU on the machine 
> are available on the market from years so there's time to propose new device
> management in IGT.
> 
> Selection using bus-device or functional specifiers.
> ---
> 1. bus-device selection
>    Current devices are mostly PCI-Express devices, but we should be able to
>    open to specify device using any bus selection. For example:
>    --device PCI:00:02.0
>    would select PCI device on 00 bus, 02 device and function 0.
> 
>    --device USB:002:003
>    would select device 003 on bus 002.
>   
>    Future buses can be specified by extending --device argument parsing.
> 
> 2. Functional selection
>    CI environments shouldn't rely on bus-device. Same devices (GPUs) may be 
>    inserted into different slots so functional selection is required. 
>    
>    Let's imagine machine have 5 cards:
>    00:02:0 - VendorX, product: A
>    00:03:0 - VendorY, product: B
>    00:04:0 - VendorX, product: A
>    00:05:0 - VendorY, product: C
>    00:06:0 - VendorX, product: D
> 
>    We should be able to select same device regardless order on the bus.
>    For example:
>    --device VendorX:A 
>    --device VendorX:A:0 
>    will select first A card of VendorX
> 
>    --device VendorY:A:1
>    will select second A card of VendorX
> 
>    If vendor would like to have its own parsing method for the product
>    overriding parsing will be possible. For example:
>    --device VendorX:GPU-Card:0
>    VendorX must then register its own function to handle 'GPU-Card:0'
>    string on his own. 
> 
> ===
> 
> Device management
> ---
> New device management should allow user to see current devices. 
> I propose IGT tool called 'lsgpu' which would discover devices in 
> any registered bus topologies scanning methods.

Hey,

Seems like a well-written verbalization of some of the ideas that were
floating around since XDC. Thanks!


Couple of points that were not captured here:

1. We should have an option to set device via .igtrc and IGT_DEVICE env
variable too. ENV variable would take precedence over .igtrc, and
--device switch would take precedence over both of the other methods.

That would help a lot with both automatization and having some defaults
for local development.


2. What to do with drm_open_driver(DRIVER_INTEL) et al?

Simplest solution would be to just skip if we are --device-ing a card
that does not meet those "constraints", but we may want to rework those
a bit?

3. VKMS and VGEM?


I have also CCed people that may be interested in running IGT with
multiple GPUs and may something to say.

Cheers,
Arek
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Kempczynski, Zbigniew]

On Tue, 2019-06-04 at 11:36 +0300, Arkadiusz Hiler wrote:
> 
> Hey,
> 
> Seems like a well-written verbalization of some of the ideas that were
> floating around since XDC. Thanks!
> 
> 
> Couple of points that were not captured here:
> 
> 1. We should have an option to set device via .igtrc and IGT_DEVICE env
> variable too. ENV variable would take precedence over .igtrc, and
> --device switch would take precedence over both of the other methods.
> 
> That would help a lot with both automatization and having some defaults
> for local development.

Acked. I'll follow this override path. 

> 
> 
> 2. What to do with drm_open_driver(DRIVER_INTEL) et al?
> 
> Simplest solution would be to just skip if we are --device-ing a card
> that does not meet those "constraints", but we may want to rework those
> a bit?
> 

IMHO --device specification should override this vendor requirement
constraint. Using globals is not pretty solution but allows easily 
adds new device selection without rewritting all tests.

Best,
Zbigniew
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Michał Winiarski]

On Tue, Jun 04, 2019 at 08:29:42AM +0000, Ser, Simon wrote:
> On Tue, 2019-06-04 at 08:09 +0000, Kempczynski, Zbigniew wrote:
> > Hi all.
> > 
> > IGT device management is a little bit old. More than one GPU on the machine 
> > are available on the market from years so there's time to propose new device
> > management in IGT.
> 
> Nice! There is definitely room for improvement here, thanks for working
> on this!
> 
> I think we already have an env variable for (some of) this:
> IGT_FORCE_DRIVER. It's not great though.
> 
> > Selection using bus-device or functional specifiers.
> > ---
> > 1. bus-device selection
> >    Current devices are mostly PCI-Express devices, but we should be able to
> >    open to specify device using any bus selection. For example:
> >    --device PCI:00:02.0
> >    would select PCI device on 00 bus, 02 device and function 0.
> > 
> >    --device USB:002:003
> >    would select device 003 on bus 002.
> 
> Why not specify the DRM node path, instead of this?

To avoid coupling DRM node with device. Even today you can run IGT without any
DRM modules loaded (IGT is going to load the modules on an attempt to open DRM
node).

As a bonus, if we follow that convention while refactoring in IGT, writing tests
for module-reload / device-unbind / device-remove will become slightly less
convoluted.

-Michał

> 
> >    Future buses can be specified by extending --device argument parsing.
> 
> Some devices like the Raspberry Pi use platform buses. vc4 uses IGT, so
> it would be nice to support that.
> 
> > 2. Functional selection
> >    CI environments shouldn't rely on bus-device. Same devices (GPUs) may be 
> >    inserted into different slots so functional selection is required. 
> >    
> >    Let's imagine machine have 5 cards:
> >    00:02:0 - VendorX, product: A
> >    00:03:0 - VendorY, product: B
> >    00:04:0 - VendorX, product: A
> >    00:05:0 - VendorY, product: C
> >    00:06:0 - VendorX, product: D
> > 
> >    We should be able to select same device regardless order on the bus.
> >    For example:
> >    --device VendorX:A 
> >    --device VendorX:A:0 
> >    will select first A card of VendorX
> > 
> >    --device VendorY:A:1
> >    will select second A card of VendorX
> 
> Hmm. What defines the order between the "first" and the "second" one?
> This doesn't seem like it's "regardless order on the bus" to me? Am I
> missing something?
> 
> >    If vendor would like to have its own parsing method for the product
> >    overriding parsing will be possible. For example:
> >    --device VendorX:GPU-Card:0
> >    VendorX must then register its own function to handle 'GPU-Card:0'
> >    string on his own. 
> 
> What is the use-case for this?
> 
> > 
> > ===
> > 
> > Device management
> > ---
> > New device management should allow user to see current devices. 
> > I propose IGT tool called 'lsgpu' which would discover devices in 
> > any registered bus topologies scanning methods.
> 
> I believe we can do everything with libdrm's drmGetDevice.
> 
> Note that some utilities (e.g. https://github.com/ascent12/drm_info)
> already expose this functionality to some extend.
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Daniel Vetter]

On Tue, Jun 04, 2019 at 08:47:12AM +0000, Kempczynski, Zbigniew wrote:
> On Tue, 2019-06-04 at 11:36 +0300, Arkadiusz Hiler wrote:
> > 
> > Hey,
> > 
> > Seems like a well-written verbalization of some of the ideas that were
> > floating around since XDC. Thanks!
> > 
> > 
> > Couple of points that were not captured here:
> > 
> > 1. We should have an option to set device via .igtrc and IGT_DEVICE env
> > variable too. ENV variable would take precedence over .igtrc, and
> > --device switch would take precedence over both of the other methods.
> > 
> > That would help a lot with both automatization and having some defaults
> > for local development.
> 
> Acked. I'll follow this override path. 
> 
> > 
> > 
> > 2. What to do with drm_open_driver(DRIVER_INTEL) et al?
> > 
> > Simplest solution would be to just skip if we are --device-ing a card
> > that does not meet those "constraints", but we may want to rework those
> > a bit?
> > 
> 
> IMHO --device specification should override this vendor requirement
> constraint. Using globals is not pretty solution but allows easily 
> adds new device selection without rewritting all tests.

We still need to follow DRIVER_INTEL/DRIVER_AMD constraints. You can't run
an i915 gem tests on amdgpu or the other way round. So allowing these
options to overwrite these test constraints sounds like a bad idea to me.
Also, we can't blindly skip either, because some tests want to have an
intel and an amdgpu (e.g. kbl-g), and then tests dma-buf sharing between
the two.

So this all gets a bit tricky, and it's the reason why have the current
somewhat silly device selection logic. I think the only option for these
is that they would indicate the preferred device, but we'll pick another
one if that is present and fullfills the criteria. At least as an initial
step.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Daniel Vetter]

On Tue, Jun 04, 2019 at 08:09:55AM +0000, Kempczynski, Zbigniew wrote:
> Hi all.
> 
> IGT device management is a little bit old. More than one GPU on the machine 
> are available on the market from years so there's time to propose new device
> management in IGT.
> 
> Selection using bus-device or functional specifiers.
> ---
> 1. bus-device selection
>    Current devices are mostly PCI-Express devices, but we should be able to
>    open to specify device using any bus selection. For example:
>    --device PCI:00:02.0
>    would select PCI device on 00 bus, 02 device and function 0.
> 
>    --device USB:002:003
>    would select device 003 on bus 002.
>   
>    Future buses can be specified by extending --device argument parsing.

Please note that not everything is a pci device, and drm and igt work on
hw without any pci devices. So if you want to specify a device, I think it
needs to be the full sysfs path. Otherwise we'll write yet another custom
device id -> sysfs path parser, and libdrm already has a few of those and
they're all terrible :-)

Full sysfs path should be fairly easy to compare with drm nodes by chasing
the the sysfs device link. For simplicity I guess we could allow substring
matching, so for pci devices you wouldn't have to give the full path
(which is usually really long).
 
> 2. Functional selection
>    CI environments shouldn't rely on bus-device. Same devices (GPUs) may be 
>    inserted into different slots so functional selection is required. 
>    
>    Let's imagine machine have 5 cards:
>    00:02:0 - VendorX, product: A
>    00:03:0 - VendorY, product: B
>    00:04:0 - VendorX, product: A
>    00:05:0 - VendorY, product: C
>    00:06:0 - VendorX, product: D
> 
>    We should be able to select same device regardless order on the bus.
>    For example:
>    --device VendorX:A 
>    --device VendorX:A:0 
>    will select first A card of VendorX
> 
>    --device VendorY:A:1
>    will select second A card of VendorX
> 
>    If vendor would like to have its own parsing method for the product
>    overriding parsing will be possible. For example:
>    --device VendorX:GPU-Card:0
>    VendorX must then register its own function to handle 'GPU-Card:0'
>    string on his own. 

Udev has this idea of matching device properties. I think we should use
that, not reinvent our own wheel here. udev can give you a list of
property=value pairs for a given device, we could just again match again
those blindly. No new device parsing logic in libdrm or igt please :-)
-Daniel

> 
> ===
> 
> Device management
> ---
> New device management should allow user to see current devices. 
> I propose IGT tool called 'lsgpu' which would discover devices in 
> any registered bus topologies scanning methods.
> 
> ---
> Best,
> Zbigniew Kempczyński
> 
> 
> 
> 
> 
> _______________________________________________
> igt-dev mailing list
> igt-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/igt-dev

-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Re: [igt-dev] [RFC] IGT device scanning and selection

[Kempczynski, Zbigniew]

On Wed, 2019-06-05 at 14:35 +0200, Daniel Vetter wrote:

<cut>
> Udev has this idea of matching device properties. I think we should use
> that, not reinvent our own wheel here. udev can give you a list of
> property=value pairs for a given device, we could just again match again
> those blindly. No new device parsing logic in libdrm or igt please :-)

I've taken a look to libudev, it looks good and matches my needs. I will send
patches with device management proposals when they will be ready. Thanks for
hints.

Zbigniew
_______________________________________________
igt-dev mailing list
igt-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/igt-dev