Accelerator/GPU top

Accelerator/GPU top

[Alex Deucher]

Has anyone given any thought on how to support something like top for
accelerators or GPUs?  We have fdinfo, but using fdinfo requires extra
privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
way to even discover what processes are using the GPU.  There is the
clients list in debugfs, but that is also admin only.  Tools like ps
and top use /proc/<pid>/stat and statm.  Do you think there would be
an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
This would duplicate much of what is in fdinfo, but would be available
to regular users.

Thanks,

Alex

Re: Accelerator/GPU top

[Gregory Carter]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

There are all kinds of accelerators out there for various sorts of things.
It would be a shame if you chose an approach just for GPU's.

On Thu, Feb 5, 2026 at 11:58 AM Alex Deucher <alexdeucher@gmail.com> wrote:

> Has anyone given any thought on how to support something like top for
> accelerators or GPUs?  We have fdinfo, but using fdinfo requires extra
> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
> way to even discover what processes are using the GPU.  There is the
> clients list in debugfs, but that is also admin only.  Tools like ps
> and top use /proc/<pid>/stat and statm.  Do you think there would be
> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
> This would duplicate much of what is in fdinfo, but would be available
> to regular users.
>
> Thanks,
>
> Alex
>

[-- Attachment #2: Type: text/html, Size: 1206 bytes --]

Re: Accelerator/GPU top

[Natalie Vock]

On 2/5/26 19:58, Alex Deucher wrote:
> Has anyone given any thought on how to support something like top for
> accelerators or GPUs?

top for accelerators/GPUs kind of exists already, see [1] or [2].
Clearly, this problem has some kind of solution (looking through the 
code, it seems like they check every fd if it has a DRM fdinfo file 
associated (which is indeed not particularly efficient)).

Maybe it's worth asking the authors of the respective tools for their 
opinions here?

Natalie

[1] https://github.com/Umio-Yasuno/amdgpu_top
[2] https://github.com/Syllo/nvtop
> We have fdinfo, but using fdinfo requires extra
> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
> way to even discover what processes are using the GPU.  There is the
> clients list in debugfs, but that is also admin only.  Tools like ps
> and top use /proc/<pid>/stat and statm.  Do you think there would be
> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
> This would duplicate much of what is in fdinfo, but would be available
> to regular users.
> 
> Thanks,
> 
> Alex

Re: Accelerator/GPU top

[Alex Deucher]

On Thu, Feb 5, 2026 at 2:25 PM Natalie Vock <natalie.vock@gmx.de> wrote:
>
> On 2/5/26 19:58, Alex Deucher wrote:
> > Has anyone given any thought on how to support something like top for
> > accelerators or GPUs?
>
> top for accelerators/GPUs kind of exists already, see [1] or [2].
> Clearly, this problem has some kind of solution (looking through the
> code, it seems like they check every fd if it has a DRM fdinfo file
> associated (which is indeed not particularly efficient)).
>
> Maybe it's worth asking the authors of the respective tools for their
> opinions here?

These tools use the existing interfaces which are available, but
require admin privileges to display the per process data for all
processes using the accelerators.  That's the part I'm interested in
solving.  For example the drm client list is admin only in debugfs.
Could we add something in sysfs which anyone could read to get a list
of processes using the GPU?  Presumably this was done for
privacy/security reasons, but as accelerators become more general
purpose, I think it's reasonable for users to see how loaded they are
similar to the CPU.

Alex

>
> Natalie
>
> [1] https://github.com/Umio-Yasuno/amdgpu_top
> [2] https://github.com/Syllo/nvtop
> > We have fdinfo, but using fdinfo requires extra
> > privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
> > way to even discover what processes are using the GPU.  There is the
> > clients list in debugfs, but that is also admin only.  Tools like ps
> > and top use /proc/<pid>/stat and statm.  Do you think there would be
> > an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
> > This would duplicate much of what is in fdinfo, but would be available
> > to regular users.
> >
> > Thanks,
> >
> > Alex
>

Re: Accelerator/GPU top

[Christian König]

On 2/5/26 20:25, Natalie Vock wrote:
> On 2/5/26 19:58, Alex Deucher wrote:
>> Has anyone given any thought on how to support something like top for
>> accelerators or GPUs?
> 
> top for accelerators/GPUs kind of exists already, see [1] or [2].
> Clearly, this problem has some kind of solution (looking through the code, it seems like they check every fd if it has a DRM fdinfo file associated (which is indeed not particularly efficient)).
> 
> Maybe it's worth asking the authors of the respective tools for their opinions here?

That is a really good point. Adding Maxime Schmitt and Umio Yasuno on CC.

Let's hope I've picked the correct mail addresses.

Christian.

> 
> Natalie
> 
> [1] https://github.com/Umio-Yasuno/amdgpu_top
> [2] https://github.com/Syllo/nvtop
>> We have fdinfo, but using fdinfo requires extra
>> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
>> way to even discover what processes are using the GPU.  There is the
>> clients list in debugfs, but that is also admin only.  Tools like ps
>> and top use /proc/<pid>/stat and statm.  Do you think there would be
>> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
>> This would duplicate much of what is in fdinfo, but would be available
>> to regular users.
>>
>> Thanks,
>>
>> Alex
> 

Re: Accelerator/GPU top

[Maxime Schmitt]

Hello,

Indeed it’s not efficient but that’s the only exposed way we have to retrieve the information at this point (for drm drivers exposing metrics through fdinfo).
It doesn’t require any extra privileges if one is only interested in the user owned processes (which is most use cases as far as I know).

I think that there was a discussion at some years ago from Intel on the kernel mailing list to expose the process list using drm. 
Maybe having a way to query which processes (and file descriptor) use drm and then only going through the relevant fdinfo files would alleviate most of the inefficiencies (modulo privilege problems for other user processes).

Also I haven’t implemented use cases where the GPU doesn’t use DRM, e.g., AMD CDNA that is going through rocm (I don’t have access to a card to test it).
I also noticed some instances, where the driver doesn’t report any usage, although it’s actually using the encode/decode engine through VA-API with a DRM handle.

Cheers,
Maxime

> On 6 Feb 2026, at 14:38, Christian König <christian.koenig@amd.com> wrote:
> 
> On 2/5/26 20:25, Natalie Vock wrote:
>>> On 2/5/26 19:58, Alex Deucher wrote:
>>> Has anyone given any thought on how to support something like top for
>>> accelerators or GPUs?
>> 
>> top for accelerators/GPUs kind of exists already, see [1] or [2].
>> Clearly, this problem has some kind of solution (looking through the code, it seems like they check every fd if it has a DRM fdinfo file associated (which is indeed not particularly efficient)).
>> 
>> Maybe it's worth asking the authors of the respective tools for their opinions here?
> 
> That is a really good point. Adding Maxime Schmitt and Umio Yasuno on CC.
> 
> Let's hope I've picked the correct mail addresses.
> 
> Christian.
> 
>> 
>> Natalie
>> 
>> [1] https://github.com/Umio-Yasuno/amdgpu_top
>> [2] https://github.com/Syllo/nvtop
>>> We have fdinfo, but using fdinfo requires extra
>>> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
>>> way to even discover what processes are using the GPU.  There is the
>>> clients list in debugfs, but that is also admin only.  Tools like ps
>>> and top use /proc/<pid>/stat and statm.  Do you think there would be
>>> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
>>> This would duplicate much of what is in fdinfo, but would be available
>>> to regular users.
>>> 
>>> Thanks,
>>> 
>>> Alex
>> 
> 

Re: Accelerator/GPU top

[Umio-Yasuno - coelacanth_dream]

Hello,

Regarding the proposed interface for gathering GPU driver statistics, I would like to provide feedback based on the following points:

​1.
Providing per-process GPU information through an interface other than /proc would significantly improve the developer experience for Flatpak-based applications and tools. Since Flatpak containers have a restricted view of the host's /proc, it is currently very difficult for sandboxed monitoring tools to gather cross-process GPU metrics.

​2.
The current reliance on /proc/<pid>/{fd,fdinfo} requires root privileges to access info for other users' processes. This is a major hurdle for non-root users when attempting to detect system-critical issues, such as VRAM leaks in a compositor. 

​3.
While ROCm/amdkfd currently provides per-process VRAM usage, it lacks an interface to report the utilization of hardware engines such as Compute or SDMA. It would be highly beneficial if the new interface could address this gap, ensuring that hardware IP utilization is consistently trackable across both KFD and DRM nodes.
​
​Note: I've used Gemini to help structure my thoughts and refine the English in this mail.

Best regards,
Umio Yasuno

>
>
> On 2/5/26 20:25, Natalie Vock wrote:
>
> > On 2/5/26 19:58, Alex Deucher wrote:
> >
> > > Has anyone given any thought on how to support something like top for
> > > accelerators or GPUs?
> >
> > top for accelerators/GPUs kind of exists already, see [1] or [2].
> > Clearly, this problem has some kind of solution (looking through the code, it seems like they check every fd if it has a DRM fdinfo file associated (which is indeed not particularly efficient)).
> >
> > Maybe it's worth asking the authors of the respective tools for their opinions here?
>
>
> That is a really good point. Adding Maxime Schmitt and Umio Yasuno on CC.
>
> Let's hope I've picked the correct mail addresses.
>
> Christian.
>
> > Natalie
> >
> > [1] https://github.com/Umio-Yasuno/amdgpu_top
> > [2] https://github.com/Syllo/nvtop
> >
> > > We have fdinfo, but using fdinfo requires extra
> > > privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
> > > way to even discover what processes are using the GPU. There is the
> > > clients list in debugfs, but that is also admin only. Tools like ps
> > > and top use /proc/<pid>/stat and statm. Do you think there would be
> > > an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
> > > This would duplicate much of what is in fdinfo, but would be available
> > > to regular users.
> > >
> > > Thanks,
> > >
> > > Alex

Re: Accelerator/GPU top

[Tvrtko Ursulin]

On 05/02/2026 18:58, Alex Deucher wrote:
> Has anyone given any thought on how to support something like top for
> accelerators or GPUs?  We have fdinfo, but using fdinfo requires extra
> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
> way to even discover what processes are using the GPU.  There is the
> clients list in debugfs, but that is also admin only.  Tools like ps
> and top use /proc/<pid>/stat and statm.  Do you think there would be
> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
> This would duplicate much of what is in fdinfo, but would be available
> to regular users.

In short yes, the inefficiency of the client discovery bugged me since 
the start of fdinfo and it was discussed a few times how to improve it.

AFAIR in those discussions a lists of clients outside of debugfs was 
mentioned, since that would allow the cost of discovery to not scale by 
the number of irrelevant processes. Something in proc was also mentioned 
but I guess for both no one had a strong enough drive to actually do it.

Challenge will be finding the threads to see if there were any 
interesting conclusions... The only one I can remember at the moment was 
this:

https://lore.kernel.org/dri-devel/20240403182951.724488-1-adrian.larumbe@collabora.com/

It looks this attempt died out due proposing a binary interface in sysfs.

In any case, I'd say it makes sense to do something.

Regards,

Tvrtko

P.S.
Just in a case you did not see it, a generic tool exists as gputop in 
IGT but as you say, you can only see data for processes the user running 
the tool can read fdinfo stats of. And there some nicer tools feeding of 
the same data.

Re: Accelerator/GPU top

[Christian König]

On 2/6/26 10:58, Tvrtko Ursulin wrote:
> 
> On 05/02/2026 18:58, Alex Deucher wrote:
>> Has anyone given any thought on how to support something like top for
>> accelerators or GPUs?  We have fdinfo, but using fdinfo requires extra
>> privileges (CAP_SYS_PTRACE) and there is not a particularly efficient
>> way to even discover what processes are using the GPU.  There is the
>> clients list in debugfs, but that is also admin only.  Tools like ps
>> and top use /proc/<pid>/stat and statm.  Do you think there would be
>> an appetite for something like /proc/<pid>/drm/stat, statm, etc.?
>> This would duplicate much of what is in fdinfo, but would be available
>> to regular users.
> 
> In short yes, the inefficiency of the client discovery bugged me since the start of fdinfo and it was discussed a few times how to improve it.
> 
> AFAIR in those discussions a lists of clients outside of debugfs was mentioned, since that would allow the cost of discovery to not scale by the number of irrelevant processes. Something in proc was also mentioned but I guess for both no one had a strong enough drive to actually do it.

The fundamental problem is that Linux only tracks in one direction who has which file open, but not the reverse. In other words even an lsof /dev/dri/render* does nothing else than going over /proc and seeing who has opened what.

We can expose the list of open DRM connections and which process was the last user of each in sysfs (for example), but that is probably not accurate and might also expose information not every process should know about.

Regards,
Christian.

> 
> Challenge will be finding the threads to see if there were any interesting conclusions... The only one I can remember at the moment was this:
> 
> https://lore.kernel.org/dri-devel/20240403182951.724488-1-adrian.larumbe@collabora.com/
> 
> It looks this attempt died out due proposing a binary interface in sysfs.
> 
> In any case, I'd say it makes sense to do something.
> 
> Regards,
> 
> Tvrtko
> 
> P.S.
> Just in a case you did not see it, a generic tool exists as gputop in IGT but as you say, you can only see data for processes the user running the tool can read fdinfo stats of. And there some nicer tools feeding of the same data.
>