Re: [PATCH 0/8] Introduce fwctl subystem

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, Jun 11, 2024 at 05:36:17PM +0200, Daniel Vetter wrote:
> reliablity/health reporting. That's a lot more vendor specific in nature
> and needs to be customized anyway per deployement. And only much higher in
> the stack, maybe in k8s, can a technically reasonable unification even
> happen.  So again we're much more lenient about infrastructure enabling
> and uapi than stuff applications will use directly.

To be clear, this is the specific niche fwctl is for. It is not for
GPU command submission or something like that, and as I said to Jiri I
would agree to agressively block such abuses.
 
> Currently that's enough of a mess in drm that I feel like enforcing
> something like fwctl is still too much. But maybe once fwctl is
> established with other subsystems/devices we can start the conversations
> with vendors to get this going a few years down the road.

I wouldn't say enforcing, but instead of having every GPU driver build
their own weird vendor'd way to access their debug/diagnostic stuff
steer them into fwctl. These data center GPUs with FW at least have
lots of appropriate stuff and all the vendor OOT stuff has tooling to
inspect the GPUs far more than DRM has code for (ie
rocm-smi/nvidia-smi are have some features that are potentially good
candidates for fwctl)

> In practice, it doesn't seem to be an issue, at least not beyond the
> intentionally pragmatic choices where we merge kernel code with known
> sub-par/incomplete userspace. I'm not sure why, but to my knowledge all
> attempts to break the spirit of our userspace rules while following the
> letter die in vendor-internal discussions, at least for all the
> established upstream driver teams.

I think the same is broadly true of RDMA as well, except we don't
bother with the kernel trying to police the command stream - direct
submission from userspace. I can't say it has been much of an issue.

> tldr; fwctl as I understand it feels like a bridge to far for drm today,
> but I'd very much like someone else to make this happen so we could
> eventually push towards adoption too.

Hahah, okay, well, I'm pushing :)

Jason