Re: [PATCH V4 0/5] mlx5 ConnectX control misc driver

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Apr 04, 2024 at 12:53:39PM -0700, Jakub Kicinski wrote:
> On Thu, 4 Apr 2024 15:33:05 -0300 Jason Gunthorpe wrote:
> > Uh no, mlx5 already has an excellent in-tree driver, thank you very
> > much. The configuration is not changing the driver, it is changing the
> > device.
> > 
> > Consider, I can ship unique devices pre-configured for each site's
> > special needs. They still work with the same inbox driver.
> > 
> > In fact that happens already for certain large customers. It is why
> > Jakub recently pointed out that Meta doesn't need any
> > provisioning/mlx5ctl/misc driver. They just buy different devices than
> > everyone else.
> 
> > Further it is really rude and inappropriate to say that some customers
> > should not be able to enjoy intree drivers because of your aesthetic
> > opinion of hacks in a device's design.
> 
> To my knowledge the "customizations" are mostly around fitting into OCP
> servers.

Nope. I understand it is significant. If Meta had to work with a COTS
environment like a HP/Dell customer then Meta would have a list of
flash configurables to set. I think you greatly underestimate the
privilege of being at a hyperscaler and having vendors create custom
products just for you..

> Those unfamiliar with how hyperscalers operate can mentally
> replace $hyperscaler with HP or Dell in your message. Minus all the
> proprietary OOB management stuff those guys also provide.

A significant Dell customer will get a server pre-populated with a NIC
with some generic Dell configuration. In most cases the customer will
have to then program the flash to match their needs. Set a specific FW
version, set site specific configurables, etc.

Similar to how a Dell customer will have to change the BIOS settings
in the Dell to match their needs.

> Discussion is about mlx5ctrl not "the device" as much as you'd like
> to equate the two.

I view mlx5ctl/fwctl as a window into the device.

Jason