Re: [PATCH iwl-next v4 08/12] ice: Save and load RX Queue head

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Dec 07, 2023 at 07:55:17AM +0000, Tian, Kevin wrote:
> > From: Cao, Yahui <yahui.cao@intel.com>
> > Sent: Tuesday, November 21, 2023 10:51 AM
> >
> > +
> > +		/* Once RX Queue is enabled, network traffic may come in at
> > any
> > +		 * time. As a result, RX Queue head needs to be loaded
> > before
> > +		 * RX Queue is enabled.
> > +		 * For simplicity and integration, overwrite RX head just after
> > +		 * RX ring context is configured.
> > +		 */
> > +		if (msg_slot->opcode == VIRTCHNL_OP_CONFIG_VSI_QUEUES)
> > {
> > +			ret = ice_migration_load_rx_head(vf, devstate);
> > +			if (ret) {
> > +				dev_err(dev, "VF %d failed to load rx head\n",
> > +					vf->vf_id);
> > +				goto out_clear_replay;
> > +			}
> > +		}
> > +
> 
> Don't we have the same problem here as for TX head restore that the
> vfio migration protocol doesn't carry a way to tell whether the IOAS
> associated with the device has been restored then allowing RX DMA
> at this point might cause device error?

Does this trigger a DMA?

> @Jason, is it a common gap applying to all devices which include a
> receiving path from link? How is it handled in mlx migration
> driver? 

There should be no DMA until the device is placed in RUNNING. All
devices may instantly trigger DMA once placed in RUNNING.

The VMM must ensure the entire environment is ready to go before
putting anything in RUNNING, including having setup the IOMMU.

> I may overlook an important aspect here but if not I wonder whether
> the migration driver should keep DMA disabled (at least for RX) even
> when the device moves to RUNNING and then introduce an explicit
> enable-DMA state which VMM can request after it restores the
> relevant IOAS/HWPT...
> with the device.

Why do we need a state like this?

Jason