Re: [PATCH net-next 2/3] virtio-net: use per-receive queue page frag alloc for mergeable bufs

[Jason Wang <jasowang@redhat.com>]

On 12/27/2013 05:56 AM, Eric Dumazet wrote:
> On Thu, 2013-12-26 at 13:28 -0800, Michael Dalton wrote:
>> On Mon, Dec 23, 2013 at 11:37 AM, Michael S. Tsirkin <mst@redhat.com> wrote:
>>> So there isn't a conflict with respect to locking.
>>>
>>> Is it problematic to use same page_frag with both GFP_ATOMIC and with
>>> GFP_KERNEL? If yes why?
>> I believe it is safe to use the same page_frag and I will send out a
>> followup patchset using just the per-receive page_frags. For future
>> consideration, Eric noted that disabling NAPI before GFP_KERNEL
>> allocs can potentially inhibit virtio-net network processing for some
>> time (e.g., during a blocking memory allocation or preemption).
> Yep, using napi_disable() in the refill process looks quite inefficient
> to me, it not buggy.
>
> napi_disable() is a big hammer, while whole idea of having a process to
> block on GFP_KERNEL allocations is to allow some asynchronous behavior.
>
> I have hard time to convince myself virtio_net is safe anyway with this
> work queue thing.
>
> virtnet_open() seems racy for example :
>
>         for (i = 0; i < vi->max_queue_pairs; i++) {
>                 if (i < vi->curr_queue_pairs)
>                         /* Make sure we have some buffers: if oom use wq. */
>                         if (!try_fill_recv(&vi->rq[i], GFP_KERNEL))
>                                 schedule_delayed_work(&vi->refill, 0);
>                 virtnet_napi_enable(&vi->rq[i]);
>
>
> What if the workqueue is scheduled _before_ the call to virtnet_napi_enable(&vi->rq[i]) ?

Then napi_disable() in refill_work() will busy wait until napi is
enabled by virtnet_napi_enable() which looks safe. Looks like the real
issue is in virtnet_restore() who calls try_fill_recv() in neither napi
context nor napi disabled context.
>
> refill_work() will happily conflict with another cpu, two cpus could 
> call try_fill_recv() at the same time, or worse napi_enable() would crash.
>
> I do not have time to make a full check, but I guess there are
> other races like this one.
>
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index c51a98867a40..b8e2adb5d0c2 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -726,16 +726,18 @@ again:
>  static int virtnet_open(struct net_device *dev)
>  {
>  	struct virtnet_info *vi = netdev_priv(dev);
> +	bool refill = false;
>  	int i;
>  
>  	for (i = 0; i < vi->max_queue_pairs; i++) {
>  		if (i < vi->curr_queue_pairs)
>  			/* Make sure we have some buffers: if oom use wq. */
>  			if (!try_fill_recv(&vi->rq[i], GFP_KERNEL))
> -				schedule_delayed_work(&vi->refill, 0);
> +				refill = true;
>  		virtnet_napi_enable(&vi->rq[i]);
>  	}
> -
> +	if (refill)
> +		schedule_delayed_work(&vi->refill, 0);
>  	return 0;
>  }
>  
>
>
>
>
>