From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42C99314A8E for ; Thu, 29 Jan 2026 18:11:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769710310; cv=none; b=TiHQXWg5mUG4DmGgpvJ5cyuFh8+hbQQbMuVZMhX2sYFfGSaIeWVTbGi2m6OL24KFKGaIcRqtulfdFiL9x4Qv6YjiS8yzq2Na6nSLl/BSrXjy8KQmRbe8c7yX6Da3vK7eeGYjIZTwyuPx0q3ijKx4pb7pTa/sq9x63mLgyzirKko= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769710310; c=relaxed/simple; bh=3kNfQon72BntdnPcyCmIVY1CnJOW18aHomAr9Tp38M4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=W6zl6djEJp4pD94ETPpJsOOQuPj+wA+oMp1bTF3S74VW0+YUfQJHxS47ei5KPfj6b9XwgDtf46jrWm+hOm4U7dPO3ViM2r4PtxKLZ4eDJGEQCJPcQGOHSH986r2qlaPY5oB7f0UNmsK+ofnBr0LjW7Y3cJ/vVll0D4k9v1Zk4Ws= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GrHpwfyG; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=oO/HqAat; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GrHpwfyG"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="oO/HqAat" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1769710308; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sR+lMNDRP3cmxx8RR95s6vZsFNJKZiISbaEJv5rMD4Q=; b=GrHpwfyGYXqe+SEEtc9Q3pE/JkhgkmhutRxbpRIvbmxopNwXPJgw2RdxVdRnpI5VC1E/g/ vKlYeh9G6yLxvaoBIFPaiWcIqsn+vyyGE7beaDhVpl9iBUdcqurWyTVT8Ah3W19dSNu1VZ IdZQlar1fsX9eZW2KCamxaOkGN5/AUw= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-314-oC1mZYi7MB2qN13hRLGagw-1; Thu, 29 Jan 2026 13:11:46 -0500 X-MC-Unique: oC1mZYi7MB2qN13hRLGagw-1 X-Mimecast-MFC-AGG-ID: oC1mZYi7MB2qN13hRLGagw_1769710305 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-480717a8e05so11510905e9.3 for ; Thu, 29 Jan 2026 10:11:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1769710305; x=1770315105; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=sR+lMNDRP3cmxx8RR95s6vZsFNJKZiISbaEJv5rMD4Q=; b=oO/HqAat0+p9PJdpFJEqxeapDYosiBYUdDDVMu7pWe1wPslasXZNPX8ey8aSl9IJ4o S+xcQZ5Ue1DXs2th0NnAqLpPi8+rnsDDugkO0st0wphOIG0KlRi3dVBUhmGyV7JcyxZk jbht6O3QQMm4pf4bTYPOa+nb+9hZGGHK2I2A3M4A8PnwTM5gvWDi8UKSu1D3kfJMgb0c pGKzMM6TdLdofulRVkL32YoXv471TKhbkQwd0q3vKuePCFH7iUQ3XimNBPHjBj07rGYh OEK17/GriBsijsByBe+WklNOig/q0YachKBGx5PC/L7pUbe6kJQKafOeXwrFShgNKz3L 8r6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1769710305; x=1770315105; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sR+lMNDRP3cmxx8RR95s6vZsFNJKZiISbaEJv5rMD4Q=; b=eVL94kOW3QnHPYiO+huQwVeiZ2Ku6uXf7Xedw5VZLpaIT4duhYBZiDJj3eKn9AWvx6 oYly+B7std3u6kepY9HkKtsI/qcA0/HLJHBxNs5iE4l5cLQYnPyu90KuNNg6fp+sIk8Y Eq4B281cu/ibjkAWbf+f/whCLgfMlb4knl5nkjQlIr26/CrL+HHw1i9vKA3PdeScznEH ru0xx98fYl0/AO2bwz3WzevHzGssPGwDqDEMLxpjFoNPxOE9ez+maUIRjZbjpa1P2FOh NDILg0YVx4jXEsREXr2UFJdA5nfjH4hOfIydZzpcpYbG/YHdnzcDWvJchzpybybeUbGW KAMw== X-Forwarded-Encrypted: i=1; AJvYcCVeyr3iJbtYwBt30zHO9+RXzV01cFOVNxG0XaJ+7Dpi6zfoMkyACB5+4oc622Gkoy0EqBENnOLvQ6aISl8=@vger.kernel.org X-Gm-Message-State: AOJu0YxMWFtPekgmsq25ilrJRhkyaXUGBb6dHAt4eIyAUAdCbkfDdouQ 3+srfwWQwjA1gP0TXs9h1VxGuzsLctteQ1uB7dNi5FTOvZlzxXHTZkxsR2AfBg/7UVCbqn3J+cD bxAHO8ZP6CsZ7bOzBZfiAcIIWuhTrXwxsQmWRAALTdqtNCacIZA3mrkmj7uYAvIh06w== X-Gm-Gg: AZuq6aKunVBnrPbqwnLV0FIl5hGZ4qNSVmvYgt3UCiK5cI+v432gzunX0/+wTx03QUX C5ixIg4kLG0PLOm+3inwtAEtjGbncu6UN3FWqeXz3y7f8ziBrapAzt6G58WcF+SJ6UkqNWtSfO4 cM0o0WI/Cy1ZG4F6bT0BIitFat05iWyuJX4aMQC2RJi21cYxb+Tqlnoulfb+dHhqAln0LIidxCz tj3XQFKF280EqAV1rprp1yfvjRjQFPL1K6yiOY2S0y0dl0/9ZbNWehb+Lv2mM/zcPPPvdm7ZWyg +KIZMYGFJCoU97MLX5vM9CgwUGVZ+CseCv9h3b27WqokSZcuNFrcxviPgV8NjjTNBdrtcCz/e4/ Kx/oRH2LGcPtFYPATEcR9n0vmUTb4wuoFPg== X-Received: by 2002:a05:600c:3f16:b0:477:54cd:200e with SMTP id 5b1f17b1804b1-482db4569c0mr779095e9.1.1769710305296; Thu, 29 Jan 2026 10:11:45 -0800 (PST) X-Received: by 2002:a05:600c:3f16:b0:477:54cd:200e with SMTP id 5b1f17b1804b1-482db4569c0mr778815e9.1.1769710304801; Thu, 29 Jan 2026 10:11:44 -0800 (PST) Received: from redhat.com (IGLD-80-230-34-155.inter.net.il. [80.230.34.155]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4806cddffe9sm196831645e9.4.2026.01.29.10.11.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Jan 2026 10:11:44 -0800 (PST) Date: Thu, 29 Jan 2026 13:11:41 -0500 From: "Michael S. Tsirkin" To: Johannes Thumshirn Cc: Alexander Graf , Jason Wang , Xuan Zhuo , Eugenio =?iso-8859-1?Q?P=E9rez?= , "open list:VIRTIO CORE" , open list Subject: Re: [PATCH v2] virtio_ring: Add READ_ONCE annotations for device-writable fields Message-ID: <20260129130818-mutt-send-email-mst@kernel.org> References: <20260129121604.745681-1-johannes.thumshirn@wdc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20260129121604.745681-1-johannes.thumshirn@wdc.com> Thanks for the patch! yet something to improve: On Thu, Jan 29, 2026 at 01:15:58PM +0100, Johannes Thumshirn wrote: > From: Alexander Graf > > KCSAN reports data races when accessing virtio ring fields that are > concurrently written by the device (host). These are legitimate > concurrent accesses where the CPU reads fields that the device updates > via DMA-like mechanisms. > > Add accessor functions that use READ_ONCE() to properly annotate these > device-writable fields and prevent compiler optimizations that could in > theory break the code. This also serves as documentation showing which > fields are shared with the device. > > The affected fields are: > - Split ring: used->idx, used->ring[].id, used->ring[].len > - Packed ring: desc[].flags, desc[].id, desc[].len > > Signed-off-by: Alexander Graf > [jth: Add READ_ONCE in virtqueue_kick_prepare_split ] > Co-developed-by: Johannes Thumshirn > Signed-off-by: Johannes Thumshirn > drop the empty line pls. > --- > Changes to v1: > - Updated comments (mst, agraf) > - Moved _read suffix to prefix in newly introduced functions (mst) > - Update my minor contribution to Co-developed-by (agraf) > - Add "in theory" to changelog > --- > drivers/virtio/virtio_ring.c | 69 ++++++++++++++++++++++++++++-------- > 1 file changed, 54 insertions(+), 15 deletions(-) > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c > index ddab68959671..66802d11d30e 100644 > --- a/drivers/virtio/virtio_ring.c > +++ b/drivers/virtio/virtio_ring.c > @@ -222,6 +222,48 @@ struct vring_virtqueue { > #endif > }; > > +/* > + * Accessors for device-writable fields in virtio rings. > + * These fields are concurrently written by the device and read by the driver. > + * Use READ_ONCE() to prevent compiler optimizations and document the > + * intentional data race and prevent KCSAN warnings. Use READ_ONCE ... , document ... and prevent > + */ > +static inline u16 vring_read_used_idx(const struct vring_virtqueue *vq) > +{ > + return virtio16_to_cpu(vq->vq.vdev, > + READ_ONCE(vq->split.vring.used->idx)); > +} > + > +static inline u32 vring_read_used_id(const struct vring_virtqueue *vq, u16 idx) > +{ > + return virtio32_to_cpu(vq->vq.vdev, > + READ_ONCE(vq->split.vring.used->ring[idx].id)); > +} > + > +static inline u32 vring_read_used_len(const struct vring_virtqueue *vq, u16 idx) > +{ > + return virtio32_to_cpu(vq->vq.vdev, > + READ_ONCE(vq->split.vring.used->ring[idx].len)); > +} > + above are only for split I think? then pls note in the name: e.g. vring_read_split_used_len > +static inline u16 vring_read_packed_desc_flags(const struct vring_virtqueue *vq, > + u16 idx) > +{ > + return le16_to_cpu(READ_ONCE(vq->packed.vring.desc[idx].flags)); > +} > + > +static inline u16 vring_read_packed_desc_id(const struct vring_virtqueue *vq, > + u16 idx) > +{ > + return le16_to_cpu(READ_ONCE(vq->packed.vring.desc[idx].id)); > +} > + > +static inline u32 vring_read_packed_desc_len(const struct vring_virtqueue *vq, > + u16 idx) > +{ > + return le32_to_cpu(READ_ONCE(vq->packed.vring.desc[idx].len)); > +} > + hmm the names are very long now. maybe drop _read_ from there. > static struct vring_desc_extra *vring_alloc_desc_extra(unsigned int num); > static void vring_free(struct virtqueue *_vq); > > @@ -736,9 +778,10 @@ static bool virtqueue_kick_prepare_split(struct virtqueue *_vq) > LAST_ADD_TIME_INVALID(vq); > > if (vq->event) { > - needs_kick = vring_need_event(virtio16_to_cpu(_vq->vdev, > - vring_avail_event(&vq->split.vring)), > - new, old); > + u16 event = virtio16_to_cpu(_vq->vdev, > + READ_ONCE(vring_avail_event(&vq->split.vring))); > + why not wrap this one then? > + needs_kick = vring_need_event(event, new, old); > } else { > needs_kick = !(vq->split.vring.used->flags & > cpu_to_virtio16(_vq->vdev, > @@ -808,8 +851,7 @@ static void detach_buf_split(struct vring_virtqueue *vq, unsigned int head, > > static bool more_used_split(const struct vring_virtqueue *vq) > { > - return vq->last_used_idx != virtio16_to_cpu(vq->vq.vdev, > - vq->split.vring.used->idx); > + return vq->last_used_idx != vring_read_used_idx(vq); > } > > static void *virtqueue_get_buf_ctx_split(struct virtqueue *_vq, > @@ -838,10 +880,8 @@ static void *virtqueue_get_buf_ctx_split(struct virtqueue *_vq, > virtio_rmb(vq->weak_barriers); > > last_used = (vq->last_used_idx & (vq->split.vring.num - 1)); > - i = virtio32_to_cpu(_vq->vdev, > - vq->split.vring.used->ring[last_used].id); > - *len = virtio32_to_cpu(_vq->vdev, > - vq->split.vring.used->ring[last_used].len); > + i = vring_read_used_id(vq, last_used); > + *len = vring_read_used_len(vq, last_used); > > if (unlikely(i >= vq->split.vring.num)) { > BAD_RING(vq, "id %u out of range\n", i); > @@ -923,8 +963,7 @@ static bool virtqueue_poll_split(struct virtqueue *_vq, unsigned int last_used_i > { > struct vring_virtqueue *vq = to_vvq(_vq); > > - return (u16)last_used_idx != virtio16_to_cpu(_vq->vdev, > - vq->split.vring.used->idx); > + return (u16)last_used_idx != vring_read_used_idx(vq); > } > > static bool virtqueue_enable_cb_delayed_split(struct virtqueue *_vq) > @@ -1701,10 +1740,10 @@ static void detach_buf_packed(struct vring_virtqueue *vq, > static inline bool is_used_desc_packed(const struct vring_virtqueue *vq, > u16 idx, bool used_wrap_counter) > { > - bool avail, used; > u16 flags; > + bool avail, used; > > - flags = le16_to_cpu(vq->packed.vring.desc[idx].flags); > + flags = vring_read_packed_desc_flags(vq, idx); > avail = !!(flags & (1 << VRING_PACKED_DESC_F_AVAIL)); > used = !!(flags & (1 << VRING_PACKED_DESC_F_USED)); > > @@ -1751,8 +1790,8 @@ static void *virtqueue_get_buf_ctx_packed(struct virtqueue *_vq, > last_used_idx = READ_ONCE(vq->last_used_idx); > used_wrap_counter = packed_used_wrap_counter(last_used_idx); > last_used = packed_last_used(last_used_idx); > - id = le16_to_cpu(vq->packed.vring.desc[last_used].id); > - *len = le32_to_cpu(vq->packed.vring.desc[last_used].len); > + id = vring_read_packed_desc_id(vq, last_used); > + *len = vring_read_packed_desc_len(vq, last_used); > > if (unlikely(id >= vq->packed.vring.num)) { > BAD_RING(vq, "id %u out of range\n", id); > -- > 2.52.0