From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E66DC1A2C0A for ; Sat, 18 May 2024 00:39:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715992774; cv=none; b=QjHO8+C8FWZC49b7886vThmQT9Rw67vbCjk44wJSF4k9w5QoYzZc5od0EQ87hdkyvyYGFqIXIMewYKo89+MH+QoRPMveW1+/Mau30J0EXUJsRII+bPRJzQoJPO03MN5K+OjhCFE5h+3cApywls3jgJ4628FibTyFfb2/wtv/u24= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715992774; c=relaxed/simple; bh=4V5YW8ORGMY45KRj8UEqSm59pF8Y+sc6mDeE+TSeGJM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=gR0XZj2/WmS4MvBiLKw1GXzSCDiJcrJgPT82eDHsMLMpl9Qz/Q+D5vuMfznhDyOrcw2omaD/2y6VIkzQi1Rp8NCxzWtRY7DlOCw0SRI4yKqEotgIBOIO5dLRaoZy8F289FMS6FBNyBRuhdUDUyqt5qG3SjkE2nM83qQJhF009qM= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=SV2zemg5; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="SV2zemg5" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1ee12baa01cso27592235ad.0 for ; Fri, 17 May 2024 17:39:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1715992772; x=1716597572; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=p7Punv+O2VI5RKAkHRkhtEk7CBj9yRK2PJX5XN6bfhc=; b=SV2zemg5NtIR3pbXcUpjQL4Nh2uV808bM0bYra4XbVqoECYENJz6/Yy1du6GUsBF1T XjClNZyHgkq1IjzGsNu0VF4KSRcFvP8duWW93pNhihxONKxIBHEuEju184pzDGSNNupq ib3YMvRdDHjiHmp2OxcZgusnPxfFWYkd34Nlw+eS/M5SIsLUf3Mrx9wPF3j4JbN0L51J 8Ns52iIbDMMJzhPw7VAE6tB2RCkaU9f1rM9auoIgPX19Bn4lqARdC5AzZbrjVYBG9xPX cdNuEoJX1TmbwOv4k+7AFSI4ow5gFWRUrYzogvoH4IJNgoo7XBVgU2ba15ADfeNZaYsV sGRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715992772; x=1716597572; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=p7Punv+O2VI5RKAkHRkhtEk7CBj9yRK2PJX5XN6bfhc=; b=KIDpcgOSmPTlSefIvHUTK7dOE3xrFMpx4vJ9ZGBq2hKAnMBCx7WQDE7MQP5lcm3LMI e2k8257f+fLclBivkUdkVTkqvp5r3rhExSgaKIiNEvJ92IcPRzDJbEX4y/XWDy7YqnR4 /N73CPsYAOfjo45gZNms1uW/AikgK16ncdO6lMOcwjRPaqLd7mPujMiNQAV0QoB7oB5Y 6pKd1DYD0W3y+LcxQTXQf2y+A804gSUwzoddNjNXT3XPRsu/KPgUw0/5BT0pf4DRQima D/+Wy8OBAcvwn6ni6I8J4QBHK33KIugaCjWP7Pd2SsSB2n3YT3DXJ6zi0QiX6D0Xvj6f 63UA== X-Forwarded-Encrypted: i=1; AJvYcCUA45wulm+zNCvq4WyIP9nSqtb9a/s/xtibrG37nV8j+6A3LVsbzV10JKjvMQ0/3WAZlOkBEuW2C3Y7oG2z69zIxn4OSk7Lec6CqwU= X-Gm-Message-State: AOJu0YycbLHkCjmHVv8Egk+9yDDFy7qUxruCEu/0HtpejQb2Z0f1z1Tt 6KLvHvbafpAx9PyEnxWdMCwrg9FecgV/duii7I444xNI1EMM7E0FLUWpLyUmyXU= X-Google-Smtp-Source: AGHT+IG2kizdOiSw0uylPEq0qK5Nujj1quEtxgB2wxc39/tXdkGx33RSfpvHjRCh3dNNGsAUN6Msjg== X-Received: by 2002:a05:6a00:1397:b0:6e6:89ad:1233 with SMTP id d2e1a72fcca58-6f4e02a6150mr30537733b3a.2.1715992772166; Fri, 17 May 2024 17:39:32 -0700 (PDT) Received: from ?IPV6:2a03:83e0:1156:1:1cbd:da2b:a9f2:881? ([2620:10d:c090:500::6:9fd9]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-6f4d2af2bccsm16503658b3a.170.2024.05.17.17.39.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 May 2024 17:39:31 -0700 (PDT) Message-ID: <090be3c0-42e6-4b97-8b03-eb64b06a2911@davidwei.uk> Date: Fri, 17 May 2024 17:39:25 -0700 Precedence: bulk X-Mailing-List: linux-alpha@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH net-next v9 05/14] netdev: netdevice devmem allocator Content-Language: en-GB To: Mina Almasry , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org, sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-arch@vger.kernel.org, bpf@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Donald Hunter , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Thomas Bogendoerfer , "James E.J. Bottomley" , Helge Deller , Andreas Larsson , Jesper Dangaard Brouer , Ilias Apalodimas , Steven Rostedt , Masami Hiramatsu , Mathieu Desnoyers , Arnd Bergmann , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , John Fastabend , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Steffen Klassert , Herbert Xu , David Ahern , Willem de Bruijn , Shuah Khan , Sumit Semwal , =?UTF-8?Q?Christian_K=C3=B6nig?= , Pavel Begunkov , Jason Gunthorpe , Yunsheng Lin , Shailend Chand , Harshitha Ramamurthy , Shakeel Butt , Jeroen de Borst , Praveen Kaligineedi , Willem de Bruijn , Kaiyuan Zhang References: <20240510232128.1105145-1-almasrymina@google.com> <20240510232128.1105145-6-almasrymina@google.com> From: David Wei In-Reply-To: <20240510232128.1105145-6-almasrymina@google.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 2024-05-10 16:21, Mina Almasry wrote: > +/* This returns the absolute dma_addr_t calculated from > + * net_iov_owner(niov)->owner->base_dma_addr, not the page_pool-owned > + * niov->dma_addr. > + * > + * The absolute dma_addr_t is a dma_addr_t that is always uncompressed. > + * > + * The page_pool-owner niov->dma_addr is the absolute dma_addr compressed into > + * an unsigned long. Special handling is done when the unsigned long is 32-bit > + * but the dma_addr_t is 64-bit. > + * > + * In general code looking for the dma_addr_t should use net_iov_dma_addr(), > + * while page_pool code looking for the unsigned long dma_addr which mirrors > + * the field in struct page should use niov->dma_addr. > + */ > +static inline dma_addr_t net_iov_dma_addr(const struct net_iov *niov) > +{ > + struct dmabuf_genpool_chunk_owner *owner = net_iov_owner(niov); > + > + return owner->base_dma_addr + > + ((dma_addr_t)net_iov_idx(niov) << PAGE_SHIFT); > +} This part feels like devmem TCP specific, yet the function is in netmem.h. Please consider moving it into devmem.{h,c} which makes it less likely that people not reading your comment will try using it. > + > +static inline struct net_devmem_dmabuf_binding * > +net_iov_binding(const struct net_iov *niov) > +{ > + return net_iov_owner(niov)->binding; > +} > + > /* netmem */ > > /** > diff --git a/net/core/devmem.c b/net/core/devmem.c > index d82f92d7cf9ce..1f90e23a81441 100644 > --- a/net/core/devmem.c > +++ b/net/core/devmem.c > @@ -54,6 +54,42 @@ void __net_devmem_dmabuf_binding_free(struct net_devmem_dmabuf_binding *binding) > kfree(binding); > } > > +struct net_iov * > +net_devmem_alloc_dmabuf(struct net_devmem_dmabuf_binding *binding) > +{ > + struct dmabuf_genpool_chunk_owner *owner; > + unsigned long dma_addr; > + struct net_iov *niov; > + ssize_t offset; > + ssize_t index; > + > + dma_addr = gen_pool_alloc_owner(binding->chunk_pool, PAGE_SIZE, > + (void **)&owner); > + if (!dma_addr) > + return NULL; > + > + offset = dma_addr - owner->base_dma_addr; > + index = offset / PAGE_SIZE; > + niov = &owner->niovs[index]; > + > + niov->dma_addr = 0; > + > + net_devmem_dmabuf_binding_get(binding); > + > + return niov; > +} > + > +void net_devmem_free_dmabuf(struct net_iov *niov) > +{ > + struct net_devmem_dmabuf_binding *binding = net_iov_binding(niov); > + unsigned long dma_addr = net_iov_dma_addr(niov); > + > + if (gen_pool_has_addr(binding->chunk_pool, dma_addr, PAGE_SIZE)) > + gen_pool_free(binding->chunk_pool, dma_addr, PAGE_SIZE); > + > + net_devmem_dmabuf_binding_put(binding); > +} > + > /* Protected by rtnl_lock() */ > static DEFINE_XARRAY_FLAGS(net_devmem_dmabuf_bindings, XA_FLAGS_ALLOC1); >