From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7798084DF1 for ; Mon, 13 May 2024 23:03:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.169 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715641406; cv=none; b=KGClKSgZ7fsEDeoe2JKyAXgw/Ck1QsLoAXU/3Zgk7CXEn9pil+qcJ7dSe057RdwOG0YI66H8+GakfJ8xeyB4ScbEzGaoKj4C09G3mQ+nMxSxYLzKhD5FR4WS82TYe/xXVWeeoa/0G3TE4nbl9SLaSkWrKHNJabqoLiYfTK637Dk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715641406; c=relaxed/simple; bh=/9z5Va8BoS10Zw3lv3iW17W4Qhvjzu7IIUUc3s4u6+E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MxVtRZmu1cJvNhu/2InqZn45sH509Udh2teZAbcdUjxh1SNdTudzx4n1NbcD0bHDXeTpAQwseocYAXsNub8OZxzE+d7sUEAh2fFQLhnpYyfe8NRO5i4YGqNiIFhQCsZLcSTJIcL3Fl3jBcGxNUlBzPkQ1MR0w9q9DOSEoE9BTzQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca; spf=pass smtp.mailfrom=ziepe.ca; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b=iKd+8fwn; arc=none smtp.client-ip=209.85.210.169 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ziepe.ca Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="iKd+8fwn" Received: by mail-pf1-f169.google.com with SMTP id d2e1a72fcca58-6f490b5c23bso4168266b3a.3 for ; Mon, 13 May 2024 16:03:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1715641404; x=1716246204; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=eiIURfigbv4jfMYVn+cegJe5X/kgu+BloNveVda8Z54=; b=iKd+8fwngYQ1a9VzFZ1uGEVrJHwiOLeTu0PzsbV4jUJ3bNunb/Hrd+Ob/apHhpW9F1 PpcyDeGKKTuX8Lbk4hb1N8fYWGHFp+1iCvt0CyhdE9tdaQXsjtG0FFvDIhbHWsu/qMpI mJx6tYUMvvYeQwFfnDSL/KByxX7SdP9uHyOlVaXU1p2xFGcaTMInakp5Z9YwdclpkCsL vp8ZFSDahv/9sI/5ChJXUPBeS2g5a2ttW6IgLxquW0ahKsdBV0wo97d708G81TM5vp0D oncFWNnaT9SS/8tJqd018OMRbp+87vuaFtRFBHrDWi4BjiWVAulZ4HPrpxPyJoo1SpE1 Ay4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715641404; x=1716246204; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=eiIURfigbv4jfMYVn+cegJe5X/kgu+BloNveVda8Z54=; b=wbymkt8PlhGxKjT2QBEfga2FFHVSorUbMk7OOpQo59LZ/3wc47TvYlgslkGnF/wG/X Du3vXzTHVZWHKj1gomWvVq6r10PUAfY9Nw0chna4AzVMPt1UmrTJ4brHW67tbXud151A cl2GZ6OlHlURT5IBCdHOoPwlSl0FcDfftaJVPFvqjCgaqHpCH18Fex/IrpngtcNb497l qA+qlHEJcx2LknGtsqZT9q8QuPJzppcfM9TEx4fnIKeT6V93dsmpPyRI5JoBZzL3G+DX U0nXCKvqlG1T2vDfZi2iIV6AauMAyqw1WlhmvgpfFnehfVvj0RtfUQtvH1L2vRlsYCrm eFYA== X-Forwarded-Encrypted: i=1; AJvYcCXLiDH7rs7Mv/ReYEVLLUcPzyt2fzabyb9SagqxOUhu4mGdjx/OCHIqc64XCejiBI/H2tFCgixosZmv2XrumWy6A4sCQO0osa5mKjp+ X-Gm-Message-State: AOJu0Yz7KWpjutztCOXQiUoQEWMyu3wQxHYodTU2GDTAfUAblfx5eBQm wPf8nF4tjt6628bAdSbHnGogLmvS0ZkLuvMI6D4AY2z7Ns1er6zbsVuvCRWGbaE= X-Google-Smtp-Source: AGHT+IGZZ05t4FHBsyrm+Iw0NDc4I2GEzxI8XRqc94KzZJOwLZEC+KqvVPk0Pm01zm+C+DAs6jMXHA== X-Received: by 2002:a05:6a21:6da1:b0:1a9:5e1f:8485 with SMTP id adf61e73a8af0-1afde1180a2mr10908731637.31.1715641403754; Mon, 13 May 2024 16:03:23 -0700 (PDT) Received: from ziepe.ca ([50.204.89.20]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-6f4d2a665c0sm7877475b3a.3.2024.05.13.16.03.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 May 2024 16:03:22 -0700 (PDT) Received: from jgg by jggl with local (Exim 4.95) (envelope-from ) id 1s6ehW-0001ej-5J; Mon, 13 May 2024 20:03:22 -0300 Date: Mon, 13 May 2024 20:03:22 -0300 From: Jason Gunthorpe To: =?utf-8?B?SMOla29u?= Bugge Cc: linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, rds-devel@oss.oracle.com, Leon Romanovsky , Saeed Mahameed , Tariq Toukan , "David S . Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Tejun Heo , Lai Jiangshan , Allison Henderson , Manjunath Patil , Mark Zhang , Chuck Lever , Shiraz Saleem , Yang Li Subject: Re: [PATCH 0/6] rds: rdma: Add ability to force GFP_NOIO Message-ID: References: <20240513125346.764076-1-haakon.bugge@oracle.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20240513125346.764076-1-haakon.bugge@oracle.com> On Mon, May 13, 2024 at 02:53:40PM +0200, Håkon Bugge wrote: > This series enables RDS and the RDMA stack to be used as a block I/O > device. This to support a filesystem on top of a raw block device > which uses RDS and the RDMA stack as the network transport layer. > > Under intense memory pressure, we get memory reclaims. Assume the > filesystem reclaims memory, goes to the raw block device, which calls > into RDS, which calls the RDMA stack. Now, if regular GFP_KERNEL > allocations in RDS or the RDMA stack require reclaims to be fulfilled, > we end up in a circular dependency. > > We break this circular dependency by: > > 1. Force all allocations in RDS and the relevant RDMA stack to use > GFP_NOIO, by means of a parenthetic use of > memalloc_noio_{save,restore} on all relevant entry points. I didn't see an obvious explanation why each of these changes was necessary. I expected this: > 2. Make sure work-queues inherits current->flags > wrt. PF_MEMALLOC_{NOIO,NOFS}, such that work executed on the > work-queue inherits the same flag(s). To broadly capture everything and understood this was the general plan from the MM side instead of direct annotation? So, can you explain in each case why it needs an explicit change? And further, is there any validation of this? There is some lockdep tracking of reclaim, I feel like it should be more robustly hooked up in RDMA if we expect this to really work.. Jason