From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A13AC04A94 for ; Fri, 28 Jul 2023 18:39:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232991AbjG1Sja (ORCPT ); Fri, 28 Jul 2023 14:39:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48698 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233941AbjG1Sj2 (ORCPT ); Fri, 28 Jul 2023 14:39:28 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DFBC4488 for ; Fri, 28 Jul 2023 11:39:26 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-686bea20652so2410880b3a.1 for ; Fri, 28 Jul 2023 11:39:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1690569566; x=1691174366; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=2flqKPY9mF456lzl7KZSlOozTCBExKcvkCqkGx9wsSE=; b=hoBR/RW0WEdqNVVswTS/7PMr8na4eRwQGTuRVV/2gXcg/fpOfuif3lVi2AHeWpL5MZ 8FVYAenG0BPDSoD+acBSjJt67i+GHDSMk1LralzU7yF9zxqOG5m9ZVj/BVXBT8a1vpXd E0Svba5qKjjmGVOgp/MrMG2FKBAjAt3KDr1kaRCea2YmLjo6tTa/nLIVuTOLYCEsFm4J sEojsEc+nXPLnyWSXpGQSoJ5Mm6ULvYM/xvh0cJ8/tdLQvjIIis9nrimhXOF6pe/MzaO yT4gAW/lyG+/BlmjmGBkeHK3KO9CDCVwYo9jlBDHw/9aL3tBp1CcPiVRqnpeYvb1r1C9 dOsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690569566; x=1691174366; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=2flqKPY9mF456lzl7KZSlOozTCBExKcvkCqkGx9wsSE=; b=fCiYgfdhSaxONMj2pRhXhPiMQE2hbGo6jYm7Xs1uToDvPl7FkcgaRCA5MA9dslnzLY Z+O8Utn04WCUtyrhkWriC8wWSlGYTyqCuWF2mSoHrM7xgzjjxXqcMlMxh5xGfIVb98uH Y3k/CkVxZHU4YkFgcXe+eoDnhiVj0VCig3WtYDBuoO0veX0v0MCiTrtXXiTzc2WdfTSy J+3Ld9goM/5FUeMd92NvOkhoqLQOMNYKuf8CuTX/it7sk+vvUzTE9R6RdVEiQc9atfk2 y9xfJ4M8JGhIIStxz3T0qFb2wPOQImv8m+wiQpIgUUxYY6z4R4sWv3b/XzZse2zlrx18 UGGw== X-Gm-Message-State: ABy/qLauZofu0FMF79xEoRu8CIcC5nrjRELka3C4T+9T5RAWc36B6HUB lCVAr8bmLqSGJp8sNteYmuVhkA== X-Google-Smtp-Source: APBJJlHg+nmPr0U2c+eIQr9bUSWzAzZD8+AsIqvrxWDqn+SqeHSWiNYJxTwFAdcBUodvt5dsdhGY2A== X-Received: by 2002:a05:6a20:96c9:b0:135:110c:c6de with SMTP id hq9-20020a056a2096c900b00135110cc6demr2662864pzc.6.1690569566294; Fri, 28 Jul 2023 11:39:26 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-25-194.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.25.194]) by smtp.gmail.com with ESMTPSA id z136-20020a63338e000000b0056001f43726sm3807409pgz.92.2023.07.28.11.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Jul 2023 11:39:25 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1qPSN1-001o9T-Vo; Fri, 28 Jul 2023 15:39:23 -0300 Date: Fri, 28 Jul 2023 15:39:23 -0300 From: Jason Gunthorpe To: Long Li Cc: Wei Hu , "netdev@vger.kernel.org" , "linux-hyperv@vger.kernel.org" , "linux-rdma@vger.kernel.org" , Ajay Sharma , "leon@kernel.org" , KY Srinivasan , Haiyang Zhang , "wei.liu@kernel.org" , Dexuan Cui , "davem@davemloft.net" , "edumazet@google.com" , "kuba@kernel.org" , "pabeni@redhat.com" , "vkuznets@redhat.com" , "ssengar@linux.microsoft.com" , "shradhagupta@linux.microsoft.com" Subject: Re: [PATCH v4 1/1] RDMA/mana_ib: Add EQ interrupt support to mana ib driver. Message-ID: References: <20230728170749.1888588-1-weh@microsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org On Fri, Jul 28, 2023 at 06:22:53PM +0000, Long Li wrote: > > Subject: Re: [PATCH v4 1/1] RDMA/mana_ib: Add EQ interrupt support to mana ib > > driver. > > > > On Fri, Jul 28, 2023 at 05:51:46PM +0000, Long Li wrote: > > > > Subject: Re: [PATCH v4 1/1] RDMA/mana_ib: Add EQ interrupt support > > > > to mana ib driver. > > > > > > > > On Fri, Jul 28, 2023 at 05:07:49PM +0000, Wei Hu wrote: > > > > > Add EQ interrupt support for mana ib driver. Allocate EQs per > > > > > ucontext to receive interrupt. Attach EQ when CQ is created. Call > > > > > CQ interrupt handler when completion interrupt happens. EQs are > > > > > destroyed when ucontext is deallocated. > > > > > > > > It seems strange that interrupts would be somehow linked to a ucontext? > > > > interrupts are highly limited, you can DOS the entire system if > > > > someone abuses this. > > > > > > > > Generally I expect a properly functioning driver to use one interrupt per CPU > > core. > > > > > > Yes, MANA uses one interrupt per CPU. One interrupt is shared among > > > multiple EQs. > > > > So you have another multiplexing layer between the interrupt and the EQ? That is > > alot of multiplexing layers.. > > > > > > You should tie the CQ to a shared EQ belong to the core that the CQ > > > > wants to have affinity to. > > > > > > The reason for using a separate EQ for a ucontext, is for preventing > > > DOS. If we use a shared EQ, a single ucontext can storm this shared EQ > > > affecting other users. > > > > With a proper design it should not be possible. The CQ adds an entry to the EQ > > and that should be rate limited by the ability of userspace to schedule to re-arm > > the CQ. > > I think DPDK user space can sometimes storm the EQ by arming the CQ > from user-mode. Maybe maliciously you can do a blind re-arm, but nothing sane should do that. > With a malicious DPDK user, this code can be abused to arm the CQ at > extremely high rate. Again, the rate of CQ re-arm is limited by the ability of userspace to schedule, I'm reluctant to consider that a DOS vector. Doesn't your HW have EQ overflow recovery? Frankly, stacking more layers of IRQ multiplexing doesn't seem like it should solve any problems, you are just shifting where the DOS can occure. Allowing userspace to create EQs is its own DOS direction, either you exhaust and DOS the number of EQs or you DOS the multiplexing layer between the interrupt and the EQ. Jason