From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============0400641375569683858==" MIME-Version: 1.0 From: Walker, Benjamin Subject: Re: [SPDK] Running nvmf target as non-root Date: Fri, 17 May 2019 17:30:28 +0000 Message-ID: In-Reply-To: 056caa86f0819e5fd99b5b2d5b5c59577065484c.camel@intel.com List-ID: To: spdk@lists.01.org --===============0400641375569683858== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Thu, 2019-05-16 at 16:21 +0000, Walker, Benjamin wrote: > On Thu, 2019-05-16 at 03:39 +0000, Stojaczyk, Dariusz wrote: > > By default, DPDK tries to retrieve physical addresses from pagemap and = that > > obviously doesn't work without root permissions. Since DPDK 18.11 (I th= ink?) > > we can provide --iova-mode=3Dva command line param that should make DPD= K rely > > on > > VFIO IOVAs instead of physical addreses. However, in SPDK we don't prov= ide > > that param today and there's no way to enable it without changing the c= ode. > > That param should be enough to start SPDK framework, but particular mod= ules > > might still malfunction later on. I'm aware of at least vhost PMD that = tries > > to read pagemap as well - it just won't work as an unpriviledged user. = > > = > > I think we could make "something" work as an unprivileged user pretty q= uick, > > but the big blocker towards supporting today it is our testing framewor= k. > = > Here's a patch that makes the tests run as an unprivileged user for nvmf = if an > IOMMU is available. It correctly reproduces the bug on the physical syste= m we > use in the test pool. The patch needs to get rebased on top of a bunch of > refactoring patches that Jim pushed out yesterday, but I think this will = be > sufficient to solve the testing problem. > = > https://review.gerrithub.io/c/spdk/spdk/+/454679 And here's a patch to DPDK that fixes the problem. I can run the nvmf targe= t as an unprivileged user with this change (no SPDK changes were required). https://review.gerrithub.io/c/spdk/dpdk/+/454926 > = > > D. > > = > > > -----Original Message----- > > > From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, > > > Benjamin > > > Sent: Wednesday, May 15, 2019 8:13 PM > > > To: spdk(a)lists.01.org > > > Subject: Re: [SPDK] Running nvmf target as non-root > > > = > > > On Tue, 2019-05-14 at 13:16 -0600, Michael Haeuptle wrote: > > > > Hello, > > > > = > > > > I was wondering if it is possible to run apps like nvmf target as a > > > > non-root user. > > > > = > > > > I read the sections about pagemap/IOMMU in both DPDK and SPDK and I= 'm > > > not > > > > sure if it is possible or not from the text. > > > > = > > > > In any case, I enabled IOMMU via the grub command line and loaded v= fio- > > > pci > > > > but I'm still getting an error: > > > > = > > > > Starting SPDK v19.04 / DPDK 19.02.0 initialization... > > > > [ DPDK EAL parameters: nvgrid --no-shconf -c 0x1 --log-level=3Dlib.= eal:6 > > > > --base-virtaddr=3D0x200000000000 --match-allocations > > > > --file-prefix=3Dspdk_pid6059 ] > > > > EAL: VFIO support initialized > > > > EAL: Cannot obtain physical addresses: Success. Only vfio will func= tion. > > > > error allocating rte services array > > > > EAL: FATAL: rte_service_init() failed > > > > EAL: rte_service_init() failed > > > > Failed to initialize DPDK > > > = > > > In theory, yes it is possible to run the NVMe-oF target as a non-root > > > user. > > > Historically we've even had it working (we contributed the patches to= DPDK > > > to > > > make it work). But we don't have automated tests for this, and DPDK > > > doesn't > > > either, so periodically the functionality goes stale. Based on the er= ror > > > message > > > you are seeing, I think that is what happened here - this new > > > rte_service_init > > > code in DPDK is doing something that requires root when it shouldn't = be. > > > = > > > I think we need to address the testing issue prior to fixing whatever= the > > > bug > > > is. We already have a set of tests running on a physical system with = the > > > IOMMU > > > enabled, so it should be as simple as dropping privileges in the test > > > script > > > when it goes to run the NVMe-oF target, if the IOMMU is enabled. I'm = going > > > to > > > give this a shot and see how it works out. > > > = > > > > I'm seeing this message in dmesg after enabling iommu in grub (but > > > nothing > > > > else). > > > > [ 0.000000] DMAR: IOMMU enabled > > > > = > > > > Thanks. > > > > = > > > > -- Michael > > > > _______________________________________________ > > > > SPDK mailing list > > > > SPDK(a)lists.01.org > > > > https://lists.01.org/mailman/listinfo/spdk > > > = > > > _______________________________________________ > > > SPDK mailing list > > > SPDK(a)lists.01.org > > > https://lists.01.org/mailman/listinfo/spdk > > _______________________________________________ > > SPDK mailing list > > SPDK(a)lists.01.org > > https://lists.01.org/mailman/listinfo/spdk > = > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk --===============0400641375569683858==--