From mboxrd@z Thu Jan 1 00:00:00 1970 From: "XU Liang" Subject: Re: =?utf-8?q?=5BPATCH=5D_eal=3A_map_PCI_memory_resources_?= =?utf-8?q?after_hugepages?= Date: Mon, 10 Nov 2014 18:01:09 +0800 Message-ID: References: <1415417532-4363-1-git-send-email-liang.xu@cinfotech.cn>, C6ECDF3AB251BE4894318F4E4512369780C07DC0@IRSMSX109.ger.corp.intel.com Reply-To: XU Liang Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable To: "=?UTF-8?B?QnVyYWtvdiwgQW5hdG9seQ==?=" , "dev-VfR2kkLFssw@public.gmane.org" Return-path: In-Reply-To: C6ECDF3AB251BE4894318F4E4512369780C07DC0-kPTMFJFq+rHjxeytcECX8bfspsVTdybXVpNB7YpNyf8@public.gmane.org List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces-VfR2kkLFssw@public.gmane.org Sender: "dev" It is a default value when the requested_addr isn't exist, not an overide. Whe= n the=C2=A0pci_map_resource is called at the primary process, the=C2=A0request= ed_addr is NULL. The default value will be provided by=C2=A0default_map_addr.=0A= When the=C2=A0pci_map_resource is called at the secondery process, the=C2=A0re= quested_addr is exist. Then everything isn't be changed.----------------------= --------------------------------------------From:Burakov, Anatoly Time:2014 Nov 10 (Mon) 17 : 54To:=E5=BE=90=E4=BA=AE , dev-VfR2kkLFssw@public.gmane.org Cc:thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org Subject:RE: [PATCH] eal: map PCI memory resources after hug= epages=0AHi Liang=0A=0AI don't think that overriding the value passed to pci_m= ap_resource as argument is the way to go. While it results in less code, it lo= oks weird, in my opinion at least, as I believe tracking the correctness of ad= dress being requested should be the responsibility of the caller, i.e. either = UIO or VFIO code. Which is why I keep insisting that you make requested_pci_ad= dr global to linuxapp EAL PCI section and put it into include/eal_pci_init.h. = Would you mind if I made a patch for this issue based on your code?=0A=0AThank= s,=0AAnatoly=0A=0A-----Original Message-----=0AFrom: Liang Xu [mailto:liang.xu= @cinfotech.cn] =0ASent: Saturday, November 8, 2014 3:32 AM=0ATo: dev-VfR2kkLFssw@public.gmane.org=0A= Cc: Burakov, Anatoly; thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org=0ASubject: [PATCH] eal: map PC= I memory resources after hugepages=0A=0AA multiple process DPDK application mu= st mmap hugepages and pci resources into same virtual addresses. By default th= e virtual addresses chosen by the primary process automatically when calling t= he mmap. But sometime the chosen virtual addresses isn't usable at secondary p= rocess. Such as the secondary process linked with more libraries than primary = process. The library has been mapped into this virtual address. The command li= ne parameter 'base-virtaddr' has been added for this situation. If it's config= ured, the hugepages will be mapped into this base address. But the virtual add= ress of pci resources mapped still does not refer to the parameter. In that ca= se "EAL: pci_map_resource(): cannot mmap" =0Awill be got.=0A=0AThis patch try = to map pci resources after hugepages. So the error can be resolved by set base= -virtaddr into free virtual address space.=0A=0ASigned-off-by: Liang Xu =0A---=0A lib/librte_eal/linuxapp/eal/eal_pci.c | 32 ++++++++= +++++++++++++++++++++++-=0A 1 file changed, 31 insertions(+), 1 deletion(-)=0A= =0Adiff --git a/lib/librte_eal/linuxapp/eal/eal_pci.c b/lib/librte_eal/linuxap= p/eal/eal_pci.c=0Aindex ddb0535..502eef2 100644=0A--- a/lib/librte_eal/linuxap= p/eal/eal_pci.c=0A+++ b/lib/librte_eal/linuxapp/eal/eal_pci.c=0A@@ -97,14 +97,= 42 @@ error:=0A return -1;=0A }=0A =0A+static void *=0A+pci_find_max_end_va(v= oid)=0A+{=0A+ const struct rte_memseg *seg =3D rte_eal_get_physmem_layout();=0A= + const struct rte_memseg *last =3D seg;=0A+ unsigned i =3D 0;=0A+=0A+ for (i = =3D 0; i < RTE_MAX_MEMSEG; i++, seg++) {=0A+ if (seg->addr =3D=3D NULL)=0A+ = break;=0A+=0A+ if (seg->addr > last->addr)=0A+ last =3D seg;=0A+=0A+ }=0A+= return RTE_PTR_ADD(last->addr, last->len); }=0A+=0A /* map a particular resou= rce from a file */ void * pci_map_resource(void *requested_addr, int fd, off= _t offset, size_t size) {=0A void *mapaddr;=0A =0A+ /* By default the PCI me= mory resource will be mapped after hugepages */=0A+ static void *default_map_a= ddr;=0A+ if (NULL =3D=3D requested_addr) {=0A+ if (NULL =3D=3D default_map_ad= dr)=0A+ default_map_addr =3D pci_find_max_end_va();=0A+ mapaddr =3D default= _map_addr;=0A+ } else {=0A+ mapaddr =3D requested_addr;=0A+ }=0A+=0A /* M= ap the PCI memory resource of device */=0A- mapaddr =3D mmap(requested_addr, s= ize, PROT_READ | PROT_WRITE,=0A+ mapaddr =3D mmap(mapaddr, size, PROT_READ | P= ROT_WRITE,=0A MAP_SHARED, fd, offset);=0A if (mapaddr =3D=3D MAP_FAILED ||= =0A (requested_addr !=3D NULL && mapaddr !=3D requested_addr)) { @@ -114,6 = +142,8 @@ pci_map_resource(void *requested_addr, int fd, off_t offset, size_t = size)=0A strerror(errno), mapaddr);=0A goto fail;=0A }=0A+ if (NULL =3D=3D= requested_addr)=0A+ default_map_addr =3D RTE_PTR_ADD(mapaddr, size);=0A =0A = RTE_LOG(DEBUG, EAL, " PCI memory mapped at %p\n", mapaddr);=0A =0A--=0A1.9.1