From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stephen Hemminger Subject: Re: Memory allocated using rte_zmalloc() has non-zeros Date: Wed, 18 Jul 2018 13:58:17 -0700 Message-ID: <20180718135817.66728c37@xeon-e3> References: <8bc76811-ac29-d7f2-e4c3-12b50fd44dba@solarflare.com> <58e5044c-3d13-9171-4168-b4d6b1d61927@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "Burakov, Anatoly" , "dev@dpdk.org" , Sergio Gonzalez Monroy To: Andrew Rybchenko Return-path: Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by dpdk.org (Postfix) with ESMTP id 1AD6723C for ; Wed, 18 Jul 2018 22:58:20 +0200 (CEST) Received: by mail-pg1-f196.google.com with SMTP id f1-v6so2526921pgq.12 for ; Wed, 18 Jul 2018 13:58:20 -0700 (PDT) In-Reply-To: List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On Wed, 18 Jul 2018 22:52:12 +0300 Andrew Rybchenko wrote: > On 18.07.2018 20:18, Burakov, Anatoly wrote: > > On 18-Jul-18 4:20 PM, Andrew Rybchenko wrote: =20 > >> Hi Anatoly, > >> > >> I'm investigating issue which finally comes to the fact that memory=20 > >> allocated using > >> rte_zmalloc() has non zeros. > >> > >> If I add memset just after allocation, everything is perfect and=20 > >> works fine. > >> > >> I've found out that memset was removed from rte_zmalloc_socket() some= =20 > >> time ago: > >> =20 > >> =C2=A0>>> =20 > >> commit b78c9175118f7d61022ddc5c62ce54a1bd73cea5 > >> Author: Sergio Gonzalez Monroy > >> Date:=C2=A0=C2=A0 Tue Jul 5 12:01:16 2016 +0100 > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 mem: do not zero out memory on zmalloc > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 Zeroing out memory on rte_zmalloc_socket is n= ot required anymore=20 > >> since all > >> =C2=A0=C2=A0=C2=A0=C2=A0 allocated memory is already zeroed. > >> > >> =C2=A0=C2=A0=C2=A0=C2=A0 Signed-off-by: Sergio Gonzalez Monroy=20 > >> > >> <<< > >> > >> but may be something has changed now that made above statement false. > >> > >> I observe the problem when memory is reallocated. I.e. I configure 7=20 > >> queues, > >> start, stop, reconfigure to 3 queues, start. Memory is allocated on=20 > >> start and > >> freed on stop, since we have less queues on the second start it is=20 > >> allocated > >> in a different way and reuses previously allocated/freed memory. > >> > >> Do you have any ideas what could be wrong? > >> > >> Andrew. > >> > >> =20 > > > > Hi Andrew, > > > > I will look into it first thing tomorrow. In general, we memset(0) on=20 > > free, and kernel gives us zeroed out pages initially, so the most=20 > > likely point of failure is that i'm not overwring some malloc headers=20 > > correctly on free. =20 >=20 > OK, at least now I know how it is supposed to work in theory. >=20 > The following region was allocated=C2=A0 (the second number below is poin= ter=20 > plus size) > ALLOC 0x7fffa3264080-0x7fffa32640b8 >=20 > Not zerod address is 16 bytes before: > (gdb) p/x ((uint64_t *)0x7fffa3264070)[0] > $4 =3D 0x4000000002 > (gdb) p/x ((uint64_t=C2=A0 *)0x7fffa3264070)[1] > $5 =3D 0x80 >=20 > then freed > FREE 0x7fffa3264080-0x7fffa32640b8 >=20 > but above values (gdb) are still the same > then it is allocated as the part of bigger memory chunk > ALLOC 0x7fffa3245b80-0x7fffa3265fd8 > which should contain zeros, but above values are still the same. >=20 > It is interesting that it looks like it was the first block freed on the= =20 > port stop. I'm not 100% sure since I've put printouts to my allocation=20 > wrapper, not EAL. >=20 > Many thanks, > Andrew. memset here is what is supposed to clear the data. struct malloc_elem * malloc_elem_free(struct malloc_elem *elem) { void *ptr; size_t data_len; ptr =3D RTE_PTR_ADD(elem, MALLOC_ELEM_HEADER_LEN + elem->pad); data_len =3D elem->size - elem->pad - MALLOC_ELEM_OVERHEAD; elem =3D malloc_elem_join_adjacent_free(elem); malloc_elem_free_list_insert(elem); elem->pad =3D 0; /* decrease heap's count of allocated elements */ elem->heap->alloc_count--; memset(ptr, 0, data_len); Maybe data_len is not correct either because of bug, or your application cl= obbered the malloc reserved regions in the element. More likely, gcc is incorrectly optimizing this away. https://wiki.sei.cmu.edu/confluence/display/c/MSC06-C.+Beware+of+compiler+o= ptimizations https://www.cryptologie.net/article/419/zeroing-memory-compiler-optimizatio= ns-and-memset_s/