From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============7106361174069274358=="
MIME-Version: 1.0
From: Walker, Benjamin <benjamin.walker at intel.com>
Subject: Re: [SPDK] Hugepages deallocation
Date: Tue, 28 May 2019 17:08:05 +0000
Message-ID: <f970f4ecbd5472baa554e0542b55b14200312bdb.camel@intel.com>
In-Reply-To: HE1PR0502MB30340449167AEABA7EA24488DB1C0@HE1PR0502MB3034.eurprd05.prod.outlook.com
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============7106361174069274358==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

On Sun, 2019-05-26 at 15:23 +0000, Gal Shalom wrote:
> Hello,
> =

> I encounter a initialization failure of SPDK v18.04, which I suspect the
> reason is the process of hugepages deallocation in SPDK.
> During my initialization I try to allocate a buffer of 1GB.
> The SPDK is able to initialized on the first few tries after a fresh rebo=
ot,
> but after I have killed it and started it over several times - it is unab=
le to
> allocate the 1GB buffer. I suspect the SPDK isn't properly release the
> hugepages and they remain fragmented which is the reason for the failure =
of
> the allocation.
> =

> I am using spdk_dma_zmalloc for the buffer allocation and hugepages of 51=
2MB,
> and would like any thoughts or suggestions about the error I get.

One suggestion I can come up with would be to debug this a bit by making a =
few
DPDK calls after initialization to walk through the allocated hugepages ("m=
emory
segments") and test whether they are physically contiguous enough to perfor=
m a
1GB allocation. DPDK has facilities for walking through the memory map - wi=
th
the version shipped in SPDK v18.04, I think the code would look like this:

	struct rte_mem_config *mcfg;
	size_t seg_idx;

	mcfg =3D rte_eal_get_configuration()->mem_config;
	for (seg_idx =3D 0; seg_idx < RTE_MAX_MEMSEG; seg_idx++) {
		struct rte_memseg *seg =3D &mcfg->memseg[seg_idx];

		if (seg->addr =3D=3D NULL) {
			break;
		}

		seg->phys_addr; // Physical address of memory segment here.
		seg->len; // Length of this segment
	}

Alternatively, the memory allocator has been rewritten in a more recent DPD=
K.
You could try upgrading to SPDK 19.04 (which uses DPDK 19.02) and see if the
problem persists.

> =

> Thanks,
> Gal
> =

> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk


--===============7106361174069274358==--