[PATCH v2] tools/testing/nvdimm: add virtual ramdisk range

[PATCH v2] tools/testing/nvdimm: add virtual ramdisk range

[Dan Williams]

Test the virtual disk ranges that platform firmware like EDK2/OVMF might
emit.

Cc: "Lee, Chun-Yi" <jlee@suse.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
Changes since v1:

* Move this test range to nfit_test.1 to not break the spa ordering
  expected by the nfit_test.0 tests.

 tools/testing/nvdimm/test/nfit.c |   18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/tools/testing/nvdimm/test/nfit.c b/tools/testing/nvdimm/test/nfit.c
index 52df3c20231d..d4a4635c1186 100644
--- a/tools/testing/nvdimm/test/nfit.c
+++ b/tools/testing/nvdimm/test/nfit.c
@@ -104,6 +104,7 @@ enum {
 	NUM_MEM = NUM_DCR + NUM_BDW + 2 /* spa0 iset */ + 4 /* spa1 iset */,
 	DIMM_SIZE = SZ_32M,
 	LABEL_SIZE = SZ_128K,
+	SPA_VCD_SIZE = SZ_4M,
 	SPA0_SIZE = DIMM_SIZE,
 	SPA1_SIZE = DIMM_SIZE*2,
 	SPA2_SIZE = DIMM_SIZE,
@@ -618,7 +619,7 @@ static int nfit_test0_alloc(struct nfit_test *t)
 
 static int nfit_test1_alloc(struct nfit_test *t)
 {
-	size_t nfit_size = sizeof(struct acpi_nfit_system_address)
+	size_t nfit_size = sizeof(struct acpi_nfit_system_address) * 2
 		+ sizeof(struct acpi_nfit_memory_map)
 		+ offsetof(struct acpi_nfit_control_region, window_size);
 
@@ -631,6 +632,10 @@ static int nfit_test1_alloc(struct nfit_test *t)
 	if (!t->spa_set[0])
 		return -ENOMEM;
 
+	t->spa_set[1] = test_alloc(t, SPA_VCD_SIZE, &t->spa_set_dma[1]);
+	if (!t->spa_set[1])
+		return -ENOMEM;
+
 	return ars_state_init(&t->pdev.dev, &t->ars_state);
 }
 
@@ -1335,7 +1340,16 @@ static void nfit_test1_setup(struct nfit_test *t)
 	spa->address = t->spa_set_dma[0];
 	spa->length = SPA2_SIZE;
 
-	offset += sizeof(*spa);
+	/* virtual cd region */
+	spa = nfit_buf + sizeof(*spa);
+	spa->header.type = ACPI_NFIT_TYPE_SYSTEM_ADDRESS;
+	spa->header.length = sizeof(*spa);
+	memcpy(spa->range_guid, to_nfit_uuid(NFIT_SPA_VCD), 16);
+	spa->range_index = 0;
+	spa->address = t->spa_set_dma[1];
+	spa->length = SPA_VCD_SIZE;
+
+	offset += sizeof(*spa) * 2;
 	/* mem-region0 (spa0, dimm0) */
 	memdev = nfit_buf + offset;
 	memdev->header.type = ACPI_NFIT_TYPE_MEMORY_MAP;

Re: [PATCH v2] tools/testing/nvdimm: add virtual ramdisk range

[joeyli]

Hi Dan, 

On Fri, Jul 15, 2016 at 12:35:58PM -0700, Dan Williams wrote:
> Test the virtual disk ranges that platform firmware like EDK2/OVMF might
> emit.
> 
> Cc: "Lee, Chun-Yi" <jlee@suse.com>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> Changes since v1:
> 
> * Move this test range to nfit_test.1 to not break the spa ordering
>   expected by the nfit_test.0 tests.
>

I reviewed and tested this patch by nvdimm testing driver.
Please feel free to add
	Reviewed-by: Lee, Chun-Yi <jlee@suse.com> 

But I have a question to run nfit testing driver on OVMF. I always got
"coherent allocation failed" as below:	

[   86.530950] platform nfit_test.0: swiotlb buffer is full (sz: 134217728 bytes)
[   86.530954] swiotlb: coherent allocation failed for device nfit_test.0 size=134217728
[   86.530956] CPU: 0 PID: 1291 Comm: insmod Tainted: G           OE   4.7.0-rc7-default+ #323
[   86.530958] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
[   86.530959]  0000000000000000 ffff880234e63bb0 ffffffff81334e5c 0000000008000000
[   86.530961]  ffff880232fa5998 ffff880234e63bf0 ffffffff8135eb79 ffff88020000000f
[   86.530963]  00000000024002c0 ffff880232fa5998 0000000008000000 ffff880234e63c68
[   86.530965] Call Trace:
[   86.530971]  [<ffffffff81334e5c>] dump_stack+0x63/0x87
[   86.530975]  [<ffffffff8135eb79>] swiotlb_alloc_coherent+0x149/0x160
[   86.530978]  [<ffffffff810611d3>] x86_swiotlb_alloc_coherent+0x43/0x50
[   86.530981]  [<ffffffffa054db3a>] dma_alloc_attrs.constprop.10+0x5a/0x80 [nfit_test]
[   86.530984]  [<ffffffffa00d618c>] nfit_test_init+0x18c/0x1000 [nfit_test]
[   86.530985]  [<ffffffffa00d6000>] ? 0xffffffffa00d6000
[   86.530988]  [<ffffffff81002190>] do_one_initcall+0x50/0x190
[   86.530992]  [<ffffffff811eef00>] ? kmem_cache_alloc_trace+0x170/0x220
[   86.530995]  [<ffffffff8118dbfc>] ? do_init_module+0x27/0x1e2
[   86.530997]  [<ffffffff8118dc35>] do_init_module+0x60/0x1e2
[   86.531000]  [<ffffffff81108fb1>] load_module+0x1411/0x1c20
[   86.531002]  [<ffffffff81105cb0>] ? __symbol_put+0x40/0x40
[   86.531006]  [<ffffffff812dfbbd>] ? ima_post_read_file+0x3d/0x80
[   86.531009]  [<ffffffff811099c9>] SYSC_finit_module+0xa9/0xd0
[   86.531011]  [<ffffffff81109a0e>] SyS_finit_module+0xe/0x10
[   86.531013]  [<ffffffff81003a62>] do_syscall_64+0x62/0x110
[   86.531016]  [<ffffffff81630a61>] entry_SYSCALL64_slow_path+0x25/0x25
[   86.531018] platform nfit_test.0: need 128M of free cma

I tried to add "swiotlb=65536" and "cma=128M" or "cma=256M", even "cma=2G" kernel
parameters, but nfit-test.ko still can NOT allocate DMA success.

So, I always add a patch to change the DIMM_SIZE to SZ_1M, that means test0
only uses 4M. Then the nfit-test.ko works for testing.
 
Did I miss anything before running the nfit testing driver?


Thanks a lot!
Joey Lee 

Re: [PATCH v2] tools/testing/nvdimm: add virtual ramdisk range

[Dan Williams]

On Sun, Jul 17, 2016 at 2:36 AM, joeyli <jlee-IBi9RG/b67k@public.gmane.org> wrote:
> Hi Dan,
>
> On Fri, Jul 15, 2016 at 12:35:58PM -0700, Dan Williams wrote:
>> Test the virtual disk ranges that platform firmware like EDK2/OVMF might
>> emit.
>>
>> Cc: "Lee, Chun-Yi" <jlee-IBi9RG/b67k@public.gmane.org>
>> Signed-off-by: Dan Williams <dan.j.williams-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> ---
>> Changes since v1:
>>
>> * Move this test range to nfit_test.1 to not break the spa ordering
>>   expected by the nfit_test.0 tests.
>>
>
> I reviewed and tested this patch by nvdimm testing driver.
> Please feel free to add
>         Reviewed-by: Lee, Chun-Yi <jlee-IBi9RG/b67k@public.gmane.org>
>
> But I have a question to run nfit testing driver on OVMF. I always got
> "coherent allocation failed" as below:
>
> [   86.530950] platform nfit_test.0: swiotlb buffer is full (sz: 134217728 bytes)
> [   86.530954] swiotlb: coherent allocation failed for device nfit_test.0 size=134217728
> [   86.530956] CPU: 0 PID: 1291 Comm: insmod Tainted: G           OE   4.7.0-rc7-default+ #323
> [   86.530958] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
> [   86.530959]  0000000000000000 ffff880234e63bb0 ffffffff81334e5c 0000000008000000
> [   86.530961]  ffff880232fa5998 ffff880234e63bf0 ffffffff8135eb79 ffff88020000000f
> [   86.530963]  00000000024002c0 ffff880232fa5998 0000000008000000 ffff880234e63c68
> [   86.530965] Call Trace:
> [   86.530971]  [<ffffffff81334e5c>] dump_stack+0x63/0x87
> [   86.530975]  [<ffffffff8135eb79>] swiotlb_alloc_coherent+0x149/0x160
> [   86.530978]  [<ffffffff810611d3>] x86_swiotlb_alloc_coherent+0x43/0x50
> [   86.530981]  [<ffffffffa054db3a>] dma_alloc_attrs.constprop.10+0x5a/0x80 [nfit_test]
> [   86.530984]  [<ffffffffa00d618c>] nfit_test_init+0x18c/0x1000 [nfit_test]
> [   86.530985]  [<ffffffffa00d6000>] ? 0xffffffffa00d6000
> [   86.530988]  [<ffffffff81002190>] do_one_initcall+0x50/0x190
> [   86.530992]  [<ffffffff811eef00>] ? kmem_cache_alloc_trace+0x170/0x220
> [   86.530995]  [<ffffffff8118dbfc>] ? do_init_module+0x27/0x1e2
> [   86.530997]  [<ffffffff8118dc35>] do_init_module+0x60/0x1e2
> [   86.531000]  [<ffffffff81108fb1>] load_module+0x1411/0x1c20
> [   86.531002]  [<ffffffff81105cb0>] ? __symbol_put+0x40/0x40
> [   86.531006]  [<ffffffff812dfbbd>] ? ima_post_read_file+0x3d/0x80
> [   86.531009]  [<ffffffff811099c9>] SYSC_finit_module+0xa9/0xd0
> [   86.531011]  [<ffffffff81109a0e>] SyS_finit_module+0xe/0x10
> [   86.531013]  [<ffffffff81003a62>] do_syscall_64+0x62/0x110
> [   86.531016]  [<ffffffff81630a61>] entry_SYSCALL64_slow_path+0x25/0x25
> [   86.531018] platform nfit_test.0: need 128M of free cma
>
> I tried to add "swiotlb=65536" and "cma=128M" or "cma=256M", even "cma=2G" kernel
> parameters, but nfit-test.ko still can NOT allocate DMA success.
>
> So, I always add a patch to change the DIMM_SIZE to SZ_1M, that means test0
> only uses 4M. Then the nfit-test.ko works for testing.
>
> Did I miss anything before running the nfit testing driver?

The CMA dependency is a hassle, so for v4.8 I'm replacing it with vmalloc:

https://patchwork.kernel.org/patch/9199761/
https://patchwork.kernel.org/patch/9199763/

Re: [PATCH v2] tools/testing/nvdimm: add virtual ramdisk range

[joeyli]

On Sun, Jul 17, 2016 at 09:05:19AM -0700, Dan Williams wrote:
> On Sun, Jul 17, 2016 at 2:36 AM, joeyli <jlee@suse.com> wrote:
> > Hi Dan,
> >
> > On Fri, Jul 15, 2016 at 12:35:58PM -0700, Dan Williams wrote:
> >> Test the virtual disk ranges that platform firmware like EDK2/OVMF might
> >> emit.
> >>
> >> Cc: "Lee, Chun-Yi" <jlee@suse.com>
> >> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> >> ---
> >> Changes since v1:
> >>
> >> * Move this test range to nfit_test.1 to not break the spa ordering
> >>   expected by the nfit_test.0 tests.
> >>
> >
> > I reviewed and tested this patch by nvdimm testing driver.
> > Please feel free to add
> >         Reviewed-by: Lee, Chun-Yi <jlee@suse.com>
> >
> > But I have a question to run nfit testing driver on OVMF. I always got
> > "coherent allocation failed" as below:
> >
> > [   86.530950] platform nfit_test.0: swiotlb buffer is full (sz: 134217728 bytes)
> > [   86.530954] swiotlb: coherent allocation failed for device nfit_test.0 size=134217728
> > [   86.530956] CPU: 0 PID: 1291 Comm: insmod Tainted: G           OE   4.7.0-rc7-default+ #323
> > [   86.530958] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.0.0 02/06/2015
> > [   86.530959]  0000000000000000 ffff880234e63bb0 ffffffff81334e5c 0000000008000000
> > [   86.530961]  ffff880232fa5998 ffff880234e63bf0 ffffffff8135eb79 ffff88020000000f
> > [   86.530963]  00000000024002c0 ffff880232fa5998 0000000008000000 ffff880234e63c68
> > [   86.530965] Call Trace:
> > [   86.530971]  [<ffffffff81334e5c>] dump_stack+0x63/0x87
> > [   86.530975]  [<ffffffff8135eb79>] swiotlb_alloc_coherent+0x149/0x160
> > [   86.530978]  [<ffffffff810611d3>] x86_swiotlb_alloc_coherent+0x43/0x50
> > [   86.530981]  [<ffffffffa054db3a>] dma_alloc_attrs.constprop.10+0x5a/0x80 [nfit_test]
> > [   86.530984]  [<ffffffffa00d618c>] nfit_test_init+0x18c/0x1000 [nfit_test]
> > [   86.530985]  [<ffffffffa00d6000>] ? 0xffffffffa00d6000
> > [   86.530988]  [<ffffffff81002190>] do_one_initcall+0x50/0x190
> > [   86.530992]  [<ffffffff811eef00>] ? kmem_cache_alloc_trace+0x170/0x220
> > [   86.530995]  [<ffffffff8118dbfc>] ? do_init_module+0x27/0x1e2
> > [   86.530997]  [<ffffffff8118dc35>] do_init_module+0x60/0x1e2
> > [   86.531000]  [<ffffffff81108fb1>] load_module+0x1411/0x1c20
> > [   86.531002]  [<ffffffff81105cb0>] ? __symbol_put+0x40/0x40
> > [   86.531006]  [<ffffffff812dfbbd>] ? ima_post_read_file+0x3d/0x80
> > [   86.531009]  [<ffffffff811099c9>] SYSC_finit_module+0xa9/0xd0
> > [   86.531011]  [<ffffffff81109a0e>] SyS_finit_module+0xe/0x10
> > [   86.531013]  [<ffffffff81003a62>] do_syscall_64+0x62/0x110
> > [   86.531016]  [<ffffffff81630a61>] entry_SYSCALL64_slow_path+0x25/0x25
> > [   86.531018] platform nfit_test.0: need 128M of free cma
> >
> > I tried to add "swiotlb=65536" and "cma=128M" or "cma=256M", even "cma=2G" kernel
> > parameters, but nfit-test.ko still can NOT allocate DMA success.
> >
> > So, I always add a patch to change the DIMM_SIZE to SZ_1M, that means test0
> > only uses 4M. Then the nfit-test.ko works for testing.
> >
> > Did I miss anything before running the nfit testing driver?
> 
> The CMA dependency is a hassle, so for v4.8 I'm replacing it with vmalloc:
> 
> https://patchwork.kernel.org/patch/9199761/
> https://patchwork.kernel.org/patch/9199763/

Those patches work great!

Thanks a lot!
Joey Lee