From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============1296849044462420448=="
MIME-Version: 1.0
From: archiemorningstar88 at gmail.com
Subject: [SPDK] Using cmb_copy in QEMU environment
Date: Sat, 18 Apr 2020 20:58:09 +0000
Message-ID: <20200418205809.2799.37485@ml01.vlan13.01.org>
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============1296849044462420448==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Hi,

I've been trying to get cmb_copy (examples/nvme/cmb_copy) to work using an =
emulated SSD with CMB in a QEMU environment, but cmb_copy cannot allocate a=
 CMB buffer. The identify (examples/nvme/identify) reports the two emulated=
 SSD's I created have 128MB of CMB. I'm running QEMU v4.2.0 with the guest =
OS Ubuntu 18.04. I've also tried various revisions of SPDK (20.01.1, 19.10.=
1, 19.04.1, 18.04.1) with similar results.

I see these two errors print out which I believe couldn't register the memo=
ry so the call to spdk_nvme_ctrlr_alloc_cmb_io_buffer() fails (pctrlr->cmb_=
bar_virt_addr is NULL):
could not get phys addr for 0x1088200000
nvme_pcie.c: 554:nvme_pcie_ctrlr_map_cmb: *ERROR*: spdk_mem_register() fail=
ed

I tried to trace the code and see that when it searches for the physical ad=
dress g_vtophys_pci_devices list is empty when trying to register the CMB m=
emory space. It only gets later populated with the two SSDs when cleanup_pc=
i_devices() is called at the end of spdk_pci_device_attach().

My qemu startup:
x86_64-softmmu/qemu-system-x86_64 \
-cpu host \
-machine q35,accel=3Dkvm,kernel-irqchip=3Dsplit \
-smp 8 \
-m 32768 \
-enable-kvm \
-drive if=3Dvirtio,file=3Dtest1.qcow2,cache=3Dnone \
-vnc :1 \
-device e1000,netdev=3Dnet0 \
-netdev user,id=3Dnet0,hostfwd=3Dtcp::5555-:22 \
-drive file=3Dnvme0.img,if=3Dnone,id=3DD22,format=3Draw \
-device nvme,drive=3DD22,serial=3D1234,cmb_size_mb=3D128 \
-drive file=3Dnvme1.img,if=3Dnone,id=3DD23,format=3Draw \
-device nvme,drive=3DD23,serial=3D5678,cmb_size_mb=3D128 \
-device intel-iommu,intremap=3Don \

If anyone has any suggestions it would be greatly appreciated.

Archie
--===============1296849044462420448==--