Re: Vfio-on-arm: the device tree does not contain the DMA device

[Shannon Zhao <zhaoshenglong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>]

Hi Bharat,


ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan  1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found

Thanks,
Shannon
On 2014/6/24 16:08, Bharat.Bhushan-KZfg59tc24xl57MIdRCFDg@public.gmane.org wrote:
> 
> 
>> -----Original Message-----
>> From: Shannon Zhao [mailto:zhaoshenglong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>> Sent: Tuesday, June 24, 2014 1:16 PM
>> To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org;
>> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org;
>> zhangchengxing-hv44wF8Li93QT0dZR+AlfA@public.gmane.org
>> Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device
>>
>> Hi Bharat,
>>
>> Thanks for your help.
>>
>> I executed the following cmd:
>> 	echo vfio-platform >
>> /sys/bus/platform/devices/2c0a0000.dma/driver_override
> 
> You have to bind the device with vfio-platform driver after the above command.
> 
>> But there is no /dev/vfio/0 and error reported starting guest
>>
>> 	-nographic can not be used with -daemonize
>> 	char device redirected to /dev/pts/3 (label pty0-base)
>> 	PL330_VFIO: Group is not viable (not all devices bound for vfio)
> 
> A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.
> 
> Also looks like there are more device attached to the said iommu-group.
> Can check by running below command:
> #ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices
> 
> Thanks
> -Bharat
> 
>> 	qemu-system-arm: Error while probing real device.
>>
>> 	qemu-system-arm: Initialization of device pl330 failed Is there something
>> wrong?
>>
>> Thank you very much.
>> Shannon
>>
>> On 2014/6/24 15:05, Bharat.Bhushan-KZfg59tc24xl57MIdRCFDg@public.gmane.org wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Shannon Zhao [mailto:zhaoshenglong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>>>> Sent: Tuesday, June 24, 2014 12:30 PM
>>>> To: Alvise Rigo; Bhushan Bharat-R65777;
>>>> a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
>>>> kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; zhangchengxing-hv44wF8Li93QT0dZR+AlfA@public.gmane.org
>>>> Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
>>>> device
>>>>
>>>> Hi Alvise,
>>>>
>>>> Thanks for your help.
>>>> Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo
>>>> 2c0a0000.dma
>>>>> /sys/bus/platform/drivers/vfio-platform/vfio_bind".
>>>> But when using qemu to start guest, there is an error :
>>>>
>>>> -nographic can not be used with -daemonize char device redirected to
>>>> /dev/pts/3 (label pty0-base)
>>>> PL330_VFIO: === VFIO device file descriptor 13 === Device has 1
>>>> region(s):
>>>> PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
>>>> PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here
>>>> ]------------ kernel BUG at mm/slub.c:3386!
>>>> Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
>>>> CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
>>>> task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
>>>> kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
>>>> pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013
>>>> sp : ed0b9e64  ip : 000800f3  fp : 000014ac
>>>> r10: 00000000  r9 : ed0b8000  r8 : edeb3b80
>>>> r7 : 0000000e  r6 : 80620584  r5 : 00000000  r4 : eea0b000
>>>> r3 : 00000400  r2 : 800f38b0  r1 : eea0ce60  r0 : 00000400
>>>> Flags: nZcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
>>>> Control: 30c53c7d  Table: eddac080  DAC: 00000000 Process
>>>> qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)
>>>> Stack: (0xed0b9e64 to 0xed0ba000)
>>>> 9e60:          ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
>>>> 9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d
>>>> 80319208
>>>> 9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024
>>>> 008b2074
>>>> 9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000
>>>> 00000000
>>>> 9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780
>>>> 803155d0
>>>> 9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020
>>>> ed0b9f50
>>>> 9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000
>>>> 806522f8
>>>> 9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0
>>>> 00000000
>>>> 9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000
>>>> 00000000
>>>> 9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d
>>>> 00000036
>>>> 9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060
>>>> 00000001
>>>> 9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220
>>>> 000014ac
>>>> 9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000
>>>> 8062c770 [<800d9e3c>] (kfree) from [<80319094>]
>>>> (vfio_set_trigger.isra.5+0x40/0x104)
>>>> [<80319094>] (vfio_set_trigger.isra.5) from [<80319208>]
>>>> (vfio_platform_set_irq_trigger+0xb0/0x144)
>>>> [<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>]
>>>> (vfio_platform_set_irqs_ioctl+0x44/0x58)
>>>> [<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>]
>>>> (vfio_platform_ioctl+0x2e0/0x314)
>>>> [<8031886c>] (vfio_platform_ioctl) from [<803155d0>]
>>>> (vfio_device_fops_unl_ioctl+0x20/0x2c)
>>>> [<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>]
>>>> (do_vfs_ioctl+0x78/0x5b8)
>>>> [<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c)
>>>> [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>]
>>>> (ret_fast_syscall+0x0/0x30)
>>>> Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)
>>>>
>>>> I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu.
>>>> Following is qemu cmdline:
>>>> 	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
>>>>        	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
>>>>        	-serial chardev:pty0 -daemonize \
>>>>        	-device
>>>> pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai
>>>> ner="/dev/v
>>>> fio/vfio" \
>>>>        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
>>>>        	-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
>>>> - nographic
>>>>
>>>> If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio".
>>>> But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".
>>>
>>> Can you Check if you have driver_override at
>> "/sys/bus/platform/devices/DEVICE-XXX/ ?
>>> If yes then you should use driver_override mechanism to bind that device to
>> vfio-platform driver.
>>>
>>> Thanks
>>> -Bharat
>>>
>>>> And if I use above qemu cmdline to start guest, there is also an error :
>>>> 	PL330_VFIO: Group is not viable (not all devices bound for vfio)
>>>> 	qemu-system-arm: Error while probing real device.
>>>>
>>>> 	qemu-system-arm: Initialization of device pl330 failed
>>>>
>>>> The usage about v4 and v6 has changed ?
>>>> And would you tell me when you will update the guideline about vfio-on-arm?
>>>>
>>>> Thanks,
>>>> Shannon
>>>> On 2014/6/20 16:37, Alvise Rigo wrote:
>>>>> Linaro has a great set of pre-built images here http://releases.linaro.org/.
>>>>>
>>>>> For example these should work fine:
>>>>> http://releases.linaro.org/13.12/ubuntu/saucy-images
>>>>>
>>>>> alvise
>>>>>
>>>>> Il 20/06/2014 10:23, Shannon Zhao ha scritto:
>>>>>> Hi Alvise,
>>>>>>
>>>>>> I'm so sorry that I can't use qemu-debootstrap because my
>>>>>> environment can't
>>>> access some links using proxy.
>>>>>> Are there other ways to get a proper fs?
>>>>>>
>>>>>> Thanks
>>>>>> Shannon
>>>>>>
>>>>>> On 2014/6/20 15:15, Alvise Rigo wrote:
>>>>>>> Hi Shannon,
>>>>>>>
>>>>>>> If you follow the steps in the section "Creating the File System"
>>>>>>> of the guide you should be able to obtain a working fs.
>>>>>>>
>>>>>>> Regards,
>>>>>>> alvise
>>>>>>>
>>>>>>> Il 20/06/2014 04:57, Shannon Zhao ha scritto:
>>>>>>>> Hi Bharat,
>>>>>>>>
>>>>>>>> Thanks for your advice .
>>>>>>>>
>>>>>>>> I'm using the rootfs getting from the url
>>>> "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali
>>>> p-
>>>> armel.cramfs".
>>>>>>>>
>>>>>>>> Can you give me another link to get a newer rootfs?
>>>>>>>>
>>>>>>>> Thank you
>>>>>>>> Shannon
>>>>>>>>
>>>>>>>> On 2014/6/19 20:20, Bharat.Bhushan-KZfg59tc24xl57MIdRCFDg@public.gmane.org wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> -----Original Message-----
>>>>>>>>>> From: Shannon Zhao [mailto:zhaoshenglong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>>>>>>>>>> Sent: Thursday, June 19, 2014 5:47 PM
>>>>>>>>>> To: Bhushan Bharat-R65777; Alvise Rigo;
>>>>>>>>>> a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org;
>>>>>>>>>> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org
>>>>>>>>>> Subject: Re: Vfio-on-arm: the device tree does not contain the
>>>>>>>>>> DMA device
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 2014/6/19 20:12, Bharat.Bhushan-KZfg59tc24xl57MIdRCFDg@public.gmane.org wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>> From: Shannon Zhao [mailto:zhaoshenglong-hv44wF8Li93QT0dZR+AlfA@public.gmane.org]
>>>>>>>>>>>> Sent: Thursday, June 19, 2014 5:38 PM
>>>>>>>>>>>> To: Alvise Rigo; Bhushan Bharat-R65777;
>>>>>>>>>>>> a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org;
>>>>>>>>>>>> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
>>>>>>>>>>>> kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org
>>>>>>>>>>>> Subject: Re: Vfio-on-arm: the device tree does not contain
>>>>>>>>>>>> the DMA device
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Alvise,
>>>>>>>>>>>>
>>>>>>>>>>>> I run following cmds :
>>>>>>>>>>>> 	echo 2c0a0000.dma >
>>>>>>>>>>>> /sys/bus/platform/drivers/vfio-platform/vfio_bind
>>>>>>>>>>>
>>>>>>>>>>> run "ls /dev/vfio/" before and after the above command
>>>>>>>>>> Hi Bharat,
>>>>>>>>>>
>>>>>>>>>> Nothing appeared either before or after the cmd "echo
>>>>>>>>>> 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
>>>>>>>>>> There is only /dev/vfio.
>>>>>>>>>
>>>>>>>>> I am not sure, but may be you are using pretty old rootfs with
>>>>>>>>> broken udev
>>>> rules. Try to fetch a newer rootfs form somewhere.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> -Bharat
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> -Bharat
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> 	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
>>>>>>>>>>>>         	-device
>>>>>>>>>>>>
>>>>>>>>>>
>>>> pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai
>>>> ner="/dev/v
>>>>>>>>>>>> fio/vfio" \
>>>>>>>>>>>>         	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
>>>>>>>>>>>>         	-kernel zImage -dtb guest-a15.dtb -initrd
>>>>>>>>>>>> filesystem.cpio.gz
>>>> -
>>>>>>>>>>>> nographic But the error still exists.
>>>>>>>>>>>> 	qemu-system-arm: -device
>>>>>>>>>>>>
>>>>>>>>>>
>>>> pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=
>>>> /dev/vfio/v
>>>>>>>>>>>> fio: Parameter 'driver' expects pluggable device type
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Shannon
>>>>>>>>>>>>
>>>>>>>>>>>> On 2014/6/19 19:26, Alvise Rigo wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Il 19/06/2014 13:11, Shannon Zhao ha scritto:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2014/6/19 18:47, Alvise Rigo wrote:
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Il 19/06/2014 11:38, Shannon Zhao ha scritto:
>>>>>>>>>>>>>>>> Hi Bharat,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 2014/6/19 17:19, Bharat.Bhushan-KZfg59tc24xl57MIdRCFDg@public.gmane.org wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>>>>>>> From: kvmarm-bounces-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org
>>>>>>>>>>>>>>>>>> [mailto:kvmarm- bounces-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org] On
>>>>>>>>>>>>>>>>>> Behalf Of Shannon Zhao
>>>>>>>>>>>>>>>>>> Sent: Thursday, June 19, 2014 2:45 PM
>>>>>>>>>>>>>>>>>> To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org;
>>>>>>>>>>>>>>>>>> iommu-cunTk1MwBs/ROKNJybVBZg@public.gmane.org foundation.org;
>>>>>>>>>>>>>>>>>> kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org
>>>>>>>>>>>>>>>>>> Subject: Re: Vfio-on-arm: the device tree does not
>>>>>>>>>>>>>>>>>> contain the DMA device
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Alvise,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks for your reply.
>>>>>>>>>>>>>>>>>> I have apply the patch and could see the DMA devices.
>>>>>>>>>>>>>>>>>> But there is no vosys_dmatest int the directory
>>>>>>>>>>>>>>>>>> "/sys/kernel/debug/" after mounting debugfs.
>>>>>>>>>>>>>>>>>> I found that the branch vfio-platform-v4 doesn't
>>>>>>>>>>>>>>>>>> contain the module vosys_dmatest.
>>>>>>>>>>>>>>>>>> Then I port it from the branch pl330_dmatest and run
>>>> vosys_dmatest
>>>>>>>>>>>> successfully.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 	echo 1 > /sys/kernel/debug/vosys_dmatest/start
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 	[ 6145.343419] dmaengine: __dma_request_channel:
>>>>>>>>>>>>>>>>>> success
>>>>>>>>>>>> (dma0chan0)
>>>>>>>>>>>>>>>>>> 	[ 6145.344000] vosys dmatest: callback: transfer completed
>>>>>>>>>>>>>>>>>> 	[ 6145.344147] vosys dmatest: tx completed
>>>>>>>>>>>>>>>>>> 	[ 6145.344589] vosys dmatest: source and destination match.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Download the qemu and use following command to start a guest:
>>>>>>>>>>>>>>>>>> 	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
>>>>>>>>>>>>>>>>>> cortex-
>>>> a15 \
>>>>>>>>>>>>>>>>>>         	-device
>>>>>>>>>>>> pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
>>>>>>>>>>>>>>>>>>         	vfio_container="/dev/vfio/vfio" \
>>>>>>>>>>>>>>>>>>         	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"
>>>> \
>>>>>>>>>>>>>>>>>>         	-kernel zImage -dtb guest.dtb -initrd
>>>> filesystem.cpio.gz
>>>>>>>>>>>>>>>>>> - nographic An error reported:
>>>>>>>>>>>>>>>>>> 	qemu-system-arm: -device
>>>>>>>>>>>>>>>>>> pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Why do we need to give "vfio_group=/dev/vfio/0" ?
>>>>>>>>>>>>>>>> I refer to the url
>>>>>>>>>>>> "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 	could not open disk image vfio_container=/dev/vfio:
>>>>>>>>>>>>>>>>>> 	Could not open 'vfio_container=/dev/vfio': No such
>>>>>>>>>>>>>>>>>> file or directory
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Are you unbinding "2c0a0000.dma" from its driver and
>>>>>>>>>>>>>>>>> binding with
>>>> "vfio-
>>>>>>>>>>>> playform" driver ?
>>>>>>>>>>>>>>>> How to unbind "2c0a0000.dma" from its driver?
>>>>>>>>>>>>>>>> I have tried following cmd to bind "2c0a0000.dma" with
>>>>>>>>>>>>>>>> "vfio-
>>>> playform"
>>>>>>>>>>>> driver.
>>>>>>>>>>>>>>>> 	echo 2c0a0000.dma >
>>>>>>>>>>>>>>>> /sys/bus/platform/drivers/vfio-platform/vfio_bind
>>>>>>>>>>>>>>>> But there is no /dev/vfio/0 or /dev/vfio/vfio .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you please verify that while configuring the host you enabled:
>>>>>>>>>>>>>>> - Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
>>>>>>>>>>>>>>> System MMU
>>>>>>>>>>>>>>> (SMMU) Support and
>>>>>>>>>>>>>>> - Device Drivers -> VFIO Non-Privileged userspace driver
>>>>>>>>>>>>>>> framework
>>>>>>>>>>>>>>> -> VFIO support for platform devices
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Alvise,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I have checked the two configuration are enabled.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Then try the following cmd, but another error reported:
>>>>>>>>>>>>>> 	qemu-system-arm: -device
>>>>>>>>>>>>>> pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
>>>>>>>>>>>>>> container=
>>>>>>>>>>>>>> /dev/vfio/vfio: Parameter 'driver' expects pluggable device
>>>>>>>>>>>>>> type
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Do I need to unbind the "2c0a0000.dma" from its driver and
>>>>>>>>>>>>>> binding
>>>> with
>>>>>>>>>>>> "vfio-platform" driver as Bharat mentioned ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, the 2c0a0000.dma should not be bound to any driver at
>>>>>>>>>>>>> boot, so you just need to:
>>>>>>>>>>>>> echo 2c0a0000.dma >
>>>>>>>>>>>>> /sys/bus/platform/drivers/vfio-platform/vfio_bind
>>>>>>>>>>>>> to bind the device to the VFIO driver.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> alvise
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Then try to update the pl330-vfio-dev branch and run QEMU
>>>>>>>>>>>>>>> with the
>>>>>>>>>> following
>>>>>>>>>>>> command:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
>>>>>>>>>>>>>>>     -device
>>>>>>>>>>>>
>>>>>>>>>>
>>>> pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai
>>>> ner="/dev/v
>>>>>>>>>>>> fio/vfio" \
>>>>>>>>>>>>>>>     -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
>>>>>>>>>>>>>>>     -kernel zImage -dtb guest.dtb -initrd
>>>>>>>>>>>>>>> filesystem.cpio.gz
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> that means without breaking the "-device pl330" options.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> alvise
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>> -Bharat
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Is there something I'm wrong or do I miss some steps?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>> Shannon
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 2014/6/18 21:24, Alvise Rigo wrote:
>>>>>>>>>>>>>>>>>>> Hi Shannon,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you for reporting this issue, the device tree
>>>>>>>>>>>>>>>>>>> file was
>>>> indeed
>>>>>>>>>>>> missing.
>>>>>>>>>>>>>>>>>>> To solve the problem, you can apply the following
>>>>>>>>>>>>>>>>>>> patch to the repository you mentioned
>>>>>>>>>>>>>>>>>>> (git://github.com/virtualopensystems/arm-dts.git).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> For your information, soon we will update the guide
>>>>>>>>>>>>>>>>>>> with a newer version of VFIO with this issue fixed.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>>>>>>>> alvise
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>>>>>  fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
>>>>>>>>>>>>>>>>>>> +++++++++++++++++++++++++++++-
>>>>>>>>>>>>>>>>>>>  1 file changed, 97 insertions(+), 3 deletions(-)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
>>>>>>>>>>>>>>>>>>> b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
>>>>>>>>>>>>>>>>>>> index ef1275b..79c6af2 100644
>>>>>>>>>>>>>>>>>>> --- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
>>>>>>>>>>>>>>>>>>> +++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
>>>>>>>>>>>>>>>>>>> @@ -142,7 +142,7 @@
>>>>>>>>>>>>>>>>>>>  		cpu7: cpu@7 {
>>>>>>>>>>>>>>>>>>>  			device_type = "cpu";
>>>>>>>>>>>>>>>>>>>  			compatible = "arm,cortex-a7";
>>>>>>>>>>>>>>>>>>> -			reg = <0x104>;
>>>>>>>>>>>>>>>>>>> +			reg = <0x103>;
>>>>>>>>>>>>>>>>>>>  			cluster = <&cluster1>;
>>>>>>>>>>>>>>>>>>>  			core = <&core7>;
>>>>>>>>>>>>>>>>>>>  		};
>>>>>>>>>>>>>>>>>>> @@ -159,7 +159,7 @@
>>>>>>>>>>>>>>>>>>>  		#address-cells = <0>;
>>>>>>>>>>>>>>>>>>>  		interrupt-controller;
>>>>>>>>>>>>>>>>>>>  		reg = <0 0x2c001000 0 0x1000>,
>>>>>>>>>>>>>>>>>>> -		      <0 0x2c002000 0 0x2000>,
>>>>>>>>>>>>>>>>>>> +		      <0 0x2c002000 0 0x1000>,
>>>>>>>>>>>>>>>>>>>  		      <0 0x2c004000 0 0x2000>,
>>>>>>>>>>>>>>>>>>>  		      <0 0x2c006000 0 0x2000>;
>>>>>>>>>>>>>>>>>>>  		interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
>>>>>>>>>>>>>>>>>>>  			     <1 10 0xf08>;
>>>>>>>>>>>>>>>>>>>  	};
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> +	smmu@2c0b0000 {
>>>>>>>>>>>>>>>>>>> +                compatible = "arm,smmu-v2";
>>>>>>>>>>>>>>>>>>> +                reg = <0x0 0x2c0b0000 0x0 0x10000>;
>>>>>>>>>>>>>>>>>>> +                #global-interrupts = <2>;
>>>>>>>>>>>>>>>>>>> +                interrupts = <0 58 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 59 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>,
>>>>>>>>>>>>>>>>>>> +                             <0 60 4>;
>>>>>>>>>>>>>>>>>>> +                mmu-masters = <&dma0 0x0 0x8>,
>>>>>>>>>>>>>>>>>>> +                              <&dma1 0x1 0x9>,
>>>>>>>>>>>>>>>>>>> +                              <&dma2 0x2 0xa>,
>>>>>>>>>>>>>>>>>>> +                              <&dma3 0x3 0xb>;
>>>>>>>>>>>>>>>>>>> +        };
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +	dma0: dma@2c0a0000 {
>>>>>>>>>>>>>>>>>>> +		compatible = "arm,pl330";
>>>>>>>>>>>>>>>>>>> +		reg = <0 0x2c0a0000 0 0x1000>;
>>>>>>>>>>>>>>>>>>> +		interrupts = <0 64 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 65 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 66 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 67 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 68 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 69 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 70 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 71 4>;
>>>>>>>>>>>>>>>>>>> +		#dma-cells = <1>;
>>>>>>>>>>>>>>>>>>> +		#stream-id-cells = <2>;
>>>>>>>>>>>>>>>>>>> +		clocks = <&oscclk6a>;
>>>>>>>>>>>>>>>>>>> +		clock-names = "apb_pclk";
>>>>>>>>>>>>>>>>>>> +	};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +	dma1: dma@2c0a1000 {
>>>>>>>>>>>>>>>>>>> +		compatible = "arm,pl330", "arm,primecell";
>>>>>>>>>>>>>>>>>>> +		reg = <0 0x2c0a1000 0 0x1000>;
>>>>>>>>>>>>>>>>>>> +		interrupts = <0 96 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 97 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 98 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 99 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 100 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 101 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 102 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 103 4>;
>>>>>>>>>>>>>>>>>>> +		#dma-cells = <1>;
>>>>>>>>>>>>>>>>>>> +		#stream-id-cells = <2>;
>>>>>>>>>>>>>>>>>>> +		clocks = <&oscclk6a>;
>>>>>>>>>>>>>>>>>>> +		clock-names = "apb_pclk";
>>>>>>>>>>>>>>>>>>> +	};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +	dma2: dma@2c0a2000 {
>>>>>>>>>>>>>>>>>>> +		compatible = "arm,pl330", "arm,primecell";
>>>>>>>>>>>>>>>>>>> +		reg = <0 0x2c0a2000 0 0x1000>;
>>>>>>>>>>>>>>>>>>> +		interrupts = <0 128 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 129 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 130 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 131 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 132 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 133 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 134 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 135 4>;
>>>>>>>>>>>>>>>>>>> +		#dma-cells = <1>;
>>>>>>>>>>>>>>>>>>> +		#stream-id-cells = <2>;
>>>>>>>>>>>>>>>>>>> +		clocks = <&oscclk6a>;
>>>>>>>>>>>>>>>>>>> +		clock-names = "apb_pclk";
>>>>>>>>>>>>>>>>>>> +	};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +	dma3: dma@2c0a3000 {
>>>>>>>>>>>>>>>>>>> +		compatible = "arm,pl330", "arm,primecell";
>>>>>>>>>>>>>>>>>>> +		reg = <0 0x2c0a3000 0 0x1000>;
>>>>>>>>>>>>>>>>>>> +		interrupts = <0 160 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 161 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 162 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 163 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 164 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 165 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 166 4>,
>>>>>>>>>>>>>>>>>>> +			     <0 167 4>;
>>>>>>>>>>>>>>>>>>> +		#dma-cells = <1>;
>>>>>>>>>>>>>>>>>>> +		#stream-id-cells = <2>;
>>>>>>>>>>>>>>>>>>> +		clocks = <&oscclk6a>;
>>>>>>>>>>>>>>>>>>> +		clock-names = "apb_pclk";
>>>>>>>>>>>>>>>>>>> +	};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>> +	oscclk6a: oscclk6a {
>>>>>>>>>>>>>>>>>>> +		/* Reference 24MHz clock */
>>>>>>>>>>>>>>>>>>> +		compatible = "fixed-clock";
>>>>>>>>>>>>>>>>>>> +		#clock-cells = <0>;
>>>>>>>>>>>>>>>>>>> +		clock-frequency = <24000000>;
>>>>>>>>>>>>>>>>>>> +		clock-output-names = "oscclk6a";
>>>>>>>>>>>>>>>>>>> +	};
>>>>>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>>>>>>  	smb {
>>>>>>>>>>>>>>>>>>>  		compatible = "simple-bus";
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> @@ -244,4 +339,3 @@
>>>>>>>>>>>>>>>>>>>  		/include/ "rtsm_ve-motherboard.dtsi"
>>>>>>>>>>>>>>>>>>>  	};
>>>>>>>>>>>>>>>>>>>  };
>>>>>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Shannon
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> kvmarm mailing list
>>>>>>>>>>>>>>>>>> kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org
>>>>>>>>>>>>>>>>>> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> .
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Shannon
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> .
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Shannon
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> .
>>>>>>>
>>>>>>
>>>>>
>>>>> .
>>>>>
>>>>
>>>> --
>>>> Shannon
>>>
>>
>> --
>> Shannon
> 

-- 
Shannon