Question regarding Virtio Console and Remoteproc

Question regarding Virtio Console and Remoteproc

[Sjur BRENDELAND]

Hi Amit and Rusty,

I've been looking into the possibility of using the Virtio Console
Driver together with the remoteproc framework to communicate with
ST-Ericsson modem over shared memory.

It seems like Virtio Console would be a good fit, except for a issue
with buffer allocation. Due to HW limitations the STE-Modem cannot
access kernel memory (no IOMMU and limited address range). Instead
we have a designated shared memory region used for IPC. 

Due to this I cannot use kmalloc() for buffer allocation, but I
have to allocate buffers from the memory region shared with the
modem.

In remoteproc this is solved by using dma_alloc_coherent() for all
memory to be shared with the modem. This works fine for me, because
I can pass the IPC memory region to dma_declare_coherent_memory() 
so dma_alloc_coherent() will allocate from this memory region.

I think I can solve this issue in Virtio Console by changing calls
to kmalloc() to something like:

	if (virtio_has_feature(vdev, VIRTIO_CONSOLE_USE_DMA_MEM)) {
		dma_addr_t dma;
		buf = dma_alloc_coherent(dev, size, &dma, GFP_KERNEL);
	} else
		buf = kmalloc(count, GFP_KERNEL);

I'd like to get the opinion from you virtualization folks on this!
If you think it looks reasonable I might start cooking some patches...

Regards,
Sjur

Re: Question regarding Virtio Console and Remoteproc

[Ohad Ben-Cohen]

On Fri, Jun 1, 2012 at 10:31 AM, Sjur BRENDELAND
<sjur.brandeland@stericsson.com> wrote:
>        if (virtio_has_feature(vdev, VIRTIO_CONSOLE_USE_DMA_MEM)) {
>                dma_addr_t dma;
>                buf = dma_alloc_coherent(dev, size, &dma, GFP_KERNEL);
>        } else
>                buf = kmalloc(count, GFP_KERNEL);

Something along those lines is also needed for remote processors which
access memory via an IOMMU (e.g. OMAP4's M3 and DSP).

Allocating the memory via the DMA API will seamlessly configure the
relevant IOMMU as needed, and will make the buffers accessible to the
remote processors.

Thanks,
Ohad.

Re: Question regarding Virtio Console and Remoteproc

[Rusty Russell]

On Fri, 1 Jun 2012 10:51:38 +0300, Ohad Ben-Cohen <ohad@wizery.com> wrote:
> On Fri, Jun 1, 2012 at 10:31 AM, Sjur BRENDELAND
> <sjur.brandeland@stericsson.com> wrote:
> >        if (virtio_has_feature(vdev, VIRTIO_CONSOLE_USE_DMA_MEM)) {
> >                dma_addr_t dma;
> >                buf = dma_alloc_coherent(dev, size, &dma, GFP_KERNEL);
> >        } else
> >                buf = kmalloc(count, GFP_KERNEL);
> 
> Something along those lines is also needed for remote processors which
> access memory via an IOMMU (e.g. OMAP4's M3 and DSP).
> 
> Allocating the memory via the DMA API will seamlessly configure the
> relevant IOMMU as needed, and will make the buffers accessible to the
> remote processors.
> 
> Thanks,
> Ohad.

It seems quite sensible.  The formal definition in the spec would be
good.  In particular, defining DMA_MEM in a generic (non-Linux) way will
be interesting.

Thanks,
Rusty.
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: Question regarding Virtio Console and Remoteproc

[Benjamin Herrenschmidt]

On Mon, 2012-06-04 at 14:15 +0930, Rusty Russell wrote:
> > Something along those lines is also needed for remote processors
> which
> > access memory via an IOMMU (e.g. OMAP4's M3 and DSP).
> > 
> > Allocating the memory via the DMA API will seamlessly configure the
> > relevant IOMMU as needed, and will make the buffers accessible to
> the
> > remote processors.
> > 
> > Thanks,
> > Ohad.
> 
> It seems quite sensible.  The formal definition in the spec would be
> good.  In particular, defining DMA_MEM in a generic (non-Linux) way
> will be interesting.

I think we could do a bit better...

We need to dbl check if all the archs we care about use the dma_ops
abstraction, but if they do, then we can remove the conditional and just
use the dma_* functions everywhere, and have virtio-pci itself tweak the
dma_ops pointer in struct device.

The default would be to swap the normal PCI one that the arch code has
put there with some "null" ops to match existing virtio, and we could
then use a negociated capability to avoid that behaviour.

It's important to keep in mind that for KVM, in most cases we want to
keep the bypass of the iommu, simply because it's always going to be
faster than emulating one & mapping/unmapping things to/from it.


Cheers,
Ben.

Re: Question regarding Virtio Console and Remoteproc

[Amit Shah]

Hi Sjur,

On (Fri) 01 Jun 2012 [09:31:30], Sjur BRENDELAND wrote:
> Hi Amit and Rusty,
> 
> I've been looking into the possibility of using the Virtio Console
> Driver together with the remoteproc framework to communicate with
> ST-Ericsson modem over shared memory.
> 
> It seems like Virtio Console would be a good fit, except for a issue
> with buffer allocation. Due to HW limitations the STE-Modem cannot
> access kernel memory (no IOMMU and limited address range). Instead
> we have a designated shared memory region used for IPC. 
> 
> Due to this I cannot use kmalloc() for buffer allocation, but I
> have to allocate buffers from the memory region shared with the
> modem.
> 
> In remoteproc this is solved by using dma_alloc_coherent() for all
> memory to be shared with the modem. This works fine for me, because
> I can pass the IPC memory region to dma_declare_coherent_memory() 
> so dma_alloc_coherent() will allocate from this memory region.
> 
> I think I can solve this issue in Virtio Console by changing calls
> to kmalloc() to something like:
> 
> 	if (virtio_has_feature(vdev, VIRTIO_CONSOLE_USE_DMA_MEM)) {
> 		dma_addr_t dma;
> 		buf = dma_alloc_coherent(dev, size, &dma, GFP_KERNEL);
> 	} else
> 		buf = kmalloc(count, GFP_KERNEL);
> 
> I'd like to get the opinion from you virtualization folks on this!
> If you think it looks reasonable I might start cooking some patches...

I don't have a problem with this.

Thanks,
		Amit