Re: remoteproc: Load coprocessor code to the specific main memory location

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> On Wed, Jan 25, 2012 at 2:41 PM, Michal Simek <monstr@monstr.eu> wrote:
>> First of all will be the best to describe what we want to achieve.
>> We have Zynq platform ARM Cortex-A9(which should be ARMv7 - 2cpus).
>> cpu0 runs Linux and cpu1 will be used for real-time os. We would like to use
>> remoteproc for loading any code(firmware if you like) for the cpu1 and start
>> it.
> 
> Sounds cool.
> 
> So you're not < ARMv6, and I now understand why you bumped into that
> code path: you're using a vanilla kernel, without CMA.

I started with pur vanilla kernel and patches from your tree
git://git.kernel.org/pub/scm/linux/kernel/git/ohad/remoteproc.git
for-next branch
but then I had to moved to for-next-acked-merged because in the origin
one were some problems.

> 
> I suggest you merge Marek's CMA branch into your tree. Though 1MB
> isn't that much really, and you can easily get away even without CMA.
> You just won't have any non-CMA public example to use as a reference
> ;).

I use 1MB for now. It is hard to say how big that real-time OS with app will be.
I will merge it. Hopefully there won't be conflicts.

Do you have any kernel tree with CMA just for sure?

> 
> For details about the CMA tree and branches, see Marek's email:
> 
> https://lkml.org/lkml/2012/1/10/48

No problem to do so. I have read CMA some time ago.

> 
>> Then in zynq_rproc_kick I copy content of this 1MB carveout to 0x0 physical
> 
> I recommend to keep the ->kick() handler as light as possible, as it
> may be frequently triggered (think of a network driver doing I/O).

It will be no problem to keep it pretty light when I am able to map firmware to
location I like.

> 
> Ideally you should do the minimum required to only wake up the other
> core when you are kicked. Copying a 1MB region every kick sounds
> excessive.

Sure I know. It is just hack to proof me that I am able to do what I want to do.
Currently it is right time to do it in way which could be possible to add to mainline
in future.

> 
>> The main my problem is that firmware base addr must be at the beginning of
>> address space
>> or selected address but it is fixed and known.
> ..
>> ok. How do you handle limitations if coprocessor expect code on any address?
> 
> Just make sure you reserve the CMA region at that fixed address.
> 
> For a reference how this can be done, look for OMAP_RPROC_CMA_BASE in
> my rpmsg_3.2_rc4 branch.

I have looked at it. Please correct me if I am wrong.

You use 0xa9800000 address which is pretty good.
What I can do is to say - all sw for cpu1 has to start at 0x0f000000 which
give me enough space for Linux kernel. And CMA can allocate this place
in bootup time. Probably it is not the best to use physical address 0x0
because it is mapped to virtual 0xC0000000 and have to solve all problems
with linux text base.


>>> Looping in Mark, who did the davinci remoteproc support, and might
>>> have the code available to share. Hopefully we'd have the patches
>>> posted soon, and then no-iommu platforms could just use them for
>>> reference.
>>
>> ok. great. Will look forward for that code and recommendation because I am
>> ARM newbie.
> 
> So I no longer think you need the davinci code, you can just follow
> the OMAP example.

Good examples are always welcomed - it can show others how to do it in the right way. :-)

>> I am also interested in suggestions about DTS binding. Have someone tried to
>> describe it or use it?
> 
> No, AFAICT. And I'm not sure there are DTS binding for CMA regions
> yet. Feel free to take a stab at it though ;)

ok. Will wait for Grants suggestions. I have added devicetree mailing list to CC list too.

What do you use for firmware replacing?
Just remove kernel module and load different one?

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Wed, Jan 25, 2012 at 4:19 PM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
> I started with pur vanilla kernel and patches from your tree
> git://git.kernel.org/pub/scm/linux/kernel/git/ohad/remoteproc.git
> for-next branch
> but then I had to moved to for-next-acked-merged because in the origin
> one were some problems.

Unlike for-next-acked-merged, the other branches aren't rebased and
therefore have merge conflicts with 3.3 material; that's intentional.

> Do you have any kernel tree with CMA just for sure?

Not a 3.3-based one, so I recommend you to just merge Marek's branch directly.

I'll soon create a full-fledged 3.3 branch though, so you may want to
wait a bit until I get to it.

> What do you use for firmware replacing?
> Just remove kernel module and load different one?

If your driver boots the remote core on its ->probe() handler, then
yes, you need to unbind and bind the device.

One way is to unload and load the driver, but you can also use the
bind/unbind sysfs files of your driver (echo -n device-name >
/sys/..../{un}bind).

Regards,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> On Wed, Jan 25, 2012 at 4:19 PM, Michal Simek <monstr@monstr.eu> wrote:
>> I started with pur vanilla kernel and patches from your tree
>> git://git.kernel.org/pub/scm/linux/kernel/git/ohad/remoteproc.git
>> for-next branch
>> but then I had to moved to for-next-acked-merged because in the origin
>> one were some problems.
> 
> Unlike for-next-acked-merged, the other branches aren't rebased and
> therefore have merge conflicts with 3.3 material; that's intentional.
> 
>> Do you have any kernel tree with CMA just for sure?
> 
> Not a 3.3-based one, so I recommend you to just merge Marek's branch directly.
> 
> I'll soon create a full-fledged 3.3 branch though, so you may want to
> wait a bit until I get to it.

I have done it and tested. It works as expected. Thanks.
I have created region at phys addr 0x10000000 with selected size.
I had to move carveout in resource table to be first in this area.
Also removed that copying from kick function.



I have one small problem which is that physical address is 0x10000000
which means that firmware entry point is the same.
In rproc_load_segments is da composed from phdr->p_paddr which is 0x10000000.
And code is designed that this load addr is offset.

Here is the code:
	/* go through the available ELF segments */
	for (i = 0; i < ehdr->e_phnum; i++, phdr++) {
		u32 da = phdr->p_paddr; // OFFSET 0x10000000
		u32 memsz = phdr->p_memsz;

But for my case is physical address correct and it is not offset 0x10000000.
I have created temporary fix which substract virtual address to get correct da.

u32 da = phdr->p_paddr - phdr->p_vaddr;

What should be correct solution?

> 
>> What do you use for firmware replacing?
>> Just remove kernel module and load different one?
> 
> If your driver boots the remote core on its ->probe() handler, then
> yes, you need to unbind and bind the device.
> 
> One way is to unload and load the driver, but you can also use the
> bind/unbind sysfs files of your driver (echo -n device-name >
> /sys/..../{un}bind).

ok. It also works.

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Michal Simek wrote:
> Ohad Ben-Cohen wrote:
>> On Wed, Jan 25, 2012 at 4:19 PM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
>>> I started with pur vanilla kernel and patches from your tree
>>> git://git.kernel.org/pub/scm/linux/kernel/git/ohad/remoteproc.git
>>> for-next branch
>>> but then I had to moved to for-next-acked-merged because in the origin
>>> one were some problems.
>>
>> Unlike for-next-acked-merged, the other branches aren't rebased and
>> therefore have merge conflicts with 3.3 material; that's intentional.
>>
>>> Do you have any kernel tree with CMA just for sure?
>>
>> Not a 3.3-based one, so I recommend you to just merge Marek's branch 
>> directly.
>>
>> I'll soon create a full-fledged 3.3 branch though, so you may want to
>> wait a bit until I get to it.
> 
> I have done it and tested. It works as expected. Thanks.
> I have created region at phys addr 0x10000000 with selected size.
> I had to move carveout in resource table to be first in this area.
> Also removed that copying from kick function.
> 
> 
> 
> I have one small problem which is that physical address is 0x10000000
> which means that firmware entry point is the same.
> In rproc_load_segments is da composed from phdr->p_paddr which is 
> 0x10000000.
> And code is designed that this load addr is offset.
> 
> Here is the code:
>     /* go through the available ELF segments */
>     for (i = 0; i < ehdr->e_phnum; i++, phdr++) {
>         u32 da = phdr->p_paddr; // OFFSET 0x10000000
>         u32 memsz = phdr->p_memsz;
> 
> But for my case is physical address correct and it is not offset 
> 0x10000000.
> I have created temporary fix which substract virtual address to get 
> correct da.
> 
> u32 da = phdr->p_paddr - phdr->p_vaddr;
> 
> What should be correct solution?

Just follow up. I have find out that moving to 0x10000000 offset will be problematic
because there must be vector table at 0x0. IIUC Linux remaps reset vectors to 0xffff0000 offset
through cp15 reg in bootup code.

Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Thu, Jan 26, 2012 at 12:44 PM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
> I have one small problem which is that physical address is 0x10000000
> which means that firmware entry point is the same.

This is what we do with the davinci DSPs, too.

> In rproc_load_segments is da composed from phdr->p_paddr which is
> 0x10000000.

Ok, I don't see any issue here.

> And code is designed that this load addr is offset.

Not sure exactly what do you mean by that ?

> Here is the code:
>        /* go through the available ELF segments */
>        for (i = 0; i < ehdr->e_phnum; i++, phdr++) {
>                u32 da = phdr->p_paddr; // OFFSET 0x10000000
>                u32 memsz = phdr->p_memsz;
>
> But for my case is physical address correct and it is not offset 0x10000000.

Again, I'm not sure what exactly is the issue. p_paddr is the physical
address where the image is expected, that sounds ok to me.

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Hi Ohad,

Ohad Ben-Cohen wrote:
> On Thu, Jan 26, 2012 at 12:44 PM, Michal Simek <monstr@monstr.eu> wrote:
>> I have one small problem which is that physical address is 0x10000000
>> which means that firmware entry point is the same.
> 
> This is what we do with the davinci DSPs, too.
> 
>> In rproc_load_segments is da composed from phdr->p_paddr which is
>> 0x10000000.
> 
> Ok, I don't see any issue here.
> 
>> And code is designed that this load addr is offset.
> 
> Not sure exactly what do you mean by that ?
> 
>> Here is the code:
>>        /* go through the available ELF segments */
>>        for (i = 0; i < ehdr->e_phnum; i++, phdr++) {
>>                u32 da = phdr->p_paddr; // OFFSET 0x10000000
>>                u32 memsz = phdr->p_memsz;
>>
>> But for my case is physical address correct and it is not offset 0x10000000.
> 
> Again, I'm not sure what exactly is the issue. p_paddr is the physical
> address where the image is expected, that sounds ok to me.

sorry for delay. I was working on different issue.
I have solved this issue by using ram address ranges 0x0 - 0x10000000.

I am using CONFIG_PHYS_OFFSET=0x10000000 in linux which give me free space for rtos.

Here is how it is achieve
	ret = dma_declare_coherent_memory(&zynq_freertos.dev, 0,
				0, 0x10000000, DMA_MEMORY_MAP);

which is fully compatible with remoteproc.
I have also got trace buffer to work - there were problem with caches on cpu with rtos.

I still have the problem to reset cpu1 but we are investigating how to do so. Maybe in future
we will add new message which jumps back to bootloader code if not possible to do it throuth any


I have looked at omap remoteproc and there is communication between cocpu through omap mbox
and IRQs. I am working on similar communication for testing
and I would like to know how vrings fit to this. Do you have any example of using it?


struct resource __resource resources[] = {
     { TYPE_CARVEOUT, 0, TEXT_BASE + 0,    0, 0, 0, SZ_1M, 0, 0,0,0,0,"IPU_MEM_TEXT"},
     { TYPE_VIRTIO_DEV,0, TEXT_BASE + 0,   0, 0, 0, SZ_1M, VIRTIO_ID_RPMSG,0,0,0,0,"vdev:rpmsg"},

     { TYPE_VRING, 0, TEXT_BASE + 0x20000, 0, 0, 0, VQ0_SIZE,0,0,0,0,0,"vring:sysm3->mpu"},
     { TYPE_VRING, 1, TEXT_BASE + 0x30000, 0, 0, 0, VQ1_SIZE,0,0,0,0,0,"vring:mpu->sysm3"},
     { TYPE_TRACE, 0, TEXT_BASE + 0x40000, 0, 0, 0, 0x8000, 0,0,0,0,0,"trace:sysm3"},
};

Below is my full bootup log.

Thanks,
Michal


Linux version 3.3.0-rc1+ (monstr@monstr.eu) (gcc version 4.5.1 (Sourcery G++ Lite 2010.09-62) ) #355 PREEMPT Mon Feb 13 15:10:48 CET 2012
CPU: ARMv7 Processor [410fc090] revision 0 (ARMv7), cr=10c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
Machine: Xilinx Zynq Platform, model: ZC702_jason
bootconsole [earlycon0] enabled
vmalloc_min ef800000, PAGE_OFFSET c0000000
Ignoring RAM at 00000000-0fffffff (vmalloc region overlap).
vmalloc_min ef800000, PAGE_OFFSET c0000000
dma_contiguous_reserve(limit 20000000)
dma_contiguous_reserve: total available: 256 MiB, size absolute: 1 MiB, size percentage: 0 MiB
dma_contiguous_reserve: reserving 1 MiB for global area
dma_declare_contiguous(size 100000, base 00000000, limit 20000000)
cma: CMA: reserved 8 MiB at 1f800000
Memory policy: ECC disabled, Data cache writeback
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 65024
Kernel command line:  console=ttyPS0,115200 earlyprintk
PID hash table entries: 1024 (order: 0, 4096 bytes)
Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
Memory: 256MB = 256MB total
Memory: 243984k/243984k available, 18160k reserved, 0K highmem
Virtual kernel memory layout:
     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
     fixmap  : 0xfff00000 - 0xfffe0000   ( 896 kB)
     vmalloc : 0xd0800000 - 0xff000000   ( 744 MB)
     lowmem  : 0xc0000000 - 0xd0000000   ( 256 MB)
     modules : 0xbf000000 - 0xc0000000   (  16 MB)
       .text : 0xc0008000 - 0xc0283000   (2540 kB)
       .init : 0xc0283000 - 0xc0753000   (4928 kB)
       .data : 0xc0754000 - 0xc076c0c0   (  97 kB)
        .bss : 0xc076c0e4 - 0xc07774c0   (  45 kB)
NR_IRQS:246
xlnx,ps7-ttc-1.00.a #0 at 0xd0800000, irq=43
Calibrating delay loop... 2531.32 BogoMIPS (lpj=12656640)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
CPU: Testing write buffer coherency: ok
hw perfevents: enabled with ARMv7 Cortex-A9 PMU driver, 1 counters available
Setting up static identity map for 0x101cc918 - 0x101cc94c
devtmpfs: initialized
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
L2x0 series cache controller enabled
l2x0: 8 ways, CACHE_ID 0x00000000, AUX_CTRL 0x02060000, Cache size: 524288 B
hw-breakpoint: debug architecture 0x0 unsupported.
Switching to clocksource xttcpss_timer1
NET: Registered protocol family 2
IP route cache hash table entries: 2048 (order: 1, 8192 bytes)
TCP established hash table entries: 8192 (order: 4, 65536 bytes)
TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 8192 bind 8192)
TCP reno registered
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
msgmni has been set to 492
e0001000.serial: ttyPS0 at MMIO 0xe0001000 (irq = 82) is a xuartps
console [ttyPS0] enabled, bootconsole disabled
console [ttyPS0] enabled, bootconsole disabled
GEM: BASEADDRESS hw: e000b000 virt: d0806000
XEMACPS mii bus: probed
GEM: phydev ceaf7800, phydev->phy_id 0x1410cc2, phydev->addr 0x7
GEM: MAC addr 00:b0:00:e0:ff:bf
eth0, pdev->id -1, baseaddr 0xe000b000, irq 54
eth0, phy_addr 0x7, phy_id 0x01410cc2
eth0, attach [Generic PHY] phy driver
zynq-rproc zynq-rproc.1: zynq_rproc_probe
zynq-rproc zynq-rproc.1: cpu1_freertos is available
zynq-rproc zynq-rproc.1: Note: remoteproc is still under development and considered experimental.
zynq-rproc zynq-rproc.1: THE BINARY FORMAT IS NOT YET FINALIZED, and backward compatibility isn't yet guaranteed.
rpmsg_init
rpmsg_probe
zynq-rproc zynq-rproc.1: powering up cpu1_freertos, firmware test
zynq-rproc zynq-rproc.1: Booting fw image test, size 203250
zynq-rproc zynq-rproc.1: iommu not found
zynq-rproc zynq-rproc.1: rsc: type 0, da 0x0, pa 0x0, len 0x100000, id 0, name IPU_MEM_TEXT, flags 0
zynq-rproc zynq-rproc.1: rsc: type 4, da 0x0, pa 0x0, len 0x100000, id 0, name vdev:rpmsg, flags 7
zynq-rproc zynq-rproc.1: rsc: type 3, da 0x20000, pa 0x0, len 0x100, id 0, name vring:sysm3->mpu, flags 0
zynq-rproc zynq-rproc.1: rsc: type 3, da 0x30000, pa 0x0, len 0x100, id 1, name vring:mpu->sysm3, flags 0
zynq-rproc zynq-rproc.1: rsc: type 2, da 0x40000, pa 0x0, len 0x8000, id 0, name trace:sysm3, flags 0
zynq-rproc zynq-rproc.1: phdr: type 1 da 0x0 memsz 0x1be80 filesz 0x1509c
zynq-rproc zynq-rproc.1: zynq_rproc_start
zynq-rproc zynq-rproc.1: zynq_device_enable Enable device - it means jump
zynq-rproc zynq-rproc.1: remote processor cpu1_freertos is now up
zynq-rproc zynq-rproc.1: zynq_rproc_kick vqid 0
zynq_mbox_msg_send 1
virtio_rpmsg_bus virtio0: rpmsg host is online
TCP cubic registered
NET: Registered protocol family 10
IPv6 over IPv4 tunneling driver
NET: Registered protocol family 17
NET: Registered protocol family 40
VFP support v0.3: implementor 41 architecture 3 part 40 variant 0 rev 0
Registering SWP/SWPB emulation handler
Freeing init memory: 4928K
Mounting proc:
Mounting var:
Populating /var:
Running local start scripts.
Mounting debugfs:
Mounting sysfs:
mdev: initialising /dev
mdev: Registering hotplug handler
/proc/mtd doesn't exist. Will not create /dev/flash/* device nodes
Mounting devpts:
Setting hostname:
Bringing up network interfaces:
GEM: lp->tx_bd cf841000 lp->tx_bd_dma 1f841000 lp->tx_skb ceb3bbc0
GEM: lp->rx_bd cf840000 lp->rx_bd_dma 1f840000 lp->rx_skb ceb3bac0
GEM: MAC 0xe000b000, 0x0000bfff, 00:b0:00:e0:ff:bf
Starting portmap:

Welcome to
  _____       _           _      _
| ___ \     | |         | |    (_)
| |_/ / ___ | |_   __ _ | |     _  _ __   _   _ __  __
|  __/ / _ \| __| / _` || |    | || '_ \ | | | |\ \/ /
| |   |  __/| |_ | (_| || |____| || | | || |_| | >  <
\_|    \___| \__| \__,_|\_____/|_||_| |_| \__,_|/_/\_\

on zynq-silicon


zynq-silicon login: root
Password:









-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Mon, Feb 13, 2012 at 3:20 PM, Michal Simek <monstr@monstr.eu> wrote:
> Here is how it is achieve
>        ret = dma_declare_coherent_memory(&zynq_freertos.dev, 0,
>                                0, 0x10000000, DMA_MEMORY_MAP);
>
> which is fully compatible with remoteproc.

Great, this is what we've been doing with davinci too.

> Do you have any example of using it?

What exactly are you looking for ? RTOS or Linux side ?

On the Linux side, you can just make the rpmsg sample work (it's part
of the rpmsg patch set). For the RTOS side, feel free to just take our
code (it's BSD licensed and hosted on github) and adapt it to your
environment.

Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> On Mon, Feb 13, 2012 at 3:20 PM, Michal Simek <monstr@monstr.eu> wrote:
>> Here is how it is achieve
>>        ret = dma_declare_coherent_memory(&zynq_freertos.dev, 0,
>>                                0, 0x10000000, DMA_MEMORY_MAP);
>>
>> which is fully compatible with remoteproc.
> 
> Great, this is what we've been doing with davinci too.

Cool.

> 
>> Do you have any example of using it?
> 
> What exactly are you looking for ? RTOS or Linux side ?

Both side will be great. Rtos is freertos.

> 
> On the Linux side, you can just make the rpmsg sample work (it's part
> of the rpmsg patch set). For the RTOS side, feel free to just take our
> code (it's BSD licensed and hosted on github) and adapt it to your
> environment.

You mean that server_sample and rpmsg_omx right?
I have also found rpmsg-omx demo application.
What I am missing is how to probe rpmsg bus.
Do you have any working example?

Communication between cpus is done through swirqs. I have test sending it from linux to rtos.

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Michal Simek wrote:
> Ohad Ben-Cohen wrote:
>> On Mon, Feb 13, 2012 at 3:20 PM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
>>> Here is how it is achieve
>>>        ret = dma_declare_coherent_memory(&zynq_freertos.dev, 0,
>>>                                0, 0x10000000, DMA_MEMORY_MAP);
>>>
>>> which is fully compatible with remoteproc.
>>
>> Great, this is what we've been doing with davinci too.
> 
> Cool.
> 
>>
>>> Do you have any example of using it?
>>
>> What exactly are you looking for ? RTOS or Linux side ?
> 
> Both side will be great. Rtos is freertos.
> 
>>
>> On the Linux side, you can just make the rpmsg sample work (it's part
>> of the rpmsg patch set). For the RTOS side, feel free to just take our
>> code (it's BSD licensed and hosted on github) and adapt it to your
>> environment.
> 
> You mean that server_sample and rpmsg_omx right?
> I have also found rpmsg-omx demo application.
> What I am missing is how to probe rpmsg bus.
> Do you have any working example?
> 
> Communication between cpus is done through swirqs. I have test sending 
> it from linux to rtos.

ok. How that rpmsg channels are created? Is it based on data sent from remoteproc?
Or based on resource table?

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Michal Simek wrote:
> Michal Simek wrote:
>> Ohad Ben-Cohen wrote:
>>> On Mon, Feb 13, 2012 at 3:20 PM, Michal Simek <monstr@monstr.eu> wrote:
>>>> Here is how it is achieve
>>>>        ret = dma_declare_coherent_memory(&zynq_freertos.dev, 0,
>>>>                                0, 0x10000000, DMA_MEMORY_MAP);
>>>>
>>>> which is fully compatible with remoteproc.
>>>
>>> Great, this is what we've been doing with davinci too.
>>
>> Cool.
>>
>>>
>>>> Do you have any example of using it?
>>>
>>> What exactly are you looking for ? RTOS or Linux side ?
>>
>> Both side will be great. Rtos is freertos.
>>
>>>
>>> On the Linux side, you can just make the rpmsg sample work (it's part
>>> of the rpmsg patch set). For the RTOS side, feel free to just take our
>>> code (it's BSD licensed and hosted on github) and adapt it to your
>>> environment.
>>
>> You mean that server_sample and rpmsg_omx right?
>> I have also found rpmsg-omx demo application.
>> What I am missing is how to probe rpmsg bus.
>> Do you have any working example?
>>
>> Communication between cpus is done through swirqs. I have test sending 
>> it from linux to rtos.
> 
> ok. How that rpmsg channels are created? Is it based on data sent from 
> remoteproc?
> Or based on resource table?

I should be more specific. Can you point me to remoteproc code which publish
remote service based on them Linux rpmsg driver will be probed? It is rtos part of code.
In your ELCE presentation is called rpmsg-client-sample service.

Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Thu, Feb 16, 2012 at 10:12 AM, Michal Simek <monstr@monstr.eu> wrote:
> I should be more specific. Can you point me to remoteproc code which publish
> remote service based on them Linux rpmsg driver will be probed? It is rtos
> part of code.
> In your ELCE presentation is called rpmsg-client-sample service.

Sorry for the late response (I'm in ELC).

Yes, use the client sample (which is part of the submitted patch set)
and not the omx/server samples (which are public, but were never
submitted nor cleaned up).

The channels are created dynamically: the rpmsg bus designates address
53 for a name service, which listens to channel creation/removal
announcements coming from the remote processor. Take a look in
virtio_rpmsg_bus.c and you'll immediately see it.

Hope this helps,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> On Thu, Feb 16, 2012 at 10:12 AM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
>> I should be more specific. Can you point me to remoteproc code which publish
>> remote service based on them Linux rpmsg driver will be probed? It is rtos
>> part of code.
>> In your ELCE presentation is called rpmsg-client-sample service.
> 
> Sorry for the late response (I'm in ELC).
> 
> Yes, use the client sample (which is part of the submitted patch set)
> and not the omx/server samples (which are public, but were never
> submitted nor cleaned up).

Client sample is clear because it is probed when services are published.

> 
> The channels are created dynamically: the rpmsg bus designates address
> 53 for a name service, which listens to channel creation/removal
> announcements coming from the remote processor. Take a look in
> virtio_rpmsg_bus.c and you'll immediately see it.

I see that values but I can't see how rtos should send that messages.

Have checked both vrings - one for rx and one for tx. They are
initialized correctly on addresses I like.

(gdb) x /80x 0xe1100000
0xe1100000:     0x31140000      0x00000000      0x00000200      0x00010002
0xe1100010:     0x31140200      0x00000000      0x00000200      0x00020002
0xe1100020:     0x31140400      0x00000000      0x00000200      0x00030002
0xe1100030:     0x31140600      0x00000000      0x00000200      0x00040002
0xe1100040:     0x31140800      0x00000000      0x00000200      0x00050002

(gdb) x /80x 0xe1104000
0xe1104000:     0x00000000      0x00000000      0x00000000      0x00010000
0xe1104010:     0x00000000      0x00000000      0x00000000      0x00020000
0xe1104020:     0x00000000      0x00000000      0x00000000      0x00030000
0xe1104030:     0x00000000      0x00000000      0x00000000      0x00040000
0xe1104040:     0x00000000      0x00000000      0x00000000      0x00050000
0xe1104050:     0x00000000      0x00000000      0x00000000      0x00060000
0xe1104060:     0x00000000      0x00000000      0x00000000      0x00070000



rtos can handle kick from Linux
Linux is able to catch kick from freertos and run rproc_vg_interrupt

 From code I see that in rpmsg_probe function is virtqueue_kick which means sending
kick to rtos. Then rtos can send the message.


Above is condition "if (virtio_has_feature(vdev, VIRTIO_RPMSG_F_NS))" which is failing for me.
Not sure if is correct.

I am still not sure how to propagate that messages from rtos side.
In TI software it is in NameMap.c file which calls MessageQCopy_send function with address 53(which fits).
If I look at that function you copy data to msg->payload, setup data length, dstAddr is 53, srcAddr is don't know yet,
flag is 0, reserverd is 0 then you call VirtQueue_addUsedBuf and kick.

How does look like that structure for vring channels? It is probably mapped any structure. Can you point me which one?
Channel description is rpmsg_channel_info struct.

Thanks,
Michal





-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Michal Simek wrote:
> Ohad Ben-Cohen wrote:
>> On Thu, Feb 16, 2012 at 10:12 AM, Michal Simek <monstr@monstr.eu> wrote:
>>> I should be more specific. Can you point me to remoteproc code which 
>>> publish
>>> remote service based on them Linux rpmsg driver will be probed? It is 
>>> rtos
>>> part of code.
>>> In your ELCE presentation is called rpmsg-client-sample service.
>>
>> Sorry for the late response (I'm in ELC).
>>
>> Yes, use the client sample (which is part of the submitted patch set)
>> and not the omx/server samples (which are public, but were never
>> submitted nor cleaned up).
> 
> Client sample is clear because it is probed when services are published.
> 
>>
>> The channels are created dynamically: the rpmsg bus designates address
>> 53 for a name service, which listens to channel creation/removal
>> announcements coming from the remote processor. Take a look in
>> virtio_rpmsg_bus.c and you'll immediately see it.
> 
> I see that values but I can't see how rtos should send that messages.
> 
> Have checked both vrings - one for rx and one for tx. They are
> initialized correctly on addresses I like.
> 
> (gdb) x /80x 0xe1100000
> 0xe1100000:     0x31140000      0x00000000      0x00000200      0x00010002
> 0xe1100010:     0x31140200      0x00000000      0x00000200      0x00020002
> 0xe1100020:     0x31140400      0x00000000      0x00000200      0x00030002
> 0xe1100030:     0x31140600      0x00000000      0x00000200      0x00040002
> 0xe1100040:     0x31140800      0x00000000      0x00000200      0x00050002
> 
> (gdb) x /80x 0xe1104000
> 0xe1104000:     0x00000000      0x00000000      0x00000000      0x00010000
> 0xe1104010:     0x00000000      0x00000000      0x00000000      0x00020000
> 0xe1104020:     0x00000000      0x00000000      0x00000000      0x00030000
> 0xe1104030:     0x00000000      0x00000000      0x00000000      0x00040000
> 0xe1104040:     0x00000000      0x00000000      0x00000000      0x00050000
> 0xe1104050:     0x00000000      0x00000000      0x00000000      0x00060000
> 0xe1104060:     0x00000000      0x00000000      0x00000000      0x00070000
> 
> 
> 
> rtos can handle kick from Linux
> Linux is able to catch kick from freertos and run rproc_vg_interrupt
> 
>  From code I see that in rpmsg_probe function is virtqueue_kick which 
> means sending
> kick to rtos. Then rtos can send the message.
> 
> 
> Above is condition "if (virtio_has_feature(vdev, VIRTIO_RPMSG_F_NS))" 
> which is failing for me.
> Not sure if is correct.
> 
> I am still not sure how to propagate that messages from rtos side.
> In TI software it is in NameMap.c file which calls MessageQCopy_send 
> function with address 53(which fits).
> If I look at that function you copy data to msg->payload, setup data 
> length, dstAddr is 53, srcAddr is don't know yet,
> flag is 0, reserverd is 0 then you call VirtQueue_addUsedBuf and kick.
> 
> How does look like that structure for vring channels? It is probably 
> mapped any structure. Can you point me which one?
> Channel description is rpmsg_channel_info struct.

ok on that addresses above is mapped vring_desc structure. I expect that
addr is address to any memory location when data are places. Next is filled
by init because of ring buffer. What flags should be used? And what is data structure?

struct vring_desc {
	/* Address (guest-physical). */
	__u64 addr;
	/* Length. */
	__u32 len;
	/* The flags as indicated above. */
	__u16 flags;
	/* We chain unused descriptors via this, too */
	__u16 next;
};

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Thu, Feb 16, 2012 at 2:22 PM, Michal Simek <monstr@monstr.eu> wrote:
> ok on that addresses above is mapped vring_desc structure. I expect that
> addr is address to any memory location when data are places. Next is filled
> by init because of ring buffer. What flags should be used? And what is data
> structure?

I'm sorry, but can you please rephrase this question too ? I'm just not
sure I understand you correctly.

In general, you don't need to fiddle with the vring_desc structure -
that's internal to virtio, and things should just work.

Or are you asking about the rtos side ?

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

[-- Attachment #1.1: Type: text/plain, Size: 2755 bytes --]

2012/2/16 Ohad Ben-Cohen <ohad-Ix1uc/W3ht7QT0dZR+AlfA@public.gmane.org>

> On Thu, Feb 16, 2012 at 2:22 PM, Michal Simek <monstr-pSz03upnqPeHXe+LvDLADg@public.gmane.org> wrote:
> > ok on that addresses above is mapped vring_desc structure. I expect that
> > addr is address to any memory location when data are places. Next is
> filled
> > by init because of ring buffer. What flags should be used? And what is
> data
> > structure?
>
> I'm sorry, but can you please rephrase this question too ? I'm just not
> sure I understand you correctly.
>
> In general, you don't need to fiddle with the vring_desc structure -
> that's internal to virtio, and things should just work.
>
> Or are you asking about the rtos side ?
>

yes, I mean that rtos side.

Communication from Linux should be pretty easy because I can use sendto
functions but
IRC in init part Linux kick rtos which send which services are
available.inany format which is unknown to me.

rtos should compose that data message and Linux will encode it and is able
to register that and use that.

My question is how that rtos should communicate. What data to send and
where and which structure to you.
For example: Use first vring buffer and write address where message is
placed and it is in ths structure (u32 flag, u32 address, u32 len,
u32data[MAX].

Two vrings buffers are setup as ring buffers and they are empty and from my
code reading rtos has to fill them.


Maybe I can say it in different way. In your ELCE example on panda board (I
will get it pretty shortly to be able to test it) ou get these messages

virtion_rpmsg_bus virtion0: rpmsg host is online - I get the same message
omap-rproc omap-rproc.1: received echo 0 - it is I think written in omap
remote proc driver

and then you get

virtion_rpmsg_bus virtio0: creating channel rpmsg-client-sample2...
virtion_rpmsg_bus virtio0: creating channel rpmsg-client-sample3...
...

Based on that when you have that channel name you can probe that driver
with match id table which has to use the same name to be able to
communication with this channel (name is rpmsg-client-sample).

I am not able to get that list of published services from rtos that's why
linux can't create that channels
and I would lke to know what to do on rtos side and how to send that
messages and which format I should
use.
Definitely rtos when this message is composed will kick Linux that there is
pending message.

Please also correct me if I read the code wrongly. that mailbox in omap
driver is used just to say which buffer contains that message. Is it
correct?
I can create 15 sw signals to tell rtos which buffer contains message.

Will be help if I clear my code and send it to you for review? Maybe you
will see where the problem is.

Thanks,
Michal

[-- Attachment #1.2: Type: text/html, Size: 3433 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Thu, Feb 16, 2012 at 9:21 PM, Michal Simek <monstr@monstr.eu> wrote:
> IRC in init part Linux kick rtos which send which services are available.in
> any format which is unknown to me.

Check out struct rpmsg_ns_msg - that's the exact format of the name
service notification messages (it's also mentioned by
Documentation/rpmsg.txt).

Of course, these messages, like any other message that is sent across,
need to begin with the usual rpmsg header. Its format is defined by
struct rpmsg_hdr.

To send the message from the rtos to Linux, you then need to grab a
buffer from the available ring (of the vring which is used by Linux to
receive messages), fill it with the message, put it in the used ring,
and kick the Linux side.

> Please also correct me if I read the code wrongly. that mailbox in omap
> driver is used just to say which buffer contains that message. Is it
> correct?

We're using the mailbox payload in order to indicate which vring has
pending buffers.

> I can create 15 sw signals to tell rtos which buffer contains message.

You can also just use a single interrupt and then, when it is
triggered, check both vrings for pending buffers. It's just another if
statement and shouldn't be too much of an overhead.

> Will be help if I clear my code and send it to you for review? Maybe you
> will see where the problem is.

I'm not sure I'll have the time to review it, but let's try to
progress this way and see how it goes.

Sorry I couldn't answer faster this week - I'm in ELC and was a bit
away from emails. I'll soon be flying back and that will keep me away
from emails again, but b/o next week I'll be online again.

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> On Thu, Feb 16, 2012 at 9:21 PM, Michal Simek <monstr@monstr.eu> wrote:
>> IRC in init part Linux kick rtos which send which services are available.in
>> any format which is unknown to me.
> 
> Check out struct rpmsg_ns_msg - that's the exact format of the name
> service notification messages (it's also mentioned by
> Documentation/rpmsg.txt).
> 
> Of course, these messages, like any other message that is sent across,
> need to begin with the usual rpmsg header. Its format is defined by
> struct rpmsg_hdr.
> 
> To send the message from the rtos to Linux, you then need to grab a
> buffer from the available ring (of the vring which is used by Linux to
> receive messages), fill it with the message, put it in the used ring,
> and kick the Linux side.

ok. Will try.

> 
>> Please also correct me if I read the code wrongly. that mailbox in omap
>> driver is used just to say which buffer contains that message. Is it
>> correct?
> 
> We're using the mailbox payload in order to indicate which vring has
> pending buffers.

ok.

> 
>> I can create 15 sw signals to tell rtos which buffer contains message.
> 
> You can also just use a single interrupt and then, when it is
> triggered, check both vrings for pending buffers. It's just another if
> statement and shouldn't be too much of an overhead.
> 
>> Will be help if I clear my code and send it to you for review? Maybe you
>> will see where the problem is.
> 
> I'm not sure I'll have the time to review it, but let's try to
> progress this way and see how it goes.
> 
> Sorry I couldn't answer faster this week - I'm in ELC and was a bit
> away from emails. I'll soon be flying back and that will keep me away
> from emails again, but b/o next week I'll be online again.

that's ok. I will be out of office next week and then I will have panda board
to check how it works on working sample. Unfortunately I don't have enough time
to play with rpmsg this week too.
IRC you mentioned somewhere that you maybe will have firmware which is possible to
compiled by gcc and which doesn't required that TI tools. Am I correct or that was
something totally different?

Thanks,
Michal

-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

Hi Michal,

On Mon, Feb 20, 2012 at 4:22 PM, Michal Simek <monstr@monstr.eu> wrote:
> IRC you mentioned somewhere that you maybe will have firmware which is
> possible to
> compiled by gcc and which doesn't required that TI tools. Am I correct or
> that was
> something totally different?

You're correct - this is in the works, but I'm afraid I don't have any
scheduling information about this, sorry.

But we have folks from another vendor now who got rpmsg working, and
are working out the legal issues so they can release the RTOS sources.
Hopefully that'll be released soon.

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Ohad Ben-Cohen wrote:
> Hi Michal,
> 
> On Mon, Feb 20, 2012 at 4:22 PM, Michal Simek <monstr@monstr.eu> wrote:
>> IRC you mentioned somewhere that you maybe will have firmware which is
>> possible to
>> compiled by gcc and which doesn't required that TI tools. Am I correct or
>> that was
>> something totally different?
> 
> You're correct - this is in the works, but I'm afraid I don't have any
> scheduling information about this, sorry.
> 
> But we have folks from another vendor now who got rpmsg working, and
> are working out the legal issues so they can release the RTOS sources.
> Hopefully that'll be released soon.

That will be great to get it. Please let me know when you have them.
After EW I will look at it anyway.


Thanks,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Wed, Feb 22, 2012 at 2:10 PM, Michal Simek <monstr@monstr.eu> wrote:
> That will be great to get it. Please let me know when you have them.

Will do.

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Michal Simek]

Hi Ohad,

Ohad Ben-Cohen wrote:
> On Wed, Feb 22, 2012 at 2:10 PM, Michal Simek <monstr@monstr.eu> wrote:
>> That will be great to get it. Please let me know when you have them.
> 
> Will do.

I have got the pandaboard (revision A3). u-boot and linux is running on it.
I just need to compile ducati-m3-core0.xem3

I have found the manual at https://github.com/GAnthony/syslink3

In sysbios-rpmsg readme I have found some TI tools which are required
and there is any compilation failure. I think it is because of missing CGT tools

        3) CGT TMS470: 4.9.0
           Public Links:-
             TMS470 Compiler is not available for free publicly. Please contact your TI representative.

Is it possible to compile firmware with this compiler?


I have also tried your bios version at
git://github.com/ohadbc/sysbios-rpmsg.git
branch: rpmsg_3.3_rc1_TLV

compilation passed but it doesn't generate bin file. :-(
I will investigate it tmr.

I would like to compile that bin file myself to be able to add some debug messages.

Thanks for your help,
Michal


-- 
Michal Simek, Ing. (M.Eng)
w: www.monstr.eu p: +42-0-721842854
Maintainer of Linux kernel 2.6 Microblaze Linux - http://www.monstr.eu/fdt/
Microblaze U-BOOT custodian

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

Hi Michal,

On Tue, Mar 6, 2012 at 5:22 PM, Michal Simek <monstr@monstr.eu> wrote:
>       3) CGT TMS470: 4.9.0
>          Public Links:-
>            TMS470 Compiler is not available for free publicly. Please
> contact your TI representative.

I think you need those tools and can't build the image without them
(Mark/Suman/Fernando, PCMIIW).

Btw I was also told that it will soon be possible to build the
firmware with gcc. Mark will know more details about this.

Thanks,
Ohad.

Re: remoteproc: Load coprocessor code to the specific main memory location

[Ohad Ben-Cohen]

On Thu, Feb 16, 2012 at 2:07 PM, Michal Simek <monstr@monstr.eu> wrote:
> Above is condition "if (virtio_has_feature(vdev, VIRTIO_RPMSG_F_NS))" which
> is failing for me.

You might want to debug why.

For the dynamic channel creation to work, your rpmsg vdev should have
VIRTIO_RPMSG_F_NS. But that's part of the generic code, and you should
not bother about it - it should just work.

> How does look like that structure for vring channels? It is probably mapped
> any structure. Can you point me which one?

I'm sorry, can you please rephrase the question ? I'm not sure I'm following.

Thanks,
Ohad.