[Xenomai-help] rt_heap and DMA32 zone

[Xenomai-help] rt_heap and DMA32 zone

[Daniel Jacques (Alten)]

[-- Attachment #1: Type: text/plain, Size: 823 bytes --]

Hi all,

 

I am working on 64 bits Linux 2.6.32.15 and xenomai 2.5.4.

I am using rt_heap to share memory between user and kernel domain. 

I tried to create a shared memory of 16 Mbytes, but it appears that I
cannot create  a heap larger than 3 Mbytes.

The cat /proc/buddyinfo command gives me the following result:

Node 0, zone      DMA      1      2      2      1      2      1      1
0      1      1      3

Node 0, zone    DMA32    103     69     47     28     12      4      2
2      2      3    471

3 Mbytes is the maximum memory on DMA zone. So, It seems that the DMA32
zone is not take into account.

I checked my kernel compilation options but I did not find anything that
could explain this behavior. 

Is rt_heap able to manage DMA32 zone? 

 

Regards,

Jacques

 

 


[-- Attachment #2: Type: text/html, Size: 4438 bytes --]

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Daniel Jacques (Alten) wrote:
> Hi all,
> 
>  
> 
> I am working on 64 bits Linux 2.6.32.15 and xenomai 2.5.4.
> 
> I am using rt_heap to share memory between user and kernel domain.
> 
> I tried to create a shared memory of 16 Mbytes, but it appears that I
> cannot create  a heap larger than 3 Mbytes.

http://www.xenomai.org/index.php/Request_for_information

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Daniel Jacques (Alten)]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

Please find bellow and in attachment the requested information:

Linux kernel: 2.6.32.15
Adeos patch: adeos-ipipe-2.6.32.15-x86-2.7-01.patch
Xenomai: 2.5.4
Host Linux distribution: Based on Debian 5.0
Compiler: gcc version 4.3.2

Piece of code:
#define HEAP_SIZE_IN_BYTES 16*1024*1024 
static int __init audioInit_module(void)
{
int status;
RT_HEAP heap;

 ...

  /* Create the global heap */
  if( (status = rt_heap_create(&heap, "audioInputHeap", HEAP_SIZE_IN_BYTES, H_SHARED|H_DMA )) < 0 ) 
  {
    printk("rt_heap_create audioInputHeap error %d\n", status );
    return status;        
  }
  else
  {
    printk ("Heap audioInputHeap created\n");
  }

...

}

Regards,
Jacques

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: mercredi 13 octobre 2010 11:37
To: Daniel Jacques (Alten)
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] rt_heap and DMA32 zone

Daniel Jacques (Alten) wrote:
> Hi all,
> 
>  
> 
> I am working on 64 bits Linux 2.6.32.15 and xenomai 2.5.4.
> 
> I am using rt_heap to share memory between user and kernel domain.
> 
> I tried to create a shared memory of 16 Mbytes, but it appears that I
> cannot create  a heap larger than 3 Mbytes.

http://www.xenomai.org/index.php/Request_for_information

-- 
					    Gilles.

[-- Attachment #2: config_Linux2.6.32.15_Xeno2.5.4.zip --]
[-- Type: application/x-zip-compressed, Size: 21918 bytes --]

[-- Attachment #3: kern.log.zip --]
[-- Type: application/x-zip-compressed, Size: 85855 bytes --]

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Daniel Jacques (Alten) wrote:
> Please find bellow and in attachment the requested information:
> 
> Linux kernel: 2.6.32.15
> Adeos patch: adeos-ipipe-2.6.32.15-x86-2.7-01.patch
> Xenomai: 2.5.4
> Host Linux distribution: Based on Debian 5.0
> Compiler: gcc version 4.3.2
> 
> Piece of code:
> #define HEAP_SIZE_IN_BYTES 16*1024*1024 
> static int __init audioInit_module(void)
> {
> int status;
> RT_HEAP heap;
> 
>  ...
> 
>   /* Create the global heap */
>   if( (status = rt_heap_create(&heap, "audioInputHeap", HEAP_SIZE_IN_BYTES, H_SHARED|H_DMA )) < 0 ) 
>   {
>     printk("rt_heap_create audioInputHeap error %d\n", status );
>     return status;        
>   }
>   else
>   {
>     printk ("Heap audioInputHeap created\n");
>   }
> 
> ...
> 
> }

So, if I understand correctly, the problem is that Linux does not use
the DMA32 pool when it is passed the GFP_DMA flag? In that case, you
probably need an H_DMA32 flag, which get translated into GFP_DMA32 by
the rtheap code.

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Daniel Jacques (Alten)]

Yes, it seems to be the solution.
I performed the following patch and the problem is solved on my application

*** xenomai-2.5.4/include/native/heap.h 2010-06-16 14:31:04.000000000 +0200
--- xenomai-2.5.4_patchDMA32/include/native/heap.h      2010-10-13 15:40:01.000000000 +0200
***************
*** 34,39 ****
--- 34,40 ----
  #define H_SINGLE   0x400      /* Manage as single-block area. */
  #define H_SHARED   (H_MAPPABLE|H_SINGLE) /* I.e. shared memory segment. */
  #define H_NONCACHED 0x800
+ #define H_DMA32     0x1000      /* Use memory suitable for DMA32. */
  
  /** Structure containing heap-information useful to users.
   *
diff -crB xenomai-2.5.4/ksrc/skins/native/heap.c xenomai-2.5.4_patchDMA32/ksrc/skins/native/heap.c
*** xenomai-2.5.4/ksrc/skins/native/heap.c      2010-05-18 20:31:16.000000000 +0200
--- xenomai-2.5.4_patchDMA32/ksrc/skins/native/heap.c   2010-10-13 15:40:24.000000000 +0200
***************
*** 205,210 ****
--- 205,215 ----
   * operations with I/O devices. The physical address of the
   * heap can be obtained by a call to rt_heap_inquire().
   *
+  * - H_DMA32 causes the block pool associated to the heap to be
+  * allocated in physically contiguous memory, suitable for DMA32
+  * operations with I/O devices. The physical address of the
+  * heap can be obtained by a call to rt_heap_inquire().
+  *
   * - H_NONCACHED causes the heap not to be cached. This is necessary on
   * platforms such as ARM to share a heap between kernel and user-space.
   * Note that this flag is not compatible with the H_DMA flag.
***************
*** 265,270 ****
--- 270,276 ----
                err = xnheap_init_mapped(&heap->heap_base,
                                         heapsize,
                                         ((mode & H_DMA) ? GFP_DMA : 0)
+                                        | ((mode & H_DMA32) ? GFP_DMA32 : 0)
                                         | ((mode & H_NONCACHED) ?
                                            XNHEAP_GFP_NONCACHED : 0));
                if (err)

This patch or similar could be included in a next release.
Thanks for your help Gilles,
Regards
Jacques

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: mercredi 13 octobre 2010 14:37
To: Daniel Jacques (Alten)
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] rt_heap and DMA32 zone

Daniel Jacques (Alten) wrote:
> Please find bellow and in attachment the requested information:
> 
> Linux kernel: 2.6.32.15
> Adeos patch: adeos-ipipe-2.6.32.15-x86-2.7-01.patch
> Xenomai: 2.5.4
> Host Linux distribution: Based on Debian 5.0
> Compiler: gcc version 4.3.2
> 
> Piece of code:
> #define HEAP_SIZE_IN_BYTES 16*1024*1024 
> static int __init audioInit_module(void)
> {
> int status;
> RT_HEAP heap;
> 
>  ...
> 
>   /* Create the global heap */
>   if( (status = rt_heap_create(&heap, "audioInputHeap", HEAP_SIZE_IN_BYTES, H_SHARED|H_DMA )) < 0 ) 
>   {
>     printk("rt_heap_create audioInputHeap error %d\n", status );
>     return status;        
>   }
>   else
>   {
>     printk ("Heap audioInputHeap created\n");
>   }
> 
> ...
> 
> }

So, if I understand correctly, the problem is that Linux does not use
the DMA32 pool when it is passed the GFP_DMA flag? In that case, you
probably need an H_DMA32 flag, which get translated into GFP_DMA32 by
the rtheap code.

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Daniel Jacques (Alten) wrote:
> Yes, it seems to be the solution.
> I performed the following patch and the problem is solved on my application
> 
> (...)
> This patch or similar could be included in a next release.
> Thanks for your help Gilles,

If you generate it with git-format-patch/git-send-mail, you can even get
your name in the logs :-)


-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Gilles Chanteperdrix wrote:
> Daniel Jacques (Alten) wrote:
>> Yes, it seems to be the solution.
>> I performed the following patch and the problem is solved on my application
>>
>> (...)
>> This patch or similar could be included in a next release.
>> Thanks for your help Gilles,
> 
> If you generate it with git-format-patch/git-send-mail, you can even get
> your name in the logs :-)

Ok, forget it, the patch is simple enough. But please next time, use
diff -p option, it allows us to apply the changes manually if patch fails.

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Daniel Jacques (Alten)]

OK. I'll do it the next time.
Regards
Jacques

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: mercredi 13 octobre 2010 16:19
To: Daniel Jacques (Alten)
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] rt_heap and DMA32 zone

Gilles Chanteperdrix wrote:
> Daniel Jacques (Alten) wrote:
>> Yes, it seems to be the solution.
>> I performed the following patch and the problem is solved on my application
>>
>> (...)
>> This patch or similar could be included in a next release.
>> Thanks for your help Gilles,
> 
> If you generate it with git-format-patch/git-send-mail, you can even get
> your name in the logs :-)

Ok, forget it, the patch is simple enough. But please next time, use
diff -p option, it allows us to apply the changes manually if patch fails.

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Daniel Jacques (Alten) wrote:
> Yes, it seems to be the solution.
> I performed the following patch and the problem is solved on my application

Applied, thanks.

-- 
                                                                Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Daniel Jacques (Alten)]

Hi,

I noticed some allocation failure using kmalloc in case of GFP_DMA32 with big size, even if I modified the CONFIG_FORCE_MAX_ZONEORDER to be coherent with my max size (32MB).
I don't know the reason of this failure. I am interesting by any idea about this point ...
Meanwhile, to avoid this trouble I performed the following additional patch in ksrc/nucleus/heap.c to force using of __get_free_pages instead of kmalloc in case of GFP_DMA32 flag.
Jacques

*** ../../../xenomai-2.5.4_old/ksrc/nucleus/heap.c      2010-10-13 17:10:27.000000000 +0200
--- heap.c      2010-10-19 09:22:13.000000000 +0200
*************** static inline void *__alloc_and_reserve_
*** 1033,1039 ****
                 * Otherwise, we have been asked for some kmalloc()
                 * space. Assume that we can wait to get the required memory.
                 */
!               if (size <= KMALLOC_MAX_SIZE)
                        ptr = kmalloc(size, kmflags | GFP_KERNEL);
                else
                        ptr = (void *)__get_free_pages(kmflags | GFP_KERNEL,
--- 1033,1039 ----
                 * Otherwise, we have been asked for some kmalloc()
                 * space. Assume that we can wait to get the required memory.
                 */
!               if (!(kmflags & GFP_DMA32) && (size <= KMALLOC_MAX_SIZE))
                        ptr = kmalloc(size, kmflags | GFP_KERNEL);
                else
                        ptr = (void *)__get_free_pages(kmflags | GFP_KERNEL,
*************** static void __unreserve_and_free_heap(vo
*** 1066,1072 ****
                for (vaddr = vabase; vaddr < vabase + size; vaddr += PAGE_SIZE)
                        ClearPageReserved(virt_to_page(vaddr));
  
!               if (size <= KMALLOC_MAX_SIZE)
                        kfree(ptr);
                else
                        free_pages((unsigned long)ptr, get_order(size));
--- 1066,1072 ----
                for (vaddr = vabase; vaddr < vabase + size; vaddr += PAGE_SIZE)
                        ClearPageReserved(virt_to_page(vaddr));
  
!               if (!(kmflags & GFP_DMA32) && (size <= KMALLOC_MAX_SIZE))
                        kfree(ptr);
                else
                        free_pages((unsigned long)ptr, get_order(size));

-----Original Message-----
From: Gilles Chanteperdrix [mailto:gilles.chanteperdrix@xenomai.org] 
Sent: vendredi 15 octobre 2010 00:30
To: Daniel Jacques (Alten)
Cc: xenomai-help@gna.org
Subject: Re: [Xenomai-help] rt_heap and DMA32 zone

Daniel Jacques (Alten) wrote:
> Yes, it seems to be the solution.
> I performed the following patch and the problem is solved on my application

Applied, thanks.

-- 
                                                                Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Gilles Chanteperdrix]

Daniel Jacques (Alten) wrote:
> Hi,
> 
> I noticed some allocation failure using kmalloc in case of GFP_DMA32 with big size, even if I modified the CONFIG_FORCE_MAX_ZONEORDER to be coherent with my max size (32MB).
> I don't know the reason of this failure. I am interesting by any idea about this point ...
> Meanwhile, to avoid this trouble I performed the following additional patch in ksrc/nucleus/heap.c to force using of __get_free_pages instead of kmalloc in case of GFP_DMA32 flag.
> Jacques

At this point, the problem seems to be in linux code. So, there should
be no need to modify xenomai code. Are you sure that kmalloc does not
fail due to fragmentation?

-- 
					    Gilles.

Re: [Xenomai-help] rt_heap and DMA32 zone

[Anders Blomdell]

On 2010-10-19 12.22, Gilles Chanteperdrix wrote:
> Daniel Jacques (Alten) wrote:
>> Hi,
>>
>> I noticed some allocation failure using kmalloc in case of GFP_DMA32 with big size, even if I modified the CONFIG_FORCE_MAX_ZONEORDER to be coherent with my max size (32MB).
>> I don't know the reason of this failure. I am interesting by any idea about this point ...
>> Meanwhile, to avoid this trouble I performed the following additional patch in ksrc/nucleus/heap.c to force using of __get_free_pages instead of kmalloc in case of GFP_DMA32 flag.
>> Jacques
> 
> At this point, the problem seems to be in linux code. So, there should
> be no need to modify xenomai code. Are you sure that kmalloc does not
> fail due to fragmentation?
I have some vague recollection that dmaable allocations larger than 2MB should
be done by pci_alloc_consistent and similar (see Documentation/DMA-mapping.txt
or Documentation/DMA-API-HOWTO.txt in newer kernels)

/Anders

-- 
Anders Blomdell                  Email: anders.blomdell@domain.hid
Department of Automatic Control
Lund University                  Phone:    +46 46 222 4625
P.O. Box 118                     Fax:      +46 46 138118
SE-221 00 Lund, Sweden