DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.

DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.

[Guzman Lugo, Fernando]

Hi All,
 
        Reloading the dspbridge several times I am getting a Segmentation fault. Seeing the log it seems that the memory was exhausted 
 
The error happens when ioremap is called

void MEM_ExtPhysPoolInit(u32 poolPhysBase, u32 poolSize)
{
            u32 poolVirtBase;

            /* get the virtual address for the physical memory pool passed */
            poolVirtBase = (u32)ioremap(poolPhysBase, poolSize);
.

Putting some printk's and printing the address returned by ioremap, I realized that address returned by ioremap each time I reload the dspbridge is different, in fact the address is increasing. I also put a printk where iounmap is called to make sure it is called and yes it is actually called. However testing with a kernel + bridge for linux 23x I always get the same address for pool memory. Any idea what the problem is? I have included the console output where you can see the address increasing.

Regards,
Fernando.


 
The first 15 loads and unloads 
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = c8000000
Releasing External memory pool, address = c8000000
interation 2
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = c9000000
Releasing External memory pool, address = c9000000
interation 3
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = c9800000
Releasing External memory pool, address = c9800000
interation 4
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = ca800000
Releasing External memory pool, address = ca800000
interation 5
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cb000000
Releasing External memory pool, address = cb000000
interation 6
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cc000000
Releasing External memory pool, address = cc000000
interation 7
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cc800000
Releasing External memory pool, address = cc800000
interation 8
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = c8800000
Releasing External memory pool, address = c8800000
interation 9
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = ca000000
Releasing External memory pool, address = ca000000
interation 10
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cd000000
Releasing External memory pool, address = cd000000
interation 11
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cd800000
Releasing External memory pool, address = cd800000
interation 12
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = ce800000
Releasing External memory pool, address = ce800000
interation 13
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = cf000000
Releasing External memory pool, address = cf000000
interation 14
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = d0000000
Releasing External memory pool, address = d0000000
interation 15
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = d0800000
Releasing External memory pool, address = d0800000
...
 
and It crashes in the 33rd reload.
 
 
interation 32
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
[PHYS_POOL]Mapping External, address = d7800000
Releasing External memory pool, address = d7800000
interation 33
INFO: For help run script as ./install_bridge help
INFO:Installed Bridgedriver from current directory:/dspbridge_os
<4>vmap allocation failed: use vmalloc=<size> to increase size.
vmap allocation failed: use vmalloc=<size> to increase size.
[PHYS_POOL]Mapping External, address = 0
<4>coherent allocation too big (requested 0x400000 mask 0xffffffff)
coherent allocation too big (requested 0x400000 mask 0xffffffff)
drivers/dsp/bridge/rmgr/drv.c, line 1546: Assertion ((DSP_SUCCEEDED(status) && p
DevNodeString != NULL && !LST_IsEmpty(pDRVObject->devNodeString)) || (DSP_FAILED
(status) && *pDevNodeString == 0)) failed.
drivers/dsp/bridge/rmgr/drv.c, line 1546: Assertion ((DSP_SUCCEEDED(status) && p
DevNodeString != NULL && !LST_IsEmpty(pDRVObject->devNodeString)) || (DSP_FAILED
(status) && *pDevNodeString == 0)) failed.
drivers/dsp/bridge/rmgr/drv_interface.c, line 484: Assertion (status == 0) faile
d.
drivers/dsp/bridge/rmgr/drv_interface.c, line 484: Assertion (status == 0) faile
d.
drivers/dsp/bridge/rmgr/drv_interface.c, line 485: Assertion (DSP_SUCCEEDED(init
Status)) failed.
drivers/dsp/bridge/rmgr/drv_interface.c, line 485: Assertion (DSP_SUCCEEDED(init
Status)) failed.

--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

> -----Original Message-----
> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> owner@vger.kernel.org] On Behalf Of Guzman Lugo, Fernando
> Sent: Thursday, March 12, 2009 2:04 AM
> To: linux-omap@vger.kernel.org
> Subject: DSPBRIDGE: segmentation fault after reloading dspbridge several
> times due to a memory leak.
>         Reloading the dspbridge several times I am getting a Segmentation
> fault. Seeing the log it seems that the memory was exhausted
> 
> The error happens when ioremap is called
> 
> void MEM_ExtPhysPoolInit(u32 poolPhysBase, u32 poolSize)
> {
>             u32 poolVirtBase;
> 
>             /* get the virtual address for the physical memory pool passed
> */
>             poolVirtBase = (u32)ioremap(poolPhysBase, poolSize);
> .
> 
> Putting some printk's and printing the address returned by ioremap, I
> realized that address returned by ioremap each time I reload the dspbridge
> is different, in fact the address is increasing. I also put a printk where
> iounmap is called to make sure it is called and yes it is actually called.
> However testing with a kernel + bridge for linux 23x I always get the same
> address for pool memory. Any idea what the problem is? I have included the
> console output where you can see the address increasing.

I duplicated this with the following dummy driver which ioremaps as per the same allocations that the bridge driver would have done:

#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/mm.h>
#include <linux/dma-mapping.h>

#define BASE 0x87000000
#define SIZE 0x600000

struct mem_s{
        void * vir;
        u32 phy;
        u32 size;
};

struct mem_s b[]={
        {0,BASE,SIZE},
        {0,0x48306000,4096},
        {0,0x48004000,4096},
        {0,0x48094000,4096},
        {0,0x48002000,4096},
        {0,0x5c7f8000,98304},
        {0,0x5ce00000,32768},
        {0,0x5cf04000,81920},
        {0,0x48005000,4096},
        {0,0x48307000,4096},
        {0,0x48306a00,4096},
        {0,0x5d000000,4096},
};

static int __init dummy_init(void)
{
        int i;
        for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
                b[i].vir = ioremap(b[i].phy,b[i].size);
                if(b[i].vir == NULL) {
                        printk(KERN_ERR "Allocation failed idx=%d\n",i);
                        /* Free up all the prev allocs */
                        i--;
                        while(i>=0) {
                                iounmap(b[i].vir);
                                i--;
                        }
                        return -ENOMEM;

                }
        }
        return 0;
}
module_init(dummy_init);
static void __exit dummy_exit(void)
{
        int i;
        for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
                iounmap(b[i].vir);
        }
}
module_exit(dummy_exit);
MODULE_LICENSE("GPL");


Regression script:
#!/bin/bash
i=0
slee()
{
        echo "Sleep "
#sleep 5
}
while [ $i -lt 100 ]; do
        echo "insmod $i"
        insmod  ./dummy.ko
        if [ $? -ne 0 ]; then
                echo "QUIT IN INSMOD $i"
                exit 1;
        fi
        slee
        echo "rmmod $i"
        rmmod dummy
        if [ $? -ne 0 ]; then
                echo "QUIT IN RMMOD $i"
                exit 1;
        fi
        i=`expr $i + 1`
        slee
done



after around 38 iterations:
<4>vmap allocation failed: use vmalloc=<size> to increase size.
vmap allocation failed: use vmalloc=<size> to increase size.
<3>Allocation failed idx=0
Allocation failed idx=0

However cat /proc/meminfo after this error is:
cat /proc/meminfo
MemTotal:          61920 kB
MemFree:           56900 kB
Buffers:               0 kB
Cached:             2592 kB
SwapCached:            0 kB
Active:             1920 kB
Inactive:           1252 kB
Active(anon):        580 kB
Inactive(anon):        0 kB
Active(file):       1340 kB
Inactive(file):     1252 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:           616 kB
Mapped:              688 kB
Slab:               1296 kB
SReclaimable:        480 kB
SUnreclaim:          816 kB
PageTables:           96 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:       30960 kB
Committed_AS:       2932 kB
VmallocTotal:     319488 kB
VmallocUsed:           8 kB
VmallocChunk:     319448 kB


We seem to have more than enough vmalloc space according to this.. am I right in thinking this is a kernel vmalloc handling issue?

Regards,
Nishanth Menon
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

> -----Original Message-----
> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> owner@vger.kernel.org] On Behalf Of Menon, Nishanth
> Sent: Thursday, March 12, 2009 6:40 PM
> To: Guzman Lugo, Fernando; linux-omap@vger.kernel.org
> Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh;
> Ameya Palande
> Subject: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> reloading dspbridge several times due to a memory leak.)
> > console output where you can see the address increasing.
> 
> I duplicated this with the following dummy driver which ioremaps as per
> the same allocations that the bridge driver would have done:
> 
> #include <linux/kernel.h>
> #include <linux/module.h>
> #include <linux/slab.h>
> #include <linux/mm.h>
> #include <linux/dma-mapping.h>
> 
> #define BASE 0x87000000
> #define SIZE 0x600000
> 
Just an update (thanks to Fernando in pointing this out)
0x87000000, 0x600000, fails with same error
0x88000000, 0x600000, fails with same error
0x87000000, 0x700000 - went thru 300 odd iterations without error!! 

Mebbe this sequence is triggering a bug?

Regards,
Nishanth Menon

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

> -----Original Message-----
> From: Menon, Nishanth
> Sent: Thursday, March 12, 2009 6:53 PM
> To: Menon, Nishanth; Guzman Lugo, Fernando; linux-omap@vger.kernel.org
> Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh;
> Ameya Palande
> Subject: RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> reloading dspbridge several times due to a memory leak.)
> 
> 
> > -----Original Message-----
> > From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> > owner@vger.kernel.org] On Behalf Of Menon, Nishanth
> > Sent: Thursday, March 12, 2009 6:40 PM
> > To: Guzman Lugo, Fernando; linux-omap@vger.kernel.org
> > Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh;
> > Ameya Palande
> > Subject: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> > reloading dspbridge several times due to a memory leak.)
> > > console output where you can see the address increasing.
> >
> > I duplicated this with the following dummy driver which ioremaps as per
> > the same allocations that the bridge driver would have done:
> >
> > #include <linux/kernel.h>
> > #include <linux/module.h>
> > #include <linux/slab.h>
> > #include <linux/mm.h>
> > #include <linux/dma-mapping.h>
> >
> > #define BASE 0x87000000
> > #define SIZE 0x600000
> >
> Just an update (thanks to Fernando in pointing this out)
> 0x87000000, 0x600000, fails with same error
> 0x88000000, 0x600000, fails with same error
Wrong attempt!!
Should have tried:
0x87100000, 0x600000 - this worked
> 0x87000000, 0x700000 - went thru 300 odd iterations without error!!
This means that if I keep end address constant, it is fine.. with this hypothesis, I tried
0x87200000, 0x500000 - this also worked

Implying the start address and size of the first allocation in my dummy driver did not have a factor in the test failure, but just the end address of the ioremap!

Regards,
Nishanth Menon

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Guzman Lugo, Fernando]

>> Just an update (thanks to Fernando in pointing this out)
>> 0x87000000, 0x600000, fails with same error
>> 0x88000000, 0x600000, fails with same error
>Wrong attempt!!
>Should have tried:
>0x87100000, 0x600000 - this worked
>> 0x87000000, 0x700000 - went thru 300 odd iterations without error!!
>This means that if I keep end address constant, it is fine.. with this >hypothesis, I tried
>0x87200000, 0x500000 - this also worked

However 0x87100000, 0x500000 also works that is the same end address than 0x87000000, 0x600000. so maybe the hypothesis is no right. 

Regards,
Fernando.

-----Original Message-----
From: Menon, Nishanth 
Sent: Thursday, March 12, 2009 10:58 AM
To: Menon, Nishanth; Guzman Lugo, Fernando; linux-omap@vger.kernel.org
Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh; Ameya Palande
Subject: RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

> -----Original Message-----
> From: Menon, Nishanth
> Sent: Thursday, March 12, 2009 6:53 PM
> To: Menon, Nishanth; Guzman Lugo, Fernando; linux-omap@vger.kernel.org
> Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh;
> Ameya Palande
> Subject: RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> reloading dspbridge several times due to a memory leak.)
> 
> 
> > -----Original Message-----
> > From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
> > owner@vger.kernel.org] On Behalf Of Menon, Nishanth
> > Sent: Thursday, March 12, 2009 6:40 PM
> > To: Guzman Lugo, Fernando; linux-omap@vger.kernel.org
> > Cc: Kanigeri, Hari; Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh;
> > Ameya Palande
> > Subject: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> > reloading dspbridge several times due to a memory leak.)
> > > console output where you can see the address increasing.
> >
> > I duplicated this with the following dummy driver which ioremaps as per
> > the same allocations that the bridge driver would have done:
> >
> > #include <linux/kernel.h>
> > #include <linux/module.h>
> > #include <linux/slab.h>
> > #include <linux/mm.h>
> > #include <linux/dma-mapping.h>
> >
> > #define BASE 0x87000000
> > #define SIZE 0x600000
> >
> Just an update (thanks to Fernando in pointing this out)
> 0x87000000, 0x600000, fails with same error
> 0x88000000, 0x600000, fails with same error
Wrong attempt!!
Should have tried:
0x87100000, 0x600000 - this worked
> 0x87000000, 0x700000 - went thru 300 odd iterations without error!!
This means that if I keep end address constant, it is fine.. with this hypothesis, I tried
0x87200000, 0x500000 - this also worked

Implying the start address and size of the first allocation in my dummy driver did not have a factor in the test failure, but just the end address of the ioremap!

Regards,
Nishanth Menon

Re: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Kevin Hilman]

"Menon, Nishanth" <nm@ti.com> writes:

>> -----Original Message-----
>> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
>> owner@vger.kernel.org] On Behalf Of Guzman Lugo, Fernando
>> Sent: Thursday, March 12, 2009 2:04 AM
>> To: linux-omap@vger.kernel.org
>> Subject: DSPBRIDGE: segmentation fault after reloading dspbridge several
>> times due to a memory leak.
>>         Reloading the dspbridge several times I am getting a Segmentation
>> fault. Seeing the log it seems that the memory was exhausted
>> 
>> The error happens when ioremap is called
>> 
>> void MEM_ExtPhysPoolInit(u32 poolPhysBase, u32 poolSize)
>> {
>>             u32 poolVirtBase;
>> 
>>             /* get the virtual address for the physical memory pool passed
>> */
>>             poolVirtBase = (u32)ioremap(poolPhysBase, poolSize);
>> .
>> 
>> Putting some printk's and printing the address returned by ioremap, I
>> realized that address returned by ioremap each time I reload the dspbridge
>> is different, in fact the address is increasing. I also put a printk where
>> iounmap is called to make sure it is called and yes it is actually called.
>> However testing with a kernel + bridge for linux 23x I always get the same
>> address for pool memory. Any idea what the problem is? I have included the
>> console output where you can see the address increasing.
>
> I duplicated this with the following dummy driver which ioremaps as per the same allocations that the bridge driver would have done:
>
> #include <linux/kernel.h>
> #include <linux/module.h>
> #include <linux/slab.h>
> #include <linux/mm.h>
> #include <linux/dma-mapping.h>
>
> #define BASE 0x87000000
> #define SIZE 0x600000
>
> struct mem_s{
>         void * vir;
>         u32 phy;
>         u32 size;
> };
>
> struct mem_s b[]={
>         {0,BASE,SIZE},
>         {0,0x48306000,4096},
>         {0,0x48004000,4096},
>         {0,0x48094000,4096},
>         {0,0x48002000,4096},
>         {0,0x5c7f8000,98304},
>         {0,0x5ce00000,32768},
>         {0,0x5cf04000,81920},
>         {0,0x48005000,4096},
>         {0,0x48307000,4096},
>         {0,0x48306a00,4096},
>         {0,0x5d000000,4096},
> };

Nishant,

Which of these physical addresses causes an increasing virtual
address?  The addresses in the 0x48xxxxxx (L4, L4_WKUP) range should
just trigger static mapping via the arch-specific ioremap, so those
should always map to the same virt address. 

Could you do the experiment with a smaller number of mappings?  Maybe
just one at a time of the non L4 mappings?  Probably starting with
only the BASE,SIZE mapping.

Kevin

>
> static int __init dummy_init(void)
> {
>         int i;
>         for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
>                 b[i].vir = ioremap(b[i].phy,b[i].size);
>                 if(b[i].vir == NULL) {
>                         printk(KERN_ERR "Allocation failed idx=%d\n",i);
>                         /* Free up all the prev allocs */
>                         i--;
>                         while(i>=0) {
>                                 iounmap(b[i].vir);
>                                 i--;
>                         }
>                         return -ENOMEM;
>
>                 }
>         }
>         return 0;
> }
> module_init(dummy_init);
> static void __exit dummy_exit(void)
> {
>         int i;
>         for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
>                 iounmap(b[i].vir);
>         }
> }
> module_exit(dummy_exit);
> MODULE_LICENSE("GPL");
>
>
> Regression script:
> #!/bin/bash
> i=0
> slee()
> {
>         echo "Sleep "
> #sleep 5
> }
> while [ $i -lt 100 ]; do
>         echo "insmod $i"
>         insmod  ./dummy.ko
>         if [ $? -ne 0 ]; then
>                 echo "QUIT IN INSMOD $i"
>                 exit 1;
>         fi
>         slee
>         echo "rmmod $i"
>         rmmod dummy
>         if [ $? -ne 0 ]; then
>                 echo "QUIT IN RMMOD $i"
>                 exit 1;
>         fi
>         i=`expr $i + 1`
>         slee
> done
>
>
>
> after around 38 iterations:
> <4>vmap allocation failed: use vmalloc=<size> to increase size.
> vmap allocation failed: use vmalloc=<size> to increase size.
> <3>Allocation failed idx=0
> Allocation failed idx=0
>
> However cat /proc/meminfo after this error is:
> cat /proc/meminfo
> MemTotal:          61920 kB
> MemFree:           56900 kB
> Buffers:               0 kB
> Cached:             2592 kB
> SwapCached:            0 kB
> Active:             1920 kB
> Inactive:           1252 kB
> Active(anon):        580 kB
> Inactive(anon):        0 kB
> Active(file):       1340 kB
> Inactive(file):     1252 kB
> Unevictable:           0 kB
> Mlocked:               0 kB
> SwapTotal:             0 kB
> SwapFree:              0 kB
> Dirty:                 0 kB
> Writeback:             0 kB
> AnonPages:           616 kB
> Mapped:              688 kB
> Slab:               1296 kB
> SReclaimable:        480 kB
> SUnreclaim:          816 kB
> PageTables:           96 kB
> NFS_Unstable:          0 kB
> Bounce:                0 kB
> WritebackTmp:          0 kB
> CommitLimit:       30960 kB
> Committed_AS:       2932 kB
> VmallocTotal:     319488 kB
> VmallocUsed:           8 kB
> VmallocChunk:     319448 kB
>
>
> We seem to have more than enough vmalloc space according to this.. am I right in thinking this is a kernel vmalloc handling issue?
>
> Regards,
> Nishanth Menon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-omap" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Alejandro Blanca G.]

My 2 cents on this, and based on previous experience (which may be
outdated), are there any limits (physical, virtual) regarding memory
space being crossed with these values?

I.e., are you in the limits of a memory page or the limits of a memory bank?

Just an idea. If I'm totally lost, my apologies. Just wanted to be of help.

Best regards,

Alex B.

On Thu, Mar 12, 2009 at 11:27 AM, Kevin Hilman
<khilman@deeprootsystems.com> wrote:
> "Menon, Nishanth" <nm@ti.com> writes:
>
>>> -----Original Message-----
>>> From: linux-omap-owner@vger.kernel.org [mailto:linux-omap-
>>> owner@vger.kernel.org] On Behalf Of Guzman Lugo, Fernando
>>> Sent: Thursday, March 12, 2009 2:04 AM
>>> To: linux-omap@vger.kernel.org
>>> Subject: DSPBRIDGE: segmentation fault after reloading dspbridge several
>>> times due to a memory leak.
>>>         Reloading the dspbridge several times I am getting a Segmentation
>>> fault. Seeing the log it seems that the memory was exhausted
>>>
>>> The error happens when ioremap is called
>>>
>>> void MEM_ExtPhysPoolInit(u32 poolPhysBase, u32 poolSize)
>>> {
>>>             u32 poolVirtBase;
>>>
>>>             /* get the virtual address for the physical memory pool passed
>>> */
>>>             poolVirtBase = (u32)ioremap(poolPhysBase, poolSize);
>>> .
>>>
>>> Putting some printk's and printing the address returned by ioremap, I
>>> realized that address returned by ioremap each time I reload the dspbridge
>>> is different, in fact the address is increasing. I also put a printk where
>>> iounmap is called to make sure it is called and yes it is actually called.
>>> However testing with a kernel + bridge for linux 23x I always get the same
>>> address for pool memory. Any idea what the problem is? I have included the
>>> console output where you can see the address increasing.
>>
>> I duplicated this with the following dummy driver which ioremaps as per the same allocations that the bridge driver would have done:
>>
>> #include <linux/kernel.h>
>> #include <linux/module.h>
>> #include <linux/slab.h>
>> #include <linux/mm.h>
>> #include <linux/dma-mapping.h>
>>
>> #define BASE 0x87000000
>> #define SIZE 0x600000
>>
>> struct mem_s{
>>         void * vir;
>>         u32 phy;
>>         u32 size;
>> };
>>
>> struct mem_s b[]={
>>         {0,BASE,SIZE},
>>         {0,0x48306000,4096},
>>         {0,0x48004000,4096},
>>         {0,0x48094000,4096},
>>         {0,0x48002000,4096},
>>         {0,0x5c7f8000,98304},
>>         {0,0x5ce00000,32768},
>>         {0,0x5cf04000,81920},
>>         {0,0x48005000,4096},
>>         {0,0x48307000,4096},
>>         {0,0x48306a00,4096},
>>         {0,0x5d000000,4096},
>> };
>
> Nishant,
>
> Which of these physical addresses causes an increasing virtual
> address?  The addresses in the 0x48xxxxxx (L4, L4_WKUP) range should
> just trigger static mapping via the arch-specific ioremap, so those
> should always map to the same virt address.
>
> Could you do the experiment with a smaller number of mappings?  Maybe
> just one at a time of the non L4 mappings?  Probably starting with
> only the BASE,SIZE mapping.
>
> Kevin
>
>>
>> static int __init dummy_init(void)
>> {
>>         int i;
>>         for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
>>                 b[i].vir = ioremap(b[i].phy,b[i].size);
>>                 if(b[i].vir == NULL) {
>>                         printk(KERN_ERR "Allocation failed idx=%d\n",i);
>>                         /* Free up all the prev allocs */
>>                         i--;
>>                         while(i>=0) {
>>                                 iounmap(b[i].vir);
>>                                 i--;
>>                         }
>>                         return -ENOMEM;
>>
>>                 }
>>         }
>>         return 0;
>> }
>> module_init(dummy_init);
>> static void __exit dummy_exit(void)
>> {
>>         int i;
>>         for (i=0;i<(sizeof(b)/sizeof(struct mem_s));i++) {
>>                 iounmap(b[i].vir);
>>         }
>> }
>> module_exit(dummy_exit);
>> MODULE_LICENSE("GPL");
>>
>>
>> Regression script:
>> #!/bin/bash
>> i=0
>> slee()
>> {
>>         echo "Sleep "
>> #sleep 5
>> }
>> while [ $i -lt 100 ]; do
>>         echo "insmod $i"
>>         insmod  ./dummy.ko
>>         if [ $? -ne 0 ]; then
>>                 echo "QUIT IN INSMOD $i"
>>                 exit 1;
>>         fi
>>         slee
>>         echo "rmmod $i"
>>         rmmod dummy
>>         if [ $? -ne 0 ]; then
>>                 echo "QUIT IN RMMOD $i"
>>                 exit 1;
>>         fi
>>         i=`expr $i + 1`
>>         slee
>> done
>>
>>
>>
>> after around 38 iterations:
>> <4>vmap allocation failed: use vmalloc=<size> to increase size.
>> vmap allocation failed: use vmalloc=<size> to increase size.
>> <3>Allocation failed idx=0
>> Allocation failed idx=0
>>
>> However cat /proc/meminfo after this error is:
>> cat /proc/meminfo
>> MemTotal:          61920 kB
>> MemFree:           56900 kB
>> Buffers:               0 kB
>> Cached:             2592 kB
>> SwapCached:            0 kB
>> Active:             1920 kB
>> Inactive:           1252 kB
>> Active(anon):        580 kB
>> Inactive(anon):        0 kB
>> Active(file):       1340 kB
>> Inactive(file):     1252 kB
>> Unevictable:           0 kB
>> Mlocked:               0 kB
>> SwapTotal:             0 kB
>> SwapFree:              0 kB
>> Dirty:                 0 kB
>> Writeback:             0 kB
>> AnonPages:           616 kB
>> Mapped:              688 kB
>> Slab:               1296 kB
>> SReclaimable:        480 kB
>> SUnreclaim:          816 kB
>> PageTables:           96 kB
>> NFS_Unstable:          0 kB
>> Bounce:                0 kB
>> WritebackTmp:          0 kB
>> CommitLimit:       30960 kB
>> Committed_AS:       2932 kB
>> VmallocTotal:     319488 kB
>> VmallocUsed:           8 kB
>> VmallocChunk:     319448 kB
>>
>>
>> We seem to have more than enough vmalloc space according to this.. am I right in thinking this is a kernel vmalloc handling issue?
>>
>> Regards,
>> Nishanth Menon
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-omap" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-omap" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-omap" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

[-- Attachment #1: Type: text/plain, Size: 3439 bytes --]

Kevin,
Some more isolation:
The l-o master branch does not have an issue, only the pm branch seems to have the problem - probably some commit(s?) missing in pm branch. Since bridge is based off the pm branch, wont make sense to regress on it :(.

> -----Original Message-----
> From: Kevin Hilman [mailto:khilman@deeprootsystems.com]
> Sent: Thursday, March 12, 2009 7:27 PM
> To: Menon, Nishanth
> Cc: Guzman Lugo, Fernando; linux-omap@vger.kernel.org; Kanigeri, Hari;
> Woodruff, Richard; Lakhani, Amish; Gupta, Ramesh; Ameya Palande
> Subject: Re: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> reloading dspbridge several times due to a memory leak.)

> Which of these physical addresses causes an increasing virtual
> address?  The addresses in the 0x48xxxxxx (L4, L4_WKUP) range should
> just trigger static mapping via the arch-specific ioremap, so those
> should always map to the same virt address.
> 
> Could you do the experiment with a smaller number of mappings?  Maybe
> just one at a time of the non L4 mappings?  Probably starting with
> only the BASE,SIZE mapping.
> 
Attached is the patch for pm branch. 
Codebase:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap-2.6.git
git checkout -b pm --track origin/pm
git branch iotest-pm
git apply 0001-MISC-dummy-driver-to-test-ioremap.patch

make omap_3430sdp_defconfig
make uImage && make modules

bootargs: console=ttyS0,115200n8 noinitrd ip=dhcp root=/dev/nfs rw nfsroot=<server>:<fsPath>,nolock,wsize=1024,rsize=1024 mem=64M

I reorganized the driver from the previous version I had send, so that we can provide base address,size and number of io_remaps to try as module params. And tested with the following shell script:
#!/bin/bash
slee()
{
        echo "Sleep "
#sleep 5
}
try=1
while [ $try -lt 13 ]; do
	  i=0
        while [ $i -lt 100 ]; do
                echo "insmod $i base=0x87000000 size=0x600000 try=$try"
                insmod  ./dummy.ko base=0x87000000 size=0x600000 try=$try
                if [ $? -ne 0 ]; then
                        echo "QUIT IN INSMOD $i"
                        exit 1;
                fi
                slee
                echo "rmmod $i"
                rmmod dummy
                if [ $? -ne 0 ]; then
                        echo "QUIT IN RMMOD $i"
                        exit 1;
                fi
                i=`expr $i + 1`
                slee
        done
        try=$(( $try + 1 ))
done

Result:
insmod 21 base=0x87000000 size=0x600000 try=1
vmap allocation failed: use vmalloc=<size> to increase size.
Map[0] - virt=0x00000000 phy=0x87000000 size=0x00600000
Allocation failed idx=0
insmod: cannot insert `./dummy.ko': Cannot allocate memory (-1): Cannot allocate memory
QUIT IN INSMOD 21

In terms of virtual addresses generated:
0xCD800000
0xCE000000
0xCE800000
0xCF000000
0xCF800000
0xD0000000
0xD0800000
0xD1000000
...
[iteration 20] 0xD7800000

The next one fails.

The same patch will apply on linux-omap master branch also, but patching drivers/misc/Makefile will break with my patch - need to trivial hand merge it..

commitIDs tested against:
linux-omap master: cc2b459c066361098704c586f3134c3c3ac13be3 (the entire script runs through)
linux-omap pm: 4f422d583e2e233c19eb20754b8cfad6ed9e460a (the script fails around 21st iteration with try=1)

Regards,
Nishanth Menon

[-- Attachment #2: 0001-MISC-dummy-driver-to-test-ioremap.patch --]
[-- Type: application/octet-stream, Size: 3997 bytes --]

From b42d7fbd85f2a9b9ea9c5ddb70a7d3bf0751a95c Mon Sep 17 00:00:00 2001
From: Nishanth Menon <nm@ti.com>
Date: Thu, 12 Mar 2009 13:46:34 -0500
Subject: [PATCH] MISC: dummy driver to test ioremap
 the script such as the following tests ioremap on sdp3430
 bootargs mem=64M for sdp3430 for the test
 slee()
 {
 	echo "Sleep "
 }
 try=1
 while [ $try -lt 13 ]; do
	i=0
 	while [ $i -lt 100 ]; do
 		echo "insmod $i base=0x87000000 size=0x600000 try=$try"
 		insmod  ./dummy.ko base=0x87000000 size=0x600000 try=$try
 		if [ $? -ne 0 ]; then
 			echo "QUIT IN INSMOD $i"
 			exit 1;
 		fi
 		slee
 		echo "rmmod $i"
 		rmmod dummy
 		if [ $? -ne 0 ]; then
 			echo "QUIT IN RMMOD $i"
 			exit 1;
 		fi
 		i=`expr $i + 1`
 		slee
 	done
 	try=$(( $try + 1 ))
 done


Signed-off-by: Nishanth Menon <nm@ti.com>
---
 drivers/misc/Kconfig  |    4 ++
 drivers/misc/Makefile |    1 +
 drivers/misc/dummy.c  |   91 +++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 96 insertions(+), 0 deletions(-)
 create mode 100644 drivers/misc/dummy.c

diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index a11e2a0..2934c4a 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -13,6 +13,10 @@ menuconfig MISC_DEVICES
 
 if MISC_DEVICES
 
+config MISC_DUMMY
+	tristate "dummy lil driver"
+	default m
+
 config ATMEL_PWM
 	tristate "Atmel AT32/AT91 PWM support"
 	depends on AVR32 || ARCH_AT91SAM9263 || ARCH_AT91SAM9RL || ARCH_AT91CAP9
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index 78be134..9e7b2a6 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -2,6 +2,7 @@
 # Makefile for misc devices that really don't fit anywhere else.
 #
 obj- := misc.o	# Dummy rule to force built-in.o to be made
+obj-$(CONFIG_MISC_DUMMY)	+= dummy.o
 
 obj-$(CONFIG_IBM_ASM)		+= ibmasm/
 obj-$(CONFIG_HDPU_FEATURES)	+= hdpuftrs/
diff --git a/drivers/misc/dummy.c b/drivers/misc/dummy.c
new file mode 100644
index 0000000..9c6500f
--- /dev/null
+++ b/drivers/misc/dummy.c
@@ -0,0 +1,91 @@
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/mm.h>
+#include <linux/dma-mapping.h>
+
+static int size;
+module_param(size, int, 0);
+MODULE_PARM_DESC(size, "size default =0x87000000");
+
+static int base;
+module_param(base, int, 0);
+MODULE_PARM_DESC(base, "Baseaddress default 0x6000000");
+
+struct mem_s {
+	void *vir;
+	u32 phy;
+	u32 size;
+};
+
+static struct mem_s b[] = {
+	{0, 0, 0},		/* Fill me up scotty */
+	{0, 0x5d000000, 4096},
+	{0, 0x5c7f8000, 98304},
+	{0, 0x5ce00000, 32768},
+	{0, 0x5cf04000, 81920},
+	{0, 0x48306000, 4096},
+	{0, 0x48004000, 4096},
+	{0, 0x48094000, 4096},
+	{0, 0x48002000, 4096},
+	{0, 0x48005000, 4096},
+	{0, 0x48307000, 4096},
+	{0, 0x48306a00, 4096},
+};
+
+static int try;
+module_param(try, int, (sizeof(b) / sizeof(struct mem_s)));
+MODULE_PARM_DESC(try, "try=12");
+
+static int __init dummy_init(void)
+{
+	int i;
+	b[0].phy = base;
+	b[0].size = size;
+	if (!try)
+		try = (sizeof(b) / sizeof(struct mem_s));
+	if (!base)
+		base = 0x87000000;
+	if (!size)
+		size = 0x600000;
+	if (try > (sizeof(b) / sizeof(struct mem_s))) {
+		printk(KERN_ERR "Give me proper try value\n");
+		return -EINVAL;
+	}
+	for (i = 0; i < try; i++) {
+		b[i].vir = ioremap(b[i].phy, b[i].size);
+		printk(KERN_INFO
+		       "Map[%d] - virt=0x%08X phy=0x%08X size=0x%08X\n", i,
+		       (u32) (b[i].vir), b[i].phy, b[i].size);
+		if (b[i].vir == NULL) {
+			printk(KERN_ERR "Allocation failed idx=%d\n", i);
+			/* Free up all the prev allocs */
+			i--;
+			while (i >= 0) {
+				iounmap(b[i].vir);
+				i--;
+			}
+			return -ENOMEM;
+
+		}
+	}
+	return 0;
+}
+
+module_init(dummy_init);
+
+static void __exit dummy_exit(void)
+{
+	int i;
+	for (i = 0; i < try; i++) {
+		printk(KERN_INFO
+		       "Unmap[%d] - virt=0x%08X phy=0x%08X size=0x%08X\n", i,
+		       (u32) b[i].vir, b[i].phy, b[i].size);
+		iounmap(b[i].vir);
+	}
+}
+
+module_exit(dummy_exit);
+
+MODULE_LICENSE("GPL");
-- 
1.5.4.3

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Woodruff, Richard]

> Kevin,
> Some more isolation:
> The l-o master branch does not have an issue, only the pm branch seems to have
> the problem - probably some commit(s?) missing in pm branch. Since bridge is
> based off the pm branch, wont make sense to regress on it :(.

This rings a bell from a couple months back.  I've not followed the thread but recall a recent fix to ioremap.

http://www.mail-archive.com/linux-omap@vger.kernel.org/msg08836.html

Regards,
Richard W.

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

> -----Original Message-----
> From: Woodruff, Richard
> Sent: Thursday, March 12, 2009 9:33 PM
> To: Menon, Nishanth; Kevin Hilman
> Cc: Guzman Lugo, Fernando; linux-omap@vger.kernel.org; Kanigeri, Hari;
> Lakhani, Amish; Gupta, Ramesh; Ameya Palande
> Subject: RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after
> reloading dspbridge several times due to a memory leak.)
> > Some more isolation:
> > The l-o master branch does not have an issue, only the pm branch seems
> to have
> > the problem - probably some commit(s?) missing in pm branch. Since
> bridge is
> > based off the pm branch, wont make sense to regress on it :(.
> 
> This rings a bell from a couple months back.  I've not followed the thread
> but recall a recent fix to ioremap.
> 
> http://www.mail-archive.com/linux-omap@vger.kernel.org/msg08836.html
> 
Ouch.. commit 24f11ec001920f1cfaeeed8e8b55725d900bbb56[1] is what you are speaking of? That commit is not present in pm branch :( 

Regards,
Nishanth Menon

Ref:
1. http://git.kernel.org/?p=linux/kernel/git/tmlind/linux-omap-2.6.git;a=commit;h=24f11ec001920f1cfaeeed8e8b55725d900bbb56 

Re: ioremap bug?

[Kevin Hilman]

"Woodruff, Richard" <r-woodruff2@ti.com> writes:

>> Kevin,
>> Some more isolation:
>> The l-o master branch does not have an issue, only the pm branch seems to have
>> the problem - probably some commit(s?) missing in pm branch. Since bridge is
>> based off the pm branch, wont make sense to regress on it :(.
>
> This rings a bell from a couple months back.  I've not followed the thread but recall a recent fix to ioremap.
>
> http://www.mail-archive.com/linux-omap@vger.kernel.org/msg08836.html
>

Yes, I was trying to remember why this sounded familiar, thanks
Richard.

The reason it works on l-o and not PM branch is because PM branch is
still 2.6.8 based, and l-o is on 2.6.29-rc which has a fix for this.

Nishant, can you try to cherry-pick commit
24f11ec001920f1cfaeeed8e8b55725d900bbb56 from Linus' tree into your PM
branch and see if that helps.

Kevin

RE: ioremap bug? (was RE: DSPBRIDGE: segmentation fault after reloading dspbridge several times due to a memory leak.)

[Menon, Nishanth]

Kevin, Richard,
> -----Original Message-----
> From: Menon, Nishanth
<snip>
> 
> > -----Original Message-----
> > From: Woodruff, Richard
> > Sent: Thursday, March 12, 2009 9:33 PM
<snip>
> >
> > This rings a bell from a couple months back.  I've not followed the
> thread
> > but recall a recent fix to ioremap.
> >
> > http://www.mail-archive.com/linux-omap@vger.kernel.org/msg08836.html
> >
> Ouch.. commit 24f11ec001920f1cfaeeed8e8b55725d900bbb56[1] is what you are
> speaking of? That commit is not present in pm branch :(
> 
Aye.. cherry-pick saves the day (yet again).. script runs without a hitch.. (though I admire the simple code that Tomi had put for the proving the same)..

Anyone know of any more similar memory - kmalloc/ioremap/dma_alloc_coherent/vm surprises since 2.6.28? ;) but that would be pretty lazy me I guess..:D

Regards,
Nishanth Menon

Re: ioremap bug?

[Kevin Hilman]

"Menon, Nishanth" <nm@ti.com> writes:

> Kevin, Richard,
>> -----Original Message-----
>> From: Menon, Nishanth
> <snip>
>> 
>> > -----Original Message-----
>> > From: Woodruff, Richard
>> > Sent: Thursday, March 12, 2009 9:33 PM
> <snip>
>> >
>> > This rings a bell from a couple months back.  I've not followed the
>> thread
>> > but recall a recent fix to ioremap.
>> >
>> > http://www.mail-archive.com/linux-omap@vger.kernel.org/msg08836.html
>> >
>> Ouch.. commit 24f11ec001920f1cfaeeed8e8b55725d900bbb56[1] is what you are
>> speaking of? That commit is not present in pm branch :(
>> 
>
> Aye.. cherry-pick saves the day (yet again).. script runs without a
> hitch.. (though I admire the simple code that Tomi had put for the
> proving the same)..

Thanks, cherry picked into PM branch as well.

> Anyone know of any more similar memory -
> kmalloc/ioremap/dma_alloc_coherent/vm surprises since 2.6.28? ;) but
> that would be pretty lazy me I guess..:D

It's always an exchange.  We get fixes for known bugs, and in exchange
get new bugs that haven't been found yet. ;)

Anyways, PM branch should be moving to 2.6.29 in the next couple
weeks.  Primary hold up is trying to get clock and PM stuff stabilized
and agreed upon for upstream merge (see recent PRCM register access
thread for details.)

Kevin