* memory problem with fec on 8250
@ 2003-12-03 7:48 Kromer
2003-12-04 8:45 ` Wojciech Kromer
0 siblings, 1 reply; 11+ messages in thread
From: Kromer @ 2003-12-03 7:48 UTC (permalink / raw)
To: Linuxppc-Embedded (E-mail)
hi there
i have some problems with using fec device on my custom 8250 board
while sending a lot of data (at least 300MB) through this device causes
lack of memory
sending same amount of data through loopback works fine
i'm using 2.4.19 kernel
any idea?
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
2003-12-03 7:48 memory problem with fec on 8250 Kromer
@ 2003-12-04 8:45 ` Wojciech Kromer
2003-12-04 9:15 ` Eugene Surovegin
0 siblings, 1 reply; 11+ messages in thread
From: Wojciech Kromer @ 2003-12-04 8:45 UTC (permalink / raw)
To: Linuxppc-Embedded (E-mail)
Użytkownik Kromer napisał:
>
> hi there
>
> i have some problems with using fec device on my custom 8250 board
> while sending a lot of data (at least 300MB) through this device causes
> lack of memory
> sending same amount of data through loopback works fine
>
> i'm using 2.4.19 kernel
> any idea?
cached memory keeps growing slowly, and free is going down same way,
after putting ~300MB ftp data (from 8250 board to another host), VM
starts killing some applications
same situation is with latest linuxppc_2_4_devel kernel from denx.de,
but there is additional information from kernel
<5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
<5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
<5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
<5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
<5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
<4>VM: killing process sshd
what's wrong ???
--
* * * * * * * * * * * *
* per pedes ad astra! *
* * * * * * * * * * * * mailto:krom@dgt-lab.com.pl
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
2003-12-04 8:45 ` Wojciech Kromer
@ 2003-12-04 9:15 ` Eugene Surovegin
2003-12-04 9:45 ` Wojciech Kromer
0 siblings, 1 reply; 11+ messages in thread
From: Eugene Surovegin @ 2003-12-04 9:15 UTC (permalink / raw)
To: Wojciech Kromer; +Cc: Linuxppc-Embedded (E-mail)
On Thu, Dec 04, 2003 at 09:45:53AM +0100, Wojciech Kromer wrote:
> >i have some problems with using fec device on my custom 8250 board
> >while sending a lot of data (at least 300MB) through this device causes
> >lack of memory
> >sending same amount of data through loopback works fine
> >
> >i'm using 2.4.19 kernel
> >any idea?
>
> cached memory keeps growing slowly, and free is going down same way,
> after putting ~300MB ftp data (from 8250 board to another host), VM
> starts killing some applications
>
> same situation is with latest linuxppc_2_4_devel kernel from denx.de,
> but there is additional information from kernel
>
> <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> <5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
> <4>VM: killing process sshd
>
> what's wrong ???
It means that this box is really out of memory.
Could you provide output from utility called 'free' and from file
/proc/slabinfo at two points of time: 1) when you just booted your
boax and 2) just before it starts failing.
Thanks,
Eugene.
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
2003-12-04 9:15 ` Eugene Surovegin
@ 2003-12-04 9:45 ` Wojciech Kromer
2003-12-04 10:19 ` Eugene Surovegin
0 siblings, 1 reply; 11+ messages in thread
From: Wojciech Kromer @ 2003-12-04 9:45 UTC (permalink / raw)
To: Linuxppc-Embedded (E-mail)
Użytkownik Eugene Surovegin napisał:
>On Thu, Dec 04, 2003 at 09:45:53AM +0100, Wojciech Kromer wrote:
>
>
>>>i have some problems with using fec device on my custom 8250 board
>>>while sending a lot of data (at least 300MB) through this device causes
>>>lack of memory
>>>sending same amount of data through loopback works fine
>>>
>>>i'm using 2.4.19 kernel
>>>any idea?
>>>
>>>
>>cached memory keeps growing slowly, and free is going down same way,
>>after putting ~300MB ftp data (from 8250 board to another host), VM
>>starts killing some applications
>>
>>same situation is with latest linuxppc_2_4_devel kernel from denx.de,
>>but there is additional information from kernel
>>
>><5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
>><5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
>><5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
>><5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
>><5>__alloc_pages: 0-order allocation failed (gfp=0x1d2/0)
>><4>VM: killing process sshd
>>
>>what's wrong ???
>>
>>
>
>It means that this box is really out of memory.
>
>
sure, but _why_? thre are no memory consuption processes,
i've just run ftp-put from this box to another machine
>Could you provide output from utility called 'free' and from file
>/proc/slabinfo at two points of time: 1) when you just booted your
>boax and 2) just before it starts failing.
>
>
>
thx for quick answer
huh, a lot of info:
==== at begining =========================================
bash-2.05# free
total used free shared buffers cached
Mem: 127888 22488 105400 0 0 15272
-/+ buffers/cache: 7216 120672
Swap: 0 0 0
bash-2.05# cat /proc/slabinfo
slabinfo - version: 1.1
kmem_cache 62 68 112 2 2 1
tcp_tw_bucket 0 0 96 0 0 1
tcp_bind_bucket 3 113 32 1 1 1
tcp_open_request 0 59 64 0 1 1
inet_peer_cache 2 59 64 1 1 1
ip_fib_hash 10 113 32 1 1 1
ip_dst_cache 27 48 160 2 2 1
arp_cache 4 30 128 1 1 1
blkdev_requests 0 0 96 0 0 1
nfs_write_data 0 10 384 0 1 1
nfs_read_data 0 40 384 0 4 1
nfs_page 0 40 96 0 1 1
devfsd_event 0 0 20 0 0 1
journal_head 0 0 48 0 0 1
revoke_table 0 0 12 0 0 1
revoke_record 0 0 32 0 0 1
dnotify_cache 0 0 20 0 0 1
file_lock_cache 2 40 96 1 1 1
fasync_cache 0 0 16 0 0 1
uid_cache 0 0 32 0 0 1
skbuff_head_cache 131 192 160 8 8 1
sock 21 25 800 5 5 1
sigqueue 0 29 132 0 1 1
kiobuf 0 0 64 0 0 1
cdev_cache 10 59 64 1 1 1
bdev_cache 0 0 64 0 0 1
mnt_cache 11 59 64 1 1 1
inode_cache 419 424 480 53 53 1
dentry_cache 601 630 128 21 21 1
filp 175 180 128 6 6 1
names_cache 0 3 4096 0 3 1
buffer_head 0 0 96 0 0 1
mm_struct 14 24 160 1 1 1
vm_area_struct 462 560 96 12 14 1
fs_cache 13 113 32 1 1 1
files_cache 14 18 416 2 2 1
signal_act 16 21 1312 6 7 1
size-131072(DMA) 0 0 131072 0 0 32
size-131072 0 0 131072 0 0 32
size-65536(DMA) 0 0 65536 0 0 16
size-65536 0 0 65536 0 0 16
size-32768(DMA) 0 0 32768 0 0 8
size-32768 0 0 32768 0 0 8
size-16384(DMA) 0 0 16384 0 0 4
size-16384 0 0 16384 0 0 4
size-8192(DMA) 0 0 8192 0 0 2
size-8192 0 0 8192 0 0 2
size-4096(DMA) 0 0 4096 0 0 1
size-4096 10 10 4096 10 10 1
size-2048(DMA) 0 0 2048 0 0 1
size-2048 5 178 2048 3 89 1
size-1024(DMA) 0 0 1024 0 0 1
size-1024 17 24 1024 5 6 1
size-512(DMA) 0 0 512 0 0 1
size-512 6 8 512 1 1 1
size-256(DMA) 0 0 256 0 0 1
size-256 14 30 256 1 2 1
size-128(DMA) 0 0 128 0 0 1
size-128 829 930 128 31 31 1
size-64(DMA) 0 0 64 0 0 1
size-64 41 59 64 1 1 1
size-32(DMA) 0 0 32 0 0 1
size-32 201 226 32 2 2 1
===after some time (ftp put ~75MB) ================================
bash-2.05# free;cat /proc/slabinfo
total used free shared buffers cached
Mem: 127888 125688 2200 0 0 81772
-/+ buffers/cache: 43916 83972
Swap: 0 0 0
slabinfo - version: 1.1
kmem_cache 62 68 112 2 2 1
tcp_tw_bucket 0 0 96 0 0 1
tcp_bind_bucket 5 113 32 1 1 1
tcp_open_request 0 0 64 0 0 1
inet_peer_cache 1 59 64 1 1 1
ip_fib_hash 10 113 32 1 1 1
ip_dst_cache 51 72 160 3 3 1
arp_cache 3 30 128 1 1 1
blkdev_requests 0 0 96 0 0 1
nfs_write_data 0 0 384 0 0 1
nfs_read_data 7 70 384 4 7 1
nfs_page 7 80 96 2 2 1
devfsd_event 0 0 20 0 0 1
journal_head 0 0 48 0 0 1
revoke_table 0 0 12 0 0 1
revoke_record 0 0 32 0 0 1
dnotify_cache 0 0 20 0 0 1
file_lock_cache 2 40 96 1 1 1
fasync_cache 0 0 16 0 0 1
uid_cache 0 0 32 0 0 1
skbuff_head_cache 16857 16872 160 703 703 1
sock 23 25 800 5 5 1
sigqueue 0 29 132 0 1 1
kiobuf 0 0 64 0 0 1
cdev_cache 10 59 64 1 1 1
bdev_cache 0 0 64 0 0 1
mnt_cache 11 59 64 1 1 1
inode_cache 435 440 480 55 55 1
dentry_cache 625 630 128 21 21 1
filp 179 180 128 6 6 1
names_cache 0 2 4096 0 2 1
buffer_head 0 0 96 0 0 1
mm_struct 15 24 160 1 1 1
vm_area_struct 485 520 96 13 13 1
fs_cache 14 113 32 1 1 1
files_cache 15 18 416 2 2 1
signal_act 17 18 1312 6 6 1
size-131072(DMA) 0 0 131072 0 0 32
size-131072 0 0 131072 0 0 32
size-65536(DMA) 0 0 65536 0 0 16
size-65536 0 0 65536 0 0 16
size-32768(DMA) 0 0 32768 0 0 8
size-32768 0 0 32768 0 0 8
size-16384(DMA) 0 0 16384 0 0 4
size-16384 0 0 16384 0 0 4
size-8192(DMA) 0 0 8192 0 0 2
size-8192 0 0 8192 0 0 2
size-4096(DMA) 0 0 4096 0 0 1
size-4096 10 10 4096 10 10 1
size-2048(DMA) 0 0 2048 0 0 1
size-2048 16749 16798 2048 8382 8399 1
size-1024(DMA) 0 0 1024 0 0 1
size-1024 18 20 1024 5 5 1
size-512(DMA) 0 0 512 0 0 1
size-512 24 32 512 4 4 1
size-256(DMA) 0 0 256 0 0 1
size-256 51 75 256 4 5 1
size-128(DMA) 0 0 128 0 0 1
size-128 910 930 128 31 31 1
size-64(DMA) 0 0 64 0 0 1
size-64 42 59 64 1 1 1
size-32(DMA) 0 0 32 0 0 1
size-32 8513 8588 32 76 76 1
--
* * * * * * * * * * * *
* per pedes ad astra! *
* * * * * * * * * * * * mailto:krom@dgt-lab.com.pl
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: memory problem with fec on 8250
2003-12-04 9:45 ` Wojciech Kromer
@ 2003-12-04 10:19 ` Eugene Surovegin
[not found] ` <3FCF0E68.4010303@dgt-lab.com.pl>
0 siblings, 1 reply; 11+ messages in thread
From: Eugene Surovegin @ 2003-12-04 10:19 UTC (permalink / raw)
To: Wojciech Kromer; +Cc: Linuxppc-Embedded (E-mail)
On Thu, Dec 04, 2003 at 10:45:54AM +0100, Wojciech Kromer wrote:
> >
> >>>i have some problems with using fec device on my custom 8250 board
> >>>while sending a lot of data (at least 300MB) through this device causes
> >>>lack of memory
> >>>sending same amount of data through loopback works fine
> >>>
[snip]
> ==== at begining =========================================
> skbuff_head_cache 131 192 160 8 8 1
> size-2048 5 178 2048 3 89 1
>
> ===after some time (ftp put ~75MB) ================================
>
> skbuff_head_cache 16857 16872 160 703 703 1
> size-2048 16749 16798 2048 8382 8399 1
It looks like skb leaks in the driver. Probably in TX path.
Could you send me by private e-mail source code for ethernet driver in
question or provide a URL.
I'm not a 8250 guy, but I'll try to look at it :)
Eugene.
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
@ 2004-01-04 15:55 Samo Pogacnik
2004-01-05 8:15 ` Wojciech Kromer
0 siblings, 1 reply; 11+ messages in thread
From: Samo Pogacnik @ 2004-01-04 15:55 UTC (permalink / raw)
To: linuxppc-embedded
Hello!
Is it possible that there is something wrong in the way how the:
"netif_wake_queue(dev)" gets called from the interrupt handler in the
"fcc_enet.c"?
I can confirm this problem for 2.4.18 as well as 2.6.0 kernels on mpc82xx.
bye, Samo
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
2004-01-04 15:55 Samo Pogacnik
@ 2004-01-05 8:15 ` Wojciech Kromer
2004-01-05 22:52 ` Samo Pogacnik
0 siblings, 1 reply; 11+ messages in thread
From: Wojciech Kromer @ 2004-01-05 8:15 UTC (permalink / raw)
To: Linuxppc-Embedded (E-mail)
Użytkownik Samo Pogacnik napisał:
>Hello!
>
>Is it possible that there is something wrong in the way how the:
>"netif_wake_queue(dev)" gets called from the interrupt handler in the
>"fcc_enet.c"?
>I can confirm this problem for 2.4.18 as well as 2.6.0 kernels on mpc82xx.
>
>bye, Samo
>
>
>
>
i don't know exactly what's wrong with 82xx code,
but i wrote a small patch that frees unfreed skbuf before allocating new
one,
and now it works fine
additionaly, i had no problem with (almost) the same code on 8xx
--
* * * * * * * * * * * *
* per pedes ad astra! *
* * * * * * * * * * * * mailto:krom@dgt-lab.com.pl
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: memory problem with fec on 8250
2004-01-05 8:15 ` Wojciech Kromer
@ 2004-01-05 22:52 ` Samo Pogacnik
2004-01-06 7:47 ` cd Wojciech Kromer
0 siblings, 1 reply; 11+ messages in thread
From: Samo Pogacnik @ 2004-01-05 22:52 UTC (permalink / raw)
To: linuxppc-embedded
On Monday 05 January 2004 09:15, Wojciech Kromer wrote:
> Użytkownik Samo Pogacnik napisał:
> >
> >Is it possible that there is something wrong in the way how the:
> >"netif_wake_queue(dev)" gets called from the interrupt handler in the
> >"fcc_enet.c"? I can confirm this problem for 2.4.18 as well as 2.6.0
> >kernels on mpc82xx.
>
> i don't know exactly what's wrong with 82xx code, but i wrote a small
> patch that frees unfreed skbuf before allocating new one, and now it
> works fine
>
> additionaly, i had no problem with (almost) the same code on 8xx
hi, Wojciech
can you send the patch?
did you free skbuffs from the irq handler or somewhere else?
when i run netperf (snapshot) test using the 100Mbps connection, this
test eats all the available RAM. the /proc/meminfo shows that most of the used
memory at that point is been held by internal kernel structures (Slab).
by, Samo
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
* cd
2004-01-05 22:52 ` Samo Pogacnik
@ 2004-01-06 7:47 ` Wojciech Kromer
2004-01-06 22:03 ` memory problem with fec on 8250 Samo Pogacnik
0 siblings, 1 reply; 11+ messages in thread
From: Wojciech Kromer @ 2004-01-06 7:47 UTC (permalink / raw)
To: Linuxppc-Embedded (E-mail)
Użytkownik Samo Pogacnik napisał:
>On Monday 05 January 2004 09:15, Wojciech Kromer wrote:
>
>
>>Użytkownik Samo Pogacnik napisał:
>>
>>
>>>Is it possible that there is something wrong in the way how the:
>>>"netif_wake_queue(dev)" gets called from the interrupt handler in the
>>>"fcc_enet.c"? I can confirm this problem for 2.4.18 as well as 2.6.0
>>>kernels on mpc82xx.
>>>
>>>
>>i don't know exactly what's wrong with 82xx code, but i wrote a small
>>patch that frees unfreed skbuf before allocating new one, and now it
>>works fine
>>
>>additionaly, i had no problem with (almost) the same code on 8xx
>>
>>
>
>hi, Wojciech
>can you send the patch?
>
sure, just 4 lines addes
>did you free skbuffs from the irq handler or somewhere else?
>
>
just before allocation
>when i run netperf (snapshot) test using the 100Mbps connection, this
>test eats all the available RAM. the /proc/meminfo shows that most of the used
>memory at that point is been held by internal kernel structures (Slab).
>
>
better reults are shwn with
grep skb /proc/slabinfo
>by, Samo
>
>
here it is:
--------------------------------------------------------------------------------------------------------
***************
*** 413,418 ****
--- 412,420 ----
/* Save skb pointer.
*/
+ //by k@m 2003-12-05
+ if(cep->tx_skbuff[cep->skb_cur])
+ dev_kfree_skb_irq (cep->tx_skbuff[cep->skb_cur]);
cep->tx_skbuff[cep->skb_cur] = skb;
cep->stats.tx_bytes += skb->len;
***************
*** 560,565 ****
--- 562,568 ----
/* Free the sk buffer associated with this last transmit.
*/
dev_kfree_skb_irq(cep->tx_skbuff[cep->skb_dirty]);
+ cep->tx_skbuff[cep->skb_dirty]=0;//by k@m 2003-12-05
cep->skb_dirty = (cep->skb_dirty + 1) & TX_RING_MOD_MASK;
/* Update pointer to next buffer descriptor to be transmitted.
***************
*** 1348,1353 ****
--- 1351,1357 ----
* It works when I do this.
*/
memset((char *)ep, 0, sizeof(fcc_enet_t));
+ memset(cep->tx_skbuff,0,sizeof(cep->tx_skbuff));
/* Allocate space for the buffer descriptors in the DP ram.
* These are relative offsets in the DP ram address space.
-------------------------------------------------------------------------------------------
ps: i send you my full version of fcc_enet.c
--
* * * * * * * * * * * *
* per pedes ad astra! *
* * * * * * * * * * * * mailto:krom@dgt-lab.com.pl
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: memory problem with fec on 8250
2004-01-06 7:47 ` cd Wojciech Kromer
@ 2004-01-06 22:03 ` Samo Pogacnik
0 siblings, 0 replies; 11+ messages in thread
From: Samo Pogacnik @ 2004-01-06 22:03 UTC (permalink / raw)
To: linuxppc-embedded
On Tuesday 06 January 2004 08:47, Wojciech Kromer wrote:
> sure, just 4 lines addes
>
> >did you free skbuffs from the irq handler or somewhere else?
>
> just before allocation
>
> >when i run netperf (snapshot) test using the 100Mbps connection, this
> >test eats all the available RAM. the /proc/meminfo shows that most
> >of the used memory at that point is been held by internal kernel
> >structures (Slab).
>
> better reults are shwn with grep skb /proc/slabinfo
>
> here it is:
.
.
.
Thank you for the tips and the patch.
Eth seems to be working now without eating the RAM.
By the way that you have solved the problem, I suspect that BDs are written
over the dirty_tx limit, before their associated skbs get freed via the irq
handler.
bye, Samo
** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2004-01-06 22:03 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2003-12-03 7:48 memory problem with fec on 8250 Kromer
2003-12-04 8:45 ` Wojciech Kromer
2003-12-04 9:15 ` Eugene Surovegin
2003-12-04 9:45 ` Wojciech Kromer
2003-12-04 10:19 ` Eugene Surovegin
[not found] ` <3FCF0E68.4010303@dgt-lab.com.pl>
[not found] ` <20031204105532.GA2065@gate.ebshome.net>
2003-12-04 11:25 ` Wojciech Kromer
2003-12-04 14:03 ` Dan Malek
-- strict thread matches above, loose matches on Subject: below --
2004-01-04 15:55 Samo Pogacnik
2004-01-05 8:15 ` Wojciech Kromer
2004-01-05 22:52 ` Samo Pogacnik
2004-01-06 7:47 ` cd Wojciech Kromer
2004-01-06 22:03 ` memory problem with fec on 8250 Samo Pogacnik
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.