Re: LinuxQuestions.org - Community Bulletin

linux-admin.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

* Re: LinuxQuestions.org - Community Bulletin
       [not found] <200605020407.a7286e265899@www.linuxquestions.org>
@ 2006-05-03  6:41 ` tom arnall
  2006-05-03  5:35   ` chuck gelm
  2006-05-03 19:34   ` Glynn Clements
  0 siblings, 2 replies; 5+ messages in thread
From: tom arnall @ 2006-05-03  6:41 UTC (permalink / raw)
  To: linux-admin

i am experiencing memory problems trying to run an application that processes 
a file of about .5GB. So far my results are:

(1) set ulimit as follows: 

	core file size        (blocks, -c) 0
	data seg size         (kbytes, -d) unlimited
	file size             (blocks, -f) unlimited
	max locked memory     (kbytes, -l) 256000
	max memory size       (kbytes, -m) 256000
	open files                    (-n) 1024
	pipe size          (512 bytes, -p) 8
	stack size            (kbytes, -s) 8192
	cpu time             (seconds, -t) unlimited
	max user processes            (-u) unlimited
	virtual memory        (kbytes, -v) 768000

This gives me 'out of memory' errors.

(2) set 'virtual memory' very high and the application hogs memory and brings 
the rest of the system to almost a halt.

Thanks in advance for any help you can give me,

Tom Arnall
north spit, ca


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: LinuxQuestions.org - Community Bulletin
  2006-05-03  6:41 ` LinuxQuestions.org - Community Bulletin tom arnall
@ 2006-05-03  5:35   ` chuck gelm
  2006-05-03 11:21     ` Al Boldi
  2006-05-03 19:34   ` Glynn Clements
  1 sibling, 1 reply; 5+ messages in thread
From: chuck gelm @ 2006-05-03  5:35 UTC (permalink / raw)
  To: kloro; +Cc: linux-admin

tom arnall wrote:

>i am experiencing memory problems trying to run an application that processes 
>a file of about .5GB. So far my results are:
>
>(1) set ulimit as follows: 
>
>	core file size        (blocks, -c) 0
>	data seg size         (kbytes, -d) unlimited
>	file size             (blocks, -f) unlimited
>	max locked memory     (kbytes, -l) 256000
>	max memory size       (kbytes, -m) 256000
>	open files                    (-n) 1024
>	pipe size          (512 bytes, -p) 8
>	stack size            (kbytes, -s) 8192
>	cpu time             (seconds, -t) unlimited
>	max user processes            (-u) unlimited
>	virtual memory        (kbytes, -v) 768000
>
>This gives me 'out of memory' errors.
>
>(2) set 'virtual memory' very high and the application hogs memory and brings 
>the rest of the system to almost a halt.
>
>Thanks in advance for any help you can give me,
>
>Tom Arnall
>north spit, ca
>  
>

Hi, Tom:

 I don't know anything about 'ulimit', but how much real memory do you have?
I'd try adding another 2 Gigabytes of virtual memory.
Also, try

 nice -n 19 <application>

so that it does not 'hog' the system so much.

Run 'top' to watch how much memory your application uses.

HTH, Chuck


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: LinuxQuestions.org - Community Bulletin
  2006-05-03  5:35   ` chuck gelm
@ 2006-05-03 11:21     ` Al Boldi
       [not found]       ` <4458677E.6000706@gelm.net>
  0 siblings, 1 reply; 5+ messages in thread
From: Al Boldi @ 2006-05-03 11:21 UTC (permalink / raw)
  To: linux-kernel; +Cc: linux-admin, chuck gelm, kloro

chuck gelm wrote:
> tom arnall wrote:
> >i am experiencing memory problems trying to run an application that
> > processes a file of about .5GB. So far my results are:
> >
> >(1) set ulimit as follows:
> >
> >	core file size        (blocks, -c) 0
> >	data seg size         (kbytes, -d) unlimited
> >	file size             (blocks, -f) unlimited
> >	max locked memory     (kbytes, -l) 256000
> >	max memory size       (kbytes, -m) 256000
> >	open files                    (-n) 1024
> >	pipe size          (512 bytes, -p) 8
> >	stack size            (kbytes, -s) 8192
> >	cpu time             (seconds, -t) unlimited
> >	max user processes            (-u) unlimited
> >	virtual memory        (kbytes, -v) 768000
> >
> >This gives me 'out of memory' errors.
> >
> >(2) set 'virtual memory' very high and the application hogs memory and
> > brings the rest of the system to almost a halt.
> >
> >Thanks in advance for any help you can give me,
> >
> >Tom Arnall
> >north spit, ca
>
> Hi, Tom:
>
>  I don't know anything about 'ulimit', but how much real memory do you
> have? I'd try adding another 2 Gigabytes of virtual memory.
> Also, try
>
>  nice -n 19 <application>
>
> so that it does not 'hog' the system so much.
>
> Run 'top' to watch how much memory your application uses.
>
> HTH, Chuck
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-admin" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


^ permalink raw reply	[flat|nested] 5+ messages in thread

[parent not found: <4458677E.6000706@gelm.net>]

* Re: LinuxQuestions.org - Community Bulletin
       [not found]       ` <4458677E.6000706@gelm.net>
@ 2006-05-03 14:39         ` Al Boldi
  0 siblings, 0 replies; 5+ messages in thread
From: Al Boldi @ 2006-05-03 14:39 UTC (permalink / raw)
  To: chuck gelm; +Cc: linux-kernel, linux-admin

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

chuck gelm wrote:
> Al Boldi wrote:
> >chuck gelm wrote:
> >>tom arnall wrote:
> >>>i am experiencing memory problems

Me too on 2.6.16, so maybe lkml can help?

> >>>trying to run an application that
> >>>processes a file of about .5GB. So far my results are:
> >>>
> >>>(1) set ulimit as follows:
> >>>
> >>>	core file size        (blocks, -c) 0
> >>>	data seg size         (kbytes, -d) unlimited
> >>>	file size             (blocks, -f) unlimited
> >>>	max locked memory     (kbytes, -l) 256000
> >>>	max memory size       (kbytes, -m) 256000
> >>>	open files                    (-n) 1024
> >>>	pipe size          (512 bytes, -p) 8
> >>>	stack size            (kbytes, -s) 8192
> >>>	cpu time             (seconds, -t) unlimited
> >>>	max user processes            (-u) unlimited
> >>>	virtual memory        (kbytes, -v) 768000
> >>>
> >>>This gives me 'out of memory' errors.
> >>>
> >>>(2) set 'virtual memory' very high and the application hogs memory and
> >>>brings the rest of the system to almost a halt.
> >>>
> >>>Thanks in advance for any help you can give me,
> >>>
> >>>Tom Arnall
> >>>north spit, ca
> >>
> >>Hi, Tom:
> >>
> >> I don't know anything about 'ulimit', but how much real memory do you
> >>have? I'd try adding another 2 Gigabytes of virtual memory.
> >>Also, try
> >>
> >> nice -n 19 <application>
> >>
> >>so that it does not 'hog' the system so much.
> >>
> >>Run 'top' to watch how much memory your application uses.
> >>
> >>HTH, Chuck

Thanks!

--
Al





[-- Attachment #2: oom.log --]
[-- Type: text/x-log, Size: 6162 bytes --]

oom-killer: gfp_mask=0x201d2, order=0
 [<c0463e40>] pci_claim_resource+0x20/0xd0
 [<c013ff25>] out_of_memory+0xa5/0xc0
 [<c0141099>] __alloc_pages+0x279/0x310
 [<c0143669>] __do_page_cache_readahead+0xe9/0x120
 [<c0143b3f>] max_sane_readahead+0x2f/0x50
 [<c0463eac>] pci_claim_resource+0x8c/0xd0
 [<c013d8cb>] filemap_nopage+0x2eb/0x370
 [<c0149ea5>] do_no_page+0x65/0x220
 [<c0463f18>] pdev_sort_resources+0x28/0x100
 [<c014a1dc>] __handle_mm_fault+0xec/0x200
 [<c0113258>] do_page_fault+0x188/0x5c5
 [<c0463fbc>] pdev_sort_resources+0xcc/0x100
 [<c0463fbc>] pdev_sort_resources+0xcc/0x100
 [<c0463fbc>] pdev_sort_resources+0xcc/0x100
 [<c01130d0>] do_page_fault+0x0/0x5c5
 [<c0103a0f>] error_code+0x4f/0x54
Mem-info:
DMA per-cpu:
cpu 0 hot: high 0, batch 1 used:0
cpu 0 cold: high 0, batch 1 used:0
DMA32 per-cpu: empty
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         476kB (0kB HighMem)
Active:96 inactive:126 dirty:0 writeback:0 unstable:0 free:119 slab:445 mapped:186 pagetables:31
DMA free:476kB min:360kB low:448kB high:540kB active:384kB inactive:504kB present:8192kB pages_scanned:257 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 23*4kB 4*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 476kB
DMA32: empty
Normal: empty
HighMem: empty
Swap cache: add 11494, delete 11444, find 2453/5972, race 0+0
Free swap  = 1016696kB
Total swap = 1020088kB
Free swap:       1016696kB
2048 pages of RAM
0 pages of HIGHMEM
1031 reserved pages
209 pages shared
50 pages swap cached
0 pages dirty
0 pages writeback
186 pages mapped
445 pages slab
31 pages pagetables
Out of Memory: Kill process 65 (bash) score 88 and children.
Out of memory: Killed process 104 (mc).
oom-killer: gfp_mask=0x201d2, order=0
 [<c044fe40>] gzip_mark+0x0/0x10
 [<c013ff25>] out_of_memory+0xa5/0xc0
 [<c0141099>] __alloc_pages+0x279/0x310
 [<c0143669>] __do_page_cache_readahead+0xe9/0x120
 [<c0143b3f>] max_sane_readahead+0x2f/0x50
 [<c044feac>] flush_window+0x4c/0x70
 [<c013d8cb>] filemap_nopage+0x2eb/0x370
 [<c0149ea5>] do_no_page+0x65/0x220
 [<c044ff18>] unpack_to_rootfs+0x48/0x200
 [<c014a1dc>] __handle_mm_fault+0xec/0x200
 [<c0113258>] do_page_fault+0x188/0x5c5
 [<c044ffbc>] unpack_to_rootfs+0xec/0x200
 [<c044ffbc>] unpack_to_rootfs+0xec/0x200
 [<c044e000>] do_name+0x150/0x1b0
 [<c044e000>] do_name+0x150/0x1b0
 [<c01130d0>] do_page_fault+0x0/0x5c5
 [<c0103a0f>] error_code+0x4f/0x54
Mem-info:
DMA per-cpu:
cpu 0 hot: high 0, batch 1 used:0
cpu 0 cold: high 0, batch 1 used:0
DMA32 per-cpu: empty
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         528kB (0kB HighMem)
Active:177 inactive:73 dirty:0 writeback:0 unstable:0 free:132 slab:440 mapped:213 pagetables:27
DMA free:528kB min:360kB low:448kB high:540kB active:708kB inactive:292kB present:8192kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 40*4kB 2*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 528kB
DMA32: empty
Normal: empty
HighMem: empty
Swap cache: add 13712, delete 13643, find 2842/7215, race 0+0
Free swap  = 1016948kB
Total swap = 1020088kB
Free swap:       1016948kB
2048 pages of RAM
0 pages of HIGHMEM
1031 reserved pages
213 pages shared
69 pages swap cached
0 pages dirty
0 pages writeback
213 pages mapped
440 pages slab
27 pages pagetables
Out of Memory: Kill process 22 (bash) score 64 and children.
Out of memory: Killed process 103 (top).
oom-killer: gfp_mask=0x201d2, order=0
 [<c013ff25>] out_of_memory+0xa5/0xc0
 [<c0141099>] __alloc_pages+0x279/0x310
 [<c0143669>] __do_page_cache_readahead+0xe9/0x120
 [<c0143b3f>] max_sane_readahead+0x2f/0x50
 [<c013d8cb>] filemap_nopage+0x2eb/0x370
 [<c0149ea5>] do_no_page+0x65/0x220
 [<c014a1dc>] __handle_mm_fault+0xec/0x200
 [<c0113258>] do_page_fault+0x188/0x5c5
 [<c01130d0>] do_page_fault+0x0/0x5c5
 [<c0103a0f>] error_code+0x4f/0x54
Mem-info:
DMA per-cpu:
cpu 0 hot: high 0, batch 1 used:0
cpu 0 cold: high 0, batch 1 used:0
DMA32 per-cpu: empty
Normal per-cpu: empty
HighMem per-cpu: empty
Free pages:         368kB (0kB HighMem)
Active:18 inactive:261 dirty:0 writeback:0 unstable:0 free:92 slab:443 mapped:246 pagetables:27
DMA free:368kB min:360kB low:448kB high:540kB active:72kB inactive:1044kB present:8192kB pages_scanned:466 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 0*4kB 2*8kB 0*16kB 1*32kB 1*64kB 0*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 368kB
DMA32: empty
Normal: empty
HighMem: empty
Swap cache: add 16443, delete 16334, find 3542/8819, race 0+0
Free swap  = 1016548kB
Total swap = 1020088kB
Free swap:       1016548kB
2048 pages of RAM
0 pages of HIGHMEM
1031 reserved pages
253 pages shared
109 pages swap cached
0 pages dirty
0 pages writeback
246 pages mapped
443 pages slab
27 pages pagetables
Out of Memory: Kill process 108 (bash) score 74 and children.
Out of memory: Killed process 144 (bash).

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: LinuxQuestions.org - Community Bulletin
  2006-05-03  6:41 ` LinuxQuestions.org - Community Bulletin tom arnall
  2006-05-03  5:35   ` chuck gelm
@ 2006-05-03 19:34   ` Glynn Clements
  1 sibling, 0 replies; 5+ messages in thread
From: Glynn Clements @ 2006-05-03 19:34 UTC (permalink / raw)
  To: kloro; +Cc: linux-admin


tom arnall wrote:

> i am experiencing memory problems trying to run an application that processes 
> a file of about .5GB. So far my results are:
> 
> (1) set ulimit as follows: 
> 
> 	core file size        (blocks, -c) 0
> 	data seg size         (kbytes, -d) unlimited
> 	file size             (blocks, -f) unlimited
> 	max locked memory     (kbytes, -l) 256000
> 	max memory size       (kbytes, -m) 256000
> 	open files                    (-n) 1024
> 	pipe size          (512 bytes, -p) 8
> 	stack size            (kbytes, -s) 8192
> 	cpu time             (seconds, -t) unlimited
> 	max user processes            (-u) unlimited
> 	virtual memory        (kbytes, -v) 768000
> 
> This gives me 'out of memory' errors.
> 
> (2) set 'virtual memory' very high and the application hogs memory and brings 
> the rest of the system to almost a halt.

It sounds as if you don't have enough physical memory for the task at
hand. If that is the case, no amount of tweaking parameters will solve
your problem; you'll just have to install more memory.

You don't mention which application you're using or what type of
processing is being performed. Depending upon the task, processing a
file may require memory far in excess of the size of the file.

-- 
Glynn Clements <glynn@gclements.plus.com>

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2006-05-03 19:34 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
     [not found] <200605020407.a7286e265899@www.linuxquestions.org>
2006-05-03  6:41 ` LinuxQuestions.org - Community Bulletin tom arnall
2006-05-03  5:35   ` chuck gelm
2006-05-03 11:21     ` Al Boldi
     [not found]       ` <4458677E.6000706@gelm.net>
2006-05-03 14:39         ` Al Boldi
2006-05-03 19:34   ` Glynn Clements

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).