Re: Slowdown on high-load machines with 3000 sockets

Christian Schmid
2005-02-27T21:20:54Z

Rik van Riel wrote:
> On Sun, 27 Feb 2005, Rik van Riel wrote:
>
>
>>Is it possible to detect when the write system call blocks?
>
>
> OK, we got that on #kernelnewbies Initially the trace looks
> innocent enough, but there is something interesting going on
> with the VM. Something we may want to investigate...
>
> <Dragony> Feb 27 21:12:34 s02 kernel: Call Trace:
> <Dragony> Feb 27 21:12:34 s02 kernel: [<803e1148>] io_schedule+0x28/0x40
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8015ac26>] sync_buffer+0x36/0x50
> <Dragony> Feb 27 21:12:34 s02 kernel: [<803e1396>] __wait_on_bit+0x66/0x70
> <Dragony> Feb 27 21:12:34 s02 kernel: [<803e1428>] out_of_line_wait_on_bit+0x88/0x90
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8015c2a5>] __bread_slow+0x65/0x90
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8015c5bd>] __bread+0x3d/0x50
> <Dragony> Feb 27 21:12:34 s02 kernel: [<801dcdc7>] ext2_get_branch+0x77/0x120
> <Dragony> Feb 27 21:12:34 s02 kernel: [<801dd187>] ext2_get_block+0x77/0x3c0
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8017d258>] do_mpage_readpage+0x1a8/0x4f0
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8017d64f>] mpage_readpages+0xaf/0x130
> <Dragony> Feb 27 21:12:34 s02 kernel: [<80140c56>] read_pages+0xf6/0x100
> <Dragony> Feb 27 21:12:34 s02 kernel: [<80140f47>] do_page_cache_readahead+0x117/0x1a0
> <Dragony> Feb 27 21:12:34 s02 kernel: [<801411ab>] page_cache_readahead+0x1db/0x240
> <Dragony> Feb 27 21:12:34 s02 kernel: [<80139b10>] do_generic_mapping_read+0x100/0x3f0
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8013a31b>] generic_file_sendfile+0x5b/0x70
> <Dragony> Feb 27 21:12:34 s02 kernel: [<8015a4b0>] do_sendfile+0x1f0/0x2c0
>
> Looks like his program is doing a sendfile, which pauses
> on reading things from disk. So far, so good.
>
> However, the big question is why the slowdowns go away
> when min_free_kbytes is larger ?
>
> As an additional hint, Christian is using sys_readahead.
>
> Christian, how big are the data blocks you sys_readahead, and
> how many do you have outstanding at a time ?

I am reading 800000 bytes of data as soon as there is less than 200000 data in the cache. I do
assume the cache doesnt kill itself. I have 8 GB machine with 7,5 GB free for caching.