From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peter Lieven Subject: Re: UPD: qemu-kvm 0.12.5 virtio-net page allocation error Date: Thu, 07 Oct 2010 17:18:02 +0200 Message-ID: <4CADE4AA.1060909@dlh.net> References: <76A3E6A6-31E5-4BB0-B545-CBAB360DBBCC@dlh.net> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Peter Lieven To: kvm@vger.kernel.org, qemu-devel@nongnu.org Return-path: Received: from ssl.dlh.net ([91.198.192.8]:56589 "EHLO ssl.dlh.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1760706Ab0JGPSF (ORCPT ); Thu, 7 Oct 2010 11:18:05 -0400 In-Reply-To: <76A3E6A6-31E5-4BB0-B545-CBAB360DBBCC@dlh.net> Sender: kvm-owner@vger.kernel.org List-ID: Peter Lieven wrote: > Am 03.10.2010 um 01:48 schrieb Peter Lieven: > > >> Hi, >> >> running 0.12.5 with a Ubuntu LTS 10.04.1 64-bit kernel I see the following error after a few days with severe load (constant 100-200mbps input). >> >> # uname -a >> Linux ubuntu-newsfeed 2.6.32-24-server #43-Ubuntu SMP Thu Sep 16 16:05:42 UTC 2010 x86_64 GNU/Linux >> > > Linux newsfeed-spool 2.6.27.48-0.2-default seems not affected. I have a machine with even higher load (about 300-400mbits constant > network load) up for 4 months without a crash. > > >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] swapper: page allocation failure. order:0, mode:0x20 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Pid: 0, comm: swapper Not tainted 2.6.32-24-server #43-Ub >> untu >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Call Trace: >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] __alloc_pages_slowpath+0x56e >> /0x580 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] __alloc_pages_nodemask+0x171/0x180 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] alloc_pages_current+0x87/0xd0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] try_fill_recv+0x182/0x200 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] virtnet_poll+0x10d/0x160 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] net_rx_action+0x10f/0x250 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] __do_softirq+0xb7/0x1e0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? handle_IRQ_event+0x60/0x170 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] call_softirq+0x1c/0x30 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] do_softirq+0x65/0xa0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] irq_exit+0x85/0x90 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] do_IRQ+0x75/0xf0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ret_from_intr+0x0/0x11 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? native_safe_halt+0xb/0x10 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? default_idle+0x3d/0x90 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? cpu_idle+0xb3/0x110 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? rest_init+0x6b/0x80 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? start_kernel+0x368/0x371 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? x86_64_start_reservations+0x125/0x129 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] [] ? x86_64_start_kernel+0xfa/0x109 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Mem-Info: >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA per-cpu: >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 0: hi: 0, btch: 1 usd: 0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 1: hi: 0, btch: 1 usd: 0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 2: hi: 0, btch: 1 usd: 0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 3: hi: 0, btch: 1 usd: 0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA32 per-cpu: >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 0: hi: 186, btch: 31 usd: 59 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 1: hi: 186, btch: 31 usd: 159 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 2: hi: 186, btch: 31 usd: 102 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 3: hi: 186, btch: 31 usd: 151 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 Normal per-cpu: >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 0: hi: 186, btch: 31 usd: 51 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 1: hi: 186, btch: 31 usd: 157 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 2: hi: 186, btch: 31 usd: 162 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] CPU 3: hi: 186, btch: 31 usd: 175 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] active_anon:1857 inactive_anon:1602 isolated_anon:0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] active_file:458063 inactive_file:467825 isolated_file:0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] unevictable:0 dirty:74195 writeback:18395 unstable:0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] free:5214 slab_reclaimable:50506 slab_unreclaimable:3256 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] mapped:4151 shmem:1592 pagetables:400 bounce:0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA free:15896kB min:28kB low:32kB high:40kB active_anon:0kB inactive_anon:0kB active_fil >> e:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15336kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB s >> lab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? >> yes >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] lowmem_reserve[]: 0 3512 4017 4017 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA32 free:4652kB min:7076kB low:8844kB high:10612kB active_anon:3832kB inactive_anon:110 >> 8kB active_file:1642896kB inactive_file:1680672kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3596500kB mlocked:0kB dirty:262008kB wri >> teback:61820kB mapped:10184kB shmem:2812kB slab_reclaimable:170892kB slab_unreclaimable:4164kB kernel_stack:8kB pagetables:512kB unstable:0kB bounce:0kB >> writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] lowmem_reserve[]: 0 0 505 505 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 Normal free:308kB min:1016kB low:1268kB high:1524kB active_anon:3596kB inactive_anon:5300 >> kB active_file:189356kB inactive_file:190628kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:517120kB mlocked:0kB dirty:34772kB writebac >> k:11760kB mapped:6420kB shmem:3556kB slab_reclaimable:31132kB slab_unreclaimable:8852kB kernel_stack:1160kB pagetables:1088kB unstable:0kB bounce:0kB wr >> iteback_tmp:0kB pages_scanned:0 all_unreclaimable? no >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] lowmem_reserve[]: 0 0 0 0 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA: 2*4kB 4*8kB 5*16kB 3*32kB 3*64kB 1*128kB 2*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB >> = 15896kB >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 DMA32: 607*4kB 72*8kB 0*16kB 1*32kB 0*64kB 1*128kB 3*256kB 1*512kB 0*1024kB 0*2048kB 0*40 >> 96kB = 4444kB >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Node 0 Normal: 27*4kB 3*8kB 2*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*409 >> 6kB = 356kB >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 927861 total pagecache pages >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 296 pages in swap cache >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Swap cache stats: add 797, delete 501, find 10963/10986 >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Free swap = 3905004kB >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] Total swap = 3906552kB >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 1048576 pages RAM >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 34105 pages reserved >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 668525 pages shared >> Oct 2 10:09:35 ubuntu-newsfeed kernel: [248407.790029] 344067 pages non-shared >> >> Is this anything known? >> >> After this error, the network device hangs. However, it can be recovered by setting ip link down and up again. >> >> BR, >> Peter >> > > -- > To unsubscribe from this list: send the line "unsubscribe kvm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > anyone? willing to help with extensive debugging.