From mboxrd@z Thu Jan 1 00:00:00 1970 From: andy123 Subject: problems with 1G hugepages and linux 3.12-rc3 Date: Sun, 6 Oct 2013 02:47:41 +0200 Message-ID: <20131006024741.61cc8512@desk.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: qemu-devel@nongnu.org To: kvm@vger.kernel.org Return-path: Received: from mail-ee0-f50.google.com ([74.125.83.50]:38462 "EHLO mail-ee0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752523Ab3JFAry (ORCPT ); Sat, 5 Oct 2013 20:47:54 -0400 Received: by mail-ee0-f50.google.com with SMTP id d51so2489298eek.9 for ; Sat, 05 Oct 2013 17:47:53 -0700 (PDT) Sender: kvm-owner@vger.kernel.org List-ID: Hi, as the subject states, I have some problems with 1G hugepages with qemu(-vfio-git) on Linux 3.12-rc3. I start qemu like this, for example: "/usr/bin/qemu-system-x86_64 -enable-kvm -m 1024 -mem-path /dev/hugepages -drive file=/files/vm/arch.img,if=virtio,media=disk -monitor stdio" where /dev/hugepages is "hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,mode=1770,gid=78,pagesize=1G,pagesize=1G)" and the kernel is booted with "hugepagesz=1G hugepages=4". This result in lots of error message in dmesg, as seen here: https://gist.github.com/ajs124/6842823 (starting at 18:04:28) After starting and stopping multiple virtual machines, the hugepages seem to "fill up" and qemu outputs "file_ram_alloc: can't mmap RAM pages: Cannot allocate memory", but works anyways. With fill up, I mean that I can start qemu 2 time with "-m 2048" and 4 times with "-m 1024", before it fails to mmap. This works without any problems in 3.11.x and also with 2M hugepages (umount /dev/hugepages && mount -t hugetlbfs -o pagesize=2048k /dev/hugepages). I'm running this in arch linux and there has already been some discussion on the arch forums (https://bbs.archlinux.org/viewtopic.php?pid=1333469#p1333469) but I tried to add everything I wrote over there in this mail. In case I missed something or you need more information, I'll happily supply it in order to get this resolved. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:57605) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VScVs-0000JG-Bw for qemu-devel@nongnu.org; Sat, 05 Oct 2013 20:48:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VScVn-000682-Lv for qemu-devel@nongnu.org; Sat, 05 Oct 2013 20:48:00 -0400 Received: from mail-ee0-x22c.google.com ([2a00:1450:4013:c00::22c]:57630) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VScVn-00067r-EK for qemu-devel@nongnu.org; Sat, 05 Oct 2013 20:47:55 -0400 Received: by mail-ee0-f44.google.com with SMTP id b47so2504547eek.31 for ; Sat, 05 Oct 2013 17:47:53 -0700 (PDT) Date: Sun, 6 Oct 2013 02:47:41 +0200 From: andy123 Message-ID: <20131006024741.61cc8512@desk.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: [Qemu-devel] problems with 1G hugepages and linux 3.12-rc3 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: kvm@vger.kernel.org Cc: qemu-devel@nongnu.org Hi, as the subject states, I have some problems with 1G hugepages with qemu(-vfio-git) on Linux 3.12-rc3. I start qemu like this, for example: "/usr/bin/qemu-system-x86_64 -enable-kvm -m 1024 -mem-path /dev/hugepages -drive file=/files/vm/arch.img,if=virtio,media=disk -monitor stdio" where /dev/hugepages is "hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,mode=1770,gid=78,pagesize=1G,pagesize=1G)" and the kernel is booted with "hugepagesz=1G hugepages=4". This result in lots of error message in dmesg, as seen here: https://gist.github.com/ajs124/6842823 (starting at 18:04:28) After starting and stopping multiple virtual machines, the hugepages seem to "fill up" and qemu outputs "file_ram_alloc: can't mmap RAM pages: Cannot allocate memory", but works anyways. With fill up, I mean that I can start qemu 2 time with "-m 2048" and 4 times with "-m 1024", before it fails to mmap. This works without any problems in 3.11.x and also with 2M hugepages (umount /dev/hugepages && mount -t hugetlbfs -o pagesize=2048k /dev/hugepages). I'm running this in arch linux and there has already been some discussion on the arch forums (https://bbs.archlinux.org/viewtopic.php?pid=1333469#p1333469) but I tried to add everything I wrote over there in this mail. In case I missed something or you need more information, I'll happily supply it in order to get this resolved.