From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:34174) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TJTfF-0001ul-7w for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:27:22 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TJTfE-000585-4H for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:27:21 -0400 Received: from mail-wi0-f175.google.com ([209.85.212.175]:50395) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TJTfD-00057z-Sw for qemu-devel@nongnu.org; Wed, 03 Oct 2012 14:27:20 -0400 Received: by mail-wi0-f175.google.com with SMTP id hq4so2158735wib.10 for ; Wed, 03 Oct 2012 11:27:19 -0700 (PDT) Sender: Paolo Bonzini Message-ID: <506C8381.9090307@redhat.com> Date: Wed, 03 Oct 2012 20:27:13 +0200 From: Paolo Bonzini MIME-Version: 1.0 References: <20120924091008.GJ18470@in.ibm.com> <20120924091340.GN18470@in.ibm.com> <506C5EB0.50508@redhat.com> <506C7CD5.6090407@redhat.com> <506C814B.1080305@redhat.com> <506C8126.7080305@redhat.com> In-Reply-To: <506C8126.7080305@redhat.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] O_DIRECT on glusterfs (was Re: [PATCH v9 4/4] block: Support GlusterFS as a QEMU block backend) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Anand Avati Cc: Kevin Wolf , Anthony Liguori , Vijay Bellur , Stefan Hajnoczi , Harsh Bora , Amar Tumballi , qemu-devel@nongnu.org, "Richard W.M. Jones" , Blue Swirl , Avi Kivity , bharata@linux.vnet.ibm.com, Daniel Veillard Il 03/10/2012 20:17, Anand Avati ha scritto: > On 10/03/2012 11:17 AM, Paolo Bonzini wrote: >> Il 03/10/2012 19:58, Anand Avati ha scritto: >>>> >>>> I think these 3 lines should be removed. We're bypassing the host >>>> buffer cache just by virtue of using a userspace driver, and that's >>>> what >>>> cache=none cares about. >>> >>> O_DIRECT also has an effect on the behavior of the "client side" (the >>> part within the qemu) of Gluster stack as well. I presume the intention >>> of O_DIRECT is to minimize use of memory (whether as host' page cache or >>> buffered data in user space). To that end it is a good idea to leave >>> O_DIRECT flag set. >>> >>> The behavior of whether gluster bricks need to get the O_DIRECT >>> propagated or not is a different issue. We are exploring the possibility >>> of not sending O_DIRECT flag over the wire to mimic NFS behavior. That >>> would be independent of the qemu block driver setting the open flag. >> >> What is the effect of O_DIRECT on the client exactly? > > To avoid caching in the io-cache module, disable read-ahead etc (if > those translators are loaded). The behavior in write-behind is tunable. > You could either disable write-behind entirely (which will happen once > libgfapi supports 0-copy/RDMA) or perform a sliding-window like > size-limited write-behind (defaults to 1MB). Thanks for the information. I think we need to benchmark it and see what is the actual difference in memory usage vs. performance. The usual reasons for cache=none (enable Linux native AIO + allow migration with shared non-coherent storage) do not apply to gluster. Paolo